Speech recognition
Speech recognition is an
Some speech recognition systems require "training" (also called "enrollment") where an individual speaker reads text or isolated vocabulary into the system. The system analyzes the person's specific voice and uses it to fine-tune the recognition of that person's speech, resulting in increased accuracy. Systems that do not use training are called "speaker-independent"[1] systems. Systems that use training are called "speaker dependent".
Speech recognition applications include
The term voice recognition[3][4][5] or speaker identification[6][7][8] refers to identifying the speaker, rather than what they are saying. Recognizing the speaker can simplify the task of translating speech in systems that have been trained on a specific person's voice or it can be used to authenticate or verify the identity of a speaker as part of a security process.
From the technology perspective, speech recognition has a long history with several waves of major innovations. Most recently, the field has benefited from advances in deep learning and big data. The advances are evidenced not only by the surge of academic papers published in the field, but more importantly by the worldwide industry adoption of a variety of deep learning methods in designing and deploying speech recognition systems.
History
The key areas of growth were: vocabulary size, speaker independence, and processing speed.
Pre-1970
- 1952 – Three Bell Labs researchers, Stephen Balashek,formants in the power spectrum of each utterance.[11]
- 1960 – source-filter model of speech production.
- 1962 – 1962 World's Fair.[12]
- 1966 – Linear predictive coding (LPC), a speech coding method, was first proposed by Fumitada Itakura of Nagoya University and Shuzo Saito of Nippon Telegraph and Telephone (NTT), while working on speech recognition.[13]
- 1969 – Funding at Bell Labs dried up for several years when, in 1969, the influential John Pierce wrote an open letter that was critical of and defunded speech recognition research.[14] This defunding lasted until Pierce retired and James L. Flanagan took over.
Raj Reddy was the first person to take on continuous speech recognition as a graduate student at Stanford University in the late 1960s. Previous systems required users to pause after each word. Reddy's system issued spoken commands for playing chess.
Around this time Soviet researchers invented the dynamic time warping (DTW) algorithm and used it to create a recognizer capable of operating on a 200-word vocabulary.[15] DTW processed speech by dividing it into short frames, e.g. 10ms segments, and processing each frame as a single unit. Although DTW would be superseded by later algorithms, the technique carried on. Achieving speaker independence remained unsolved at this time period.
1970–1990
- 1971 – This revived speech recognition research post John Pierce's letter.
- 1972 – The IEEE Acoustics, Speech, and Signal Processing group held a conference in Newton, Massachusetts.
- 1976 – The first ICASSP was held in Philadelphia, which since then has been a major venue for the publication of research on speech recognition.[19]
During the late 1960s
- By the mid-1980s IBM's Fred Jelinek's team created a voice activated typewriter called Tangora, which could handle a 20,000-word vocabulary[22] Jelinek's statistical approach put less emphasis on emulating the way the human brain processes and understands speech in favor of using statistical modeling techniques like HMMs. (Jelinek's group independently discovered the application of HMMs to speech.[21]) This was controversial with linguists since HMMs are too simplistic to account for many common features of human languages.[23] However, the HMM proved to be a highly useful way for modeling speech and replaced dynamic time warping to become the dominant speech recognition algorithm in the 1980s.[24][25]
- 1982 – Dragon Systems, founded by James and Janet M. Baker,[26] was one of IBM's few competitors.
Practical speech recognition
The 1980s also saw the introduction of the n-gram language model.
- 1987 – The back-off model allowed language models to use multiple length n-grams, and CSELT[27] used HMM to recognize languages (both in software and in hardware specialized processors, e.g. RIPAC).
Much of the progress in the field is owed to the rapidly increasing capabilities of computers. At the end of the DARPA program in 1976, the best computer available to researchers was the PDP-10 with 4 MB ram.[28] It could take up to 100 minutes to decode just 30 seconds of speech.[29]
Two practical products were:
- 1984 – was released the Apricot Portable with up to 4096 words support, of which only 64 could be held in RAM at a time.[30]
- 1987 – a recognizer from Kurzweil Applied Intelligence
- 1990 – Dragon Dictate, a consumer product released in 1990[31][32] AT&T deployed the Voice Recognition Call Processing service in 1992 to route telephone calls without the use of a human operator.[33] The technology was developed by Lawrence Rabiner and others at Bell Labs.
By this point, the vocabulary of the typical commercial speech recognition system was larger than the average human vocabulary.[28] Raj Reddy's former student, Xuedong Huang, developed the Sphinx-II system at CMU. The Sphinx-II system was the first to do speaker-independent, large vocabulary, continuous speech recognition and it had the best performance in DARPA's 1992 evaluation. Handling continuous speech with a large vocabulary was a major milestone in the history of speech recognition. Huang went on to found the speech recognition group at Microsoft in 1993. Raj Reddy's student Kai-Fu Lee joined Apple where, in 1992, he helped develop a speech interface prototype for the Apple computer known as Casper.
Lernout & Hauspie, a Belgium-based speech recognition company, acquired several other companies, including Kurzweil Applied Intelligence in 1997 and Dragon Systems in 2000. The L&H speech technology was used in the Windows XP operating system. L&H was an industry leader until an accounting scandal brought an end to the company in 2001. The speech technology from L&H was bought by ScanSoft which became Nuance in 2005. Apple originally licensed software from Nuance to provide speech recognition capability to its digital assistant Siri.[34]
2000s
In the 2000s DARPA sponsored two speech recognition programs: Effective Affordable Reusable Speech-to-Text (EARS) in 2002 and
In the United States, the
In the early 2000s, speech recognition was still dominated by traditional approaches such as
The use of deep feedforward (non-recurrent) networks for acoustic modeling was introduced during the later part of 2009 by Geoffrey Hinton and his students at the University of Toronto and by Li Deng[51] and colleagues at Microsoft Research, initially in the collaborative work between Microsoft and the University of Toronto which was subsequently expanded to include IBM and Google (hence "The shared views of four research groups" subtitle in their 2012 review paper).[52][53][54] A Microsoft research executive called this innovation "the most dramatic change in accuracy since 1979".[55] In contrast to the steady incremental improvements of the past few decades, the application of deep learning decreased word error rate by 30%.[55] This innovation was quickly adopted across the field. Researchers have begun to use deep learning techniques for language modeling as well.
In the long history of speech recognition, both shallow form and deep form (e.g. recurrent nets) of artificial neural networks had been explored for many years during 1980s, 1990s and a few years into the 2000s.[56][57][58] But these methods never won over the non-uniform internal-handcrafting Gaussian mixture model/hidden Markov model (GMM-HMM) technology based on generative models of speech trained discriminatively.[59] A number of key difficulties had been methodologically analyzed in the 1990s, including gradient diminishing[60] and weak temporal correlation structure in the neural predictive models.[61][62] All these difficulties were in addition to the lack of big training data and big computing power in these early days. Most speech recognition researchers who understood such barriers hence subsequently moved away from neural nets to pursue generative modeling approaches until the recent resurgence of deep learning starting around 2009–2010 that had overcome all these difficulties. Hinton et al. and Deng et al. reviewed part of this recent history about how their collaboration with each other and then with colleagues across four groups (University of Toronto, Microsoft, Google, and IBM) ignited a renaissance of applications of deep feedforward neural networks for speech recognition.[53][54][63][64]
2010s
By early 2010s speech recognition, also called voice recognition[65][66][67] was clearly differentiated from speaker recognition, and speaker independence was considered a major breakthrough. Until then, systems required a "training" period. A 1987 ad for a doll had carried the tagline "Finally, the doll that understands you." – despite the fact that it was described as "which children could train to respond to their voice".[12]
In 2017, Microsoft researchers reached a historical human parity milestone of transcribing conversational telephony speech on the widely benchmarked Switchboard task. Multiple deep learning models were used to optimize speech recognition accuracy. The speech recognition word error rate was reported to be as low as 4 professional human transcribers working together on the same benchmark, which was funded by IBM Watson speech team on the same task.[68]
Models, methods, and algorithms
Both acoustic modeling and language modeling are important parts of modern statistically based speech recognition algorithms. Hidden Markov models (HMMs) are widely used in many systems. Language modeling is also used in many other natural language processing applications such as document classification or statistical machine translation.
Hidden Markov models
Modern general-purpose speech recognition systems are based on hidden Markov models. These are statistical models that output a sequence of symbols or quantities. HMMs are used in speech recognition because a speech signal can be viewed as a piecewise stationary signal or a short-time stationary signal. In a short time scale (e.g., 10 milliseconds), speech can be approximated as a stationary process. Speech can be thought of as a Markov model for many stochastic purposes.
Another reason why HMMs are popular is that they can be trained automatically and are simple and computationally feasible to use. In speech recognition, the hidden Markov model would output a sequence of n-dimensional real-valued vectors (with n being a small integer, such as 10), outputting one of these every 10 milliseconds. The vectors would consist of
Described above are the core elements of the most common, HMM-based approach to speech recognition. Modern speech recognition systems use various combinations of a number of standard techniques in order to improve results over the basic approach described above. A typical large-vocabulary system would need
Decoding of the speech (the term for what happens when the system is presented with a new utterance and must compute the most likely source sentence) would probably use the
A possible improvement to decoding is to keep a set of good candidates instead of just keeping the best candidate, and to use a better scoring function (
Dynamic time warping (DTW)-based speech recognition
Dynamic time warping is an approach that was historically used for speech recognition but has now largely been displaced by the more successful HMM-based approach.
Dynamic time warping is an algorithm for measuring similarity between two sequences that may vary in time or speed. For instance, similarities in walking patterns would be detected, even if in one video the person was walking slowly and if in another he or she were walking more quickly, or even if there were accelerations and deceleration during the course of one observation. DTW has been applied to video, audio, and graphics – indeed, any data that can be turned into a linear representation can be analyzed with DTW.
A well-known application has been automatic speech recognition, to cope with different speaking speeds. In general, it is a method that allows a computer to find an optimal match between two given sequences (e.g., time series) with certain restrictions. That is, the sequences are "warped" non-linearly to match each other. This sequence alignment method is often used in the context of hidden Markov models.
Neural networks
Neural networks emerged as an attractive acoustic modeling approach in ASR in the late 1980s. Since then, neural networks have been used in many aspects of speech recognition such as phoneme classification,
One approach to this limitation was to use neural networks as a pre-processing, feature transformation or dimensionality reduction,[75] step prior to HMM based recognition. However, more recently, LSTM and related recurrent neural networks (RNNs),[39][43][76][77] Time Delay Neural Networks(TDNN's),[78] and transformers[48][49][50] have demonstrated improved performance in this area.
Deep feedforward and recurrent neural networks
Deep neural networks and denoising
A success of DNNs in large vocabulary speech recognition occurred in 2010 by industrial researchers, in collaboration with academic researchers, where large output layers of the DNN based on context dependent HMM states constructed by decision trees were adopted.[81][82] [83] See comprehensive reviews of this development and of the state of the art as of October 2014 in the recent Springer book from Microsoft Research.[84] See also the related background of automatic speech recognition and the impact of various machine learning paradigms, notably including deep learning, in recent overview articles.[85][86]
One fundamental principle of deep learning is to do away with hand-crafted feature engineering and to use raw features. This principle was first explored successfully in the architecture of deep autoencoder on the "raw" spectrogram or linear filter-bank features,[87] showing its superiority over the Mel-Cepstral features which contain a few stages of fixed transformation from spectrograms. The true "raw" features of speech, waveforms, have more recently been shown to produce excellent larger-scale speech recognition results.[88]
End-to-end automatic speech recognition
Since 2014, there has been much research interest in "end-to-end" ASR. Traditional phonetic-based (i.e., all HMM-based model) approaches required separate components and training for the pronunciation, acoustic, and language model. End-to-end models jointly learn all the components of the speech recognizer. This is valuable since it simplifies the training process and deployment process. For example, a n-gram language model is required for all HMM-based systems, and a typical n-gram language model often takes several gigabytes in memory making them impractical to deploy on mobile devices.[89] Consequently, modern commercial ASR systems from Google and Apple (as of 2017[update]) are deployed on the cloud and require a network connection as opposed to the device locally.
The first attempt at end-to-end ASR was with
An alternative approach to CTC-based models are attention-based models. Attention-based ASR models were introduced simultaneously by Chan et al. of
Applications
In-car systems
Typically a manual control input, for example by means of a finger control on the steering-wheel, enables the speech recognition system and this is signaled to the driver by an audio prompt. Following the audio prompt, the system has a "listening window" during which it may accept a speech input for recognition. [citation needed]
Simple voice commands may be used to initiate phone calls, select radio stations or play music from a compatible smartphone, MP3 player or music-loaded flash drive. Voice recognition capabilities vary between car make and model. Some of the most recent[when?] car models offer natural-language speech recognition in place of a fixed set of commands, allowing the driver to use full sentences and common phrases. With such systems there is, therefore, no need for the user to memorize a set of fixed command words.[citation needed]
Education
Automatic
Assessing authentic listener intelligibility is essential for avoiding inaccuracies from
Health care
Medical documentation
In the
One of the major issues relating to the use of speech recognition in healthcare is that the
A more significant issue is that most EHRs have not been expressly tailored to take advantage of voice-recognition capabilities. A large part of the clinician's interaction with the EHR involves navigation through the user interface using menus, and tab/button clicks, and is heavily dependent on keyboard and mouse: voice-based navigation provides only modest ergonomic benefits. By contrast, many highly customized systems for radiology or pathology dictation implement voice "macros", where the use of certain phrases – e.g., "normal report", will automatically fill in a large number of default values and/or generate boilerplate, which will vary with the type of the exam – e.g., a chest X-ray vs. a gastrointestinal contrast series for a radiology system.
Therapeutic use
Prolonged use of speech recognition software in conjunction with
Military
High-performance fighter aircraft
Substantial efforts have been devoted in the last decade to the test and evaluation of speech recognition in
Working with Swedish pilots flying in the JAS-39 Gripen cockpit, Englund (2004) found recognition deteriorated with increasing g-loads. The report also concluded that adaptation greatly improved the results in all cases and that the introduction of models for breathing was shown to improve recognition scores significantly. Contrary to what might have been expected, no effects of the broken English of the speakers were found. It was evident that spontaneous speech caused problems for the recognizer, as might have been expected. A restricted vocabulary, and above all, a proper syntax, could thus be expected to improve recognition accuracy substantially.[115]
The
Speaker-independent systems are also being developed and are under test for the F35 Lightning II (JSF) and the Alenia Aermacchi M-346 Master lead-in fighter trainer. These systems have produced word accuracy scores in excess of 98%.[118]
Helicopters
The problems of achieving high recognition accuracy under stress and noise are particularly relevant in the
As in fighter applications, the overriding issue for voice in helicopters is the impact on pilot effectiveness. Encouraging results are reported for the AVRADA tests, although these represent only a feasibility demonstration in a test environment. Much remains to be done both in speech recognition and in overall speech technology in order to consistently achieve performance improvements in operational settings.
Training air traffic controllers
Training for air traffic controllers (ATC) represents an excellent application for speech recognition systems. Many ATC training systems currently require a person to act as a "pseudo-pilot", engaging in a voice dialog with the trainee controller, which simulates the dialog that the controller would have to conduct with pilots in a real ATC situation. Speech recognition and synthesis techniques offer the potential to eliminate the need for a person to act as a pseudo-pilot, thus reducing training and support personnel. In theory, Air controller tasks are also characterized by highly structured speech as the primary output of the controller, hence reducing the difficulty of the speech recognition task should be possible. In practice, this is rarely the case. The FAA document 7110.65 details the phrases that should be used by air traffic controllers. While this document gives less than 150 examples of such phrases, the number of phrases supported by one of the simulation vendors speech recognition systems is in excess of 500,000.
The USAF, USMC, US Army, US Navy, and FAA as well as a number of international ATC training organizations such as the Royal Australian Air Force and Civil Aviation Authorities in Italy, Brazil, and Canada are currently using ATC simulators with speech recognition from a number of different vendors.[citation needed]
Telephony and other domains
ASR is now commonplace in the field of
The improvement of mobile processor speeds has made speech recognition practical in smartphones. Speech is used mostly as a part of a user interface, for creating predefined or custom speech commands.
People with disabilities
People with disabilities can benefit from speech recognition programs. For individuals that are Deaf or Hard of Hearing, speech recognition software is used to automatically generate a closed-captioning of conversations such as discussions in conference rooms, classroom lectures, and/or religious services.[119]
Students who are blind (see Blindness and education) or have very low vision can benefit from using the technology to convey words and then hear the computer recite them, as well as use a computer by commanding with their voice, instead of having to look at the screen and keyboard.[120]
Students who are physically disabled have a Repetitive strain injury/other injuries to the upper extremities can be relieved from having to worry about handwriting, typing, or working with scribe on school assignments by using speech-to-text programs. They can also utilize speech recognition technology to enjoy searching the Internet or using a computer at home without having to physically operate a mouse and keyboard.[120]
Speech recognition can allow students with learning disabilities to become better writers. By saying the words aloud, they can increase the fluidity of their writing, and be alleviated of concerns regarding spelling, punctuation, and other mechanics of writing.[121] Also, see Learning disability.
The use of voice recognition software, in conjunction with a digital audio recorder and a personal computer running word-processing software has proven to be positive for restoring damaged short-term memory capacity, in stroke and craniotomy individuals.
Speech recognition is also very useful for people who have difficulty using their hands, ranging from mild repetitive stress injuries to involve disabilities that preclude using conventional computer input devices. In fact, people who used the keyboard a lot and developed
This type of technology can help those with dyslexia but other disabilities are still in question. The effectiveness of the product is the problem that is hindering it from being effective. Although a kid may be able to say a word depending on how clear they say it the technology may think they are saying another word and input the wrong one. Giving them more work to fix, causing them to have to take more time with fixing the wrong word.[126]
Further applications
- Aerospace (e.g. space exploration, spacecraft, etc.) NASA's Mars Polar Lander used speech recognition technology from Sensory, Inc. in the Mars Microphone on the Lander[127]
- Automatic subtitling with speech recognition
- Automatic emotion recognition[128]
- Automatic shot listing in audiovisual production
- Automatic translation
- eDiscovery(Legal discovery)
- Hands-free computing: Speech recognition computer user interface
- Home automation
- Interactive voice response
- Mobile telephony, including mobile email
- Multimodal interaction[64]
- Real Time Captioning[129]
- Robotics
- Security, including usage with other biometric scanners for multi-factor authentication[130]
- Speech to text (transcription of speech into text, real time video captioning, Court reporting )
- Telematics (e.g. vehicle Navigation Systems)
- Transcription (digital speech-to-text)
- Video games, with Tom Clancy's EndWar and Lifelineas working examples
- Apple's Siri)
Performance
The performance of speech recognition systems is usually evaluated in terms of accuracy and speed.
Speech recognition by machine is a very complex problem, however. Vocalizations vary in terms of accent, pronunciation, articulation, roughness, nasality, pitch, volume, and speed. Speech is distorted by a background noise and echoes, electrical characteristics. Accuracy of speech recognition may vary with the following:[133][citation needed]
- Vocabulary size and confusability
- Speaker dependence versus independence
- Isolated, discontinuous or continuous speech
- Task and language constraints
- Read versus spontaneous speech
- Adverse conditions
Accuracy
As mentioned earlier in this article, the accuracy of speech recognition may vary depending on the following factors:
- Error rates increase as the vocabulary size grows:
- e.g. the 10 digits "zero" to "nine" can be recognized essentially perfectly, but vocabulary sizes of 200, 5000 or 100000 may have error rates of 3%, 7%, or 45% respectively.
- Vocabulary is hard to recognize if it contains confusing letters:
- e.g. the 26 letters of the English alphabet are difficult to discriminate because they are confusing words (most notoriously, the E-set: "B, C, D, E, G, P, T, V, Z — when "Z" is pronounced "zee" rather than "zed" depending on the English region); an 8% error rate is considered good for this vocabulary.[134]
- Speaker dependence vs. independence:
- A speaker-dependent system is intended for use by a single speaker.
- A speaker-independent system is intended for use by any speaker (more difficult).
- Isolated, Discontinuous or continuous speech
- With isolated speech, single words are used, therefore it becomes easier to recognize the speech.
With discontinuous speech full sentences separated by silence are used, therefore it becomes easier to recognize the speech as well as with isolated speech.
With continuous speech naturally spoken sentences are used, therefore it becomes harder to recognize the speech, different from both isolated and discontinuous speech.
- Task and language constraints
- e.g. Querying application may dismiss the hypothesis "The apple is red."
- e.g. Constraints may be semantic; rejecting "The apple is angry."
- e.g. Syntactic; rejecting "Red is apple the."
Constraints are often represented by grammar.
- Read vs. Spontaneous Speech – When a person reads it's usually in a context that has been previously prepared, but when a person uses spontaneous speech, it is difficult to recognize the speech because of the disfluencies (like "uh" and "um", false starts, incomplete sentences, stuttering, coughing, and laughter) and limited vocabulary.
- Adverse conditions – Environmental noise (e.g. Noise in a car or a factory). Acoustical distortions (e.g. echoes, room acoustics)
Speech recognition is a multi-leveled pattern recognition task.
- Acoustical signals are structured into a hierarchy of units, e.g. Phonemes, Words, Phrases, and Sentences;
- Each level provides additional constraints;
e.g. Known word pronunciations or legal word sequences, which can compensate for errors or uncertainties at a lower level;
- This hierarchy of constraints is exploited. By combining decisions probabilistically at all lower levels, and making more deterministic decisions only at the highest level, speech recognition by a machine is a process broken into several phases. Computationally, it is a problem in which a sound pattern has to be recognized or classified into a category that represents a meaning to a human. Every acoustic signal can be broken into smaller more basic sub-signals. As the more complex sound signal is broken into the smaller sub-sounds, different levels are created, where at the top level we have complex sounds, which are made of simpler sounds on the lower level, and going to lower levels, even more, we create more basic and shorter and simpler sounds. At the lowest level, where the sounds are the most fundamental, a machine would check for simple and more probabilistic rules of what sound should represent. Once these sounds are put together into more complex sounds on upper level, a new set of more deterministic rules should predict what the new complex sound should represent. The most upper level of a deterministic rule should figure out the meaning of complex expressions. In order to expand our knowledge about speech recognition, we need to take into consideration neural networks. There are four steps of neural network approaches:
- Digitize the speech that we want to recognize
For telephone speech the sampling rate is 8000 samples per second;
- Compute features of spectral-domain of the speech (with Fourier transform);
computed every 10 ms, with one 10 ms section called a frame;
Analysis of four-step neural network approaches can be explained by further information. Sound is produced by air (or some other medium) vibration, which we register by ears, but machines by receivers. Basic sound creates a wave which has two descriptions: amplitude (how strong is it), and frequency (how often it vibrates per second). Accuracy can be computed with the help of word error rate (WER). Word error rate can be calculated by aligning the recognized word and referenced word using dynamic string alignment. The problem may occur while computing the word error rate due to the difference between the sequence lengths of the recognized word and referenced word.
The formula to compute the word error rate (WER) is:
where s is the number of substitutions, d is the number of deletions, i is the number of insertions, and n is the number of word references.
While computing, the word recognition rate (WRR) is used. The formula is:
where h is the number of correctly recognized words:
Security concerns
Speech recognition can become a means of attack, theft, or accidental operation. For example, activation words like "Alexa" spoken in an audio or video broadcast can cause devices in homes and offices to start listening for input inappropriately, or possibly take an unwanted action.[135] Voice-controlled devices are also accessible to visitors to the building, or even those outside the building if they can be heard inside. Attackers may be able to gain access to personal information, like calendar, address book contents, private messages, and documents. They may also be able to impersonate the user to send messages or make online purchases.
Two attacks have been demonstrated that use artificial sounds. One transmits ultrasound and attempt to send commands without nearby people noticing.[136] The other adds small, inaudible distortions to other speech or music that are specially crafted to confuse the specific speech recognition system into recognizing music as speech, or to make what sounds like one command to a human sound like a different command to the system.[137]
Further information
Conferences and journals
Popular speech recognition conferences held each year or two include SpeechTEK and SpeechTEK Europe,
Books
Books like "Fundamentals of Speech Recognition" by
A good and accessible introduction to speech recognition technology and its history is provided by the general audience book "The Voice in the Machine. Building Computers That Understand Speech" by Roberto Pieraccini (2012).
The most recent book on speech recognition is Automatic Speech Recognition: A Deep Learning Approach (Publisher: Springer) written by Microsoft researchers D. Yu and L. Deng and published near the end of 2014, with highly mathematically oriented technical detail on how deep learning methods are derived and implemented in modern speech recognition systems based on DNNs and related deep learning methods.[84] A related book, published earlier in 2014, "Deep Learning: Methods and Applications" by L. Deng and D. Yu provides a less technical but more methodology-focused overview of DNN-based speech recognition during 2009–2014, placed within the more general context of deep learning applications including not only speech recognition but also image recognition, natural language processing, information retrieval, multimodal processing, and multitask learning.[80]
Software
In terms of freely available resources, Carnegie Mellon University's Sphinx toolkit is one place to start to both learn about speech recognition and to start experimenting. Another resource (free but copyrighted) is the HTK book (and the accompanying HTK toolkit). For more recent and state-of-the-art techniques, Kaldi toolkit can be used.[139] In 2017 Mozilla launched the open source project called Common Voice[140] to gather big database of voices that would help build free speech recognition project DeepSpeech (available free at GitHub),[141] using Google's open source platform TensorFlow.[142] When Mozilla redirected funding away from the project in 2020, it was forked by its original developers as Coqui STT[143] using the same open-source license.[144][145]
Google Gboard supports speech recognition on all Android applications. It can be activated through the microphone icon.[146]
The commercial cloud based speech recognition APIs are broadly available.
For more software resources, see List of speech recognition software.
See also
- AI effect
- ALPAC
- Applications of artificial intelligence
- Articulatory speech recognition
- Audio mining
- Audio-visual speech recognition
- Automatic Language Translator
- Automotive head unit
- Braina
- Cache language model
- Dragon NaturallySpeaking
- Fluency Voice Technology
- Google Voice Search
- IBM ViaVoice
- Keyword spotting
- Kinect
- Mondegreen
- Multimedia information retrieval
- Origin of speech
- Phonetic search technology
- Speaker diarisation
- Speaker recognition
- Speech analytics
- Speech interface guideline
- Speech recognition software for Linux
- Speech synthesis
- Speech verification
- Subtitle (captioning)
- VoiceXML
- VoxForge
- Windows Speech Recognition
- Lists
References
- ^ "Speaker Independent Connected Speech Recognition- Fifth Generation Computer Corporation". Fifthgen.com. Archived from the original on 11 November 2013. Retrieved 15 June 2013.
- S2CID 13482115.
- ^ "British English definition of voice recognition". Macmillan Publishers Limited. Archived from the original on 16 September 2011. Retrieved 21 February 2012.
- ^ "voice recognition, definition of". WebFinance, Inc. Archived from the original on 3 December 2011. Retrieved 21 February 2012.
- ^ "The Mailbag LG #114". Linuxgazette.net. Archived from the original on 19 February 2013. Retrieved 15 June 2013.
- S2CID 220665533.
- (PDF) from the original on 8 March 2014. Retrieved 21 February 2014.
- ^ "Speaker Identification (WhisperID)". Microsoft Research. Microsoft. Archived from the original on 25 February 2014. Retrieved 21 February 2014.
When you speak to someone, they don't just recognize what you say: they recognize who you are. WhisperID will let computers do that, too, figuring out who you are by the way you sound.
- ^ "Obituaries: Stephen Balashek". The Star-Ledger. 22 July 2012.
- ^ "IBM-Shoebox-front.jpg". androidauthority.net. Retrieved 4 April 2019.
- ^ Juang, B. H.; Rabiner, Lawrence R. "Automatic speech recognition–a brief history of the technology development" (PDF). p. 6. Archived (PDF) from the original on 17 August 2014. Retrieved 17 January 2015.
- ^ a b Melanie Pinola (2 November 2011). "Speech Recognition Through the Decades: How We Ended Up With Siri". PC World. Retrieved 22 October 2018.
- ISSN 1932-8346.
- .
- ISBN 978-3540491255.
- ^ John Makhoul. "ISCA Medalist: For leadership and extensive contributions to speech and language processing". Archived from the original on 24 January 2018. Retrieved 23 January 2018.
- ^ Blechman, R. O.; Blechman, Nicholas (23 June 2008). "Hello, Hal". The New Yorker. Archived from the original on 20 January 2015. Retrieved 17 January 2015.
- doi:10.1121/1.381666.
- ^ Rabiner (1984). "The Acoustics, Speech, and Signal Processing Society. A Historical Perspective" (PDF). Archived (PDF) from the original on 9 August 2017. Retrieved 23 January 2018.
- ^ "First-Hand:The Hidden Markov Model – Engineering and Technology History Wiki". ethw.org. 12 January 2015. Archived from the original on 3 April 2018. Retrieved 1 May 2018.
- ^ a b "James Baker interview". Archived from the original on 28 August 2017. Retrieved 9 February 2017.
- ^ "Pioneering Speech Recognition". 7 March 2012. Archived from the original on 19 February 2015. Retrieved 18 January 2015.
- S2CID 6175701. Archived from the originalon 8 December 2023.
- ^ Juang, B. H.; Rabiner, Lawrence R. "Automatic speech recognition–a brief history of the technology development" (PDF): 10. Archived (PDF) from the original on 17 August 2014. Retrieved 17 January 2015.
{{cite journal}}
: Cite journal requires|journal=
(help) - S2CID 259502346.
- ^ "History of Speech Recognition". Dragon Medical Transcription. Archived from the original on 13 August 2015. Retrieved 17 January 2015.
- .
- ^ a b Xuedong Huang; James Baker; Raj Reddy. "A Historical Perspective of Speech Recognition". Communications of the ACM. Archived from the original on 20 January 2015. Retrieved 20 January 2015.
- ^ Kevin McKean (8 April 1980). "When Cole talks, computers listen". Sarasota Journal. AP. Retrieved 23 November 2015.
- ^ "ACT/Apricot - Apricot history". actapricot.org. Retrieved 2 February 2016.
- ^ Melanie Pinola (2 November 2011). "Speech Recognition Through the Decades: How We Ended Up With Siri". PC World. Archived from the original on 13 January 2017. Retrieved 28 July 2017.
- ^ "Ray Kurzweil biography". KurzweilAINetwork. Archived from the original on 5 February 2014. Retrieved 25 September 2014.
- ^ Juang, B.H.; Rabiner, Lawrence. "Automatic Speech Recognition – A Brief History of the Technology Development" (PDF). Archived (PDF) from the original on 9 August 2017. Retrieved 28 July 2017.
{{cite journal}}
: Cite journal requires|journal=
(help) - ^ "Nuance Exec on iPhone 4S, Siri, and the Future of Speech". Tech.pinions. 10 October 2011. Archived from the original on 19 November 2011. Retrieved 23 November 2011.
- ^ "Switchboard-1 Release 2". Archived from the original on 11 July 2017. Retrieved 26 July 2017.
- ^ Jason Kincaid (13 February 2011). "The Power of Voice: A Conversation With The Head Of Google's Speech Technology". Tech Crunch. Archived from the original on 21 July 2015. Retrieved 21 July 2015.
- ^ Froomkin, Dan (5 May 2015). "THE COMPUTERS ARE LISTENING". The Intercept. Archived from the original on 27 June 2015. Retrieved 20 June 2015.
- ^ Herve Bourlard and Nelson Morgan, Connectionist Speech Recognition: A Hybrid Approach, The Kluwer International Series in Engineering and Computer Science; v. 247, Boston: Kluwer Academic Publishers, 1994.
- ^ S2CID 1915014.
- S2CID 11715509.
- ^ Alex Graves, Santiago Fernandez, Faustino Gomez, and Jürgen Schmidhuber (2006). Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural nets. Proceedings of ICML'06, pp. 369–376.
- ^ Santiago Fernandez, Alex Graves, and Jürgen Schmidhuber (2007). An application of recurrent neural networks to discriminative keyword spotting[permanent dead link]. Proceedings of ICANN (2), pp. 220–229.
- ^ a b Haşim Sak, Andrew Senior, Kanishka Rao, Françoise Beaufays and Johan Schalkwyk (September 2015): "Google voice search: faster and more accurate." Archived 9 March 2016 at the Wayback Machine
- arXiv:2010.11929 [cs.CV].
- arXiv:2103.15808 [cs.CV].
- ^ Vaswani, Ashish; Shazeer, Noam; Parmar, Niki; Uszkoreit, Jakob; Jones, Llion; Gomez, Aidan N; Kaiser, Łukasz; Polosukhin, Illia (2017). "Attention is All you Need". Advances in Neural Information Processing Systems. 30. Curran Associates.
- arXiv:1810.04805 [cs.CL].
- ^ arXiv:2104.01778 [cs.SD].
- ^ arXiv:2203.09581 [cs.CV].
- ^ arXiv:2104.00120 [eess.AS].
- ^ "Li Deng". Li Deng Site.
- ^ NIPS Workshop: Deep Learning for Speech Recognition and Related Applications, Whistler, BC, Canada, Dec. 2009 (Organizers: Li Deng, Geoff Hinton, D. Yu).
- ^ S2CID 206485943.
- ^ S2CID 13953660.
- ^ a b Markoff, John (23 November 2012). "Scientists See Promise in Deep-Learning Programs". New York Times. Archived from the original on 30 November 2012. Retrieved 20 January 2015.
- ^ Morgan, Bourlard, Renals, Cohen, Franco (1993) "Hybrid neural network/hidden Markov model systems for continuous speech recognition. ICASSP/IJPRAI"
- S2CID 62446313.
- ^ Waibel, Hanazawa, Hinton, Shikano, Lang. (1989) "Phoneme recognition using time-delay neural networks. IEEE Transactions on Acoustics, Speech, and Signal Processing."
- S2CID 357467.
- ^ Sepp Hochreiter (1991), Untersuchungen zu dynamischen neuronalen Netzen Archived 6 March 2015 at the Wayback Machine, Diploma thesis. Institut f. Informatik, Technische Univ. Munich. Advisor: J. Schmidhuber.
- ^ Bengio, Y. (1991). Artificial Neural Networks and their Application to Speech/Sequence Recognition (Ph.D.). McGill University.
- .
- ^ Keynote talk: Recent Developments in Deep Neural Networks. ICASSP, 2013 (by Geoff Hinton).
- ^ a b Keynote talk: "Achievements and Challenges of Deep Learning: From Speech Analysis and Recognition To Language and Multimodal Processing Archived 5 March 2021 at the Wayback Machine," Interspeech, September 2014 (by Li Deng).
- ^ "Improvements in voice recognition software increase". TechRepublic.com. 27 August 2002. Archived from the original on 23 October 2018. Retrieved 22 October 2018.
Maners said IBM has worked on advancing speech recognition ... or on the floor of a noisy trade show.
- ^ "Voice Recognition To Ease Travel Bookings: Business Travel News". BusinessTravelNews.com. 3 March 1997.
The earliest applications of speech recognition software were dictation ... Four months ago, IBM introduced a 'continual dictation product' designed to ... debuted at the National Business Travel Association trade show in 1994.
- ^ Ellis Booker (14 March 1994). "Voice recognition enters the mainstream". Computerworld. p. 45.
Just a few years ago, speech recognition was limited to ...
- ^ "Microsoft researchers achieve new conversational speech recognition milestone". Microsoft. 21 August 2017.
- from the original on 25 July 2011. Retrieved 28 March 2011.
- (PDF) from the original on 18 March 2012. Retrieved 28 March 2011.
- S2CID 9563026.
- S2CID 216472225.
- .
- ^ S. A. Zahorian, A. M. Zimmer, and F. Meng, (2002) "Vowel Classification for Computer based Visual Feedback for Speech Training for the Hearing Impaired," in ICSLP 2002
- ^ Hu, Hongbing; Zahorian, Stephen A. (2010). "Dimensionality Reduction Methods for HMM Phonetic Recognition" (PDF). ICASSP 2010. Archived (PDF) from the original on 6 July 2012.
- ^ Fernandez, Santiago; Graves, Alex; Schmidhuber, Jürgen (2007). "Sequence labelling in structured domains with hierarchical recurrent neural networks" (PDF). Proceedings of IJCAI. Archived (PDF) from the original on 15 August 2017.
- ]. ICASSP 2013.
- (PDF) from the original on 29 June 2016.
- ^ Maas, Andrew L.; Le, Quoc V.; O'Neil, Tyler M.; Vinyals, Oriol; Nguyen, Patrick; Ng, Andrew Y. (2012). "Recurrent Neural Networks for Noise Reduction in Robust ASR". Proceedings of Interspeech 2012.
- ^ (PDF) from the original on 22 October 2014.
- ^ Yu, D.; Deng, L.; Dahl, G. (2010). "Roles of Pre-Training and Fine-Tuning in Context-Dependent DBN-HMMs for Real-World Speech Recognition" (PDF). NIPS Workshop on Deep Learning and Unsupervised Feature Learning.
- S2CID 14862572.
- ^ Deng L., Li, J., Huang, J., Yao, K., Yu, D., Seide, F. et al. Recent Advances in Deep Learning for Speech Research at Microsoft. ICASSP, 2013.
- ^ a b Yu, D.; Deng, L. (2014). "Automatic Speech Recognition: A Deep Learning Approach (Publisher: Springer)".
{{cite journal}}
: Cite journal requires|journal=
(help) - S2CID 16585863.
- .
- ^ L. Deng, M. Seltzer, D. Yu, A. Acero, A. Mohamed, and G. Hinton (2010) Binary Coding of Speech Spectrograms Using a Deep Auto-encoder. Interspeech.
- ^ Tüske, Zoltán; Golik, Pavel; Schlüter, Ralf; Ney, Hermann (2014). "Acoustic Modeling with Deep Neural Networks Using Raw Time Signal for LVCSR" (PDF). Interspeech 2014. Archived (PDF) from the original on 21 December 2016.
- ^ Jurafsky, Daniel (2016). Speech and Language Processing.
- ^ Graves, Alex (2014). "Towards End-to-End Speech Recognition with Recurrent Neural Networks" (PDF). ICML. Archived from the original (PDF) on 10 January 2017. Retrieved 22 July 2019.
- arXiv:1512.02595 [cs.CL].
- ^ "LipNet: How easy do you think lipreading is?". YouTube. Archived from the original on 27 April 2017. Retrieved 5 May 2017.
- arXiv:1611.01599 [cs.CV].
- arXiv:1807.05162 [cs.CV].
- ^ Chan, William; Jaitly, Navdeep; Le, Quoc; Vinyals, Oriol (2016). "Listen, Attend and Spell: A Neural Network for Large Vocabulary Conversational Speech Recognition" (PDF). ICASSP.
- arXiv:1508.04395 [cs.CL].
- arXiv:1612.02695 [cs.NE].
- arXiv:1610.03035 [stat.ML].
- S2CID 1662180.
- S2CID 264426545
- S2CID 209353525.
- ^ Loukina, Anastassia; et al. (6 September 2015), "Pronunciation accuracy and intelligibility of non-native speech" (PDF), INTERSPEECH 2015, Dresden, Germany: International Speech Communication Association, pp. 1917–1921,
only 16% of the variability in word-level intelligibility can be explained by the presence of obvious mispronunciations.
- S2CID 86440885.
pronunciation researchers are primarily interested in improving L2 learners' intelligibility and comprehensibility, but they have not yet collected sufficient amounts of representative and reliable data (speech recordings with corresponding annotations and judgments) indicating which errors affect these speech dimensions and which do not. These data are essential to train ASR algorithms to assess L2 learners' intelligibility.
- ^ Eskenazi, Maxine (January 1999). "Using automatic speech processing for foreign language pronunciation tutoring: Some issues and a prototype". Language Learning & Technology. 2 (2): 62–76. Retrieved 11 February 2023.
- ^ Tholfsen, Mike (9 February 2023). "Reading Coach in Immersive Reader plus new features coming to Reading Progress in Microsoft Teams". Techcommunity Education Blog. Microsoft. Retrieved 12 February 2023.
- ^ Banerji, Olina (7 March 2023). "Schools Are Using Voice Technology to Teach Reading. Is It Helping?". EdSurge News. Retrieved 7 March 2023.
- S2CID 13790002.
- ^ "Computer says no: Irish vet fails oral English test needed to stay in Australia". The Guardian. Australian Associated Press. 8 August 2017. Retrieved 12 February 2023.
- ^ Ferrier, Tracey (9 August 2017). "Australian ex-news reader with English degree fails robot's English test". The Sydney Morning Herald. Retrieved 12 February 2023.
- ^ Main, Ed; Watson, Richard (9 February 2022). "The English test that ruined thousands of lives". BBC News. Retrieved 12 February 2023.
- ^ Joyce, Katy Spratte (24 January 2023). "13 Words That Can Be Pronounced Two Ways". Reader's Digest. Retrieved 23 February 2023.
- ^ E.g., CMUDICT, "The CMU Pronouncing Dictionary". www.speech.cs.cmu.edu. Retrieved 15 February 2023. Compare "four" given as "F AO R" with the vowel AO as in "caught," to "row" given as "R OW" with the vowel OW as in "oat."
- . Retrieved 17 December 2023.
- OCLC 1090351600.
- ^ Englund, Christine (2004). Speech recognition in the JAS 39 Gripen aircraft: Adaptation to speech at different G-loads (PDF) (Masters thesis). Stockholm Royal Institute of Technology. Archived (PDF) from the original on 2 October 2008.
- ^ "The Cockpit". Eurofighter Typhoon. Archived from the original on 1 March 2017.
- ^ "Eurofighter Typhoon – The world's most advanced fighter aircraft". www.eurofighter.com. Archived from the original on 11 May 2013. Retrieved 1 May 2018.
- ^ Schutte, John (15 October 2007). "Researchers fine-tune F-35 pilot-aircraft speech system". United States Air Force. Archived from the original on 20 October 2007.
- ^ "Overcoming Communication Barriers in the Classroom". MassMATCH. 18 March 2010. Archived from the original on 25 July 2013. Retrieved 15 June 2013.
- ^ a b "Speech Recognition for Learning". National Center for Technology Innovation. 2010. Archived from the original on 13 April 2014. Retrieved 26 March 2014.
- ^ Follensbee, Bob; McCloskey-Dale, Susan (2000). "Speech recognition in schools: An update from the field". Technology And Persons With Disabilities Conference 2000. Archived from the original on 21 August 2006. Retrieved 26 March 2014.
- ^ "Speech recognition for disabled people". Archived from the original on 4 April 2008.
- ^ Friends International Support Group
- S2CID 142730664.
- ^ Forgrave, Karen E. "Assistive Technology: Empowering Students with Disabilities." Clearing House 75.3 (2002): 122–6. Web.
- S2CID 143159997.
- ^ "Projects: Planetary Microphones". The Planetary Society. Archived from the original on 27 January 2012.
- ISBN 978-0-387-74160-4.
- ^ "What is real-time captioning? | DO-IT". www.washington.edu. Retrieved 11 April 2021.
- ISBN 978-981-10-3237-0.
- ^ Ciaramella, Alberto. "A prototype performance evaluation report." Sundial workpackage 8000 (1993).
- S2CID 57374050.
- ^ National Institute of Standards and Technology. "The History of Automatic Speech Recognition Evaluation at NIST Archived 8 October 2013 at the Wayback Machine".
- ^ "Letter Names Can Cause Confusion and Other Things to Know About Letter–Sound Relationships". NAEYC. Retrieved 27 October 2023.
- ^ "Listen Up: Your AI Assistant Goes Crazy For NPR Too". NPR. 6 March 2016. Archived from the original on 23 July 2017.
- ^ Claburn, Thomas (25 August 2017). "Is it possible to control Amazon Alexa, Google Now using inaudible commands? Absolutely". The Register. Archived from the original on 2 September 2017.
- ^ "Attack Targets Automatic Speech Recognition Systems". vice.com. 31 January 2018. Archived from the original on 3 March 2018. Retrieved 1 May 2018.
- ISBN 978-0-387-77591-3. Archivedfrom the original on 31 January 2018.
- ^ Povey, D., Ghoshal, A., Boulianne, G., Burget, L., Glembek, O., Goel, N., ... & Vesely, K. (2011). The Kaldi speech recognition toolkit. In IEEE 2011 workshop on automatic speech recognition and understanding (No. CONF). IEEE Signal Processing Society.
- ^ "Common Voice by Mozilla". voice.mozilla.org. Archived from the original on 27 February 2020. Retrieved 9 November 2019.
- ^ "A TensorFlow implementation of Baidu's DeepSpeech architecture: mozilla/DeepSpeech". 9 November 2019 – via GitHub.
- ^ "GitHub - tensorflow/docs: TensorFlow documentation". 9 November 2019 – via GitHub.
- ^ "Coqui, a startup providing open speech tech for everyone". GitHub. Retrieved 7 March 2022.
- ISSN 1357-0978. Retrieved 16 October 2021.
- ^ "Why you should move from DeepSpeech to coqui.ai". Mozilla Discourse. 7 July 2021. Retrieved 16 October 2021.
- ^ "Type with your voice".
Further reading
- Cole, Ronald; ISBN 978-0-521-59277-2.
- Junqua, J.-C.; Haton, J.-P. (1995). Robustness in Automatic Speech Recognition: Fundamentals and Applications. Kluwer Academic Publishers. ISBN 978-0-7923-9646-8.
- Karat, Clare-Marie; Vergo, John; Nahamoo, David (2007). "Conversational Interface Technologies". In ISBN 978-0-8058-5870-9.
- Pieraccini, Roberto (2012). The Voice in the Machine. Building Computers That Understand Speech. The MIT Press. ISBN 978-0262016858.
- Pirani, Giancarlo, ed. (2013). Advanced algorithms and architectures for speech understanding. Springer Science & Business Media. ISBN 978-3-642-84341-9.
- Signer, Beat and Hoste, Lode: SpeeG2: A Speech- and Gesture-based Interface for Efficient Controller-free Text Entry, In Proceedings of ICMI 2013, 15th International Conference on Multimodal Interaction, Sydney, Australia, December 2013
- Woelfel, Matthias; McDonough, John (26 May 2009). Distant Speech Recognition. Wiley. ISBN 978-0470517048.
External links
- Speech Technology at Curlie