Multilingual Speech Recognition Platform - Dual Channel Speech Recognition SystemNTT Human Informatics Laboratories

NTT Media Intelligence Laboratories has continuously worked to develop the VoiceRex series of speech recognition engines, and has strong records of introducing this series to meeting minutes recording support systems, call center call analysis, and voice dialog agents.
Recently, we have developed, as the latest "multilingual speech recognition platform" with the latest multilingual speech recognition engine "VoiceRex NX 2020" as the core, "Dual Channel Speech Recognition System" that realizes the enhancement of the speech recognition of spoken languages using the context of the conversations by two people and the reduction of the calculation resources by the size reduction of speech recognition models. Furthermore, we have constructed an integrated API with spoken language identification and non-linguistic information recognition. As a result of this, the usage of voice input interfaces will be expanded.

While the voice application programs that operate with a short speech such as smartphone apps, smart speakers, car navigations, and so forth is widely spreading partly because of the establishment of practical, high precision speech recognition, expectations are getting higher for the speech recognition of the spoken languages by people as the next target. A rapid expansion of the business scale can be expected by envisioning the application not only to the analyses of the telephone conversations at call centers, which has been already established as a business field of speech recognition, but also to sales telephone conversations and to the speech recognition of business conversations in general.

• It uses cutting edge DNN technology to realize highly precise speech recognition even in noisy environments or from distant speakers, and also supports multiple languages.
• It realizes the mechanism for grasping the context of the conversation by two people and the sound model lightening technology for considerably reducing the amount of the processing increased accordingly
• It is possible to add words for each company or user, including product name, personal telephone directories, etc. without increasing server resources (especially memory).

• Analyses of call center telephone conversations and business telephone calls
• AI voice dialog agents with smartphones, smart speakers, robots, vehicle-equipped terminals, etc.
• Support to generating meeting minutes at meetings

To make highly accurate spontaneous speech recognition, it is necessary to reduce incorrect recognitions deviating from context by the prediction/analogy of phrases and words by grasping the context of a conversation like humans. We assumed the scene of a conversation made by two people like a call center telephone call or a business telephone and have realized the highly accurate enhancement of the spontaneous speech recognition by grasping the flow (context) of the conversation and introducing a mechanism to make prediction/analogy about the speech.

• Processing the speech recognition at the same time in the two channels of a conversation made by two people and using the past recognition result text for the next speech recognition
• Judging the recent topics and selecting language models dynamically and appropriately

Furthermore, we have realized the lightening of the acoustic model size directly linked with the processing without degradation in the speech recognition accuracy as a mechanism that considerably reduces the amount of the processing of the speech recognition increasing in the simultaneous processing of the two channels.

Besides that, this technology has been developed as a software program / upward compatible with the "multilingual speech recognition platforms" developed so far and, thus, continues to have the functions of the predecessors.

• Besides the adoption of CNN-NIN (Convolutional Neural Network and Network In Network), one of the DNN technologies resistible to noise, which has achieved the top-level accuracy at CHiME-3, an international event for evaluating speech recognition technologies under the noisy circumstances in public areas, in consideration to the use scenes with noise or at a remote distance, the high accuracy has been realized by constructing/intensifying the large-scale learning data simulating a variety of the environments assumable and examining the optimum DNN learning parameters.
• Twelve languages mainly of the Asian countries overseas from which there are a large number of visitors to Japan are supported. As a result of this, it can be possible to easily cope with the multiple languages in speech recognition services targeting words and/or short speeches used for addressing mainly a machine like a dialog with a robot, for example. At the same time with this, as for the speech recognition targeting the long sentences/spoken languages found in conversations mainly between a person and another person like those at a call center for example, two languages, Japanese and English, are supported.
• A immediate word adding function has been realized that can easily add the product names and service names, music play lists, and telephone directories customized for corporations and/or individuals, the nicknames added to robots and/or agents, and so forth.
• A spoken language identifying technology is equipped to discriminate the language by using only the features of speech.
• A hybrid speech recognition is possible to make a server speech recognition operate in parallel and, thus, to make responses to the user in appropriate timings and to present the best result.

We have constructed an API to make possible to acquire a variety of information included in voice at a shot by integrating each of the following API's we have developed so far as the recognizing technology by using voice: "speech recognition," "spoken language identification," and "non-linguistic information recognition."
We, as a result of this, think that this will lead to the planning and/or development for the new services leading to the differentiation from those of other parties by using the voice media processing.

Speech Recognition
This technology uses a machine to determine what was said from words spoken by people. Using speech recognition, instructions and data can be input to computers and machines just by speaking.

DNN (Deep Neural Network)
Deep Neural Network (DNN) is a large-scale neural network that imitates human neural circuit. In the case of speech recognition, a large amount of voice data and text data are used to do deep learning, a type of machine learning, and build this DNN.

CNN-NIN(Convolutional Neural Network and Network In Network)
This is one type of DNN widely used in image recognition, and is used as an acoustic model that is well-suited for speech distortions in noisy environments.

Acoustic Model
This is a voice model created from a large number of samples, which forms patterns of the feature quantity of voices. It is possible to recognize one specific speaker if the model is created with voice samples from one specific speaker, or recognize multiple specific speakers if the model is created with voice samples from multiple speakers.

Hybrid Speech Recognition
This is speech recognition performed by doing recognition processing of input speech on the local terminal and the server simultaneously, then compares the results of each to select the best result for output.

Immediate Word Addition
Normally, when adding words, it is necessary to add that word to a word dictionary offline beforehand, convert it to WFST (Weighted Finite-State Transducer), load it on the speech recognition server. Immediate word addition is a technique that enables recognition adding words to WFST immediately before speech recognition, rather than doing it offline.

Word Dictionary
This is a list of words and readings that are the target of recognition. Words not registered to the word dictionary will not be recognized, and be recognized as a registered word instead, so it is necessary to create a word dictionary according to the use scene.

NTT Media Intelligence Laboratories - Cognitive Information Processing Laboratory