A Comprehensive Review on Machine Learning Approaches for Enhancing Human Speech Recognition

Speech recognition, commonly referred to as automatic speech recognition (ASR), is an area of research that tries to make it possible for computers to comprehend and interpret spoken language. Due to the rising need for intelligent voice-controlled systems and the spread of speech-enabled gadgets like smartphones, virtual assistants, and smart home appliances, it has attracted considerable interest and achieved tremendous breakthroughs in recent years. Numerous fields, including human-computer interaction, natural language processing, telecommunications, healthcare, and automotive technology, can benefit from the capacity to effectively transcribe and understand spoken language. The development of algorithms and models that can automatically translate spoken language into written text is the main objective of voice recognition, which enables computers to interpret and comprehend human speech. By offering more practical and effective ways to enter data, conduct searches, and issue commands, this technology has altered the way people engage with machines. Real-time voice recognition and interpretation capabilities of speech recognition systems make it easier for people with physical limitations to use digital services and enable hands-free communication. Speech recognition technology has developed over time, moving from straightforward isolated word recognition systems to more robust, robust models that can handle continuous speech and adapt to different speakers and languages. Traditional methods merged acoustic and language models with statistical techniques like Hidden Markov Models (HMMs). However, with the introduction of deep learning and neural networks, the industry has made major strides, which have resulted in notable advances in voice recognition performance and accuracy. The goal of this survey article is to give a thorough overview of the most recent developments and methods in the field of voice recognition. The development of speech recognition systems will be known, from the best datasets. Features extraction, classification methods to more recent advancements based on deep learning.

We can learn more about machine learning's potential to revolutionize numerous industries and enhance human-computer interaction by examining how it affects human speech recognition. Developed a method for locating a speaker in an audio stream based on the biometric characteristics of the human voice, such as pitch, loudness, and frequency, a model for unsupervised learning that can learn speech representation from a little dataset. This study made use of the Librispeech dataset, and were able to attain a word error rate of 1.8 [1].

Speech recognition is a field that tries to make it possible for computers to comprehend and interpret human speech while taking into consideration the distinctive qualities that set each person apart from the other, such as speaking style, accent, method of pronunciation, and rhythm. In this field, speaker recognition, which includes speaker verification and speaker identification, is extremely important. Determine the identity of a speaker from a known collection of voices by using speaker identification [2]. The development of speech recognition technology has allowed individuals to control their digital gadgets only with voice commands, doing away with the need for more conventional input devices like a mouse or keyboard [3]. Speech recognition technology has been shown to be a useful tool for language development, helping people to pronounce words well and express themselves more clearly. These systems have also helped to increase the accessibility of education for students who are blind [4]. Speech recognition systems frequently use machine learning techniques, particularly when training speaker utterances using datasets. By examining test utterances, these trained algorithms can then recognize speakers [5]. Recent studies have shown the value of utilizing the signal's power spectral density as a way to improve speaker verification. A unique approach has been put forth that makes use of the signal's power spectrum density and a feature vector with reduced Mel-frequency cepstral coefficients (MFCC) [6]. In speaker recognition, deep learning techniques have also shown promising results. Deep Locally Connected Networks (LCN) and Convolutional Neural Networks (CNN) have both been studied for their efficacy in identifying speakers from text [7]. Additionally, a whole deep learning-based speech system has been created, and it has the potential to outperform existing recognition pipelines, especially in difficult situations like talks and noisy surroundings [8]. CNNs have also been used for speech recognition, directly enabling different types of voice variability through the network's structure [9]. A thorough examination into domain robustness has been carried out [10], which involved training a single model utilizing data from several application domains, sampling rates, noise conditions, and codec settings. In the fields of object and pattern recognition, datasets are essential. we will examine the most popular types of datasets, the best feature extraction strategies, and the most often used algorithms in this subject.

This paper is organized as section 1 shows the introduction, section 2 shows related works of past researchers, section 3 comparsion of speech recognition and speaker recognition, section 4 Types of speaker recognition 5 Types of ASR methods 6 classification of ASR system 7shows the state-of-the-art works, and section 8 shows the discussion and conclusion.

5.1 Open set

Open set ASR techniques are intended to handle speech recognition tasks where the system must convert spoken words into text while allowing for the potential to come across words or expressions that are not already part of the system's established vocabulary. The following are some essential traits and methods related to open set ASR:

Flexible Vocabulary: Open set ASR systems can recognize a variety of words and expressions, even those that aren't expressly listed in their vocabulary.

Out-of-Vocabulary (OOV) Handling: By utilizing contextual information, methods including language models, statistical language models, and neural language models are frequently utilized to enhance OOV word recognition.

Open set ASR systems need to be capable of handling recognition faults well because OOV terms might increase error rates. These errors are minimized using methods like confidence scoring and error correction procedures.

Benefits: Open set ASR offers more flexibility in addressing different speech recognition jobs, especially in situations where the vocabulary is dynamic or constantly changing.

Inaccuracy and vocabulary coverage are hampered by the inclusion of OOV words because the system must successfully generalize and handle unfamiliar words.

5.2 Closed set

Closed set ASR methods are intended for voice recognition jobs where the system only recognizes words or phrases that fall under the scope of a specified vocabulary. Here are some crucial traits and methods related to closed set ASR. Closed set ASR systems are limited in their ability to recognize only the words and phrases that are part of their vocabulary, which is frequently compiled from a single domain or application.

Pronunciation Modeling: Closed set ASR systems frequently use methods like using phonetic dictionaries or acoustic models trained on domain-specific data to model word pronunciations accurately in order to increase accuracy.

Closed set ASR systems can gain from language model optimization that is explicitly customized to the vocabulary and domain of interest.

Closed set ASR systems have the advantage of high recognition accuracy within the limited vocabulary, resulting in more accurate and trustworthy transcriptions.

The limitation of a predefined vocabulary makes closed set ASR less appropriate for applications needing vocabulary expansion or dynamic speech recognition tasks since it restricts the system's flexibility in handling new or unfamiliar terms. It's important to note that some ASR systems can combine the benefits of both open and closed set techniques to meet particular application requirements. The task-specific requirements, including vocabulary quantity, vocabulary variability, and the system's ability to handle new or unfamiliar terms, will determine whether open set or closed set ASR should be used.

As shown in Table 1 about the Librispeech dataset for different authors and numerous time there are authors used the Librispeech dataset with machine learning algorithms and deep learning, in study [5] Findings indicated that on the LibriSpeech dataset, MFCCT features in combination with DNN outperformed the baseline MFCC and time-domain features. In study [1] Librispeech was good but require a more detailed dataset with labels for specific individuals on their audio file.

Table 1. The literature survey on researchers that adopted the libri speech dataset in speech recognition

Table 2. The literature survey on researchers that adopted the TIMIT dataset in speech recognition

Table 3. The literature survey on researchers that adopted the voxforge dataset in speech recognition in speech recognition

Table 4. The literature survey on researchers that adopted the OK Google dataset in speech recognition in speech recognition

The authors used the TIMIT data set, and they reached good results without the need for handcrafted features.

The authors used an OK google dataset collected from anonymized voice search logs. For improved noise robustness they perform multi-style training.

In Table 3 the authors used the Voxforge dataset, which is a free and open-source voice database where various speakers have freely provided speech data towards the creation of speech recognition software. The Voxforge database was picked because it was created with issues like channel variance, session variation, and noise robustness in mind, whereas the majority of corpora for speaker verification were not. Thus, achieving the best results. In Table 4 the authors used an OK google dataset collected from anonymized voice search logs. For improved noise robustness they perform multi-style training.

As shown in Table 5 the authors used CNN classification methods, in study [29] present convolutional neural networks (CNNs) can model raw and tonal speech signals the experimental results show that the current CNN architecture performs significantly, with an accuracy rate of 89.15% and a WER of 10.56% for continuous and broad vocabulary sentences of speech signals with various tones. And in study [27] present a speaker verification method that uses end-to-end CNNs to learn speaker discriminative data straight from the raw audio signal, results have been obtained 98.8%, in study [30] introduced Effective speaker recognition in emotional and noisy talking situations: GMM-CNN model, results have been obtained 84.69%. The previous study [7] analyzes the effectiveness of deep Locally Connected Networks (LCN) and Convolutional Neural Networks (CNN) at recognizing speakers from text, results have been obtained is 96.48%.

Finally in terms of the neural network when CNN is used good results were obtained, but other methods that gave better results.

As shown in Table 6 the authors used the SVM classification method, in study [24] Mel-Frequency Cepstral Coefficients (MFCC), and statistical features are employed as the models' input features in Support Vector Machine (SVM) and Random Forest (RF) models, Compared to Random Forest, the Support Vector Machine has encouraging results with an accuracy of 94%. The authors [25] used a comparative analysis of various classifiers to concentrate on discovering appropriate voice signal properties. The results have shown that the best accuracy in voice pathology detection is achieved using the Support Vector Machine algorithm this approach has an accuracy of roughly 86% when classifying a voice as pathological or healthy. the authors used the output of a genetic algorithm (GA) and the inputs of a NN algorithm were merged to develop a hybrid feature selection approach. SVM, neural networks, and GMM were used for classification. In terms of SVM, the best results were 94.55% accuracy in identifying illnesses [31].

Table 5. The literature survey on researchers that adopted the CNN classification methods in speech recognition in speech recognition

Table 6. The literature survey on researchers that adopted the SVM classification methods in speech recognition in speech recognition

As shown in Table 7 the authors used the RF classification method, in study [6] to demonstrate various audio preprocessing techniques, such as noise reduction and vocal augmentation, to enhance the audios that are now available in real scenarios. The results that the classification procedure was more accurate when a machine learning classifier was used, with (RF) classifiers achieving 97.9% accuracy. Mel-Frequency Cepstral Coefficients (MFCC) [39] and statistical features are employed as the models' input features in Support Vector Machine (SVM) and Random Forest (RF) models, the result has been obtained with RF is 83%. When the authors used RF and SVM achieving the good results but there are other methods achieve better results.

Table 7. The literature survey on researchers that adopted the RF classification methods in speech recognition in speech recognition

As shown in Table 8 the authors used the GMM classification method, study [41] presented Mel-Frequency To model speakers, the Gaussian mixture model-universal background model is used, and cepstral coefficients are used for feature extraction, results show 97.8%. Paper [6] presented a reduced feature vector that makes use of fresh information gleaned from the speaker's speech to carry out GMM-based text-free speaker verification applications, results show 88.3%.

As shown in Table 9, the authors used the DNN classification method. Jahangir et al. [5] introduced a novel fusion of time-based and MFCC features (MFCCT), which combines the efficiency of time-domain and MFCC features to boost the precision of text-independent speaker identification (SI) systems. The speaker identification model was created using the retrieved MFCCT features as input from a deep neural network (DNN), DNN obtained better classification results compared with five machine learning algorithms that were recently utilized in speaker recognition, results show 99.94%. Kabir et al. [20] introduced a novel end-to-end method for speaker verification that uses the same loss for training and evaluation and directly maps the utterance to a score while jointly optimizing the internal speaker representation and the speaker model, proposed an approach improved our best small footprint DNN baseline from over 3% to 2% equal error rate on our internal “Ok Google” benchmark. By looking at the results of the classification methods, it was found that DNN is the method that achieved the highest results.

Table 8. The literature survey on researchers that adopted the GMM classification methods in speech recognition in speech recognition

Table 9. The literature survey on researchers that adopted the DNN classification methods in speech recognition in speech recognition

Table 10. State-of-the-art researcher that adopted MFCC feature extraction in Speech recognition

As shown in Table 10, the authors used MFCC feature extraction, Dua et al. [29] introduced CNNs, MFCCs, and LibROSA were incorporated to reveal the best system performance for recognizing uncommon input speech signals. And in study [24] the key benefit of the MFCC is that it is good at error reduction and capable of producing a robust feature when the signal is influenced by noise. It is a leading strategy and frequently used algorithm in speech feature extraction. MFCC is a technique for detecting frequencies above 1kHz that makes use of human hearing activity [41]. Looking at previous research, we found that MFCC has the best features.

The following important issues are highlighted in this review, which concentrates on the field of speech recognition:

Datasets: TIMIT, LibriSpeech, and Voxforge are the three most often used datasets in speech recognition.

The frequently used dataset TIMIT contains recordings of speakers from different parts of the United States, representing diverse dialects and speech patterns.

English audiobooks make up the dataset LibriSpeech, which was created especially for Large Vocabulary Continuous Speech Recognition (LVCSR) problems. A free and open-source voice database called Voxforge deals with speech recognition problems such channel variance, session variance, and noise robustness. Methods of classification: In speech recognition, DNN, CNN, RF and SVM are excellent classification techniques. DNN's performance in effectively identifying speech patterns and differentiating between speakers is demonstrated by its approximate result of 99% accuracy. CNN's efficiency in speech recognition tasks was demonstrated by the approximate result of 90.69% accuracy.

SVM demonstrated its capacity to categorize speech patterns by achieving an approximate result of 90% accuracy.

Strong Qualities: Mel-Frequency Cepstral Coefficients (MFCC) are the most effective characteristic for voice recognition, especially in distinguishing people. This paper emphasizes the value of speech recognition research datasets like TIMIT, LibriSpeech, and Voxforge. While highlighting the usefulness of classification techniques like DNN, CNN, and SVM, it points out that MFCC is the most effective feature for distinguishing speech patterns and specific people. These discoveries help the speech recognition industry and its applications continue to advance.