Utilizing reinforcement learning and support vector machine to enhance feedback mechanism in English speech training

Abstract

This study proposes an innovative approach to enhance the feedback mechanism in English speech training by combining reinforcement learning and support vector machine (SVM) technologies. Traditional feedback mechanisms for English speech training face challenges such as feedback delay, subjectivity, insufficient personalization, technical limitations, and lack of long-term analysis. To address these issues, the SVM model is used to accurately recognize English phonemes or words, enabling real-time evaluation of students’ pronunciation. The reinforcement learning mechanism leverages these real-time evaluations to make personalized adjustments, offering immediate feedback and dynamic training plans. The integration of these two technologies resulted in a multi-dimensional feedback system for assessing pronunciation performance and providing continuous improvement suggestions through long-term data analysis. Experimental results show that the reinforcement learning system using a Q-learning model maintained a high reward mean, significantly improving training effectiveness. Additionally, the SVM model achieved a pronunciation classification accuracy of over 86.54%, with an F1-score of 94.65%. In an 8-week experiment, the experimental group (using the combined Q-learning and SVM system) outperformed the control group (using traditional methods) in pronunciation scores. The experimental group’s score range expanded to 85–94, with a median increase to 89, while the control group’s range was 78–87, with a median increase to 82. The learning progress in the experimental group was significantly higher, demonstrating the effectiveness and superiority of the proposed method.

Keywords

feedback mechanism reinforcement learning support vector machine English pronunciation training long-term data analysis

Introduction

English pronunciation training is the core link to improve the pronunciation accuracy and fluency of language learners. In the context of globalization, good English pronunciation has become particularly important in the professional and academic fields. Fluent and accurate English speaking not only affects a person’s confidence in cross-cultural communication, but is also directly related to his or her career development in an international work environment. However, existing English pronunciation training methods face many challenges, which makes it a pressing issue to effectively improve learners’ pronunciation quality and fluency. Traditional speech training methods^1,2 mainly rely on real-time feedback from human teachers^3,4 and rule-based speech recognition techniques.^5,6 However, current speech training and evaluation methods face challenges such as feedback delay, strong subjectivity, insufficient personalization, technical limitations, and lack of long-term data analysis, which affect training effectiveness and tracking of student progress.

This article proposes a combination of reinforcement learning^7,8 and support vector machine^9,10 techniques to enhance the feedback mechanism in English speech training.^11,12 The innovation of this article is to provide real-time personalized feedback through reinforcement learning, so as to make precise adjustments based on the pronunciation characteristics of students, overcoming the feedback delay problem in traditional methods. At the same time, SVM performs well in pronunciation classification and pattern recognition, providing students with objective pronunciation evaluation and specific improvement suggestions. Reinforcement learning provides real-time personalized feedback through real-time evaluation and dynamic adjustment, improving training effectiveness; SVM, on the other hand, provides objective evaluation and specific improvement suggestions based on its precise pronunciation classification^13,14 and pattern recognition^15,16 capabilities. Combining the two can achieve multi-dimensional comprehensive evaluation of students’ pronunciation and provide continuous optimization guidance based on long-term data analysis. This article aims to explore the specific applications and effects of this combined technology, providing new ideas for future speech training systems. This article elaborates on how to use reinforcement learning techniques to achieve real-time feedback and personalized training plans, and introduces a method for accurate pronunciation evaluation using support vector machines. The experimental section demonstrates the actual effectiveness of the technical solution and analyzes its advantages and disadvantages. Finally, the main advantages and limitations of the method proposed in this article are summarized, and specific suggestions are put forward for future research directions, in order to further optimize speech training technology and enhance its practical application effectiveness.

Related work

The traditional English speech training methods^17,18 mainly include manual teacher guidance and rule-based speech recognition systems.^19,20 Li S²¹ pointed out that integrating English speeches into English phonetics classroom teaching could effectively improve the effectiveness of phonetics teaching, help students master English phonetics knowledge, and expand teachers’ evaluation mechanism for teaching effectiveness. Jia Jikang et al.²² proposed rule-based sentence analysis and recognition, which could quickly identify correct sentence patterns. Artificial teachers provide personalized pronunciation correction but have limited feedback, while rule-based speech recognition systems provide objective feedback but struggle to handle complex situations.

Reinforcement learning optimizes strategies through reward mechanisms, evaluates pronunciation in real-time during speech training, and adjusts training plans. In recent years, it has been widely used in education to provide personalized training recommendations. Chen Yaodong²³ proposed a personalized learning path recommendation algorithm based on reinforcement learning, which optimized the learning path by integrating feedback data. Experimental results showed that it could effectively improve the efficiency of knowledge point learning. Ren Weiwu²⁴ proposed an adaptive learning path generation mechanism based on reinforcement learning. By matching learner characteristics and needs and applying the concept of “fit,” personalized learning paths can be automatically generated and adaptively adjusted. The feasibility and effectiveness of this mechanism were verified in teaching practice. Meanwhile, the application of support vector machines in speech recognition showed excellent performance. By constructing decision boundaries to classify pronunciation, high-dimensional data and complex pronunciation patterns can be effectively handled. Chen Dong et al.²⁵ used support vector machine combined with Mel-frequency cepstral coefficient features to increase the number and feature dimension of Mel filters, effectively improving the accuracy of identifying car horn sounds in traffic noise, especially in low signal-to-noise ratio environments. This article combined reinforcement learning and support vector machine to enhance the feedback mechanism in English speech training.

Methods

Reinforcement learning model design

Basic concepts of reinforcement learning

Reinforcement learning is a learning method based on the interaction between an agent and the environment to learn optimal strategies. Its core goal is to select optimal actions in different states to maximize future cumulative rewards. The agent gradually adjusts its strategy to optimize the cumulative reward by performing actions in the environment and receiving feedback rewards. In English pronunciation training, the agent (speech training system) adjusts based on the student’s pronunciation data and learning progress. Whenever a student performs pronunciation training, the agent selects appropriate feedback and adjustment strategies based on the current pronunciation characteristics and training status. The reward signal of the environmental feedback reflects the accuracy of the student’s pronunciation. Immediate feedback rewards help students understand the pros and cons of their pronunciation, thereby motivating them to improve; long-term progress rewards promote students’ continuous learning and progress. The agent continuously adjusts its strategy and optimizes the training process based on these feedback signals, ultimately significantly improving the student’s pronunciation ability. Figure 1 shows the application of reinforcement learning in English speech training, clearly revealing the interaction process between the agent and the environment.

Figure 1.

Reinforcement learning structure.

Model structure and algorithm

This article uses the Q-learning algorithm to optimize the feedback mechanism in English speech training to give full play to the advantages of Q-learning in strategy optimization. Q-learning is a model-free reinforcement learning algorithm that finds the optimal strategy by learning the state-action value function (Q-value function). In Q-learning, the agent updates the Q-value table by constantly interacting with the environment, and gradually learns which action to take in different states to maximize the cumulative reward. One of the challenges faced by the agent in the Q-learning model is to balance exploration and utilization to optimize the learning effect. In order to balance exploration and utilization, this article adopts the ε-greedy strategy. In this strategy, the agent selects random actions with a certain probability ε for exploration to discover state-action pairs that may not have been fully explored, and selects the currently considered optimal action for utilization with a probability of 1-ε in order to maximize the reward. By dynamically adjusting the value of ε, exploration and utilization can be effectively balanced to ensure that the algorithm can explore new possibilities and make full use of existing knowledge during the learning process. Specifically, the ε value is usually set higher at the beginning of training to promote more exploration, and gradually decreases as training progresses to increase the proportion of utilization. In Q-learning, the agent continuously interacts with the environment, updates the Q-value table, and gradually learns which action to take in different states to maximize cumulative rewards. In the Q-learning model, the goal of the agent is to optimize the Q-value table through continuous exploration and utilization. The Q-value function estimates the expected value of future rewards that can be obtained by taking action in a state. The intelligent agent updates the Q-value through the following formula:

Q (s, a) \leftarrow Q (s, a) + α [r + γ \max_{a^{'}} Q (s^{'}, a^{'}) - Q (s, a)]

(1)

Among them, $a$ is the learning rate; $r$ is an instant reward; $γ$ is the discount factor; $s^{'}$ and $a^{'}$ are the next state and action, respectively. During each training cycle, the intelligent agent selects an action based on its current state. After executing the action, the environment provides an immediate reward and transitions to the next state. The intelligent agent uses the above formula to update the Q-value table and gradually optimize the strategy.

The algorithm flow of Q-learning is shown in Figure 2, and its initial step is to initialize the Q-value table to zero or assign a small random value. Subsequently, the intelligent agent interacts with the environment, captures the sequence of state action reward subsequent state, and iteratively optimizes the Q-value based on the update rules of Q-learning. During this process, the system incorporates a refined and personalized feedback mechanism to cope with different training stages and types of pronunciation errors. This feedback mechanism includes a detailed personalized training plan design process. Specifically, the system first develops an initial training plan for each student by analyzing the user’s pronunciation characteristics and learning progress. In the early stages of training, the system focuses on basic pronunciation errors, provides clear demonstration audio and key points, and motivates students to actively correct their pronunciation through instant rewards. In the middle stage, the system accurately points out the location and type of errors based on the user’s pronunciation data, and provides targeted practice materials to improve pronunciation accuracy and fluency. Finally, in the later stage, the system mainly uses positive feedback and positive evaluation to enhance students’ sense of achievement and learning motivation, while recommending advanced practice materials to promote students’ long-term progress. The personalized adjustment of the entire training process enables students to get the most suitable practice content and difficulty at each stage, thereby more effectively improving learning results. The overall reward mechanism design promotes students’ continuous progress through differentiated feedback and rewards at different stages.

Figure 2.

Q-learning algorithm process.

Reward mechanism design

In English phonetic training, this article optimizes the reward mechanism, divides it into immediate feedback rewards and long-term progress rewards, and further refines the immediate feedback reward mechanism. The immediate feedback reward mechanism is adjusted according to the type and severity of pronunciation errors. For example, for minor pronunciation errors, such as slight deviations in syllables, low-intensity feedback can be given, while for serious pronunciation errors, such as pronunciations that cannot be recognized by the speech recognition system, higher-intensity feedback and correction suggestions can be given. Through this hierarchical immediate feedback mechanism, students can improve their pronunciation problems in a more targeted manner. In addition, long-term progress rewards still focus on the overall learning progress of students over a period of time. The system conducts a comprehensive evaluation based on their ability to correct errors and the overall improvement in pronunciation, thereby more comprehensively motivating students to continue to make progress. The design of the reward function needs to be adjusted according to the specific tasks and goals to ensure that the intelligent agent can obtain effective feedback during the training process, thereby optimizing its strategy and behavior choices.

In addition, the system introduces a user feedback collection mechanism that allows students to evaluate the feedback provided by the model and make suggestions for improvement. Based on the user’s active feedback, the system can dynamically adjust the model parameters or strategies to further optimize the feedback effect and learning experience. This user-involved feedback mechanism not only improves the adaptability of the model, but also enhances the students’ sense of participation and autonomy in the learning process. The overall reward mechanism is designed to promote continuous progress of students through differentiated feedback and rewards at different stages.

Support vector machine model design

Basic concepts of SVM

Support vector machine (SVM), as a supervised learning algorithm, is widely used in classification and regression problems, especially in English speech training for pronunciation classification and scoring. SVM improves classification accuracy by constructing optimal hyperplanes to separate data points and maximize class spacing. For nonlinear data, SVM uses radial basis function (RBF) mapping to high-dimensional space to achieve linear separability of data. In the process of pronunciation classification, SVM uses Mel-frequency cepstral coefficients (MFCCs)^26,27 as features for model training, which can accurately classify pronunciation samples into categories such as “correct pronunciation” and “close pronunciation,” providing effective improvement guidance for personalized speech training.^28,29

Model training and optimization

This article selects radial basis function (RBF) as the kernel function of the support vector machine (SVM) model because the RBF kernel can effectively process nonlinear data with complex decision boundaries and shows good pattern capture ability in the process of processing pronunciation features. For the RBF kernel, this article uses a grid search method to optimize two key hyperparameters, γ and C. The role of γ is to control the influence range of the sample in the high-dimensional space, while C is used to balance the training error and complexity of the model. The optimization range of γ is [0.001, 1] and the range of C is [0.1, 10], which are determined based on a comprehensive analysis of data characteristics and previous experience. Specifically, a γ value that is too small may lead to underfitting of the model, while a too large value may cause overfitting; the C value affects the fault tolerance and fitting degree of the model. In the pronunciation classification and evaluation stage, the SVM model can accurately classify pronunciation samples into four categories: “correct pronunciation,” “close pronunciation,” “pronunciation deviation,” and “pronunciation error,” and evaluate the pronunciation quality based on these classification results. Based on the classification results, the system gives corresponding suggestions for improvement. Especially for the “mispronunciation” category, the system provides special exercises. Combined with the reinforcement learning method, the system uses SVM scoring information to provide instant and long-term feedback rewards, and dynamically adjusts the training strategy, thereby achieving personalized and accurate feedback and significantly improving the students’ pronunciation training effect. This method not only improves misclassification handling, but also improves the overall performance and application effect of the model. In order to ensure the stability of model performance, this article implements 10-fold cross-validation. This article divides the dataset into 10 equal parts, with 9 sets of data used for each training session and the remaining 1 set used for validation. The entire process is repeated 10 times, and the hyperparameter combination with the best average performance is ultimately selected as the final configuration of the model.

Pronunciation classification and evaluation

In pronunciation classification and evaluation, the SVM model accurately divides pronunciation samples into four categories: “correct pronunciation,” “close pronunciation,” “pronunciation deviation,” and “mispronunciation,” and evaluates the pronunciation quality accordingly. The classification results prompt the system to provide targeted improvement suggestions, especially to provide special exercises for “pronunciation errors.” Combined with reinforcement learning, SVM scoring information is used for immediate and long-term feedback rewards, dynamically adjusting training strategies, achieving personalized and accurate feedback, and significantly improving the effectiveness of students’ pronunciation training.

Experiments

Experimental setup

Dataset selection

This study selects high-quality datasets from GigaSpeech and CommonVoice to comprehensively cover the English speech field and enhance cross-regional representation, serving as a solid foundation for training and evaluation. This article carefully selects datasets that have been strictly annotated, has high accuracy and a unified format, and screens clear and noise-free speech samples, which significantly improves the quality of the dataset. In order to enhance diversity and robustness, custom speech samples are added to cover speakers of different genders and age groups, and samples from non-English speaking countries are introduced to fully simulate the actual language environment and help the model learn richer speech features, improving recognition ability. All data is public and anonymized. The collection of custom voice samples is informed and consent is obtained to ensure personal information protection. Security measures during data storage and transmission are also guaranteed. The composition of the dataset used in this article is shown in Table 1.

Table 1.

Overview of the dataset.

Data source	Number of samples	Percentage (%)	Comments
GigaSpeech	4000	61.5	Large-scale, diverse dataset
CommonVoice	2000	30.8	Global perspective, open community support
Customized voice samples	500	7.7	Specific samples for additional diversity

Experimental environment and tools

The choice of experimental environment and tools is crucial to the training effect of the model. This experiment is equipped with a set of high-performance hardware and software tools, as shown in Table 2, to ensure the efficient implementation of reinforcement learning and SVM models. The computer used in the experiment is equipped with an Intel Core i9-13900K CPU with 16 cores and 24 threads, which provides powerful computing power for data preprocessing and model training. The reason for choosing this processor is that its high core count and multi-threading support can effectively accelerate parallel computing tasks, especially in complex reinforcement learning training processes. At the same time, the system is installed with an NVIDIA RTX 4090 GPU, whose 24 GB video memory and a large number of CUDA cores (Compute Unified Device Architecture Core) support fast training and reasoning of deep learning models, accelerating the training process. 64GB DDR5 memory avoids training bottlenecks, and 1TB NVMe SSD ensures high-speed data reading and writing. Its video memory capacity and computing power can cope with the training needs of large-scale models and reduce latency during reasoning, thereby improving the efficiency of the experiment and the accuracy of the results. The experiment is based on Ubuntu 22.04 LTS, using Python 3.10.6 and TensorFlow 2.10.0 to implement reinforcement learning, and Scikit-learn 1.1.2 to train SVM. Jupyter Notebook 6.4.12 is used to manage and visualize experiments, and Matplotlib 3.5.3 is used to display training indicators. The overall architecture efficiently supports strategy optimization and model training.

Table 2.

Experimental environment and tools.

Hardware		Software
Category	Description	Category	Description
CPU	Intel Core i9-13900K	Operating system	Ubuntu 22.04 LTS
GPU	NVIDIA RTX 4090	Programming language	Python
Memory	64GB DDR5	Deep learning framework	TensorFlow
Storage	1TB NVMe SSD	Machine learning library	Scikit-learn
Motherboard	ASUS ROG MAXIMUS Z790 HERO	Experiment management tool	Jupyter notebook
Power supply	Corsair RM1000x	Visualization tool	Matplotlib

Experimental procedure

Data preprocessing

Data preprocessing is an important step in improving the quality of speech data and the effectiveness of model training, including denoising, feature extraction, annotation, and partitioning. This article uses spectral subtraction^30,31 to transform audio into the frequency-domain using Fourier transform, and estimate and remove background noise to improve signal clarity. In Fourier transform, this article uses 1024-point fast Fourier transform (FFT). The 1024-point FFT is selected to obtain higher resolution in the frequency domain, which is suitable for the audio signal sampling rate. A larger number of FFT points can improve the frequency domain resolution, but it also increases the computational complexity. The choice of FFT points directly affects the accuracy of the spectrum and the noise suppression effect. In the experiment, the 1024-point FFT balances the computational efficiency and frequency domain resolution, and can effectively capture the frequency components in the audio signal. Figure 3 shows a comparison of the frequency spectra of the original signal and the denoised signal. It can be seen that after spectral subtraction processing, the denoised signal waveform is smoother, and the noise components are significantly reduced, making the signal clearer.

Figure 3.

Noise reduction processing.

In order to extract the features of audio signals, this article adopts the Mel-frequency cepstral coefficient method. The specific steps are as follows: firstly, the audio signal is divided into multiple small frames, with each frame typically set to 25 milliseconds in length and a frame shift of 10 milliseconds to ensure short-term stability. Next, the Hamming window function is applied to each frame to reduce signal edge effects, which is defined as:

w (n) = 0.54 - 0.46 \cos (\frac{2 π n}{N - 1})

(2)

Among them, $N$ is the window length, usually equal to the frame length. The application of window functions helps to reduce spectral leakage at frame boundaries and improve the accuracy of fast Fourier transform (FFT) results.

Then, each windowed frame is subjected to a fast Fourier transform, and the signal is transformed from the time-domain to the frequency-domain. This article uses a 1024 point FFT, with the formula:

X (k) = \sum_{n = 0}^{N - 1} x (n) e^{- j (2 π / N) k n}

(3)

Among them, $x (n)$ is the time-domain signal; $X (k)$ is a frequency-domain signal; $N$ is the number of FFT points. By using FFT, the spectral information of each frame is obtained and the frequency components of the signal are revealed.

In frequency-domain signal processing, the signal is passed through a set of Mel filters. The design of the Mel filter bank is to simulate the sensitivity of the human ear to different frequencies, and its frequency transformation formula is:

m (f) = 2595 \log_{10} (1 + \frac{f}{700})

(4)

Among them, $f$ is the linear frequency and $m (f)$ is the Mel frequency. This article uses 26 Mel filters, each with an equally spaced bandwidth on the Mel frequency scale.

Finally, the output of the Mel filter bank is subjected to discrete cosine transform (DCT) to extract low-order cepstral coefficients, typically using the first 13 coefficients as features. The formula for DCT is:

C (k) = \sum_{n = 0}^{M - 1} \log (S (n)) \cos [\frac{π k}{M} (n + \frac{1}{2})]

(5)

Among them, $S (n)$ is the output of the Mel filter bank; $C (k)$ is the cepstral coefficient; $M$ is the number of filters. Low-order cepstral coefficients contain the main features of the signal, while high-order coefficients typically represent noise and detail information.

These features (MFCC) are used as input feature vectors for training and evaluating support vector machine models and Q-learning models. By extracting MFCC features, key information in audio signals can be effectively captured, helping the model accurately classify and optimize the quality of English pronunciation.

The dataset in this article is divided into 70% training, 15% validation, and 15% testing. Stratified sampling techniques are used to ensure balanced distribution of categories in each subset to address the issue of class imbalance.

Model training and testing

Q-learning is a reinforcement learning algorithm based on value iteration. Its core is to find the optimal strategy by continuously updating the Q value table. Through the ε-greedy strategy, the algorithm gradually tends to use the known best actions while ensuring exploration. This method continuously corrects the Q value through the reward signal to maximize the cumulative reward obtained in the long run. Support vector machine (SVM) is a supervised learning model that aims to find the best hyperplane to maximize the boundary between categories. The RBF kernel function effectively handles nonlinear problems by mapping data to a high-dimensional space. The regularization parameter C controls the complexity of the model to balance the training error and model complexity. Adjusting the values of C and γ can optimize the performance of the model on different datasets, thereby improving classification accuracy.

During the training of the Q-learning model, the Q-table is first initialized, whose dimension depends on the number of pronunciation features and the number of strategies. The initial Q-value of the Q-table is set to 0 to ensure that all state-action pairs have the same initial value at the beginning of training. To optimize the training of the Q-learning model, the following hyperparameters are set: learning rate (α) is 0.1; discount factor (γ) is 0.9; exploration rate (ε) is 0.1. These parameters control the speed of learning, the weight of future rewards, and the exploration frequency of new strategies, respectively. The learning rate α determines the step size of each Q-value update; the discount factor γ determines the influence of future rewards on current decisions; the exploration rate ε determines the probability of selecting a random action. During the training process, each student’s pronunciation feature is regarded as a state, and the system selects actions through the ε-greedy strategy. Specifically, the system selects a random action with a probability of ε and selects the action with the highest current Q value with a probability of 1-ε. The purpose of this is to balance the trade-off between exploring new strategies and exploiting existing strategies. After executing the selected action, the system obtains a reward signal based on the student’s pronunciation improvement and then updates the Q value in the Q-table.

During the training process, the system defines the state based on the student’s pronunciation characteristics. After executing the action, it obtains a reward signal based on the pronunciation improvement and updates the Q-table. The model gradually optimizes the training strategy through multiple iterations until convergence or a predetermined training epoch is reached. In order to optimize the performance of the Q-learning model, this article prepares multiple sets of parameter combinations for tuning, covering learning rate, discount factor and exploration rate. It can be seen from the results in Figure 4 that different parameter combinations have a significant impact on model performance. By analyzing the final cumulative reward value and excluding parameter groups 6, 7, and 9 with poor performance, and further comparing the convergence speed of the remaining parameter combinations, parameter group 2 (α = 0.1, γ = 0.85, ε = 0.15) is selected. Because it has the fastest convergence speed and the best final reward value, it shows that it performs best in optimizing training strategies and improving model performance. The parameter selection process of the SVM model is similar, mainly involving the selection of the kernel function, the adjustment of the penalty parameter (C) and the kernel function parameter (γ). The selection of kernel function affects the classification performance; the penalty parameter C controls the complexity of the classifier; the γ value (in the RBF kernel function) determines the range of influence of the sample on the classification boundary. By systematically adjusting these parameters and using methods such as cross-validation to evaluate the performance of different parameter combinations, the classification effect of the SVM model can be effectively optimized.

Figure 4.

Parameter tuning.

Support vector machine (SVM) is a supervised learning model that aims to find the best hyperplane to maximize the boundary between categories. The RBF kernel function effectively handles nonlinear problems by mapping data to a high-dimensional space. The regularization parameter C controls the complexity of the model to balance the training error and model complexity. Adjusting the values of C and γ can optimize the performance of the model on different datasets, thereby improving classification accuracy. In the training of the SVM model, this article chooses the RBF (radial basis function) kernel function, mainly due to its advantage in handling complex nonlinear decision boundaries. The RBF kernel function, in its Gaussian form, can construct flexible and highly nonlinear decision boundaries in the feature space, which is essential for handling complex nonlinear relationships in pronunciation features. Specifically, the parameter γ of the RBF kernel function controls the range of influence of a single sample on the decision boundary. A smaller γ value results in a smoother decision boundary, while a larger γ value can better capture subtle changes in the data. The hyperparameters such as regularization parameter C and kernel function parameter γ are set to C = 1.0 and γ = 0.5. The goal of SVM model is to distinguish samples of different categories by finding an optimal hyperplane. The SVM model is trained using the feature vectors and labels in the training set. SVM solves optimization problems through minimum optimization algorithms, minimizing classification errors and maximizing classification boundaries. To improve the generalization ability of the model, this article uses cross-validation to optimize hyperparameters C and γ, and selects the optimal combination through multiple rounds of training validation. During the testing phase, the SVM model’s classification accuracy and adaptability to pronunciation features are evaluated using a test set, and hyperparameters are adjusted based on the results to optimize performance.

The mathematical expression for the radial basis function kernel function is:

K (x, x^{'}) = \exp (- γ ∥ x - x^{'} ∥^{2})

(6)

Among them, $K (x, x^{'})$ represents the kernel function value between two samples $x$ and $x^{'}$ ; $γ$ is a parameter of the kernel function that controls the width of the Gaussian distribution; $∥ x - x^{'} ∥^{2}$ represents the square of the Euclidean distance between samples $x$ and $x^{'}$ .

The role of regularization parameter C is reflected by the following formula:

\min \frac{1}{2} ∥ w ∥^{2} + C \sum_{i = 1}^{N} ξ_{i}

(7)

It is subjected to the following constraints:

y_{i} (w^{T} ϕ (x_{i}) + b) 1 - ξ_{i}

(8)

ξ_{i} \geq 0

(9)

Among them, $\frac{1}{2} ∥ w ∥^{2}$ is the complexity term of the model, representing the two norm of the model parameter $w$ ; C is a regularization parameter that controls the trade-off between complexity and loss terms; $ξ_{i}$ is a slack variable, indicating the degree of misclassification of the $i$ -th sample; $\sum_{i = 1}^{N} ξ_{i}$ is the total misclassification loss of all samples. By adjusting the value of C, the optimal balance between complexity and misclassification loss can be found, thereby improving the performance of the model on unseen data.

Evaluation indicators

When evaluating the performance of Q-learning model and support vector machine (SVM) model, this article adopts multiple evaluation indicators to comprehensively measure the performance of these two models in speech training tasks. These indicators include accuracy, recall, F1-score, and the reward mean of Q-learning models.

Accuracy quantifies the proportion of correct predictions made by the model. Among them, TP (True Positive), TN (True Negative), FP (False Positive), and FN (False Negative), respectively, represent the number of positive samples correctly classified by the model, the number of negative samples correctly classified, the number of negative samples misclassified as positive samples and the number of positive samples misclassified as negative samples. In speech training, the accuracy rate reflects the ability of the SVM model to classify pronunciation features.

Accuracy = \frac{T P + T N}{T P + T N + F P + F N}

(10)

The recall rate measures the model’s ability to identify all actual positive samples, and the calculation formula is:

Recall = \frac{T P}{T P + F N}

(11)

The precision measures the proportion of positive samples predicted by the model that are actually positive samples:

Precision = \frac{T P}{T P + F P}

(12)

F1-score is the harmonic mean of precision and recall, used to comprehensively evaluate the precision and comprehensiveness of a model. The calculation formula is:

F 1 - score = 2 \cdot \frac{Precision \cdot Recall}{Precision + Recall}

(13)

The average reward is a unique evaluation metric for Q-learning models, used to measure the average reward obtained by the model during the training process. The average reward is calculated using the following formula:

\bar{R} = \frac{1}{N} \sum_{i = 1}^{N} R_{i}

(14)

Among them, $R_{i}$ represents the reward value in the $i$ -th training session, and $N$ is the total number of training epochs. The average reward reflects the training effectiveness of the Q-learning model, and a higher average reward indicates that the model performs well in improving pronunciation.

Results

This article investigates the comprehensive evaluation of SVM model performance in speech training tasks using 50 different sets of data. In 50 evaluations, each evaluation result is recorded in detail and statistically analyzed to comprehensively measure the performance of the model. To evaluate the performance of the Q-learning model, this article records the mean reward of the model over 100 iterations of training.

Effectiveness of reinforcement learning models

In English pronunciation training, the Q-learning model significantly enhances the effectiveness of pronunciation training by optimizing its feedback mechanism. As shown in Figure 5, in the early stages of training (rounds 1–20), due to the model being in the exploration phase and attempting multiple pronunciation correction strategies, the average reward shows significant fluctuations, which precisely reflects the uncertainty of the model in finding the best strategy; subsequently (rounds 20–50), as the model gradually masters effective feedback mechanisms and continuously optimizes its correction methods using Q-value update strategies, the reward mean significantly increases and stabilizes, providing stable and accurate pronunciation guidance for students; in the final stage (rounds 80–100), the sustained high reward mean demonstrates the comprehensive optimization of the feedback mechanism. The model can provide continuous and efficient pronunciation feedback, helping students achieve high-level pronunciation standards. This series of optimization results is a direct manifestation of the technical advantages of Q-learning model exploration and utilization mechanism, Q-value update strategy, and continuous optimization of feedback mechanism.

Figure 5.

Mean reward of reinforcement learning model.

Support vector machine model effectiveness

In the evaluation of support vector machine (SVM) model performance, this article comprehensively measures the model’s performance in speech training tasks through multiple indicators. This article uses four indicators, namely, accuracy, recall, precision, and F1-score, to evaluate the performance of the SVM model, in order to more intuitively demonstrate the performance changes of the SVM model. Figure 6 shows the changes in accuracy, recall, precision, and F1-score, respectively. As shown in Figure a, the accuracy of the SVM model remains at a high level in multiple evaluations, with the highest reaching 94.00% and the lowest reaching 86.54%, indicating that the model can stably provide high-quality classification results under different datasets and testing conditions. Figure b shows that the recall rate varies relatively steadily in different evaluations, with most evaluation results concentrated above 92%, indicating that the SVM model can effectively capture most pronunciation errors and provide comprehensive feedback. Figure c shows that the precision fluctuates greatly, but overall it remains between 82.22% and 92.31%, which may be due to the difference in the proportion of positive and negative samples in the dataset. However, overall, the SVM model still performs well in reducing false positives. Figure d shows that F1-score remains at a high level in multiple evaluations, reaching up to 94.65%, indicating that the SVM model performs well in balancing precision and comprehensiveness, providing stable and efficient performance.

Figure 6.

SVM model performance chart.

Comprehensive system effectiveness

This article selects 50 non-native English speakers aged 18 to 45 with good English skills and no pronunciation or hearing impairment as experimental participants. These participants come from different educational backgrounds, including college students, graduate students, and professionals who have worked for several years. The participants’ professional fields cover a variety of industries, including education, information technology, and finance. The determination of the sample size is based on the consideration of statistical power to ensure that the experimental results can reach the expected level of statistical significance and have sufficient power to effectively detect the experimental effect. In addition, the experimental design needs to consider the comparison between the experimental group and the control group as well as individual differences. The sample size of 50 participants ensures comparison between groups while also meeting the resource and time constraints of practical operations. Before the experiment, all participants record and read aloud texts, which are evaluated by experts as a benchmark. The experimental group uses a system combining Q-learning and SVM to optimize pronunciation through signal processing, feature extraction, and model feedback; the traditional method used by the control group is a training program based on manual feedback, audio recording and playback, fixed course materials, and traditional pronunciation training software. In order to comprehensively evaluate the effectiveness of combining Q-learning and support vector machine (SVM) system in English speech training, this article designs an 8-week learning cycle. The learning materials used in the experiment cover various English phonetic characteristics. Through the training of these materials, the pronunciation quality and learning progress of the experimental group and the control group are recorded and compared at different time points.

This article uses box plots to compare the pronunciation quality scores of the experimental group (combined with Q-learning and SVM English speech training system) and the control group (traditional method) before and after the experiment. The results are shown in Figure 7. Before the experiment, the median scores of the experimental group and the control group are approximately 79 and 80, respectively, with similar interquartile ranges and overlapping scoring ranges (75–84 for the experimental group and 76–85 for the control group). The distribution of scoring data between the two groups is similar and normal. However, the changes in scores after the experiment are significantly different. The scoring box range of the experimental group is expanded to 85 to 94, and the median increases to 89, showing a significant improvement. In contrast, the rating range of the control group increases to 78 to 87, with a median increase of only 82, indicating a relatively small improvement. Overall, the experimental group shows a significant improvement in pronunciation quality after using a training system that combines Q-learning and SVM, while the improvement effect of the control group is limited.

Figure 7.

Comparison of pronunciation quality.

In order to verify the significance of these changes, this article conducts a statistical significance test. An independent sample t test is conducted on the pronunciation quality scores of the experimental group and the control group to compare the performance differences between the two groups after the experiment. The test results show that the median score of the experimental group is significantly higher than that of the control group (p < 0.05), indicating that the pronunciation quality of the experimental group has been significantly improved after using the Q-learning and SVM system. At the same time, the intra-group comparison shows that the score of the experimental group improves significantly after 8 weeks (p < 0.05), while the improvement of the control group is relatively small. These results support the effectiveness of combining Q-learning and SVM systems in improving pronunciation quality.

This study fully demonstrates the advantages of a speech training system combining Q-learning and SVM in improving pronunciation quality and learning efficiency. Although there are certain limitations in sample size and long-term effect evaluation in the study, the effectiveness and performance of the system are clarified through comprehensive analysis.

Conclusions

This article explores an innovative solution that integrates reinforcement learning and support vector machine technology to address the challenges faced by traditional English speech training feedback mechanisms, such as feedback delay, strong subjectivity, insufficient personalization, technical limitations, and lack of long-term analysis, in the context of globalization. By using SVM model to accurately recognize English phonemes or words, real-time evaluation of students’ pronunciation can be achieved, and combined with reinforcement learning mechanism, personalized adjustments can be made based on these real-time evaluation results to provide learners with immediate feedback and dynamic training plans. This multi-dimensional comprehensive evaluation feedback system for students’ pronunciation performance has significantly improved the training effectiveness. The experimental results show that the reinforcement learning system achieves sustained maintenance of high reward mean; the accuracy of SVM model in pronunciation classification exceeds 86.54%; F1-score remains at a high level. Compared with the control group, the experimental group shows significant advantages in pronunciation scores and significantly improves learning progress, verifying the effectiveness and superiority of the method. Despite the above achievements, this study still has some limitations. The complexity of the system leads to high computing resource requirements, which may limit its application in resource-limited environments. The robustness of the model in dealing with extreme or abnormal pronunciation data also needs to be further improved. The small size of the experimental sample may affect the generalization ability of the results. Future research can be improved in the following directions: the efficiency of the algorithm is optimized to reduce the computational complexity and improve the processing speed; the scale of the experiment is expanded to cover more samples and diverse pronunciation data to enhance the generalization ability of the model; the integration with other technologies is explored, such as deep learning and transfer learning, to further improve the adaptability and accuracy of the system; a long-term tracking analysis is conducted to evaluate the performance of the system in different application scenarios, and continuously improve the personalized function.

Statements and declarations

Footnotes

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

The authors received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Rui Wang

References

Alshangiti

Evans

. Learning English vowels: the effects of different phonetic training modes on Arabic learners’ production and perceptiona. J Acoust Soc Am 2024; 156(1): 284–298.

Yang

Wang

, et al. Mobile application-based phonetic training facilitates Chinese-English learners’ learning of L2. Learn InStruct 2024; 93: 101967.

Xiwen

Wei

Benjamin

, et al.

Immediate text-based feedback timing on foreign language online assignments: how immediate should immediate feedback be?

Comput Educ Open 2023; 5(1): 100148.

Grabsch

Moore

. Does testing type matter? comparing the impact of instant and traditional feedback methods on student learning. Nacta J 2019; 63(2): 344–348.

YiChao

SiuYung

WingFat

, et al. Effects of automatic speech recognition technology on EFL learners’ willingness to communicate and interactional features. Educ Technol Soc 2023; 26(3): 37–52.

Rustam

Jiawen

. Review of research on applications of speech recognition technology to assist language learning. ReCALL 2022; 35(1): 74–88.

Marco

Alessandra

Francesco

, et al. Toward robots' behavioral transparency of temporal difference reinforcement learning with a human teacher. IEEE Trans Hum-Mach Sys 2021; 51(6): 578–589.

Peiying

Chao

Chunxiao

, et al. Deep reinforcement learning assisted federated learning algorithm for data management of IIoT. IEEE Trans Ind Inf 2021; 17(12): 8475–8484.

Kanisha

Lokesh

Kumar

, et al. Retraction Note: speech recognition with improved support vector machine using dual classifiers and cross fitness validation. Personal Ubiquitous Comput 2023; 28: 47.

10.

Karthikeyan

Priyadharsini

Balamurugan

, et al. Speaker identification using hybrid neural network support vector machine classifier. Int J Speech Technol 2022; 25(4): 1041–1053.

11.

Liu

Zuo

. Constructing and experimental research on the feedback mechanism of mathematics teaching based on error analysis. Appl Math Nonlinear Sci 2024; 9(1): 1712.

12.

. Evaluative and feedback mechanisms in writing for English as a second language (ESL) learners. Lect Notes Lang Lite 2023; 6(15): 28–31.

13.

Wei

, et al. Research on sound classification based on SVM. Neural Comput Appl 2020; 32(6): 1593–1607.

14.

Rupesh

Bodade

Divya

. Efficient classification of the adventitious sounds of the lung through a combination of SVM-LSTM-Bayesian optimization algorithm with features based on wavelet bi-phase and bi-spectrum. Biomed Eng Res 2023; 39(2): 349–363.

15.

Jingjing

Jianfeng

Lulin

, et al. Crack pattern recognition based on acoustic emission waveform features. Rock Mech Rock Eng 2022; 56(2): 1063–1076.

16.

Sun

Xingchen

Yunquan

. Coal mine gas and coal dust explosion recognition method based on spectrogram and SVM. Coal Sci Technol 2023; 51(02): 366–376.

17.

Mahdi

Khateeb

. The effectiveness of computer‐assisted pronunciation training: a meta‐analysis. Rev Educ 2019; 7(3): 733–753.

18.

Fouz-González

. Using apps for pronunciation training: an empirical evaluation of the English File Pronunciation app. Lang Learn Technol 2020; 24(1): 62–85.

19.

Alqadasi

AMA

Sunar

Turaev

, et al. Rule-based embedded HMMs phoneme classification to improve qur’anic recitation recognition. Electronics 2022; 12(1): 176.

20.

Piotr

. A rule-based grapheme-to-phoneme conversion system. Appl Sci 2022; 12(5): 2758.

21.

. Thinking about the way of English pronunciation training penetrating into English speech training. Frontiers in Educational Research 2023; 6(4): 1–5.

22.

Jia

Shao

Long

, et al. A hierarchical sentence analysis and recognition algorithm based on rule-based and syntactic synthesis. J Jilin Univ (Sci Ed) 2020; 58(04): 885–892. DOI: 10.13413/j.cnki.jdxblxb.2019298.

23.

Chen

. Research on personalized learning path recommendation algorithm based on reinforcement learning. Tech Wind 2023; 34: 82–84. DOI: 10.19392/j.cnki.1671-7341.202334028.

24.

Weiwu

Fanglin

Xiaoqiang

. Research on adaptive learning path generation mechanism based on reinforcement learning. Modern Dist Educ 2020; 06: 88–96. DOI: 10.13927/j.cnki.yuan.20201111.001.

25.

Chen

Huang

. Car horn sound recognition based on Mel frequency cepstral coefficients and support vector machine. Sci Technol Eng 2021; 21(11): 4486–4491.

26.

Saxena

Farooqui

Ali

. Extricate features utilizing mel frequency cepstral coefficient in automatic speech recognition system. Int J Manuf Eng 2022; 12(6): 14–21.

27.

Yuanlin

Jing

Hongbo

, et al. Heart sound classification based on improved Mel frequency cepstrum coefficient and integrated decision network method. J Biomed Eng 2022; 39(6): 1140–1148.

28.

Igor

Nevena

Nikola

, et al. A novel approach of determining the risks for the development of hyperinsulinemia in the children and adolescent population using radial basis function and support vector machine learning algorithm. Healthcare 2022; 10(5): 921.

29.

Ghosh

Mandal

. PigB: intelligent pig breeds classification using supervised machine learning algorithms. Int J Artif Intell Soft Comput 2022; 7(3): 242–266.

30.

Gupta

Singh

. Analysis of optimized spectral subtraction method for single channel speech enhancement. Wirel Pers Commun 2022; 128(3): 2203–2215.

31.

Zhang

, et al. Global digital compact: a mechanism for the governance of online discriminatory and misleading content generation. Int J Hum Comput Interact 2024; 2(3): 1–16. DOI: 10.1080/10447318.2024.2314350.