Adaptive vocal performance training using transfer learning and genetic algorithm: Personalization and optimization

Abstract

This study proposes an adaptive vocal performance training system based on transfer learning and genetic algorithms to address the limitations of traditional training methods, such as insufficient personalization and low efficiency. The system extracts high-level performance techniques from existing vocal datasets using transfer learning, and fine-tunes the pre-trained ResNet-50 convolutional neural network to cater to the specific needs of vocal training. A genetic algorithm is then employed to optimize the training plan, which is continuously refined through multiple iterations to meet individual student needs. Personalized training plans are provided based on the students’ voice characteristics and learning requirements. The system’s performance was evaluated, showing significant improvements in training effectiveness, with a training improvement score of 4.5 ± 0.4 and a user satisfaction score of 4.7 ± 0.3. The system also achieved a personalization score of 4.8 ± 0.2 and reduced training time to 28 ± 3 min. Compared to traditional vocal training methods, this system demonstrates superior performance in enhancing both training efficiency and effectiveness.

Keywords

transfer learning genetic algorithm vocal performance training personalization and optimization system deployment and testing

Introduction

Vocal performance¹ training refers to the process of improving the skills, sound quality, and performance ability of vocal performers through systematic practice and guidance, including training in vocal techniques, range extension, pitch accuracy, emotional expression, and other aspects of skills. The effective vocal performance training can improve students’ vocal skills, enhance their performance confidence, and promote the development of artistic expression. However, there are some shortcomings in the current vocal performance training methods. In the actual scene of music education and vocational training, it is difficult to achieve efficient personalized training due to the individual differences and diversified learning needs of students. In large class teaching, teachers often face the challenge of taking into account the different learning progress and needs of all students, resulting in uneven training results. Even in one-on-one instruction, the instructor’s experience and teaching style cannot fully adapt to each student’s unique vocal characteristics and learning style. This limitation affects the effectiveness of the training, reducing the enthusiasm of the learners and the motivation to persist in the long term. This limitation is reflected in the advanced vocal training which requires continuous improvement and fine feedback. Traditional training methods mainly rely on experienced mentors for one-on-one guidance, which is not only time-consuming and laborious, but also difficult to meet the personalized needs of each student. Due to the varying vocal foundations, ranges, characteristics, and learning progress of each student, a unified training plan is difficult to adapt to individual differences. Moreover, traditional training methods also suffer from subjectivity and lack of scientific quantitative standards in evaluation and feedback, which affects the objective evaluation and optimization improvement of training effectiveness. The field of music education is in urgent need of a training system that can combine individual needs with scientific quantitative standards, improve the overall teaching effect, and enhance students’ learning motivation and sense of achievement.

Based on the existing problems mentioned above, this article clarifies the research objective, which is to study an adaptive vocal performance training system based on the transfer learning² and the genetic algorithm,^3,4 to achieve personalized and optimized training. It is to obtain performance skills from existing vocal datasets through transfer learning techniques and optimize training plans using genetic algorithms to provide tailored vocal training plans for different individuals. This method can analyze the voice characteristics⁵ and learning needs of students, provide personalized training plans, and meet the special needs of different individuals. On this basis, the genetic algorithm can also be used to optimize training plans and continuously improve and enhance training effectiveness, ensuring that each student can obtain the best training results. This method utilizes artificial intelligence (AI) technology⁶ to automatically generate training plans, reducing reliance on experienced mentors and saving valuable educational resources. Through this innovative method, this article can provide some reference and guidance for future training and optimization in other art forms.

The main work of this paper includes the following aspects: Based on transfer learning, extract high-level performance skills from large-scale vocal music datasets, capture the core elements of vocal music training, including vocal range extension, intonation control, emotional expression, etc. Transfer learning applications improve initial model performance and reduce training time. Genetic algorithm is used to optimize the training scheme, design the fitness function, select crossover, mutation, and other genetic operations, and dynamically adjust the training scheme according to the individual characteristics of students to meet the individual needs. The optimization process considers the students’ vocal music foundation, learning progress, music style preference, and generates accurate and efficient training programs. Design and implement an adaptive vocal performance training system, automatically analyze the student’s voice characteristics, generate the initial training plan, and optimize the training effect through several iterations. The system has real-time feedback function, adjusts the training intensity and content according to the progress of students, and provides quantitative feedback indicators to help students understand the progress. The effectiveness and superiority of the system are verified by extensive experimental tests.

Literature review

Transfer learning, as a machine learning technique,^7,8 aims to improve the learning efficiency and performance of target tasks by applying knowledge obtained from one task or domain to different but related tasks or domains. In recent years, transfer learning has had certain applications in voice recognition. Li and Zhang⁹ proposed a transfer learning-based singing timbre evaluation method, and it was successfully applied to singing tone evaluation by constructing a multidimensional instrument timbre evaluation model. The transfer learning model based on instrument timbre evaluation rules performed well in singing timbre evaluation, with high applicability and accuracy, effectively solving machine learning problems under small dataset conditions and demonstrating the enormous potential of transfer learning in the field of vocal music. Kumar et al.¹⁰ applied pre-trained models to bird sound datasets through transfer learning, significantly improving the accuracy and robustness of sound recognition. Although the application fields of their study were different, the methods and conclusions had certain reference value for vocal training. Kheddar et al.¹¹ explored the application of deep transfer learning in automatic speech recognition, studying the use of transfer learning techniques to extract features from a large number of pre-trained models and apply them to specific speech recognition tasks. The experimental results showed that transfer learning significantly improved the performance of speech recognition models, enabling them to achieve good recognition results on different speech datasets. In their research, Peng et al.¹² proposed a transfer learning framework based on pre-trained convolutional neural networks and support vector machines for automatic recognition and classification of speech disorders. Their research overcomes the difficulty of model training due to the scarcity of medical data, and effectively improves the accuracy of speech disorder classification through transfer learning. Mavaddati¹³ used ResNet deep learning models and transfer learning techniques to develop a system for detecting the speaker’s age, gender, and language. By using spectral time transform, the system can improve the recognition performance in noisy environment. Fu et al.¹⁴ explored the use of instrument audio to improve the calibration of automatic singing labels. They successfully improved the accuracy of singing audio labeling by treating the human voice as a special instrument and using transfer learning techniques.

As a bionic optimization algorithm, genetic algorithm (GA) has demonstrated strong solving ability in complex optimization problems by simulating biological evolution processes such as natural selection and genetic variation. Remarkable progress has been made in the application research of this algorithm in speech processing and feature optimization. Zhang¹⁵ optimized the design of speech data system by improving genetic algorithm. Their research shows that the improved genetic algorithm can significantly improve the efficiency of speech recognition, providing important support for the development of intelligent systems industry. Chen et al.¹⁶ used genetic algorithm to optimize the sound quality prediction model of tractor noise. By optimizing the initial weights and thresholds of the neural network, they successfully improved the prediction accuracy and stability of the model. Majidi and Toroghi¹⁷ applied multi-objective genetic algorithm to music harmony generation and proposed a method that could generate polyphonic music fragments in line with music theory and audience satisfaction. The scores of experts and listeners were modeled using bidirectional long short-term memory neural networks, respectively. The proposed music generation system attempts to maximize the mentioned objective function in order to generate a new piece of music, including melody and harmony. Xiao et al.¹⁸ adopted the genetic algorithm-based beamforming technology. For this newly established method, the results show that it has higher resolution capability than traditional beamforming, and better accuracy and robustness than compressed sensing beamforming.

The previous research has significant advantages in transfer learning in handling small datasets and improving model generalization ability, which has reference value for fields such as vocal training where data acquisition is difficult. However, the transfer learning effectiveness of the methods used in this literature depends on the domain relevance of the pre-trained model, and the complex deep networks involved increase computational costs. The genetic algorithms used in these studies show excellent performance in optimizing feature selection and classification performance, improving the accuracy and efficiency of recognition tasks and demonstrating high solving efficiency and solution quality in dealing with high-dimensional and complex constraints. However, the performance of genetic algorithms designed by these research institutes depends on the characteristics of the problem and algorithm parameter settings and may fall into local optima in large-scale optimization problems. Therefore, this article studies specialized pre-trained models, combined with lightweight deep learning models and optimization algorithms, aiming to improve the effectiveness of transfer learning and reduce computational costs, solve domain relevance and resource constraints, and ultimately achieve personalized and optimized vocal training solutions, thereby enhancing the effectiveness and applicability of vocal training.

Data processing

Dataset description

The DALI dataset is used in this article, which is a large dataset containing synchronized audio, lyrics, and vocal pitch. It consists of 5358 tracks, each with its time-aligned vocal melody notes and lyrics, divided into four granularity levels. These tracks cover a variety of music styles and singing techniques, providing a rich foundation of vocal training data. The dataset covers a wide range of music styles and singing techniques, including pop, classical, jazz, etc., providing support for the diversity of vocal training. All audio samples are recorded at a high sampling rate, ensuring clarity and purity of sound quality. To ensure the accuracy and consistency of the data, the labels of the DALI dataset include the following aspects. Each audio sample is labeled with specific vocal techniques, indicating the vocal style and performer information to which the sample belongs. The dataset is divided into training set, testing set, and validation set according to a division ratio of 70%, 15%, and 15%. The training set is used for model training; the testing set is used for model evaluation; the validation set is used for system testing.

Data preprocessing

In this section, audio files in the DALI dataset are preprocessed. The data preprocessing consists of 5 steps: denoising, unifying sampling rate, audio editing, normalization, and data augmentation. In this section, a 2-minute and 7-second audio file from the DALI dataset is selected for processing as an example, and the processing results are shown in Figure 1.

Figure 1.

Data preprocessing.

Denoising

Silent segments in the audio samples are selected to estimate the spectral characteristics. This process is determined by analyzing the energy level of the audio signal, with the lower-energy portion typically representing background noise.¹⁹ The specific operation is as follows. Silent segments are extracted from the audio samples, and the average spectrum of these segments is calculated as the noise estimate. Then, the audio samples are subjected to Short-Time Fourier Transform (STFT)^20,21 to transform the audio signals into a frequency-domain representations. STFT divides audio signals into multiple short time periods and performs Fourier transform on each time period to generate a spectrogram of the audio signals. The STFT of the audio samples is calculated to obtain spectral representations. The noise spectrum is subtracted frame by frame based on the previously estimated noise spectrum. The Inverse Short-Time Fourier Transform (ISTFT)²² is applied to transform the denoised spectrogram back into time-domain signals, obtaining the denoised audio signals.

Unifying sampling rate

The Lanczos resampling algorithm is adopted, which can efficiently perform resampling while maintaining audio quality.²³ Each audio sample is resampled, and the sampling rates of all audio samples are adjusted to 44.1 kHz uniformly.

Audio editing

This step aims to standardize the length of audio samples and eliminate the impact of sample length differences on analysis. All audio samples are edited into a fixed length of 30 seconds. This length can be adjusted according to research needs, but consistency should be ensured. When extracting samples, for long samples, the most representative part is extracted, and for short samples, a uniform length is achieved by repeating or filling in silent segments.

Normalization

Each audio sample’s maximum amplitude value is calculated, and formula (1) is used to standardize the amplitude of all audio samples to the same range. Finally, the amplitude of all audio samples is normalized to ensure consistent volume for each sample. Among them, $x$ represents the original audio sample; $x_{norm}$ represents normalized audio samples; $\max (| x |)$ represents the maximum absolute amplitude value of the audio sample.

x_{norm} = \frac{x}{\max (| x |)}

(1)

Data augmentation

To improve the robustness and generalization ability of the model, data augmentation is performed on the audio samples, including the application of time scaling algorithms to generate audio samples at different speeds. The frequency transformation algorithm is used to adjust the frequency range of audio samples. Noise signals are randomly generated and superimposed onto audio samples.

Feature extraction

Discrete wavelet transform

Discrete wavelet transform (DWT) is a method of decomposing a signal into wavelet coefficients of different scales, which can capture the time-frequency characteristics of the signal.^24,25 The specific steps are as follows. Daubechies 4 is selected as the mother wavelet and the decomposition level is set to 5. The discrete wavelet transform is performed on audio signals to obtain wavelet coefficients and decomposition levels. The detail coefficients for each level are drawn as shown in Figure 2, displaying different levels of detail coefficients; each level of detail coefficient reflects the detailed information of the audio signal at different scales.

Figure 2.

Detail coefficient diagrams at each level.

Figure 2 shows the detail coefficients at different levels, each level reflecting the detailed information of the audio signal at different scales. The detail coefficient contains the high-frequency components of the audio signal and is an important basis for fine analysis of the audio signal. In the system of this article, by analyzing the detail coefficients at different levels, the high-frequency characteristics and detail changes of the audio signals can be further understood.

Mel frequency cepstral coefficients

Mel frequency cepstral coefficients (MFCCs) are an important step in speech recognition, voiceprint recognition, and vocal performance analysis by converting audio signals to the Mel frequency domain and extracting key features that reflect human ear perception.²⁶ The first step is to perform high-frequency enhancement pre-emphasis processing on the audio signals, using a first-order high-pass filter to process the input signal, enhancing high-frequency components and suppressing low-frequency components. The preprocessed audio signals are divided into several frames, each containing 25 milliseconds of samples, and the frames are shifted to 10 milliseconds. The Hamming window²⁷ is applied to each frame of the signals, aiming to reduce the discontinuity of the signals at the boundary and to ensure the smoothness of the spectrum before performing fast Fourier transform (FFT). Then a fast Fourier transform is performed on each frame of the signals, converting the time-domain signals into frequency-domain signals to obtain spectral representations. The spectrum is passed through a set of Mel filters to simulate the auditory characteristics of the human ear. The discrete cosine transform (DCT) is performed on the logarithmic energy spectrum²⁸ to extract MFCC. This section uses text.mp3 files for MFCC feature extraction, and finally the MFCC feature map is drawn, as shown in Figure 3, which displays the MFCC coefficients at different time periods.

Figure 3.

3D MFCC feature map of the vocal audio.

Model selection and training

Transfer learning model

In terms of transfer learning models, this article chooses ResNet-50²⁹ as the pre-trained model. ResNet-50 consists of 50 layers and introduces residual connections to alleviate the gradient vanishing problem through skip connections, allowing information to propagate more efficiently in the network. This is crucial for processing complex vocal performance data, as deep networks can capture richer feature representations. In contrast, other commonly used pre-training models, such as VGG-16 (Visual Geometry Group-16) and Inception-V3, are inferior to ResNet-50 in feature extraction capability and computational efficiency. VGG-16 has fewer network layers, which makes it difficult to capture complex vocal features. Inception-V3 has a strong feature separation ability, but its structure is more complex and the calculation cost is higher. ResNet-50 is pre-trained on large-scale datasets such as ImageNet and has strong feature extraction capabilities.³⁰ These pre-trained features can be well transferred to other tasks, and their stability and efficiency are validated in multiple studies, making them the preferred model for transfer learning tasks. However, this model needs to be fine-tuned to adapt to the characteristics of vocal performance data. By fine-tuning the upper convolutional layer of ResNet-50, it retains its general image feature extraction capability, while adjusting the parameters of the model to better adapt to the characteristics of vocal data (frequency changes of sound and emotional expression), the specific design is as follows.

Freezing lower convolutional layers

In ResNet-50, the first 45 convolutional layers are frozen and do not participate in training, only fine-tuning the last 5 convolutional layers. This step ensures that the lower convolutional layer retains the common features learned from large-scale datasets, while the higher convolutional layers are optimized for vocal data.

Adding custom layer

The output of the convolutional layer is flattened and input into the fully connected layer. Multiple fully connected layers are used, with Rectified Linear Unit (ReLU) activation function following each layer. The specific structure is as follows. The input dimension of fully connected layer 1 is the flattening size of the output of the convolutional layer, and the output dimension is 256. The ReLU activation function introduces nonlinearity, enabling the model to learn complex features. The Dropout layer randomly discards some neurons with a probability of 0.5 to prevent overfitting. The input dimension of fully connected layer 2 is 256, and the output dimension is 128.

After the last fully connected layer in the output layer, a Softmax layer is added to output category probabilities. The structure is as follows. The input dimension of fully connected layer 3 is 128, and the output dimension is the number of categories (vocal classification). The Softmax activation function converts the output into a probability distribution, suitable for multi-classification tasks.

Training strategy

Adopting a segmented training strategy, the newly added custom layers are first trained, and then some of the lower convolutional layers are gradually unfrozen to ensure the adaptability and stability of the model on new tasks. The first step is to train only the newly added custom layers and to keep the pre-trained convolutional layers frozen. It is trained for 10 epochs with a learning rate of 0.001. The second step is to thaw some of the convolutional layers (such as the last 5 layers) and perform joint training. It is trained for 20 epochs, and the learning rate is reduced to 0.0001.

Genetic algorithm optimization

Genetic algorithm (GA) is an optimization algorithm based on natural selection and genetic mechanisms, suitable for complex nonlinear optimization problems. It continuously evolves individuals in the population by simulating genetic operations such as selection, crossover, and mutation, ultimately finding the global optimal solution. The basic process of GA is shown in Figure 4.

Figure 4.

Flowchart of genetic algorithm.

Initializing population

Generating initial population

The initial population contains 100 training plans, each consisting of a set of hyperparameters and training strategies. Hyperparameters include learning rate, batch size, and iteration count. The real number encoding is used to represent the features of each individual (training plan). Individuals are represented as $[η, b, n, a]$ , where $η$ is the learning rate; $b$ is the batch size; $n$ is the number of iterations; $a$ is a data augmentation strategy.

Defining fitness function

The fitness function is used to evaluate the advantages and disadvantages of each training plan. The fitness value of each individual can be calculated based on the accuracy of the model, and it is used to select the next generation population. The definition of the fitness function is shown in formula (2). Among them, $α$ , $β$ , and $γ$ are weight parameters used to balance the contributions of different indicators. Then several individuals are randomly selected for comparison. The individuals with the highest fitness are selected as the parent and enter the next generation. In different training tasks, the importance of these indicators is different to fully reflect the performance of the model. It is worth explaining that in vocal performance training, the accuracy rate is directly related to the model’s correct recognition of sound features, and it needs to be given a higher weight.

F (x) = α \cdot Accuracy + β \cdot Recall + γ \cdot F 1 Score

(2)

Crossover operation

Single-point crossover operation is performed on selected parent individuals to bring diversity and promote evolution by exchanging some genes of the parent individuals. The specific operation is to select the intersection point for two individuals $[η_{1}, b_{1}, n_{1}, a_{1}]$ and $[η_{2}, b_{2}, n_{2}, a_{2}]$ and to generate offspring individuals $[η_{1}, b_{2}, n_{2}, a_{1}]$ and $[η_{2}, b_{1}, n_{1}, a_{2}]$ .

Mutation and termination

Mutation operations are performed on newly generated offspring individuals, with a mutation rate set at 0.05. These operations involve randomly changing some genes of individuals to avoid the population falling into local optima and increase exploration space. Specifically, for individual $[η, b, n, a]$ , a gene $η$ is randomly selected for mutation, resulting in a new value of $η^{'}$ . When the fitness of the optimal individual in the population reaches the preset target, or when the number of iterations reaches 100, the algorithm is terminated. The mutation strategy needs to increase the mutation probability when the search space is large and the problem complexity is high. When the problem is relatively simple, it is necessary to reduce the probability of variation and improve the accuracy of the solution.

Training process

Initial training

The training epochs are 10, with a learning rate set to 0.001, and only the newly added custom layers are trained. In this way, it is ensured that the network can adapt to specific features of vocal data while avoiding excessive adjustments to the general features learned in the pre-trained model.

Joint training and optimization

The last 5 convolutional layers of the ResNet-50 model are thawed, and the joint training is conducted to adapt to the characteristics of vocal performance data. During the training process, fine-tuning is carried out with a lower learning rate to ensure the optimization of the model’s detailed features. This part has 20 training epochs, and the learning rate is reduced to 0.0001. This can gradually adapt the model to the complex characteristics of vocal data, while avoiding gradient instability caused by high learning rates. Then the abnormal algorithm optimization process is entered for iterative optimization.

Model evaluation

Evaluation indicators

To comprehensively evaluate the performance of the model, this article uses accuracy, recall, precision, and F1 score as evaluation indicators to reflect its performance in different aspects. The accuracy is the ratio of the number of correctly classified samples to the total number of samples. The recall rate is the ratio of the number of correctly classified positive samples to the total number of actual positive samples. The precision is the ratio of the number of correctly classified positive samples to the total number of samples classified as positive. The F1 score is the harmonic mean of precision and recall.

Cross-validation

To ensure the reliability of the evaluation results, the K-fold cross-validation method is used to evaluate the model. Specifically, the dataset is divided into K subsets, with K-1 subsets used for training each time and the remaining subset used for validation. This process is repeated K times, selecting a different subset as the validation set each time. If a 5-fold cross-validation is used here, then 5 experiments are conducted, and the results of each experiment are used to evaluate the stability and performance of the model. The experimental results are shown in Table 1.

Table 1.

Evaluation indicators for model optimization.

Experiment number	Accuracy	Recall	Precision	F1 score
1	0.9003	0.8805	0.9102	0.8907
2	0.9204	0.9006	0.9301	0.9105
3	0.9402	0.9103	0.9404	0.9309
4	0.9301	0.8907	0.9206	0.9102
5	0.9508	0.9205	0.9503	0.9404

It can be seen that the accuracy of all experiments is above 0.9000, with the highest reaching 0.9508. The recall rate remains at a high level in each experiment, with a minimum of 0.8805 and a maximum of 0.9205. The precision is above 0.9100 in all experiments. The F1 score combines the precision and the recall, exceeding 0.8900 in each experiment. Overall, the adaptive vocal performance training model based on the transfer learning and the genetic algorithm in this article performs well in various evaluation indicators, demonstrating its advantages in improving vocal performance training effectiveness and personalized training.

System deployment and testing

After the model training is completed, the performance of the system needs to be verified in real scenarios. Next, the system is deployed and comprehensively tested to evaluate the application effect of the model in actual vocal performance tasks. In practical applications, in the face of extremely complex and large stylistic differences in vocal music data, it is necessary to pay attention to the accuracy and adaptability of the system, which can be optimized.

System architecture

The system architecture is shown in Figure 5. The data input module is responsible for importing audio data from the DALI dataset and performing preliminary preprocessing. The feature extraction module utilizes the MFCC method to extract audio features, which are standardized and stored for subsequent use. The model training module uses pre-trained ResNet-50 model for the transfer learning and optimizes the training plan through the genetic algorithm to select the optimal combination of hyperparameters. The result output module generates personalized training plans and outputs evaluation metrics. The system interface module provides a user-friendly interface, allowing users to upload audio samples and obtain personalized training plans.

Figure 5.

System architecture.

System performance optimization

Parallel processing

The multi-threading technology is used to parallel process feature extraction from multiple audio samples, fully utilizing the computing capability of the multi-core CPU (Central Processing Unit) and quickly processing large amounts of audio data from DALI datasets. By utilizing parallel computing libraries in Python, feature extraction time can be significantly reduced when processing large amounts of audio data.

Cache mechanism

The extracted MFCC features are stored in Redis, and a cache eviction strategy is designed. Cache resources are reasonably managed based on usage frequency and cache space to ensure efficient utilization of cache. During the model training process, key intermediate results (such as gradients, model parameters, etc.) are stored in cache for quick access during model tuning and evaluation.

System performance monitoring

The key performance indicators of a system include CPU and GPU utilization, memory usage, and data processing speed. Prometheus monitoring tools are used to set performance alarms, promptly identify and resolve performance bottlenecks. System operation logs are recorded, detailing the execution time and resource consumption of each module. The log data is regularly analyzed, and the system performance is optimized.

User interface design

This article explores a concise and intuitive platform to achieve personalized vocal training. The user interface consists of three modules: user login and registration, audio data upload, and personalized training plan generation. The system uses a combination of charts, images, and text to visually display training results and evaluation metrics. It provides a detailed evaluation report download function, allowing users to analyze training results in depth and continuously improve training plans. Educators can quickly access different functional modules according to teaching needs, reducing unnecessary operational steps. The system provides real-time prompt and guidance functions to help educators get support at any time during the use process. In the process of interface design, feedback from educators and students is collected, and multiple rounds of optimization are carried out based on this feedback in terms of rapid learning and operation.

Registration and login interface

Figure 6 shows the login and registration interface of the system, which are simple and intuitive entrances. The interface is divided into two parts: login and register, which can be switched through tabs. The login interface allows users to enter their username and password for authentication, and provides “remember me” and “forget password?” functions to ensure the convenience and security of the user experience. The registration interface guides new users to create an account, enter their username, email, and password, and they need to agree to the terms and conditions of use and privacy policy to ensure the security and compliance of the new account creation process. After successful login, users can access the “adaptive vocal performance training using transfer learning and genetic algorithm” system.

Figure 6.

Audio upload interface

The interface shown in Figure 7 is the audio data upload interface, where users can easily select audio files by clicking the “choose file” button. Supported formats include mp3 (MPEG-1 Audio Layer 3), wav (Waveform Audio File Format), and flac (Free Lossless Audio Codec). Then a brief description of the input file needs to be provided to better understand and manage the uploaded audio content. The interface also provides input options for audio metadata, allowing users to fill in information such as artist name, recording date, etc., increasing file searchability and completeness. Users can add tags (such as classical, pop, training, etc.) to help classify and retrieve audio files, further improving management efficiency. The privacy setting function allows users to choose to set files as public, private, or restricted to ensure data security and privacy protection. The agree checkbox of terms and conditions ensures that users agree to the system’s terms of use before uploading files, enhancing legal compliance. The upload history section displays the files previously uploaded by the user, including filename, upload time, and status, allowing users to easily track and manage all uploaded files.

Figure 7.

Audio upload interface.

Personalized training plan generation interface

As shown in Figure 8, the personalized training plan generation interface in the “adaptive vocal performance training system using transfer learning and genetic algorithm” provides users with an interface for generating personalized training plans. This interface analyzes user-uploaded vocal performance audio and provides multiple input options, including user profile (beginner, intermediate, and advanced), preferred vocal style, training duration (weeks), weekly training frequency, and duration of each training session. Users also need to select training focus areas, such as breathing techniques, pitch control, volume dynamics, sound quality, range extension, and expressiveness, and need to fill in additional notes to explain specific goals or preferences. Finally, clicking the “generate plan” button can bring up a specific personalized training plan interface, as shown in Figure 9.

Figure 8.

Personalized training plan generation interface.

Figure 9.

Specific training plan interface.

Figure 9 presents the specific personalized training plan interface that is accessed after clicking on “generate plan.” This interface displays a detailed training plan, including the training summary and the weekly training schedule. The training summary section lists key information such as the user’s training profile, preferred vocal style, training duration, weekly training frequency, and duration of each training session. The weekly training schedule section provides a detailed list of the training course content for each week, including the specific steps of each training session. Users can save, print, or edit the training plan through the operation buttons at the bottom of the interface.

System testing

System function testing

A total of 9 functions are tested for effectiveness in this system function test, using the validation set divided from the DALI dataset in this article for testing. A total of 9 testing processes are completed.

(1) The entire process of a user from registering a new account to successfully logging into the system is tested;

(2) User-uploaded audio files in different formats (mp3, wav, flac) are tested;

(3) After uploading the audio files, the system extracts their features and stores them in the database to verify the accuracy of feature extraction and the integrity of the storage process;

(4) Personalized training plans are generated based on user-input training parameters, and the accuracy of the plan generation and the correctness of the display are verified;

(5) The generated training plans are displayed correctly in the user interface, including training summary, weekly training schedule, and specific content of each training session;

(6) The generated training plans saved by users are tested and viewed in the history to ensure successful saving of the plans. The printing function is also tested to ensure that the print output formats are correct;

(7) The user-submitted training feedback is tested to verify the effectiveness of feedback collection and storage;

(8) The system performance is monitored under high-load conditions, and the CPU and the memory usage are recorded. Performance reports are generated;

(9) The security of the system’s storage and access to user data are verified to ensure effective privacy settings and to protect user data.

After the testing is completed, the test results are generated as shown in Table 2.

Table 2.

System function test results.

Function	Expected result	Test status
User registration & login	Successful registration and login	Passed
Audio file upload	Successful upload, shows in history	Passed
Audio feature extraction	Features extracted and stored	Passed
Personalized plan generation	Plan generated and displayed	Passed
Plan display	Plan details displayed correctly	Passed
Plan save	Plan saved, shows in history	Passed
Plan print	Plan printed, correct format	Passed
User feedback collection	Feedback submitted and stored	Passed
System performance monitoring	Real-time monitoring, report generated	Passed
Data security & privacy	Secure storage, restricted access	Passed

System performance testing

This section tests four performance metrics: audio feature extraction and processing speed, personalized training plan generation speed, system response time, and concurrent user processing capability. The corresponding test contents are testing the processing speed of audio feature extraction on datasets of different scales; testing the time of the system when generating personalized training plans; testing the response time of the system to process user requests; simulating a multi-user simultaneous operating system; and testing the stability and response speed of the system under high-concurrency conditions.

As shown in Figure 10, as the dataset size increases from 100 to 500, the feature extraction time increases from 5 s to 27 s. The linear growth trend indicates that the system can maintain a stable state as the dataset size expands. In terms of personalized training plan generation speed, the generation time for multiple tests is around 2 s, which indicates a very high response speed. In the system response time, the response time of all operations is in milliseconds, indicating that the system has extremely high response efficiency in handling common user operations. In the graph of response time changing with the number of concurrent users, as the number of users increases, the response time also maintains a linear growth trend and is relatively stable. These results validate the outstanding performance of the system and meet practical application requirements.

Figure 10.

Performance test results.

Comparative testing

To scientifically evaluate the superiority of the adaptive vocal performance training system using the transfer learning and the genetic algorithm in this article, a comparative test is conducted with traditional vocal performance training methods, and 50 testers are invited to conduct the test. In order to avoid bias in the testing process, each student takes a vocal basic test before training, balancing the differences between different students. The test is evaluated using four indicators: training improvement score, user satisfaction score, personalization score, and training time (minutes). The training improvement score, user satisfaction score, and personalization evaluation are all calculated based on feedback from testers, and the scores are all on a 5-point scale. Finally, based on the four types of data collected above, a statistical analysis is conducted to calculate the mean and standard deviation of each indicator. The comparative results are obtained as shown in Table 3.

Table 3.

Comparison between the traditional vocal training method and the adaptive vocal training system.

Indicator	Mean ± SD of traditional method	Mean ± SD of adaptive system
Training improvement score	3.2 ± 0.5	4.5 ± 0.4
User satisfaction score	3.8 ± 0.6	4.7 ± 0.3
Personalization score	3.5 ± 0.7	4.8 ± 0.2
Training time (minutes)	60 ± 10	28 ± 3

Table 3 shows that the system studied in this article outperforms the traditional vocal performance training method in all four indicators. In terms of training improvement score, user satisfaction score, and personalization score, it is about 1 point ahead of the traditional vocal performance training method, and the training time is even less than half of that of the traditional method. Upon further observation of the standard deviations of various indicators, it can be found that the standard deviations of the system in this article are smaller, indicating that the system performs consistently among the 50 participants in the test and has high stability in the results.

Conclusion

This article explores an adaptive vocal performance training system based on the transfer learning and the genetic algorithm. The transfer learning is utilized to extract high-level performance skills from existing vocal datasets and is combined with the genetic algorithms to optimize training plans. The system can provide customized vocal training plans for different individuals. In practical applications, the system proposed in this article can generate personalized training plans by analyzing and testing the voice characteristics and learning needs of students. Various system tests also verify the superiority of the system proposed in this article. This method not only reduces reliance on experienced mentors and saves valuable educational resources, but also scientifically evaluates training effectiveness through quantitative evaluation indicators and data analysis, thereby providing objective feedback and improvement suggestions. The automated and intelligent nature of the system is expected to reduce the burden on educators, allowing them to focus more on artistic expression and creation. The innovation and flexibility of the system make it have other application capabilities, which can be extended to other fields requiring personalized optimization and intelligent adjustment, such as sports training, rehabilitation treatment and personalized education. However, despite significant achievements in this study, there are still some limitations. The system is not effective in dealing with specific vocal styles, and the limitation of experimental scale affects the universality and representativeness of the results. In the future, improvements can be made in four directions: increasing dataset diversity, applying more advanced feature extraction techniques (self-supervised learning and contrastive learning), improving user interface design, and conducting long-term tracking and data analysis of users. The algorithm parameters will be further optimized to improve the generalization ability and adaptability of the system.

Statements and declarations

Footnotes

Conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Juan Guo

References

Doganyigit

Islim

. Virtual reality in vocal training: a case study. Music Educ Res 2021; 23(3): 391–401.

Zhuang

Duan

, et al. A comprehensive survey on transfer learning. Proc IEEE 2020; 109(1): 43–76.

Katoch

Chauhan

Kumar

. A review on genetic algorithm: past, present, and future. Multimed Tools Appl 2021; 80: 8091–8126.

Alhijawi

Awajan

. Genetic algorithms: theory, genetic operators, solutions, and applications. Evol Intell 2024; 17(3): 1245–1256.

Deng

Meng

Cao

, et al. Heart sound classification based on improved MFCC features and convolutional recurrent neural networks. Neural Netw 2020; 130: 22–32.

Wang

. Design of vocal music teaching system platform for music majors based on artificial intelligence. Wirel Commun and Mob Comput 2022; 2022(1): 5503834–5503911.

Khan

Tarimer

Alwageed

, et al. Effect of feature selection on the accuracy of music popularity classification using machine learning algorithms. Electronics 2022; 11(21): 3518.

Boughanmi

Ansari

. Dynamics of musical success: a machine learning approach for multimedia data fusion. J Mark Res 2021; 58(6): 1034–1057.

Zhang

. Singing-voice timbre evaluations based on transfer learning. Appl Sci 2022; 12(19): 9931.

10.

Kumar

Gupta

Singh

. A novel deep transfer learning models for recognition of birds sounds in different environment. Soft Comput 2022; 26(3): 1003–1023.

11.

Kheddar

Himeur

Al-Maadeed

, et al. Deep transfer learning for automatic speech recognition: towards better generalization. Knowl Based Sys 2023; 277: 110851.

12.

Peng

Liu

, et al. Voice disorder classification using convolutional neural network based on deep transfer learning. Sci Rep 2023; 13(1): 7264.

13.

Mavaddati

. Voice-based age, gender, and language recognition based on ResNet deep model and transfer learning in spectro-temporal domain. Neurocomputing 2024; 580: 127429.

14.

Rui

Deng

, et al. Transfer learning using musical instrument audio for improving automatic singing label calibration. IEEJ Trans Elec Engng 2024; 19(5): 707–715.

15.

Zhang

. Speech data system and computer database design based on improved genetic algorithm. J Comput Met Sci Eng 2023; 23(3): 1691–1703.

16.

Chen

Tang

, et al. Research on prediction model of tractor sound quality based on genetic algorithm. Appl Acous 2022; 185: 108411.

17.

Majidi

Toroghi

. A combination of multi-objective genetic algorithm and deep learning for music harmony generation. Multimed Tools Appl 2023; 82(2): 2419–2435.

18.

Xiao

Fan

Chen

, et al. High-resolution acoustic beamforming based on genetic algorithms. Mec Sys and Signal Proc 2023; 204: 110840.

19.

Ishiwaka

, et al. Listening to sounds of silence for speech denoising. Adv Neural Inf Proc Sys 2020; 33: 9633–9648.

20.

Manhertz

Bereczky

. STFT spectrogram based hybrid evaluation method for rotating machine transient vibration analysis. Mec Sys Signal Proc 2021; 154: 107583.

21.

Yin

Huang

, et al. Environmental sound classification using temporal-frequency attention based convolutional neural network. Sci Rep 2021; 11(1): 21552.

22.

Zhang

, et al. Global digital compact: a mechanism for the governance of online discriminatory and misleading content generation. Int J Hum-Comput Interact. 2020; 130(10): 22–32. DOI:10.1080/10447318.2024.2314350.

23.

Sharan

Xiong

Berkovsky

. Benchmarking audio signal representation techniques for classification with convolutional neural networks. Sensors 2021; 21(10): 3434.

24.

Osadchiy

Kamenev

Saharov

, et al. Signal processing algorithm based on discrete wavelet transform. Designs 2021; 5(3): 41.

25.

Hameed

. Speech compression and encryption based on discrete wavelet transform and chaotic signals. Multimed Tools Appl 2021; 80(9): 13663–13676.

26.

Albadr

MAA

Tiun

Ayob

, et al. Mel-frequency cepstral coefficient features based on standard deviation and principal component analysis for language identification systems. Cognit Comput 2021; 13: 1136–1153.

27.

Hannah

AAS

Agordzo

. A design of a low-pass FIR filter using hamming window functions in matlab. Comput Eng Intell Syst 2020; 11(2): 24–30.

28.

Ulicny

Krylov

Dahyot

. Harmonic convolutional networks based on discrete cosine transform. Pattern Recognit 2022; 129: 108707.

29.

Panda

Sharma

Bajpai

, et al. Encoder and decoder network with ResNet-50 and global average feature pooling for local change detection. Comput Vis Image Underst 2022; 222: 103501.

30.

Purnama

. Music genre recommendations based on spectrogram analysis using convolutional neural network algorithm with RESNET-50 and VGG-16 architecture. JISA (Jurnal Informatika dan Sains) 2022; 5(1): 69–74.