Intelligent English teaching prediction system based on SVM and heterogeneous multimodal target recognition

3.1 Feature extraction of EEG signals using wavelet packet decomposition

In recent years, wavelet transform is often applied to signal feature extraction and signal de-noising. The base of the wavelet transform has a shorter time duration than the sine wave base employed by the Fourier transform. The component of a signal is usually not 0 in many time intervals. Similarly, many important features of EEG signals and noise appear only in local small spaces. Moreover, these components are not like the Fourier base. Therefore, there is no efficiency in representation. The emergence of wavelet transform is precisely for this shortcoming.

Wavelet decomposition is a signal feature extraction method with higher frequency domain resolution. Wavelet packet decomposition is an extension of wavelet decomposition. The wavelet packet decomposition is to further decompose the high frequency detail signal which is not processed by the wavelet decomposition after the wavelet transform, thereby further improving the frequency resolution. A schematic diagram of wavelet decomposition and wavelet packet decomposition of the signal is shown in Figs. 1 and 2. In the wavelet packet decomposition of the signal, since the orthogonal decomposition is employed, the two frequency bands generated after the decomposition do not overlap with the frequency bands before the decomposition. Since the bandwidth of the two generated frequency bands is halved, even if the adoption rate is reduced by half in the wavelet packet decomposition, no information is lost.

OPEN IN VIEWER

Fig. 1

Wavelet decomposition of the signal.

OPEN IN VIEWER

Fig. 2

Wavelet packet decomposition of the signal.

For any wavelet series:

D j x ( t 1 ) = ∑ p d n j φ ( 2 j - X 1 )(1) can be decomposed into wavelet packet components:

D j , k , m x ( t 1 ) = ∑ p d n j , k , m φ 2 k + m ( 2 j - X 1 )(2) In the formula, D _j x (t₁) is the wavelet series of the signal x (t₁), d n j is the j wavelet packet subspace of the n scale, φ () is the scale function, X₁ is the translation factor, k is the number of layers of wavelet packet decomposition, m is the decomposition signal obtained by k-layer decomposition, and j is the resolution level.

In the signal X (t), if j = 0, k = 0 and m = 0, which represents the undecomposed source signal X (t) at the resolution level j and is denoted as X₁. Multi-layer wavelet packet decomposition can be performed. If we decompose the signal for the first time, that is, k = 1, m = 0,1, we can obtain the signal after the wavelet packet 1 decomposition, denoted as X₂ and X₃. The signal is further decomposed for the second time. At this point, k = 2, m = 0,1,2,3, and we obtain the signal after the wavelet packet decomposition, which is recorded as X₄, X₅, X₆ and X₇, and so on, as shown in Fig. 3.

OPEN IN VIEWER

Fig. 3

Structure diagram of the wavelet decomposition tree.

The wavelet packet decomposition and reconstruction algorithm are as follows:

1) Equation (3) and Equation (4) show that a signal sequence s ={ x (k) , k ∈ z } is decomposed into two signal subsequences: an even sequence s _e  ={ s _e  (k) , k ∈ z }and an odd sequence s _o  ={ s _o  (k) , k ∈ z }.

s e ( k ) = x ( 2 k ) k ∈ z(3)

s o ( k ) = x ( 2 k + 1 ) k ∈ z(4) k is the sample number in subsequences S _e and S _o .

2) By calculating equation (5 and 6), we can get the signal of each frequency band obtained after the wavelet packet layer is decomposed.

s l ( 2 l - 1 ) = s ( l - 1 ) 2 l - 1 o - p ( s ( l - 1 ) 2 l - 1 e )(5)

s l 2 l = s ( l - 1 ) 2 l - 1 e + u ( s l 2 l - 1 )(6)

3) We reconstruct the wavelet packet according to the formulas (7, 8, 9 and10). In the reconstruction process, we reserve the corresponding frequency band signal and set the remaining frequency band signals to 0:

s ( l - 1 ) 2 l - 1 e = s l 2 l - u ( s l ( 2 l - 1 ) )(7)

s ( l - 1 ) 2 l - 1 o = s l ( 2 l - 1 ) + p ( s ( l - 1 ) 2 l - 1 e )(8)

s ( l - 1 ) 2 l - 1 ( 2 k ) = s ( l - 1 ) 2 l - 1 e ( k ) k ∈ z(9)

s ( l - 1 ) 2 l - 1 ( 2 k + 1 ) = s ( l - 1 ) 2 l - 1 o ( k ) k ∈ z(10)

Each frequency band decomposed in the wavelet packet decomposition of the signal is independent of each other, and no redundant signal occurs, so the signal wavelet packet decomposition is energy balance. Equation (11) represents the energy E (n) in the wavelet packet band:

E ( n ) = ∑ k N | g n ( k ) | 2(11) In the formula, n is the band serial number.

Studies have shown that EEG signals generated by brains collected using EEG scanners are non-stationary random signals, and wavelet packet decomposition can analyze such non-stationary random signals in both time and frequency domains. In the full frequency domain, the wavelet packet orthogonally decomposes the signal, and the wavelet packet decomposition can adaptively select the appropriate frequency band, which improves the time-frequency resolution of the analysis. Therefore, wavelet packet decomposition is an analytical method that is very suitable for EEG signals.

Studies have shown that the brain produces an EEG signal with a frequency range of 9–13 Hz in a state of calm and without any stimulus input. This signal is called rhythm. If the brain is stimulated, such as when the subject uses the brain to perform or simply imagines performing a muscle exercise task (sports, talking, etc.), the 9–13 Hz EEG signal rhythm will be significantly enhanced or weakened. Therefore, the general brain-computer interface system understands the instructions that the brain wants to emit by recognizing this change in rhythm. Because we need to focus on 9–13 Hz EEG signals, we use the Db2 wavelet function to perform 6-layer wavelet packet decomposition on the experimentally acquired EEG signals, and the signals are orthogonally decomposed into 32 independent bands of 0–7.8125 Hz, 7.8125–15.6250 Hz,... In this way, the 9 to 13 Hz sinusoidal signal is assigned to the corresponding second frequency band, and we take the decomposition coefficient and energy of the second layer as the feature vector.

In the brain-computer interface system based on motion imagination, the co-space mode is a widely used EEG signal processing method. The main feature of the co-space model in processing EEG signals is that the characteristics of EEG signals can be extracted based on multiple channels, and the co-space mode depends on a sufficient number of channels. If there is sufficient spatial domain information, the co-space model is ideal for extracting features of EEG signals. This method is based on the simultaneous diagonalization of two covariance matrices to design an optimal spatial filter to resolve the two EEG populations. It was originally used to detect abnormal EEG and has been successfully used in the classification of motion-related EEG in recent years. The basic idea of the co-space model is that the original EEG signal generates a new time series through spatial mode, and its variance is best suited to distinguish between two different types of EEG signals.

The advantage of CSP is that there is no need to pre-select the frequency band specific to the subject. The disadvantage is that it requires a large number of electrodes and multi-channel analysis of EEG. Moreover, the multi-channel EEG amplification system limits the use of CSPs in portable BCI systems, so further optimization of the number of electrodes is needed.

The principle of classifying EEG signals using the co-space mode algorithm is to find a projection of the optimal space, so that the power of the two types of signals is the largest. Therefore, it can estimate two spatial filters to extract the task-related signal components, and at the same time remove the task-unrelated components and noise.

We collect EEG signals through an EEG scanner, and the signal is represented as a matrix E of N * T. N is the number of channels, and T is a sample of all EEG signals collected by each channel. Then, the regularized space covariance of the EEG signal is as shown in equation (12):

C = E E T trace ( E E T )(12) We averaged the EEG signals collected during the two Chinese pinyin pronunciations to obtain their respective average covariances c 1 ¯ and c 2 ¯ . By adding the two, we get the mixed space covariance c m ¯ = c 1 ¯ + c 2 ¯ . After that, we break down c m ¯ into the form of U _m λ _m U _m . min ∥ w ∥ 2 + C ∑ k = 1 k ξ k U m is the eigenvector of the matrix, and then we whiten it.

P = λ m - 1 U m T(13) After that, we make the following transformations for c 1 ¯ and c 2 ¯ .

S 1 = P C 1 ¯ P(14)

S 2 = P C 2 ¯ P(15)S₁ and S₂ have a common eigenvector. Then, when S₁ = Bλ₂, there is B ^T S₂ = Bλ₁B ^T and λ₁ + λ₂ = I (I is the identity matrix). If the sum of the two eigenvalues is 1, when eigenvector B has the largest eigenvalue for S₁, there is a minimum eigenvalue for S₂. The projection matrix we obtained is:

W = ( B T P ) T(16) After that, we decompose the matrix E to obtain the eigenvector of the EEG signal:

Z = WE(17) The column vector of W^-1 is a co-space mode and can be regarded as a distribution vector of the EEG signal source.

4.1 Support vector machine model (SVM)

The support vector machine model is a statistically based machine learning algorithm. It was first proposed in 1992 and quickly gained the attention of researchers and developed rapidly. The support vector machine model is a generalized linear classifier, which is mainly used in the field of visual information acquisition and processing. The basic idea of the SVM model is to map the input vector to a high-dimensional feature space through a kernel function. The sample set given is:

( x i , y i ) i = 1 , 2 , … n(18) In Equation (18), x _i is the eigenvector, n is the number of samples, y _i is the category label, and (x _i , y _i ) is a sample point.

The support vector machine model allows the hyperplane to separate the two types of data in the sample space by finding the optimal classification hyperplane of a sample space. The support vector machine model introduces an error penalty mechanism and performs trade-off optimization in the process of classification, so that the two separate types of data points are farthest from the classification surface. As shown in Fig. 4, the essence of the support vector machine model is to solve the quadratic programming problem with linear inequality constraints as shown in Equation (19), and obtain an optimal classification hyperplane, so as to obtain the decision function to classify the sample space.

OPEN IN VIEWER

Fig. 4

Schematic diagram of the classification of the support vector machine model.

min ∥ W ∥ 2 + C ∑ k - 1 k ξ k , s . t . y k ( W T x k + b ) ⩾ 1 - ξ k , ξ k > 0(19) In the formula, W is the weight vector, (x _k , y _k ) is the sample point, and C > 0 is the error penalty parameter, which can be selected by the cross-validation process. Moreover, ξ _k is the slack variable and b is the bias value. Since the two types of data boundaries are found to be proportional to the ∥W ∥ ², the first term of the objective function is used to maximize the classification boundary, and the second term in the objective function is a regularization term, indicating that training errors are allowed in the inseparable case.

The LibSVM toolbox is a simple, easy-to-use and fast and efficient SVM pattern recognition and regression software package developed by Associate Professor Zhiren Lin of Taiwan University. The SVM parameter adjustments involved in the software are relatively small, and many default parameters are provided. Moreover, the software uses these default parameters to solve many problems and provides interactive verification capabilities. In this study, we used the LibSVM toolbox to identify EEG signal feature vectors.

In order to test the performance of two EEG signal feature extraction methods in the speech brain-computer interface signal feature extraction module, we trained two support vector ma-chine models using wavelet packet decomposition and common space signal eigenvectors extracted from the co-space model algorithm, and tested the classification accuracy of two feature vector extraction models extracted by two support vector machine models. In this paper, we use the wavelet packet decomposition and co-space model algorithm to extract the features of 40 times a pronunciation and 40 times j pronunciation. We select the first 20 a pronunciation and the first 20 j-grams of EEG signal feature vectors as training sets to train two SVM classifiers. Moreover, we use two different algorithm-extracted feature vectors trained SVM classifiers to identify the remaining 20 feature vectors. The classification and recognition results of the feature vectors extracted by the SVM for the two algorithms are shown in Figs. 5 and 6.

OPEN IN VIEWER

Fig. 5

Recognition results of a training support vector machine model for extracting signal features using a co-space model.

OPEN IN VIEWER

Fig. 6

Recognition results of a training support vector machine model for extracting signal features using a wavelet packet decomposition.

We found that the SVM classifier trained by the CSP algorithm extracted the feature vector with 65% accuracy, while the SVM classifier using the wavelet packet 6-layer decomposition training has a recognition accuracy of 75%. Obviously, the feature vectors extracted by the wavelet packet decomposition algorithm have higher recognition accuracy. Then, the English speech brain-computer interface designed by wavelet packet decomposition algorithm will have better performance.