Abstract
Keywords
Introduction
The mouse is a computer input device and it is the navigator for horizontal ordinate and vertical ordinate of the computer screen. It can replace the movement and click instructions input by keyboard. People can click and move the elements by the buttons and wheel of mouse. However, for physical disabled people, such operations are very difficult. Because of physical accidents, birth defects, diseases, natural disasters, etc., the number of physical disabled people is increasing. The prosthetic hand plays an important role in the lives of the physically disabled people. The main functional prosthesis includes cable-operated prosthesis and myoelectric prosthesis [1]. The former has the problems of single function, slow control, clumsy movement and being difficult to maintain. On the contrary, the myoelectric prosthesis is a controllable prosthesis which directly operated by the brain.
A surface electromyography (sEMG) signal is composed of many motor unit action potentials (MUAPs) from many recruited motor units under surface electrode and has background noises [2]. The signal can be recorded by attaching surface electrodes on the skin [3]. The sEMG is often used for diagnosing patients with neuromuscular disorders [4]. Besides, it is also applied to man-machine interface and industrial applications [5]. Because of the existence of sEMG, researchers proposed to use EMG recognition algorithms to control prosthesis [6, 7]. Amputees contract muscle to produce sEMG signals by the motor nerve. And then, sEMG could be recognized to know the movement the amputees want. Last but not least, the prosthesis is controlled to move as the recognized movement, and therefore, sEMG has great importance in the control of prosthesis.
The sEMG has been used in prosthesis control by many researchers [8–12]. Chu et al. [8] applied real-time sEMG pattern recognition for multifunction myoelectric hand control. Momen et al. [9] also did this work. Chan et al. [10] did fuzzy sEMG classification while Pradeep et al. [11] worked on online sEMG control. While other researchers devoted themselves on controlling the hand, Zhao et al. [12] concentrated on controlling five fingers of the prosthesis hand. They combined Levenberg Marquardt (LM) or variable learning rate (VLR) based neural network with parametric Autoregressive (AR) model and wavelet transform. Results showed that the method had fast learning speed and high recognition capability. In general, the process of sEMG classification includes: data acquisition, feature extraction and classification.
Data acquisition
The EMG signal can be acquired by needle electrode [4, 13] or surface electrode [2, 14]. Also, it can be obtained through single or multi channels. Subasi et al. [4, 13] recorded EMG signals from the biceps brachii muscle of 27 subjects using a concentric needle electrode. Chu et al. [8] used four surface electrodes to measure the sEMG signals. Each subject needed to perform each motion once for a duration of 5 s. Nagata et al. [14] measured sEMG using 96 channels of multi-channel surface electrode. Therefore, in order to maintain a quick response of the system, they selected 16 using channels from it.
Feature extraction
Although discrete wavelet transform (DWT) was a common used method to extract a large number of features [4, 15], feature reduction is also important to select an optimal set of effective features [16]. For example, Chu et al. [8] used linear discriminant analysis (LDA) to reduce and cluster the WPT features. Phinyomark et al. [5] removed redundant features in time and frequency domain by scatter plot of features, statistical analysis and classifier. Kilby et al. [17] employed continuous wavelet transform to extract effective features while Hu et al. [18] employed wavelet packet transform and wavelet coefficient entropy for feature extraction. Unlike other researchers, Pradeep et al. [11] sampled the sEMG signals and then calculated the RMS of windowed steady state sEMG signals from each channel as feature extraction. Autoregressive (AR) parametric model and wavelet transform were used by Zhao et al. [12] in feature extraction. Liu et al. [19] also used autoregressive model to accomplish the operator.
Classification
Some other algorithms were used for sEMG classification by researchers, such as particle swarm optimization (PSO) [4], neural network [13, 21], support vector machine (SVM) [4, 21]. Subasi [4] used PSO optimized SVM to classify sEMG signals while Lucas et al. [15] used discrete wavelet transform and SVM. And Alkan et al. [22] used discriminant analysis and SVM to classify. Hu et al. [2] classified sEMG based on energy spectra change. Liu et al. [19] proposed a novel sEMG classifier called cascaded kernel learning machine to achieve the classification. And then, Huang et al. [23] improved classification accuracy by spatial filtering. Nagata et al. [14] used Euclidean distance to classify the sEMG of 12 movements of hands. Bu et al. [24] classified EMG signals by a probabilistic neural network and combined it with the Bayesian network motion prediction to generate motion commands for control. Wang et al. [25] implemented the classification by optimal wavelet packet method based on Davies-Bouldin criterion.
The paper is organized as follows. In Section 2, we describe the methods used for sEMG signal classification. It includes data acquisition, sample extraction, feature extraction, and classical classification methods. Section 3 presents experiments and analyses the performance. Finally, we conclude the paper in Section 4.
Method
In this paper, we proposed a window-based data acquisition method and a two-dimensional matrix image based feature extraction method for sEMG. We acquire and analyze the sEMG signals by four steps: Firstly, sEMG signals from fingers movement are acquired by electrodes; Secondly, because the sEMG is an analog signal, it is transformed to digital signal by capture devices. Then, data samples are extracted from the sEMG digital signals. Thirdly, feature maps are generated from the samples. Finally, the movement samples are classified according to the extracted features, which is the recognition process of the movement signals. The classification is based on the SVM, KNN and RBF-NN. The whole process is shown in Fig. 1.
Data collection
We try to recognize the click of left and right mouse button by index and middle fingers. At present, there are many sEMG acquisition equipments in the market with the frequency ranging from tens to thousands hertz [26, 27]. However, because of noises, the main energy changes are smaller. Muscle Sensor V3 (Advancertechnologies Inc., http://www.advancertechnologies.com/p/muscle-sensor-v3.html) is used to acquire signals of the forearm flexor retinaculum muscles. As shown in Fig. 2, the Muscle Sensor V3 with a micro-controller’s analog-to-digital module can transform the high frequency raw sine wave to rectified and smooth sine wave with a frequency of 10 Hz.
The time to accomplish one click of mouse button takes about 1-2 seconds. But, the speed of clicking mouse button is unique for different people. Figure 3 illustrates three different sEMG signals. The first sEMG represents the situation of clicking once per second. The second one is about one click every two seconds, while the third is based on the same operation every three seconds.
Figure 4 shows the sEMG collected from the device (National Instruments, BNC-2090) and the sample frequency was 4000 Hz. It’s easy to see the energy of index finger’s sEMG is greater than that of middle finger. However, if there is no software to rectify and smooth the high frequency signals, it is difficult to extract features from them and classify them. So this study adopted Muscle Sensor V3 to acquire the sEMG for analysis.
Samples extraction
As shown in Fig. 5, when clicking mouse button, the sEMG will produce a pulse. Extracting the pulse of each click will generate a sample. The samples with tag can be used to train the classification models. Here, we use a slide window with a fixed-length of N to extract samples.
The processes of samples extraction are as follows:
amax = max(A, i, j, N) #get the maximum amplitude in the window
amin = min(A, i, j, N) #get the minimum amplitude in the window
L.add(S) #put the sample S into List L
Feature extraction
After data acquisition, we have a serial of data samples as shown by Fig. 6. Each red line indicates a sample of index finger click while black dotted line indicates a middle finger click. In the experiment, the length of window is set to 9, which can effectively acquire and extract the data samples.
In Fig. 6, we can see that the energy of each sample is concentrated in the middle. That is to say, the energy of the signal is larger when closer to middle of sample. At the same time, to a certain extent the energy of the middle point can distinguish two different classes of samples. We propose a two-dimensional matrix image to separate signals according to the process of the energy changes, with the hope of using the generated two-dimensional matrix as the feature vector for classification. For a sample S (a1, a2, …, a l ) with the length of l, we generate a diagonal matrix M, M ii = a i , i ∈ [1, l]. For every i ∈ [2, l - 1], assume that Mj,2i-j = M2i-j,j = a i , for every j ∈ [1, i - 1] (see Fig. 7). Then, we use the matrix as the feature vector of the sample. We further enlarge the energy of middle point of the sample and transform the matrix into a grey-scale image with l×l pixels by min-max normalization, which is regarded as a feature map. The value of each pixel in the image is a normalized value within [0, 255]. Within the image, the partial white pixel has more energy while the partial dark one has less energy.
Classification based on SVM
Support vector machine (SVM) was first proposed by Boser et al. in 1992 [28]. It is a classification algorithm that is based on statistical learning theory. The essence of SVM is to find a separating hyperplane in the space to separate two kinds of samples and the margin of the data as maximum as possible. The two-dimensional plane in Fig. 8, for example, has two kinds of data that are indicated by circles and squares respectively. L is the classification line. L1 and L2 go through points nearest to L and are parallel with L. The distance between L1 and L2 is the margin. The objective of the SVM is to find the optimal classification line. It means the line can classify the two classes correctly (the training error rate is 0) and the margin is maximum.
In the two dimensional plane, the classification line can be represented with f (x) = (w * x) + b = 0. Substituting a point into the classification function can get the class it belongs to: If f (x) < 0, the class is 1; If f (x) > 0, the class is -1; If f (x) = 0, the point is on the line.
We can normalize the classification function. So, the linearly separable samples (x i , y i ) , x i ∈ R n , y i ∈ { 1, - 1 } , i = 1, 2, …, l meet Eqn. (6):
Therefore, the margin equals to 2/ . We maximize the margin to be equivalent in order to minimize . The separate plane can satisfy condition (6) and minimizing is achieved by the optimal separate hyperplane.
For linear inseparable situations, SVM will calculate in low-dimensional feature space first. Then, it maps low-dimensional feature space to a high-dimensional feature space by kernel function. Finally, the optimal separating hyperplane is located in the high-dimensional feature space. So, the linear inseparable situation is transformed to linear separable.
K-Nearest Neighbors (KNN) is a classic statistical-based classification algorithm. It was first proposed by Cover and Hart in 1967 [29]. The core concept of KNN is that if most of the k nearest samples of a sample belonging to a class, the sample also belongs to it. Besides, the sample has the characteristics of the class. KNN decides the class of a sample only according to the one or several nearest samples’ classes. Therefore, the decision only relates to a few neighbor samples.
Nearest Neighbor (NN) treats all training samples as representative points. So, it needs to calculate the distances between sample x and all training samples. The result is the class of the nearest sample. Assume that there are c classes of ω1, ω2, …, ω
c
to recognize and each class has N
i
, i = 1, 2, …, c identified samples, we can define the discrimination function of ω
i
as:
Here, the subscript i of is the class ω i and k is the k-th sample in N i samples of class ω i . The decision rule is:
if g j (x) = min g i (x) , i = 1, 2, …, c, then x ∈ ω j .
KNN is an extension of NN. KNN selects k nearest neighbors of x and its class is the one that most of its k nearest neighbors belong to.
We define the discrimination function as:
if g j (x) = max(k i ), then x ∈ ω j .
Now, Fig. 9 represents an example of KNN.
As shown in Fig. 10, RBF (Radial Basis Function) neural network is a three layered feedforward neural network. Hidden layer neurons respond to the input by activation function. Activation function is often RBF, such as the Gaussian kernel function:
In Eqn. (9), ω i , u i and σ are the parameters that the network needs to learn.
The activation function of RBF neural network is RBF. After going through the input buffer, the hidden units are unable to accept the input vector directly but the measured ‘distance’, which is the distance between input vector and the center of RBF. Assuming that the number of hidden units is L, the center vector of RBF can be determined by a dynamic recursive algorithm [30]. The width of Basis Function and the weights between hidden layer and output layer can also be determined. The RBF-NN works as follows:
(1) The input vector of sample i is:
Here, M is the dimension of input vector and central vector of hidden units.
(2) Choose RBF as the Gaussian function. So, for sample i, the input of the j-th hidden unit is:
In Eqn. (11), x ik is the k-th component of input vector X i . u jk is the k-th component of the central vector of j-th hidden unit’s basis function. σ j is the width of j-th hidden unit’s basis function. And L is the number of hidden units.
(3) The output model of hidden units is:
(4) The real output of the network is:
In the formula above, ωk′j is the weight between hidden unit j and output unit k′. S is the number of output units. By Eqn. (13), we can obtain the real output result of sample i:
The error target function of the sample is:
Now, yik′ is the expected output. Here, E is the squared error between real output and expected output of sample i, which is used to evaluate our results.
The value of true positives (TP), true negatives (TN), false positives (FP) and false negatives (FN) can be used to measure the performance of classification method. The prediction accuracy (Acc) for positive compounds is sensitivity (Sen), and the prediction accuracy for negative compounds is specificity (Spe).
Results
Data acquisition and environment
In this paper, Muscle Sensor V3 is used to acquire signals from the forearm flexor retinaculum muscles. In the experiments, we collected the signals of five persons while they clicked mouse buttons at different frequencies (once per second, once every two seconds, and once every three seconds). Each collection lasted two minutes. Figure 11 was the position of electrodes while collecting sEMG signals. Therefore, we obtained 500 samples of index and middle fingers each and this makes up a total of 1000 samples.
All the algorithms are written using Python programming language and operated on a computer (Ubuntu 14.04 LTS, Core i7-6850K CPU, 3.6 GHz, 128 GB memory space, GTX Titan X GPU). The algorithms are accelerated by GPU.
Feature extraction
We used the two-dimensional matrix image based signal energy separation method to extract features. Figure 12 illustrates the feature maps transformed from clicking samples of the index finger and the middle finger. We can see that the feature maps from index finger are brighter while the feature maps from the middle finger are darker. That is to say the energy produced by the index finger was larger than energy produced by middle finger. So, we can effectively distinguish these two types of samples by using feature maps.
Comparison of classification results
For the feature map of each sample, we classified them by three different classical classification algorithms. In SVM, c is set to 2 and γ is set to 0.2. In KNN, k = 5. And in RBF-NN, spread = 0.5. The SVM classifier used the LIBSVM library of National Taiwan University [31], KNN classifier used Scikit-learn library [32] and RBF-NNE used the neural network toolbox of Matlab. Table 1 shows the classification results of different classifiers, with different proportions of training set and testing set. We could find that the accuracy of each classifier did not have significant change in different experimental conditions. It shows that the feature map extraction method could reflect data distribution well, even when the size of training data is small. When the size of training data is small, the accuracy of the SVM classifier could also reach up to 100%. And when the training data becomes more, the accuracy doesn’t fluctuate. That is to say, by using the features the performance of SVM classifier is stable.
Figure 13 illustrates the results of three classifiers in ROC space. We could find that SVM is superior to RBF-NN and the RBF-NN is better than KNN.
Conclusion
The paper presents research on the sEMG signals that are produced by clicking mouse buttons. It includes four steps: data acquisition, sample extraction, feature extraction, and classification. In data acquisition, we used the Muscle Sensor V3 to acquire signals produced by each clicking. Afterwards, we employed length-fixed sliding window to extract samples from signals. A two-dimensional matrix image based feature extraction method is proposed, which could transform samples to feature maps for classification. In the experiments, we used three classical classifiers (SVM, KNN, RBF-NN) to classify the generated features. The feature map is helpful for effectively distinguishing the samples and the average accuracy of the classifiers could reach up to 90%. Besides, the accuracy of the SVM classifier can reach up to 100% even when the size of training data is small. That is to say, the feature extraction method is effective.
We performed binary classification on the single-channel sEMG signals. The future work is to validate that whether the proposed feature extraction approach is suitable for multi-classification of multi movements or not. At the same time, because the number of different movements is increasing, multi-channel sEMG is also needed. How to effectively extract features from multi-channel sEMG is also an important work that we hope to engage on in the future.
Footnotes
Acknowledgments
This work was supported by NSFC (No. 61402387); NSFC (No. 61402390); Science and Technology Key Project of Fujian Province, China (2014H0044); Science and Technology Guiding Project of Fujian Province, China (2015H0037, 2016H0035); Enterprise Technology Innovation Project of Fujian Province; Education and Research Project of Young and Middle-aged Teacher of Fujian Province, China (JAS151230, JA15018); Overseas Study Scholarship of Fujian Province; and Science and Technology Project of Xiamen, China (3502Z20153026).
