Abstract
In this work, angular displacement and angular velocity of the elbow during continuous flexion and extension movement are estimated using three different models, with Surface Electromyography (SEMG) time domain parameters as model inputs, and the results are compared to select the model that gives the most accurate results. Surface Electromyography (SEMG) is recorded using surface electrodes placed on the biceps brachii muscles during continuous flexion and extension of the elbow joint. SEMG recording is done with elbow angle changing at different angular velocities. The obtained SEMG signals are segmented into 250-millisecond duration windows using disjoint windowing technique. Two time-domain parameters, Integrated Electromyography (IEMG) and Zero crossing (ZC) are derived from each windowed SEMG signals. The obtained value of IEMG and ZC are fed as the inputs to the Multiple Input Multiple Output (MIMO) model for the estimation of elbow angular displacement and elbow angular velocity. In this work three Multiple Input Multiple Output (MIMO) nonlinear black box model are developed using a Nonlinear Auto Regressive with eXogenous inputs (NARX) structure: (1) Multi-Layered Perceptron Neural Network (MLPNN) model based on NARX input, (2) Elman neural network model based on NARX input and (3) Adaptive Neuro-Fuzzy Inference System (ANFIS) model based on NARX input. The results obtained from the three different models are compared using statistical parameters like regression coefficient and root mean square error (RSME). Based on this comparison the paper proposes that the estimation of elbow kinematics using ANFIS NARX model gives more accurate results when compared with Elman NARX model and Multi-Layered Perceptron Neural Network (MLPNN) NARX model.
Keywords
Introduction
Electromyography (EMG) is the branch that deals with the study of muscle functions by analysing the electrical signals generated by the muscles. All the body movements achieved by human beings are due to the results of extension and flexion of skeletal muscles. Motor units are the basic fictional units of muscle fibre. The starting point of any voluntary movement of body parts is by the generation of stimulus nerve signal from the central nervous system. The stimulus signals from the central nervous system are transmitted to the nerve endings of the muscle. These stimuli excite several individual motor unit by generating electrical potentials across its cell membrane and leads to flexion and extension of muscle that intern leads to movements of human body parts. The Electromyography signal detected from the surface of the skin above the skeletal muscles are referred as Surface Electromyography signal (SEMG) [1]. It is considered as the sum total of the electrical potential produced by different motor units of muscle fibres. SEMG signals are packed with information regarding both static and dynamic conditions of the muscles [2]. Conducting surface electrode can be used to detect electromyography signals from the skin surface. For the past few years, SEMG is widely used for diagnosing neural and muscular disorders [3], analysing fatigue and non-fatigue conditions of muscles [4, 5] and also for human intention detection [6, 7] and biomechanical control for partially affected amputees [2, 8].
SEMG signals are considered to be the most suitable solution for the rehabilitation of partial hand amputees [9]. Several research works have been conducted in recent years for predicting human intentions by considering it as a classification problem and predicting the kinematics of elbow, shoulders, fingers, etc. by considering it as an estimation or identification problem.
Rami N. Khushaba et al. attempted to predict human intentions by classifying the individual and combined finger movements of subjects using time domain features obtained from the SEMG signals [6]. They achieved a classification accuracy of 90% for different subjects. Different hand movement like left, right, up and down are classified with an accuracy of 88.4% by Ahsan et al. [10]. They classified the hand motion using Artificial neural network (ANN) based classifiers.
Along with several classification works, works were also conducted for the estimation of human kinematics continuously using the parameters derived from the SEMG signals. Some of the research activities held in the area of estimation of human kinematics are based on white box modelling [11], which require a real knowledge about the system under study and are very complex. A large number of system parameters has to be considered during modelling, and it makes the white box modelling impractical. Works were also done in the area of black box modelling that consider the system as a black box and the model are derived from the input and output values of the system [11]. Lizhi Pan et al. predicted finger joint angle using EMG for partially affected hand amputees [12]. They made subjects to move their finger from maximum flexion to maximum extension while performing different static wrist motions. Here a state space model is used for the prediction. The predicted angle is validated using regression value and obtained a regression value of 0.843. Nikhil A. Shrirao et al. did work to estimate the finger joint angle by extracting normalised root mean square (NRMS) value from SEMG signal [13]. Here different neural network structures are used to predict the joint angle. The results are validated using normalised root mean square error (NRMSE). The obtained maximum NRMSE is 0.147 for finger extension and 0.163 for finger flexion. Another attempt was made by Arthur T.C. et al. for the prediction of the kinematics of shoulder and elbow for both able bodied and patients with spinal cord injuries [3]. They estimated joint angle, angular velocity and angular acceleration using rectified raw EMG signals. A time delayed artificial neural network model is used for the prediction. The model is validated using RMSE. They obtained an RMSE value less than 20% for both able bodied and subjects with spinal cord injuries while estimating the elbow joint angle during flexion and extension of the elbow. Nor Anija Jalaludin et al. conducted experiments to determine the thumb joint angle and thumb tip force using RMS of raw SEMG signal [14]. Here a feed forward neural network structure is used as the model and trained it using the Levenberg-Marquardt algorithm. The performance of the system is validated using RMSE value. Muye Pang et al. estimated the elbow joint angle using a white box model approach [15]. Here a white box type muscular model is used for the estimation of elbow joint angle. This study achieved RMSE values of less than 10%. Yu et al. suggested a method to estimate the elbow joint angle using a third order polynomial function model and they achieved a mean square error (MSE) of 1.4883 while conducting experiments using experienced subjects [16]. Here they used an additional optimisation technique to reduce the MSE. Panagiotis K. Artemiadis et al. developed robotic arm where the position and force are controlled using EMG signal [17]. Here a Multiple Input Multiple Output (MIMO) black box state space model is used for the estimation. Panagiotis K. Artemiadis et al. also did work on teleoperation of a robotic arm for planar catching movements using SEMG signal [18]. Here an Auto Regressive moving average with exogenous output (ARMAX) model is used for the estimation and control. Another black box type modelling for estimating elbow joint angle was conducted by Li Dapeng et al. using a three layered feedforward neural network trained using back-propagation algorithm [19]. Yee Mon Aung et al. worked on the estimation of upper limb joint angle using SEMG signal [20]. RMS value of SEMG signal taken from anterior deltoid, posterior deltoid, biceps brachii and triceps brachii muscles are used as the input parameters for the estimation. A three layered back propagation neural network model trained using the Levenberg-Marquardt algorithm is used in this work for finding the joint angle. Results indicate that they achieved an average MSE of 0.0176 while estimating the elbow joint angle. S. Hussain et al. classified the intention of the human being using an Adaptive Neuro-Fuzzy Inference System (ANFIS). They achieved appreciable classification accuracy in their work. Abdulhamid Subasi also achieved a classification accuracy of 95% while classifying the subjects as normal, neurogenic or myopathic using ANFIS [21]. In their work, they also compared the accuracy of classification using Multi-Layered Perceptron Neural Networks (MLPNN), Dynamic Fuzzy Neural Network (DFNN) and Adaptive Neuro-Fuzzy Inference System (ANFIS) based classifiers.
From the literature survey it is clear that there exists a relationship between the SEMG signal parameters, velocity of movement and angular position of joint and also that the relationship is strictly non-linear [22, 23]. At the same time, very limited number of research happened on the combined estimation of elbow joint angle and elbow movement velocity.
In this work, three Multiple Input Multiple Output (MIMO) nonlinear black box models are developed using a Nonlinear Auto Regressive with eXogenous inputs (NARX) structure for the estimation of elbow kinematics. (1) Multi-Layered Perceptron Neural Network model based on NARX input, (2) Elman neural network model based on NARX input and (3) Adaptive Neuro-Fuzzy Inference System model based on NARX input. Two-time domain parameters, Integrated Electromyography (IEMG) and Zero crossing (ZC) extracted from the SEMG signals obtained from the biceps brachii muscles during self-paced movements of the elbow are used as the two inputs to the MIMO system. The estimated elbow angle and the estimated angular velocity of the elbow angle change are the two outputs of the MIMO system.
The rest of the paper is arranged as follows. Section 2 explains the experimental setup and feature extraction, which includes data acquisition, signal processing, experimental protocol and method of feature extraction. Section 3 explains the methodology of estimation which includes basic dynamic model structure, Multi-layered perceptron neural network (MLPNN) NARX model structure, Elman neural network NARX model structure and Adaptive Neuro-Fuzzy Inference System (ANFIS) NARX model structure. The experimental results, validation of all the three models and comparison of models are included in Section 4, and Section 5 explains the conclusions drawn from the work.
Experimental setup and feature extraction
Signal acquisition and signal processing
SEMG signal from the biceps brachii muscle during the change of elbow angle are acquired using a custom built Bio amplifier. A standard biosignal acquiring circuit using an instrumentation amplifier (INA 128P) and a high-speed operational amplifier (OPA2132P) are used for the SEMG signal acquisition. Custom designed bio-amplifier has a gain of 10, a common mode rejection ratio of 100 dB (minimum) and a bandwidth of 200 KHz. The amplified SEMG signal from the bioamplifier is acquired using MyDAQ data acquisition card, National Instruments make, having a maximum sampling rate 200 KSamples/sec and ADC resolution of 16 bit. SEMG data is sampled at 10 K samples/sec and filtered using LabVIEW software. A third order IIR bandpass filter with a lower cut-off frequency 20 Hz and upper cut-off frequency 400 Hz is implemented for the removal of baseline shifting and high-frequency noise. Elbow angle is measured using a tilt sensor consisting of a triple axis analog accelerometer ADXL 335 and a supporting program. The accelerometer module is attached to the forearm of the subject. ADXL 335 outputs three analog voltages proportional to the accelerations in the X, Y and Z directions caused by the change of elbow angle. The analog voltage corresponding to Z directions acceleration is read through one of the channels of the data acquisition card at a sampling rate 10 K samples/sec. A program coded using LabVIEW computes the elbow angle from this acceleration value. The angular velocity of elbow angle change is derived from the computed elbow angle by differentiating elbow angle with respect to time, using LabVIEW program. The accelerometer based elbow angle measurement setup is calibrated using a protractor chart hanging on the wall. The schematic diagram of the experimental setup is shown in Fig. 1.
Experimental protocol
Four able-bodied subjects are permitted to partake in the experiments. The average age of the participants is 28 years, average height is 1.68 m, and an average weight is 69 Kg. A prior consent is taken from all the subjects before participating in the experiments and also made sure that none of the subjects are having any neuromuscular problems. The subjects are instructed to stay in standing position, and the accelerometer module is attached to the forearm tightly. Three disc type surface electrodes made of silver-silver chloride (Ag-AgCl) is used for the acquisition of SEMG data. One of the electrodes (positive electrode) is placed at the centre of the biceps brachii muscle and the second electrode (negative electrode) is placed at the lower end of biceps brachii muscle [19]. The distance between the electrodes is kept as 3 cm [5]. The third electrode, the reference electrode is kept at upper side of human palm [23]. The subjects are allowed to take complete rest before conducting the experiment and are told to relax while undergoing experiments. The subjects are asked to perform continuous elbow extension and flexion for some time with varying velocity of flexion and extension. Subjects are instructed to follow a predetermined trajectory displayed on the monitor during the experimentation to get SEMG data at clearly separable and different velocities and at the same time told to make the movements as smooth as possible. SEMG data are taken from the dominant hand of the subjects while non-dominant hand is directed to keep in normal anatomical position, i.e., the elbow is kept in fully extended condition. The picture of the experimental setup and the position of electrodes are shown in Fig. 2.
Feature extraction
One of the main constraint before feature extraction is the length of the SEMG data used for extracting the features. The classification or estimation accuracy increases with increase in segment length of SEMG data. But as the segment length increases the response will become more discontinuous [2]. Data segmentation can be implemented using either sliding window method [9] or disjoint window method [24]. Although sliding window method with maximum overlap was assumed to be the one that provides good accuracy than disjoint window method, it is not frequently used because of high computational cost and time requirements [25]. Here in this work, disjoint windowing method with 250-millisecond window size is used as a compromise to above said problems. Raw SEMG signals, considered as a time series signals that are non-repeatable and stochastic in nature, but at the same time, it is observed that certain features of SEMG signals are varying with the kinematics of human muscles [26]. Several works have been conducted for determining the human kinematics using time domain features [27], frequency domain feature [28], wavelet based features [29] and also with combinations of different features [21].
IEMG is defined as the mathematical integral of absolute value of SEMG signal amplitude. IEMG gives a good measure of muscle kinematics. The equation for finding IEMG is given by,
Where N is the number of samples of SEMG signals in one window and xk is the SEMG signal.
Zero crossing (ZC) is the measure of number times the SEMG signal amplitude crosses zero line. The formula for ZC is given as
Where N is the number of samples of SEMG signals in one window and xk is the SEMG signal.
Here, in this work the SEMG data is segmented into 250-millisecond windows using disjoint window method. Several time domain parameters like root mean square value (RMS), zero crossing (ZC), Wilson amplitude (WAMP), integrated EMG (IEMG), moving average value (MAV) and waveform length (WL) are derived from each window of SEMG data. A correlation experiment is done between angular displacement and corresponding value of SEMG features during continuous flexion and extension movement. The obtained value of correlation coefficient while correlating angular displacement and SEMG features are shown in Table 1. It is observed from the table that IEMG and ZC are showing better correlation with angular displacement when compared with other parameters. So these two parameters are used for later experimentation.
Basic dynamic model structure
Several dynamic model structures are used for prediction and estimation. The selection of appropriate model is one of the major difficulties in estimation. NARX network designed as the feed forward time delay neural network can be successively used for the prediction of time series systems [30]. Mathematical representation of an NARX model structure is given as
Where y(t) is the predicted output, y(t-1), y(t-2),etc. are time delayed outputs, u(t) is the input to the model structure, u(t-1), u(t-2), etc. are time delayed inputs. n u and n y are the input and output order of the system [31, 32] and f(.) is considered as a nonlinear approximation function. The nonlinear approximation function f(.) is approximated using several methods like neural networks, fuzzy logic and also by using the combination of both fuzzy logic and neural networks, i.e., by using ANFIS methods.
Multiple Layered Perceptron Neural Network, also called Multiple Layered Feed Forward Neural Network, is the simplest class of neural network. MLPNN consists of more than two layers of neuron. The first layer is called input layer where the input data are applied, and the last layer is called output layer, where the estimated outputs are taken out. A number of hidden layers are kept in between the input and output layer, which helps the neural network to learn complex task [33]. The signal propagates from the input layer to output layer through hidden layers in one direction. MLPNN is most commonly used for estimation of time series systems. Steepest descent backpropagation algorithm is used commonly for the training of MLPNN [31]. The simplicity of algorithm leads to the selection of steepest descent backpropagation algorithm despite its slow converging speed. The structure of proposed MLPNN NARX model for the estimation of elbow angle and angular velocity of elbow angle change is shown in Fig. 3 [34]. The value of n u and n y are taken as 1 for simplicity of the diagram, but in actual implementation the value of n u and n y is taken as 3. A three layered MLPNN is used in this work. The number neurons in the input layer is taken as 14 and that of output layer is 2, since the model consists of 14 inputs (IEMG, ZC, time delayed IEMG and ZC and time delayed outputs) and 2 outputs (angular displacement and angular velocity). The number of neurons in the hidden layer is taken as 25 after several trails of experiment with the aim of getting optimum results (i.e., minimum RMSE). The NARX MLPNN model is trained using steepest descent back propagation algorithm.
Elman Neural Network NARX model
Another important classification of neural network is Recurrent Neural Network (RNN). RNN is a neural network with at least one feedback between layers. This type of network has close resemblance with neural connection of human brain. One of the simplest types of RNN is Elman Neural Network (ENN) [35, 36]. ENN is almost similar to MLPNN, the only difference is that along with the input layer, output layer and hidden layer, here exits an additional layer called context layer. Feedback connections are made from middle hidden layer to this context layer, which makes the system sensitive to the history of input data. RNN type neural network will give a better prediction accuracy for time series system [37].
The proposed structure of Elman neural network NARX model for the estimation is shown in Fig. 4. Here also the value of n u and n y are taken as 1 for simplicity of the diagram, but in actual implementation the value of n u and n y is taken as 3. The number of neurons in the input layer is 14, output layer is 2 and hidden layer is 25. The model is trained using Levenberg-Marquardt (LM) back propagation algorithm.
Adaptive Neuro-Fuzzy Inference System (ANFIS) NARX model
ANFIS is another method that is commonly used for the prediction of the time series signal system [21, 38]. Here the function, f(.) in the NARX model is approximated using ANFIS. ANFIS combines the benefits of both neural network and fuzzy logic system. ANFIS model uses Takagi Sugeno’s type fuzzy method at the same time it has neural network architecture makes the system learn from the training data.
Here in this work an ANFIS NARX model is used for the estimation of elbow angle and elbow velocity with 14 inputs and 2 outputs. The structure of proposed ANFIS NARX model is shown in Fig. 5 and fuzzy rules for estimating elbow angle and elbow velocity is shown in Table 2. The value of n u and n y are taken as 1 for making the diagram and rules simple. But in actual implementation the value of n u and n y is taken as 3.
Results and discussion
During the conduct of experiments, the subjects are asked to perform continuous elbow flexion and extension with different angular velocity. SEMG signals are recorded from the biceps brachii muscles of hand using Ag-AgCl surface electrode. The obtained SEMG signals are segmented using disjoint window technique into windows, each with 250-millisecond sizes. Then the time domain parameters IEMG and ZC are extracted from each window segment. The actual value of elbow angle is measured using a calibrated three axis accelerometer, and the angular velocity of elbow angle change is computed by differentiating the measured elbow angle with respect to time.
The representative graph of IEMG and ZC for continuous flexion and extension of elbow joint at different angular velocity are shown in Figs. 6 and 7. The representative graph of actual angle values measured using calibrated accelerometers for different elbow flexion and extension of subjects is shown in Fig. 8. The obtained value of IEMG, ZC, angle and velocity data are divided randomly using LABVIEW software, 50% of the data are used as the training data, and full datasets are used for the validation of the model. The IEMG and ZC training data are fed as the input to three different models, MLPNN NARX model, Elman NARX model and ANFIS NARX model. The corresponding known values of angle and velocity data are used as the target for training the model. Figures 9 to 11 indicates the predicted values and actual values of elbow joint angle obtained from all the three models. Figures 12 to 14 indicates the predicted values and actual values of the elbow joint angular velocity obtained from all the three models. The error between predicted values and actual values during the estimation of elbow angle and velocities for all the three dynamic models are shown in Figs. 15 and 16.
The obtained models are validated and compared using two criteria, Root mean square error (RMSE) and regression coefficient (R). The RMSE results during the estimation of angular displacement and angular velocity using three different models calculated for 4 subjects (sub1-sub4) with each performing 3 trails (T1-T3) is summarized in Figs. 17 and 18. Above figures also shows the average, maximum and minimum RMSE values obtained during the estimation of angle and velocity from the above experiment. The regression results during the estimation of angular displacement and angular velocity using three different models calculated for 4 subjects (sub1-sub4) with each performing 3 trails (T1-T3) is summarized in Figs. 19 and 20. The figures also shows the average, maximum and minimum regression values obtained during the estimation of angle and velocity from the above experiment. From the above figures, it is clear that the estimation using ANFIS NARX model is giving lowest value of RMSE and the regression value for ANFIS NARX model is almost close to 1. The regression plot for elbow angle estimation and elbow angular velocity estimation are also shown in Figs. 21 to 23. A two-way analysis on variance (ANOVA) was done to find the interaction of two factors, model (MLPNN, Elman and ANFIS) and experimental regression value during angular displacement estimation for different subjects (sub1-sub4) [39]. The null hypothesis is set as “there is no interaction between subjects and model”. The above analysis found that there is no interaction between different models and subjects. Since the p value > 0.05 and also f < fcritical (p value = 0.483, f = 0.845 and fcritical = 3.009). So fail to reject the null hypothesis. A similar ANOVA study is also done during the angular velocity estimation and obtained same result. A two-way ANOVA was also done to find the interaction of different models (MLPNN, Elman, and ANFIS) and repeated trials on same subject (T1-T3) during the estimation of angular displacement. Here the null hypothesis is set as “there is no interaction between repeated trials and models”. The above analysis found that there is no interaction between different models and repetition of experiment (p value = 0.263, f = 1.902 and fcritical = 6.944).∥A similar ANOVA study is also done during the angular velocity estimation and obtained same result. At the same time a significant interaction is found between the models (MLPNN, Elman and ANFIS) on subjects (sub1-sub4) (p value = 0.034, f = 3.693 and fcritical = 3.403). Tukey comparison was also conducted to find the best among the three models [40]. Tukey comparison found that the regression value of the three models has following significant difference: Elman<MLPNN<ANFIS. Tukey comparison is also done using RMSE value of three different model and found that the RMSE value of three different model has following significant difference: Elman>MLPNN>ANFIS. Results show that a better performance is obtained for ANFIS NARX model when compared with Elman NARX and MLPNN NARX model while estimating the elbow kinematics.
Conclusion
In this paper, the elbow angle and angular velocity of the elbow angle change during continuous flexion and extension of elbow joint are estimated from time domain parameters of SEMG signal using three different models, MLPNN NARX model, Elman neural network NARX model and ANFIS NARX model. Two time domain parameters IEMG and ZC, extracted from the SEMG signal are given as the input to the MIMO model, and two outputs elbow joint angle and elbow movement angular velocity are estimated from the proposed model. The performance of all the three models are compared using two statistical parameters, Root mean square error (RMSE) and regression coefficient. Results shows that the elbow joint angle and elbow movement angular velocity can be estimated using all the three models with good accuracy. Results also indicate that the ANFIS NARX model is giving higher accuracy in estimation of elbow kinematics when compared with the other two models.
Footnotes
Acknowledgments
The financial support and laboratory facilities provided by the parent institution for conducting this research are gratefully acknowledged. Authors of all the journals listed in the reference section are also acknowledged gratefully. The support and financial assistance provided by TEQIP-II is also acknowledged.
