Estimation of Upper Limb Joint Angle Using Surface EMG Signal

Abstract

In the development of robot-assisted rehabilitation systems for upper limb rehabilitation therapy, human electromyogram (EMG) is widely used due to its ability to detect the user intended motion. EMG is one kind of biological signal that can be recorded to evaluate the performance of skeletal muscles by means of a sensor electrode. Based on recorded EMG signals, user intended motion could be extracted via estimation of joint torque, force or angle. Therefore, this estimation becomes one of the most important factors to achieve accurate user intended motion. In this paper, an upper limb joint angle estimation methodology is proposed. A back propagation neural network (BPNN) is developed to estimate the shoulder and elbow joint angles from the recorded EMG signals. A Virtual Human Model (VHM) is also developed and integrated with BPNN to perform the simulation of the estimated angle. The relationships between sEMG signals and upper limb movements are observed in this paper. The effectiveness of our developments is evaluated with four healthy subjects and a VHM simulation. The results show that the methodology can be used in the estimation of joint angles based on EMG.

Keywords

sEMG based Upper Limb Rehabilitation Back Propagation Neural Network Joint Angle Estimation Virtual Reality

1. Introduction

Physical rehabilitation is the process of physical training that someone uses to improve or recover from their lost physical functions due to spinal cord injury (SCI), traumatic brain injury (TBI) or cerebrovascular accident (CVA). The loss of physical functions normally limits the performance of a person's daily life activities and leads to poor quality of life. Therefore, numerous researchers have developed physical rehabilitation systems to restore the lost functions and promote the quality of patients' lives. In terms of physical rehabilitation systems, robot-assisted approaches [1 –3] have been widely developed to restore the lost functions of patients with server impairments. However, it has been proved that a normal rehabilitation system will not provide fast recovery or an effective system unless the system is integrated with human biological signals, especially with EMG signal [4]. EMG is a human biological signal that can be recorded to evaluate the human skeletal muscle tension and control the related movement. Therefore, it becomes one of the most important biological signals that directly reflect the user intended motion and it is often employed as an indication tool for a user's paralysed arm performance. As a result, EMG is used to detect the intended movement by estimating force, torque and angle to generate a robot arm movement that will guide the user's paralysed arm as a rehabilitation therapy.

As far as the estimation method is concerned, Arif et al. [5] developed an estimation method of shoulder and elbow joint torque by means of a genetic algorithm (GA) process. In their work, GA was used to find the mathematical model that fits the processed EMG signals into the joint torque. The number of possible models was provided by a user in the model library that served as the search space for GA. Edward et al. proposed nonlinear dynamic models to estimate the joint torque for the elbow [6]. In their estimation method, second or third degree polynomial functions (EMGσ) were added to a nonlinear parametric model and model parameters were estimated via a pseudoinverse and ridge regression method to improve the accuracy of the estimated elbow torque value. Another elbow joint estimation was carried out by Khalil et al. [7]. In their work, a new mathematical model was developed with the consideration of an exponential and nonlinear nature. The adjustable parameters were obtained by means of nonlinear regression. In the work of Farid et al. [8], the elbow-induced wrist force is estimated with a fast orthogonal search (FOS), which is a time-domain method for rapid nonlinear system identification. Estimation of the elbow joint angle was performed by Suryanarayanan et al. [9]. In this work, the real joint angle was measured by a goniometer to calibrate and train the developed artificial neural network and estimate the elbow joint angle from EMG signals. Another estimation method of joint angle was developed by Masairo et al. [10]. In his development, the joint angle was estimated by an EMG-Joint angle model, which expresses the linear relationship between EMG signals and joint angles. We have been developing an EMG based upper limb rehabilitation system and this work can be found in [11 –13]. In this paper, a BPNN model and VHM model are developed. The estimation of the upper limb joint angle is performed by a sEMG based BPNN and the VHM is developed to simulate the estimated joint angle, which is an output of BPNN.

The rest of the paper is organized as follows. Section 2 presents the methodology of our development including biosignal measurement and processing, development of back propagation neural network and development of EMG controlled virtual human model (VHM). Section 3 provides the experiment, results and discussion while Section 4 presents the conclusion and future works.

2. Methodology

2.1 System Overview

The overview of the developed system is depicted in Figure 1. The complete system consists of three steps, where Step 1 deals with biosignal measurement and processing, Step 2 takes care of finding the relationship between sEMG signals and joint angle by means of BPNN and Step 3 performs the simulation of estimated shoulder and elbow joint angle of VHM that mimics the real human's arm movements.

2.2 Biosignal acquisition and processing

Biosignals consist of all kinds of electrical signals such as electroencephalogram (EEG), electrocardiogram (ECG) or electromyogram (EMG) that can be monitored and measured from a biological organism. Among these biosignals, EMG is widely utilized in the development of physical rehabilitation systems due to its ability to provide information regarding muscle activities such as contracting of the muscles, muscle fatigue, rate of tension build-up and reflex activities. Therefore, it becomes very important in many clinical and biomedical applications. There are two methods to obtain EMG signals, namely surface EMG (sEMG) where a sensor electrode is attached to the user skin and intramuscular EMG (iEMG) where a needle electrode is inserted through the skin into the muscle tissue. In this work, sEMG was utilized to extract information on muscle performance using FlexComp Infiniti from Thought Technology [14]. Before the signals were employed for further processing, pre-processing of the recorded signals was performed to reduce the movement artefacts and increase the signal to noise ratio. The raw signal was detected via active triode electrode with the distance between the electrodes set to 2cm to prevent muscle crosstalk. The detected signal was then amplified with differential amplifiers, which take the electrode reading as inputs; electronics circuitry subtracts the two signals and then amplifies the difference. Subsequently, the signal is rectified with Root Mean Square (RMS) and converted it to an amplitude envelope with the following equation (1).

R M S_{e m g} (n) = \sqrt{\frac{1}{n} \sum_{s = 1}^{n} s E M G {(s)}^{2}}

(1)

where sEMG(s) represents the amplitude of the signal in s_th sampling and n is the number of samples. After that, a low pass filter (LPF) is employed by means of Equation 2, which is a simple discrete form of LPF that applies the forward difference rule to the continuous LPF [15].

Figure 1.

Overview of developed system

L P F (n) = R M S_{e m g} (n) * θ + L P F (n - 1) * (1 - θ)

(2)

where LPF(n) represents the low-pass-filtered signals at the n^th sample, RMS_emg(n) is an RMS value that is obtained from Equation (1) at the n_th sample and θ is defined as follows in Equation 3.

θ = 2 * π * f_{c} * T

(3)

where f_c represents the cut-off frequency, in this case f_c = 1 and T is a sampling period of 1/2048(s). In this work, a sampling frequency of 2048 Hz was set. The signals were collected from the anterior deltoid, posterior deltoid, biceps brachii and triceps brachii. The positions of the electrode sensors are as depicted in Figure 2. After the low-pass filter, normalization of the signal was performed with the following equation (4).

y = (y_{max} {- y}_{min}) * ({x - x}_{min}) / (x_{max} {- x}_{min}) {+ y}_{min}

(4)

In this paper, minimum and maximum values were set to −1 and +1. The raw and pre-processed signals can be found as depicted in Figure 3. The pre-processed data will then serve as an input to the back propagation neural network (BPNN), which will be described in the next section.

Figure 2.

Position of electrode sensors

Figure 3.

Signal obtained by taking Raw, performed RMS and LPF pre-processing during muscle contraction

2.3 Back propagation Neural Network (BPNN)

In general, artificial neural networks (ANN) are able to work out a variety of problems such as prediction, optimization, pattern recognition, etc. Taking advantage of its benefits, in this work BPNN was utilized to find the relationship between sEMG and the joint angle of the upper limb. The nature of BPNN is to minimize the error of the network using the derivatives of the error function. It calculates the derivatives' flows backwards through the network.

In this paper, three layers of BPNN controllers were developed. The structure of the constructed BPNN is portrayed in Figure 4. From the figure it can be seen that the first layer is an input layer, which consists of four nodes for sEMG signal of anterior deltoid (A. Deltoid), posterior deltoid (P. Deltoid), biceps brachii (B. Brachii) and triceps brachii (T. Brachii) muscles. The second layer is a hidden layer constructed from 20 nodes. In this layer the Levenberg-Marquardt algorithm was employed. This algorithm is the combination of Gauss-Newton algorithm and the steepest descent method to take over the speed benefit of the Gauss-Newton algorithm and stability of the steepest descent method. The equation of the Levenberg-Marquardt algorithm can be found in Equation 5. During the training process, this algorithm will switch between the steepest decent and the Gauss-Newton algorithm depending on the combination coefficient, µ, as shown in Equation 6. If µ is very small, the Gauss-Newton algorithm is used. If µ is very large, the steepest decent method is employed. The equations of the Gauss-Newton algorithm and steepest decent method can be found in Equation 7 and 8.

w_{k + 1} = w_{k} - {(J_{k}^{T} J_{k} + μ I)}^{- 1} J_{k}^{T} e_{k}

(5)

α = \frac{1}{μ}

(6)

w_{k + 1} = w_{k} - {(J_{k}^{T} J_{k})}^{- 1} J_{k}^{T} e_{k}

(7)

Figure 4.

Structure of developed BPNN controller

w_{k + 1} = w_{k} - α g_{k}

(8)

where w is the weight factor, k is the index of iterations, J is the Jacobian matrix, e is a training error and α is the learning constant. The last layer, the output layer, provides the estimated joint angle for the upper limb. Although the target joint angle of the upper limb movements is able to move according to the anatomical range of motion of the human upper limb [3], in this paper the maximum range of motion of upper limb is set as in Table 1 and developed as a sine wave form in Matlab. This target joint angle is set as an ideal joint angle of BPNN where the input, four sEMG signals, are trained to achieve the estimated joint angle. The performance of the BPNN was evaluated with mean square error (MSE). The output of BPNN will then send the command to VHM to mimic the subject movements. The detail development of VHM will be presented in the following section.

2.4 EMG controlled Virtual Human Model

A Virtual Human Model (VHM) was developed to imitate real human movements. The expected output of the VHM is to simulate an upper limb movement that mimics the upper limb movement of the subject. The VHM was developed in swirlX3D and exported as VRML to receive the command from BPNN, which was developed in Matlab. To mimic the normal human joint rotation and arm movements, a “Humanoid” node in swirlX3D was created for the VHM upper limb. The joint rotation in the VHM is achieved by means of joint.rotation = [fq_axis(1) fq_axis(2) fq_axis(3) fq_angle] command. The value of fq_axis(1), fq_axis(2) and fq_axis(3) are Booleans where one represents the choice for linear movement, which permits the rotation of the segment in three axes. As the value of fq_axis(1), fq_axis(2) and fq_axis(3) are generally Booleans, integrating the quaternion angles is required to incorporate a dynamic aspect in the movement of the VHM that will appropriately simulate the arm in a virtual reality (VR) environment. The fragment of code that converts to the quaternion angle can be found in Figure 5. In the VR environment, the fq_axis(1) direction is left to right, the fq_axis(2) direction is up and down and the fq_axis(3) direction is into and out of the display screen. The output value of BPNN, which is a joint angle, will apply in an fq_angle of joint.rotation command to perform the VHM simulation.

3. Experiment, results and discussion

The experiment was carried out to evaluate the effectiveness of our development with four healthy subjects: two females and two males, with ages ranging from 25 – 80 years old. They were asked to perform four different kinds of upper limb motions, as shown in Table 1.

Table 1.

Types of movements and its corresponding range of motion

Subject	Shoulder Movements			Elbow
	(degree)			(Degree)
	Flexion-Extension	Vertical Abduction-Adduction	Horizontal Abduction-Adduction	Flexion-Extension
S1	±90	±90	±90	±130
S2	±90	±90	±90	±130
S3	±90	±90	±90	±130
S4	±90	±90	±90	±130

Figure 5.

Matlab code that converts to quaternion and VRML angle

Hardware set up was performed before starting the experiment. A standard personal computer (PC), webcam and Biograph Infiniti system were used in this experiment. Four MyoScan sensors from Thought Technology were connected to four channels of a FlexComp Infiniti encoder. The encoder was then connected to a TT-USB interface via a fibre-optic cable and the interface was connected to a USB port on the computer. Four MyoScan sensors were attached to the anterior deltoid, posterior deltoid, biceps brachii and triceps brachii muscles according to Figure 2. Subjects were asked to carry out four types of movement as stated in Table 1. In each session there were six repeats of continuous movement with almost the same speed and amplitude and there were five sessions for each type of movement. Therefore, a total of 30 groups of sEMG data for individual movements were collected from each subject. From the collected data, two sessions of each movement were used as a training dataset and the rest served as a testing dataset. The sEMG signals of these muscles were recorded and extracted with the root mean square (RMS), which filtered and normalized the signals. The pre-processed signals were then trained with BPNN. After the network is trained, the testing data were sent to BPNN to estimate the joint angle. The three steps of the experiment were conducted as followed.

First, the training dataset was inputted into the BPNN, where the output of this network results in the angle of the shoulder or elbow joint. Figure 6 illustrates the result of the target angle and the trained angle for the shoulder abduction/adduction movement of subject 3. It shows that the trained angle from the developed BPNN nearly coincides with the target angle. This shows that the developed system based on BPNN is working well.

Figure 6.

Result of trained angle with BPNN and target angle for shoulder abduction/adduction movement of subject 3

In the second step, the experiment was carried out to test sEMG dataset and the trained BPNN. In Figure 7, RMS, sEMG and the estimated joint angle of the shoulder abduction/adduction movement of subject 3 is plotted to observe the relationship between them. The figure shows the sEMG signals and predicted angles where the peak signal corresponds to the most contraction of the muscle and peak angle corresponds to the maximum angle of the shoulder or elbow joint, in this case the shoulder joint. This proves that the developed BPNN is able to correlate the relationship between sEMG and the joint angle very well. The validation of the network performance in the estimation of the joint angle in each motion was also performed with the MSE method, which takes the mean square error between the ideal joint angle and the estimated joint angle and the results are listed in Table 2. From this table, it can be clearly seen that the joint angles are estimated quite accurately using developed BPNN.

In the final step, the predicted angle from the output of BPNN is sent to the VHM for upper limb simulations: shoulder flexion/extension, shoulder vertical abduction/adduction, horizontal abduction, adduction and elbow flexion/extension and the result of the VHM simulation is portrayed in Figure 8 (a-d).

In addition to this, the relationship between sEMG and the type of movements was analysed in this work. From Figure 9 (a-d) it can be clearly seen that the type of movements and the contributing muscles, together with the pre-processed signals are the input for the developed BPNN. During shoulder flexion/extension movement, the anterior deltoid muscle is most contracted. As for shoulder abduction/adduction movement, the anterior deltoid and posterior deltoid muscles contribute more significantly than other muscles. Although anterior deltoid contributes in both shoulder flexion/extension and abduction/adduction, the amplitude of the EMG is stronger in flexion/extension movement. During elbow movement, the biceps brachii muscle is the most contributed muscle. During shoulder horizontal abduction/adduction motion, the most contracted muscles are similar to that of shoulder abduction/adduction movement. However, the anterior deltoid muscle is always above the certain sEMG value. It is because the anterior deltoid is contracted all the time to maintain the arm in the horizontal plane. This will be very constructive information for our future work when we are required to define the intended type of arm movement.

Figure 7.

Relationship between estimated angle and RMS sEMG during shoulder abduction/adduction movement of subject 3

Table 2.

The result of MSE for estimation of shoulder and elbow joint for each movement

Subject	Session Information	Type of Upper Limb Movements
Subject	Session Information	Shoulder Flexion-Extension	Shoulder Abduction-Adduction	Shoulder Horizontal Abduction-Adduction	Elbow Flexion-Extension
S1	S1	0.0216	0.0191	0.0098	0.0125
	S2	0.0169	0.0169	0.0141	0.0174
	S3	0.0238	0.0063	0.0159	0.0228
	Avg MSE	0.0208	0.0141	0.0133	0.0176
S2	S1	0.0181	0.0205	0.0288	0.0136
	S2	0.0086	0.0089	0.0085	0.0159
	S3	0.0118	0.0174	0.0187	0.0149
	Avg MSE	0.0128	0.0156	0.0187	0.0148
S3	S1	0.0077	0.0216	0.0098	0.0117
	S2	0.0102	0.0169	0.0359	0.0125
	S3	0.0287	0.0238	0.0259	0.0109
	Avg MSE	0.0155	0.0208	0.0239	0.0117
S4	S1	0.0082	0.0094	0.0307	0.0164
	S2	0.0179	0.0192	0.0119	0.0123
	S3	0.0205	0.0201	0.0175	0.0171
	Avg MSE	0.0155	0.0162	0.0200	0.0153

Figure 8 (a).

Result of VHM simulation in shoulder flexion/extension movement

Figure 8 (b).

Result of VHM simulation in shoulder abduction/adduction movement

Figure 8 (c).

Result of VHM simulation in shoulder horizontal abduction/adduction movement

Figure 8 (d).

Result of VHM simulation in elbow flexion/extension movement

Figure 9 (a).

Muscle performance during shoulder flexion/extension: Raw EMG (left) and Processed EMG (right)

Figure 9 (b).

Muscle performance during shoulder abduction/adduction: Raw EMG (left) and Processed EMG (right)

Figure 9 (c).

Muscle performance during shoulder horizontal abduction/adduction: Raw EMG (left) and Processed EMG (right)

Figure 9 (d).

Muscle performance during elbow flexion/extension: Raw EMG (left) and Processed EMG (right)

4. Conclusion and future work

In this paper, an sEMG based back propagation neural network (BPNN) and a virtual human model (VHM) were developed. Four sEMG signals were collected from each of four healthy subjects and then sent to a BPNN controller to estimate the upper limb joint angle. The estimated angle was then displayed by the developed VHM. The evaluation results show that the developed BPNN is able to represent the relationship between sEMG and joint angle successfully and the simulation of VHM mimicked the human arm movements. The experiment provided very positive results and this leads us to work on the next step to perform the intended combined movements. To determine the intended combined movements, the nature of muscle contributions and their movements were observed as expressed in Section 3. The result of this development will be reported in the near future.

References

Jordan

Sampson

Hijmans

King

Hale

(2011) ImAble system for upper limb stroke rehabilitation, Virtual Rehabilitation (ICVR), 2011 International Conference on, pp. 1–2.

Nef

Guidali

Riener

(2009) ARMin III - arm therapy exoskeleton with an ergonomic shoulder actuation, Applied Bionics & Biomechanics, vol. 6, pp. 127–142.

Perry

J.C.

Rosen

Burns

(2007) Upper-Limb Powered Exoskeleton Design, Mechatronics, IEEE/ASME Transactions on, vol. 12, pp. 408–417.

Huang

Wolf

(2006) Recent developments in biofeedback for neuromotor rehabilitation, Journal of NeuroEngineering and Rehabilitation, vol. 3, pp. 11.

Oyong

A.W.

Parasuraman

Jauw

V.L.

(2010) Robot assisted stroke rehabilitation: Estimation of muscle force/joint torque from EMG using GA, Biomedical Engineering and Sciences (IECBES), 2010 IEEE EMBS Conference on.

Clancy

E.A.

Liu

Moyer

D.V.Z.

(2012) Identification of Constant-Posture EMG-Torque Relationship About the Elbow Using Nonlinear Dynamic Models, Biomedical Engineering, IEEE Transactions on, vol. 59, pp. 205–212.

Ullah

Jung-Hoon

(2009) A mathematical model for mapping EMG signal to joint torque for the human elbow joint using nonlinear regression, 4th International Conference on Autonomous Robots and Agents, ICARA 2009.

Mobasser

Eklund

J.M.

Hashtrudi-Zaad

(2007) Estimation of Elbow-Induced Wrist Force With EMG Signals Using Fast Orthogonal Search, Biomedical Engineering, IEEE Transactions on, vol. 54, pp. 683–693.

Suryanarayanan

Reddy

N.P.

Gupta

(1995) Artificial neural networks for estimation of joint angle from EMG signals, Engineering in Medicine and Biology Society, 1995., IEEE 17th Annual Conference.

10.

Yoshikawa

Mikawa

Tanaka

(2007) Hand Pose Estimation Using EMG Signals, Engineering in Medicine and Biology Society, 2007. EMBS 2007. 29th Annual International Conference of the IEEE.

11.

Mon

A. Yee

Al-Jumaily

, “AR based upper limb rehabilitation system,” in Biomedical Robotics and Biomechatronics (BioRob), 2012 4th IEEE RAS & EMBS International Conference on, pp. 213–218.

12.

Mon

A. Yee

Al-Jumaily

(2012) Shoulder rehabilitation with biofeedback simulation, Mechatronics and Automation (ICMA), 2012 International Conference on, pp. 974–979.

13.

Kaluarachchi

Mon

A. Yee

Al-Jumaily

(2011) Virtual games based self rehabilitation for home therapy system, Hybrid Intelligent Systems (HIS), 2011 11th International Conference on, pp. 653–657.

14.

Thought Technology. (May, 2012). Available: http://www.thoughttechnology.com/.

15.

H.J.

Lee

A.Y.

Choi

(2011) Human elbow joint angle estimation using electromyogram signal processing, Signal Processing, IET, vol. 5, pp. 767–775.