Neuro-Based Thumb-Tip Force and Joint Angle Modelling for Development of Prosthetic Thumb Control

Abstract

Human fingers have a specific role that contributes to different hand functions. Among these fingers, the thumb plays the most special function as an anchor to many hand activities. As a result, the loss of the thumb due to traumatic accidents can be catastrophic as proper hand function will be severely limited. In order to solve this problem, a prosthetic thumb is developed to be worn in complementing the function of the rest of the fingers. The movement of the prosthetic device can be naturally controlled by using electromyogram (EMG) signals. In this work, the EMG signals from the human muscles were measured in different thumb configurations and thumb-tip forces in flexion movement. The muscles involved are the Adductor Pollicis (AP), Flexor Pollicis Brevis (FPB), Abductor Pollicis Brevis (APB) and First Dorsal Interosseous (FDI). The classification of the EMG signals based on different force and thumb configurations is performed using an Artificial Neural Network (ANN). From a series of experiments, the results show that the neural network efficiently classified the signals and a unique set of EMG signals was generated for each thumb movement and force. Therefore, EMG signals were used to control the prosthetic movement with aid from the developed neural network.

Keywords

Electromyogram (EMG) Signal Prosthetic Thumb Artificial Neural Network (ANN)

1. Introduction

Amputation is the loss of a human body part due to traumatic accidents or the removal of an injured or diseased body part. The numbers of amputees are high and increasing every year, as reported in [1]. The amputation of body parts can be classified into lower limb and upper limb amputation. Lower limb amputation is the loss of body parts from waist below. While the loss of body parts from the shoulder below is considered as upper limb amputation and this includes amputation of the fingers and hand [2].

Every normal-born human has fingers connected to each hand. Each of the digits has their own function and the thumb plays the role of an anchor to many hand functions due to its unique anatomy. In addition, the thumb has the same flexion, extension, abduction and adduction function as the other fingers with the addition of a unique opposability function that allows for proper object grasping [3]. According to clinicians, the thumb is responsible for at least 50% of overall hand functions [4]. Therefore, the function of the thumb is critical to overall hand function and is uniquely endowed with anatomical features that allow for circumduction and opposition movement [5]. The loss of the thumb will affect human daily activities as the proper hand function is limited. Whereby, the thumb is the most important digit for the pinch and grasps function of the hand. As a result, research in developing a prosthetic thumb is needed in order to help the amputee to regain the complete hand functions so as to live efficiently.

The movement of the thumb is controlled by the central nervous system in the brain. The brain sends a signal to the motor neurons in the form of action potentials through the nervous system. Upon receiving the signal, the motor neurons stimulate several muscle fibres that are located inside the muscle and cause muscle contraction [6]. The contraction of muscles then generates forces to move the thumb. During contraction, the activated muscle fibres generate electrical potential known as Electromyogram (EMG) signals that can be measured non-invasively from the skin surface. [7]. The amplitudes of these signals are small, ranging from 0 to 10 mV peak-to-peak and the frequencies are between 0 to 500 Hz [8].

In the prosthetic research field, considerable efforts have been made to improve the performance of the artificial devices to be more naturally operable [9 –11]. In [9], the researchers used the surface EMG signals to control the force of a German Aerospace Centre (DLR) prosthetic hand for more dexterous operation. While in [10] the researchers have carried out a study on the surface EMG signals for controlling the individual fingers of a prosthetic hand non-invasively. Additionally, in [11] the researchers have used the surface EMG signals for controlling the movement of the robotic hand to perform regular hand activities such as grasping and holding. Furthermore, in [12 –15] various techniques such as Artificial Neural Network (ANN), RQA, Genetic Algorithm (GA), Wavelet transform and Support Vector Machines (SVM) have been used to classify the EMG signals for better prosthetic function.

According to Park et al. [4], the precision of thumb function is closely related to knowledge of the muscle signals governing the operation of the thumb. In [11] the authors reported that for each individual finger, the related EMG signals can be used to control its movement. Besides that, different finger movements require different amounts of force and [9] investigates the contribution of EMG signals to a number of finger movements. Based on the above discussion, the EMG signals can be used to control the prosthetic thumb for more natural operation and force is required by the prosthetic thumb for proper object handling, which can be measured at the thumb-tip using force sensor.

In this paper, the relationship between the EMG signal of the thumb muscles and the thumb tip force at various thumb bending angles was first established. The model of the relationship will be obtained and used for estimating the joint angle and thumb-tip force from EMG signals. The estimated angle and force is the desired joint angle and thumb-tip force. The estimated joint angle will be used to control the angle of the prosthetic thumb movement via a system controller. The movement of the prosthetic thumb will cause force generation at the prosthetic thumb-tip. The prosthetic thumb-tip force will be measured and compared with the estimated force. If there is a difference between the estimated and measured force, the system controller will move the prosthetic thumb by either adding or reducing the angle based on the requirement to increase or decrease the force. The joint angle and thumb-tip force are very much related and both of them are controlled by EMG signals. Therefore, the model of the relationship between EMG signals, the joint angle and the thumb-tip force at various joint angles is important in order to develop proper control of the prosthetic thumb device.

2. Experimental setup

2.1 Muscle selection

Human thumb movement is controlled by nine muscles that are connected to the bone and are known as skeletal muscles. These muscles can be classified into two groups, known as intrinsic muscles and extrinsic muscles. Intrinsic muscles are shorter muscles that originate primarily in the human hand. The longer muscles, called extrinsic muscles, are located inside the human forearm. The muscles in the intrinsic group are the Adductor Pollicis (AP), First Dorsal Interosseous (FDI), Flexors Pollicis Brevis (FPB), Opponens Pollicis (OPP) and Abductors Pollicis Brevis (APB). While the Flexors Pollicis Longus (FPL), Abductors Pollicis Longus (APL), Extensors Pollicis Longus (EPL) and Extensors Pollicis Brevis (EPB) are in the extrinsic group [16]. Among the nine muscles, only AP, APB, FDI and FPB are chosen for collecting EMG signals, since these intrinsic muscles are located at the outermost layer and are easier to measure using surface electrodes. Besides that, the intrinsic muscles are used to assist in power gripping and help to control the stability and movements during pinching and gripping [6].

The experiments to measure thumb-tip force and EMG signals were conducted on five female subjects aged between 25 and 28 years old. The experimental setup for thumb-tip force and EMG signals measurement consists of surface electrodes, a G.tec data acquisition system, a force sensor, an ADC circuit, a computer and angled blocks with a 0° angle, 15° angle, 30° angle and 45° angle, as shown in Figure 1.

Figure 1.

Experimental setup

During measurement, the subjects were seated on a chair with their forearm rested on a desk and the thumb-tip was placed on the force sensor that was secured on the flat surface of the angled block. The placement of the thumb-tip on the angled block caused the thumb joint angle to produce an angle that follows the angle of the block. Then the subjects were instructed to relax and press the thumb-tip against the angled block with the increasing force until it reached the maximum force. The thumb-tip forces were measured using a Force Sensing Resistor (FSR) force sensor. The measured force is the Maximum Voluntary Contraction (MVC) value and the subjects were asked to maintain the MVC force for five seconds. The MVC value differs between subjects. The forces values were transferred into the computer using an Analogue to Digital Converter (ADC) circuit to observe the value of the exerted force. The thumb movement is related to the muscle contraction and the EMG signals generated during muscle contraction were measured simultaneously with the thumb-tip force measurement. For EMG signal measurement purposes, the Ag/AgCl surface electrodes were placed above the related muscles as shown in Figure 2a and Figure 2b. The placements of the electrodes referred to the human anatomy. The signals captured by the electrodes were recorded using the G.tec data acquisition system with a sampling frequency of 1200Hz and were saved on the computer. The EMG signals were band pass filtered between 0.5Hz and 500Hz to remove unwanted signals and the notch filter was also applied at 50Hz to remove the artefacts from the power line. The thumb-tip force and EMG signals measurement was repeated for four different thumb configurations, which were at 0, 15, 30 and 45 degrees of the angled block, as shown in Figure 2 and at four different thumb tip forces (100% MVC, 75% MVC, 50% MVC and 25% MVC) on each subject.

Figure 2.

Placement for (a) Electrode on FDI muscle and (b) Electrodes on AP, APB and FPB muscles

2.3 Pre-processing of EMG signals

The recorded EMG signals from the muscles, as shown in Figure 3, contain important information about muscle contraction during thumb operation. However, these signals can easily be contaminated by external interference such as electrode noise, motion artefacts, power line noise, ambient noise and inherent noise in surrounding electrical and electronic equipment. Therefore, in order to remove the noise, the recorded signals were pre-processed using a band pass filter with a cut-off frequency between 20 and 500Hz [4], [17 –19].

The strength of the filtered EMG signals that can be observed through signal energy is important for prosthetic thumb control. This energy was computed using Eq. 1 and the example of the signal energy is plotted in Figure 4.

Figure 3.

Raw EMG signal from APB muscle

E_{s} = \sum_{n = 0}^{N} x (n)^{2}

(1)

where E_s is the energy of the signal, x(n) is the EMG signal and N is the number of data.

Figure 4.

Signal energy from APB muscle

2.4 Feature Extraction

After transforming the EMG signals into the form of signal energy, the signals were extracted using the Root Mean Square (RMS) using Eq. 2. This method is used to extract the useful information that is hidden in the EMG signals and remove the unwanted parts and interferences.

R M S = \sqrt{\frac{\sum_{n = 1}^{N} x_{n}^{2}}{N}}

(2)

where N is the number of EMG data, n is data index and x is the value of the data

The signals are later smoothed using the Moving Average (MA) in Eq. 3 to obtain the enveloped EMG signals.

M A = \frac{\sum_{i = 1}^{k} x_{i}}{k}

(3)

where k is the size of the moving window, i is data index and x is the value of the data

The amplitudes of the enveloped EMG signals from all the muscles are different between subjects. The differences need to be unified for better EMG signal classification for the purpose of controlling the prosthetic thumb. One of the techniques used to control these variables is through normalization. The normalization can be performed by using the max-min normalization technique in Eq. 4 and shown in Figure 4.

X_{n o r m} = \frac{x - \min A}{\max A - \min A} (n e w \max A - n e w \min A) + n e w \min A

(4)

Figure 4.

Normalized EMG signals of 100 % MVC force at 0, 15, 30 and 45 degree of thumb configuration for subject 1

where Xnorm is the normalized data, X is the data to be normalize, min A is the minimum value in a set of data, max A is the maximum value in a set of data, new max A is the new maximum range and usually set to one and new min A is the new minimum range and usually set to zero.

2.5 Classification of EMG signals using Artificial Neural Network (ANN)

The enveloped EMG signals of each muscle have a unique pattern that can be used in signal classification. The common technique for EMG signal classification is using an Artificial Neural Network (ANN) [19]. In developing the network, the recorded signal was first divided into a training data set and a testing data set. The training dataset from four muscles, targeted forces and targeted angles that give a total of 19200 data sets were trained in the network (i.e., 4800 input datasets from each muscle).

The type of network used in this work is a feed forward network with a Levenberg-Marquardt training algorithm, as shown in Figure 5. The designed network consists of four neurons in the input layer, one hidden layer and two neurons in the output layer with a sigmoid activation function. Four neurons at the input layer, represented as x₁, x₂, x₃ and x₄, are the EMG signals from the AP muscle, APB muscle, FDI muscle and FPB muscle. The y₁ and y₂ notation in the output layer represent the targeted thumb-tip force and joint angle. There were two types of activation function used in the system that known as tan-sigmoid transfer function and linear transfer function. The tan-sigmoid transfer function that was used to classify the signals into classes was applied in a hidden layer and the linear transfer function that was used to identify and approximate the relationship between the EMG signals, the thumb-tip force and the joint angles was applied in the output layer.

Figure 5.

Artificial Neural Network architecture

To optimize the generalization, an early stopping method was applied to randomly divide the dataset into 70% learning set, 15% validation set and 15% testing set. The learning dataset was trained in the network together with the targeted outputs. These datasets were used to compute the gradient and update the network weights and biases. The weights of the ANN were adjusted using the LM algorithm provided by the MATLAB Neural Network Toolbox. The algorithm can be represented as follows:

w = w + Δ w

(5)

Δ w = {[J^{T} J + η I]}^{- 1} J^{T} e

(6)

e = R - z

(7)

where w is the weight vector, δw is the difference between the weight vector, J is the Jacobian matrix that contains the first derivatives of the network error with respect to the weight, ^η is a scale parameter, I is the identity matrix, R is the vector of the reference motion, z is a vector of the estimated motion and e is a vector network errors.

For the developed network, the choice of the optimal number of hidden neurons used in the hidden layer is based on the evaluation of the ANN performance using Normalized Root Mean Square Error (NRMSE). The best ANN architecture is shown by the smallest value of NRMSE or test error [20]. It should be noted that an insufficient number of neurons could cause the network to be unable to model the complex data, resulting in poor fitting of the model. Meanwhile excessive neurons could cause the training time to become very long, resulting in over fitting of the model from the data supplied.

Therefore, a series of tests was performed using various numbers of hidden neurons from 10 to 20 for evaluation, as shown in Figure 6. The number of hidden nodes used in the neural network that gives the smallest test error for force and angle is selected as the optimal number of hidden neurons. In this research five neural networks from five subjects were trained to observe individual responses rather than a combination of all the subjects. The optimal numbers of hidden neurons were unique for each subject. The selected hidden neurons are 16 neurons, 19 neurons, 15 neurons, 15 neurons and 14 neurons for Subject 1 to Subject 5, respectively.

Figure 6.

ANN simulation result to determine the optimal number of hidden neurons for subject 1

3. Results and Discussion

The movement of the thumb produces a joint angle and force at the thumb-tip. Therefore, for proper control of the prosthetic thumb movement, the relationship between the EMG signals and the joint angles, as well as the thumb-tip force, must be established. This relationship can be modelled using ANN through the classification of EMG signals at different joint angles and thumb-tip forces. The performance of the developed neural network can be evaluated by applying testing data to the network. Testing data is another set of enveloped EMG signals that has not been seen by the network. The outputs from the testing procedure are in the form of % MVC for thumb-tip force and degrees for the joint angles, as shown in Figure 7. The patterns of the responses from all five subjects are similar to this figure. Based on the figure, the neural network has successfully classified the EMG signals according to the angle and force. It also has effectively estimated the joint angles and thumb-tip forces from the testing data. The response shows that EMG amplitude is increasing with the increases in joint angle and % MVC. Furthermore, the combinations of enveloped EMG signals from four muscles are unique for different angles and thumb-tip forces.

The performances of the estimation were made based on the Root Mean Square Error (RMSE) obtained from Eq. 8.

R M S E = \sqrt{\frac{\sum {(x_{i} - y_{i})}^{2}}{N}}

(8)

where x, y and N indicate the measured force by the force sensor, the estimated force from ANN and the total data, respectively.

Table 1.

Performance of force and angle estimation based on the error (RMSE) obtained from verification step for Subject 1.

Actual Angle (Degree)	Actual Force (% MVC)	Angle Error (RMSE)	Force Error (RMSE)
0	25	9.18 × 10⁻⁴	0.0029
	50	0.0132	0.0279
	75	0.0021	2.08 × 10⁻⁴
	100	0.0103	0.0139
15	25	0.0051	0.009
	50	0.008	0.016
	75	0.0116	0.0157
	100	0.0017	0.0011
30	25	0.0127	0.0351
	50	0.0043	0.0072
	75	0.0034	0.0069
	100	3.2 × 10⁻³	0.0131
45	25	0.0065	0.0083
	50	0.0069	0.0119
	75	3.8 × 10⁻⁴	0.0025
	100	0.0038	2.2 × 10⁻⁴

Figure 7.

Thumb-tip force and joint angles values predicted using neural networks for Subject 1

The RMSE is calculated by determining the error between the actual values and the predicted values using the neural network. The small errors obtained from the testing step are reflected in Table 1. The table shows that the estimated values are quite close to the actual values. However, there are some errors that are slightly bigger compared the other values due to outliers that occurred at the estimated values and only small portion are affected. The performances of the estimations for all the subjects are similar to this table. Therefore, it can be concluded that the ANN model of the relationship between EMG signals, thumb-tip forces and joint angles have been successfully developed.

The ability of the neural network to classify the signals and perform estimations makes it suitable to be included in the system that controls the operation of the prosthetic thumb. The input to the prosthetic control system is the enveloped EMG signals from the four muscles. These signals contain information about a particular value of force and joint angle.

The information from the signals was extracted using the developed neural network and set as the desired values for the prosthetic control. The desired joint angle value was sent to the motor controller to move the servomotor by a 1-degree step. The servomotor was coupled with the prosthetic thumb and caused this device to move towards the angled block, as shown in Figure 8.

Figure 8.

Prosthetic thumb movement toward 45° angle block

The movement of the prosthetic thumb generates force at the thumb-tip. The thumb-tip force was measured using a force sensor that was placed on the flat surface of the angled block. Then, the measured force was compared with the desired force from the ANN. The prosthetic thumb-tip force response is shown in Figure 9. If the measured force is less than the desired force, the controller will send a signal to the motor controller to move the prosthetic thumb in a forward direction by five degrees. This process continues until the error between the desired force and the measured force has subsided to a minimum value. If the measured force is bigger than the desired force, the prosthetic thumb will send a command to the motor controller to move the prosthetic thumb backward by five degrees until the error is very small and steady in that position, as shown in Figure 10.

Figure 9.

Measured and desired thumb-tip force at 25% MVC (7N)

Figure 10.

Measured and desired joint angle at 45° angle

4. Conclusion

In this research work, the mapping model between the signals from the AP, FPB, APB and FDI muscles to the thumb-tip forces under four different thumb configurations described by 0, 15, 30, 45 degree blocks is established. The estimated force is obtained from 25%, 50%, 75% and 100% of the MVC. The selected muscles are intrinsic types of muscle and are directly involved in controlling the operation of the thumb. The result shows the efficacy of using the ANN model to classify the EMG signals for different values of force and thumb angles. Furthermore, the result also shows that the EMG signals can be used to control a prosthetic thumb with help from a neural network.

Footnotes

9. Acknowledgement

The authors would like to acknowledge the grant number EDW A-12-479-1270 and FRGS 0510-130 for supporting this research work.

References

PPUM (2009) Laporan Tahunan Pusat Perubatan Universiti Malaya 2009. Kuala Lumpur: Pusat Perubatan Universiti Malaya. 272p.

Toledo

Leija

Munoz

Vera

Ramirez

, (2009) Upper limb prostheses for amputations above elbow: A review. Pan American Health Care Exchanges (PAHCE), pp. 104–108.

Chang

L Y

Matsuoka

, (2006) A kinematic thumb model for the ACT hand. Proceedings of the 2006 IEEE International Conference on Robotics and Automation, pp. 1000–1005.

Park

Kwon

Lee

Kim

, (2009) Thumb-tip force estimation from sEMG and a musculoskeletal model for real-time finger prosthesis. 2009 IEEE International Conference on Rehabilitation Robotics, pp. 305–310.

Salah

M M

Khalid

K N

, (2008) Thumb reconstruction by grafting skeletonized amputated phalanges and soft tissue cover – A new technique: A case series. Cases Journal, 1: Pp. 725–730.

Cael

, (2010) Functional Anatomy: Musculoskeletal Anatomy, Kinesiology and Palpation for Manual Therapists. Pennsylvania: Lippincott Williams and Wilkins. 452p.

Rogers

, (2011) Bone and Muscles: Structure, Force and Motion, New York: Brittanica Educational Publishing. 269p.

Soo

Sugi

Yokoi

Arai

Nishino

Kato

Nakamura

Ota

, (2010) Estimation of handgrip force using frequency-band technique during fatiguing muscle contraction. J Electromyogr Kinesiol. 20: Pp. 888–895.

Castellini

van der Smagt

Sandini

Hirzinger

, (2008) Surface EMG for force control of mechanical hands. 2008 IEEE International Conference on Robotics and Automation. pp. 725–730.

10.

Harada

Nakakuki

Hikita

Ishii

, (2010) Robot finger design for myoelectric prosthetic hand and recognition of finger motions via surface EMG. 2010 IEEE International Conference on Automation and Logistics (ICAL). pp. 273–278.

11.

Tenore

Ramos

Fahmy

Acharya

Etienne-Cummings

Thakor

N K

, (2007) Towards the Control of Individual Fingers of a Prosthetic Hand Using Surface EMG Signals. Proceedings of the 29*th International Conference of the IEEE EMBS. pp. 6145–6148.

12.

Jung

K K

Kim

J W

Lee

H K

Chung

S B

Eom

K H

, (2007) EMG pattern classification using spectral estimation and neural network. SICE Annual Conference. pp. 1108–1111.

13.

Yuan

Zhu

Liu

Lei

, (2008) Classification of the surface EMG signal using RQA based representations. 2008 IEEE International Joint Conference on Neural Networks. pp. 2106–2111.

14.

Karimi

Pourghassem

Shahgholian

, (2011) A novel prosthetic hand control approach based on genetic algorithm and wavelet transform features. 7th International Colloquium on Signal Processing and its Applications (CSPA). pp. 287–292.

15.

Chu

J U

Moon

Mun

M S

, (2006) A Real-Time EMG Pattern Recognition System Based on Linear-Nonlinear Feature Projection for a Multifunction Myoelectric Hand. IEEE Transactions on Biomedical Engineering, 53: Pp. 2232–2239.

16.

McKinley

O'Loughlin

, (2011) Human Anatomy. New York: McGraw-Hill Science Engineering. 966p.

17.

Konrad

, (2005) The ABC of EMG: A Practical Introduction to kinesiological Electromyography. USA: Noraxon Inc. 60p.

18.

Ibrahim

Nagarajan

Rizon

Hazry

Ruslizam

Azlin

C O

, (2008) Electromyography Signal Based For Intelligent Prosthesis Design. Proceeding for the 4th International Conference on Biomedical Engineering. pp. 187–190.

19.

Ahsan

Ibrahimy

M I

Khalifa

O O

, (2011) Hand motion detection from EMG signals by using ANN based classifier for human computer interaction. 4th International Conference on Modeling, Simulation and Applied Optimization (ICMSAO). pp. 1–6.

20.

Choi

Kwon

Park

Shin

Kim

, (2008) Real-time isometric pinch force prediction from sEMG. Medical Engineering & Physics, 32: Pp. 429–436.