Artificial neural network EMG classifier for functional hand grasp movements prediction

Abstract

Objective

To design and implement an electromyography (EMG)-based controller for a hand robotic assistive device, which is able to classify the user's motion intention before the effective kinematic movement execution.

Methods

Multiple degrees-of-freedom hand grasp movements (i.e. pinching, grasp an object, grasping) were predicted by means of surface EMG signals, recorded from 10 bipolar EMG electrodes arranged in a circular configuration around the forearm 2–3 cm from the elbow. Two cascaded artificial neural networks were then exploited to detect the patient's motion intention from the EMG signal window starting from the electrical activity onset to movement onset (i.e. electromechanical delay).

Results

The proposed approach was tested on eight healthy control subjects (4 females; age range 25–26 years) and it demonstrated a mean ± SD testing performance of 76% ± 14% for correctly predicting healthy users' motion intention. Two post-stroke patients tested the controller and obtained 79% and 100% of correctly classified movements under testing conditions.

Conclusion

A task-selection controller was developed to estimate the intended movement from the EMG measured during the electromechanical delay.

Keywords

Electromyography (EMG)EMG controller artificial neural networks hand rehabilitation movement prediction

Introduction

People who experience a sudden or progressive loss of motor capabilities attribute high value to being able to directly interact with the objects of daily life.¹In this context, current upper limb rehabilitation results are unsatisfying, with 75% of stroke survivors still showing chronic upper limb symptoms after acute stroke.² The amount of time spent in rehabilitation training has been identified as a key factor that can improve rehabilitation outcomes, but usually patients are engaged in the activity for only a small part of the day, around 13%.³ In this context, considerable effort has been made in the field of robotic rehabilitation in order to be able to deliver a much higher amount of time for rehabilitation training, not only in specialized centres, but also in home environments, and to enable patients to perform precise and repeatable therapeutic exercises.⁴ However, it is known from the literature that an effective rehabilitation treatment should combine intensive and repetitive training with the subject's active participation, in order to promote neuroplasticity and to facilitate the recovery of functional motor skills.⁵ This is in agreement with neuroimaging evidence that shows that an effective rehabilitation exercise, which activates the correct brain areas through the active participation of the patient in the physical therapy, should improve motor learning,^6,7 even if there is a certain level of inter-subject variability.⁸ In recent years, immersive training, biofeedback solutions, virtual gaming and brain–computer interfaces have been extensively designed and developed in an attempt to actively involve the patient during rehabilitation exercises.^9–11 When dealing with neuromotor rehabilitation, residual muscular activity that is measured by superficial electromyography (EMG) signals is useful accessible information that can detect the motion intention of the patient, even when movement kinematics are affected.^12,13 The practice of task-specific, functional upper limb movements are only performed in 51% of upper limb rehabilitation sessions.¹⁴ Furthermore, specific hand rehabilitation sessions are even less frequently performed, though current available technology has yielded multi-fingered exoskeletons capable of independently moving the fingers (e.g. Gloreha hand rehabilitation glove; Idrogenet Srl, Lumezzane, Italy).

In the literature, studies have been conducted that involved the use of surface forearm muscle EMG signals to design myoelectric controllers. For example, different approaches have been proposed to discriminate between upper limb and hand/finger movements including principal component analysis,^15,16 support vector machines,^17,18 and linear discrimination analysis.¹⁹ Artificial neural networks (ANN) have been reported to show a promising performance in the classification of motions based on biosignal patterns,²⁰ they have been applied to the movement classification problem to superficial EMG signals,²¹ and they are able to capture system nonlinearity.²² Moreover, ANNs have low computational load, since, once defined, consist of additions and multiplications, and this is important when developing real-time applications.

However, as discussed previously in the context of hand prosthesis,²³ current myoelectric control is not adequate for simultaneous actuating multiple degrees-of-freedom, as required for functional movement execution. The goal of the present study was to design and implement an EMG-based controller for a robotic assistive device for the hand, which was able to classify the user's motion intention in order to drive the robot toward a synergic action with the residual user's activation. The myoelectric controller was designed to provide a prediction before the real execution of the hand grasp task, and therefore exploiting the EMG signal temporal window going from muscle activation to kinematic effective movement, i.e. the electromechanical delay phase.²⁴ The implementation of a myoelectric controller based on the electromechanical latency allowed the detection of the user's motion intention and the integration of it into the closed-loop controller of the robotic assistance device. Throughout the design phase of the controller, special attention was paid to the usability and applicability of the system in a nonstructured and dynamic environment (i.e. the patient's home), especially in terms of what affects electrode placement.

Subjects and methods

Study participants

This prospective study enrolled healthy volunteers with no neurological or orthopaedic impairment from the local population in the Lombardy region, Italy. In addition, the EMG-based controller was tested on chronic post-stroke patients, who had inefficient control of the hand, in order to test the effectiveness of the proposed approach. The study was conducted at Villa Beretta Rehabilitation Centre, Valduce Hospital, Costamasnaga, Italy between June 2014 and June 2015. The experiments were conducted with the approval of the local Ethics Committee of Villa Beretta Rehabilitation Centre (ethics approval number: 0050310/14U 3.11 28/11/2014) and all study participants gave verbal informed consent after personal illustration of the procedure given by the principal investigator (M.G.).

Task definition

Participants were asked to sit comfortably in a seat in front of a table, with their arm placed parallel to the horizontal table surface, and the hand open with the palm perpendicular with respect to the table surface (i.e. the resting position). The wrist was inserted in a custom-made support that prevented the subject from prono-supinating the wrist. Three hand grasp functional tasks were selected (Figure 1): (i) pinching: a grasping action performed with the thumb and the forefinger to grasp small objects; (ii) grasp an object: a grasping action that depends upon the movement of all of the fingers to grasp an object (e.g. a bottle of water); (iii) grasping: a grasping action with an empty hand that results in a fist.

Figure 1.

Experimental protocol showing the wrist inserted in a custom-made support that prevented the subject from prono-supinating the wrist. Three hand grasp functional tasks were selected: (a) pinching: a grasping action performed with the thumb and the forefinger to grasp small objects; (b) grasp an object: a grasping action that depends upon the movement of all of the fingers to grasp an object (e.g. a bottle of water); (c) grasping: a grasping action with an empty hand that results in a fist. Each participant, after a period of familiarization with the protocol, performed 20 trials of each hand grasp task. Movements were auditory paced every 10 s. Each hand grasp task (i.e. pinching, grasp an object, grasping) was acquired in a different run. At least 5 min of rest was provided between each run and it was extended upon the subject's request.

Control subject experimental procedure

Each participant, after a period of familiarization with the protocol, performed 20 trials of each hand grasp task. Movements were auditory paced every 10 s. Each hand grasp task (i.e. pinching, grasp an object, grasping) was acquired in a different run. At least 5 min of rest was provided between each run and it was extended upon the subject's request (Figure 1). To investigate the repeatability of the approach, each healthy participant performed the experimental protocol twice after the electrodes were repositioned.

EMG recordings

Electromyography signals were recorded with a multi-channel signal amplifier system (Porti™; Twente Medical System International, Oldenzaal, The Netherlands). The sampling frequency was set to 2048 Hz. Ten superficial self-adhesive electrodes arranged in a bipolar configuration resulting in five acquired EMG channels were fixed with Velcro on the dominant forearm. The design of the EMG electrode set-up was driven by the priority for the ease of use and fitting, allowing at the same time to record the muscular activity from a variety of muscles that control hand/finger movements. In this configuration, the electrodes were not placed specifically on a single muscle,^15,22,23,25 but instead the information recorded from the electrodes was global, and the overall signal was processed to record the patient's motion intention. To facilitate this, the five pairs of EMG electrodes were placed around the dominant forearm in a circular configuration 2–3 cm from the elbow (Figure 2a). The EMG electrodes were placed in order to maximize the inter-electrode pair distance. The forearm circumference at the position 2–3 cm from the elbow was measured with the hand in the resting position (i.e. arm placed parallel to the horizontal table surface, and the hand open with the palm perpendicular with respect to the table surface). The circumference was divided by five to determine the specific places to attach the electrodes. Since electrode placement was not dependent on the need to record the signal from particular muscles, the starting point was not fixed (Figure 2b). The ground electrode was placed on the opposite wrist.

Figure 2.

Placement of the electromyography (EMG) electrodes: (a) forearm in the resting position showing the EMG electrode placement along the forearm itself 2–3 cm from the elbow; (b) schematic of the forearm cross-section showing the EMG electrodes equally spaced, with the crosses indicating where the five pairs of EMG electrodes were positioned. The colour version of this figure is available at: http://imr.sagepub.com.

EMG task-classifier design

The EMG task-selection controller design was based on the results obtained from a pilot study on healthy controls previously reported.²² In particular, the system predicts the intention to perform a certain hand grasp functional task among a predefined selection from the EMG signals measured in a 100 ms window after the EMG onset. The 100 ms window represents the EMG portion corresponding to the electromechanical delay, i.e. the temporal delay between muscles fibre depolarization and effective kinematic onset of movement.²⁴ The task-classifier architecture was based on a sequence of ANNs. In particular, each trial to be classified was provided as input in the form of EMG signal portions corresponding to the electromechanical delay – the pattern vector. The pattern vector was provided as input to successive ANNs with one hidden layer detailed in the following sections. The first ANN classifies the pattern vector in clusters, defined by a subject specific clustering algorithm in charge of defining subsets of classification groups. Pattern vectors associated with clusters that contain more than one hand grasp task were input to a second ANN in charge of classifying hand grasp tasks within the cluster (Figure 3). For example, let us suppose that the subject-specific algorithm identifies two clusters for subject X, cluster 1 that includes pinching and grasping tasks, and cluster 2 that includes grasp an object task. Cluster 1 pattern vectors (i.e. pinching and grasping tasks) are input to a second ANN that classifies them as pinching and grasping. Cluster 2 output directly corresponds to the final classification since it only includes one hand grasp task. The EMG task-classifier specific architecture will be outlined in three steps: (1) EMG processing; (2) task-classifier calibration; and (3) task-classifier testing. All analysis steps were performed offline in order to test the goodness of the approach. The entire EMG signal (i.e. all 20 trials) underwent EMG preprocessing procedures (i.e. STEP 1), which was then partitioned into calibration trials and testing trials. In task-classifier calibration, and task-classifier testing steps (i.e. STEPS 2, 3), only a 100 ms window after movement onset for each trial was considered.

Figure 3.

Graphical outline of the electromyography (EMG) task-classifier architecture. Suppose that the subject-specific algorithm identifies for the depicted subject two clusters, namely cluster 1 (C1), which includes pinching (T1) and grasping tasks (T1), and cluster 2 (C2), which includes grasp an object task (T3). Cluster 1 pattern vectors (i.e. pinching and grasping tasks) are input to a second artificial neural network (ANN) that classifies them as pinching and grasping. The Cluster 2 output directly corresponds to the final classification since it only includes one hand grasp task. The colour version of this figure is available at: http://imr.sagepub.com.

STEP 1: EMG processing

Graphical representation of the EMG processing step is presented in Figure 4 and described below.

Figure 4.

Graphical representation of the electromyography (EMG) processing step (i.e. STEP 1) for study subject S01, pinching task (20 trials). For clarity, only one EMG channel is represented (channel 1). (a) Graphic representation of the experimental protocol. Each vertical dashed line represents an auditory cue, which were spaced 10 s apart; (b) EMG raw signal; (c) preprocessed EMG signal; (d) preprocessed EMG signal (black line) with low-pass mean EMG signal used for onset identification as described in STEP 1.2 section superimposed (red line). Vertical red dashed lines represent EMG onsets. Grey shaded windows represent the 100 ms windows used as input in the task-classifier. The colour version of this figure is available at: http://imr.sagepub.com.

STEP 1.1: Preprocessing

The five EMG signals (i.e. five acquired channels) were independently preprocessed as follows: (i) high-pass filtered (third order analogic Butterworth filter, cut-off frequency = 10 Hz) to remove the offset; (ii) rectified; (iii) low-pass filtered (third order analogic Butterworth filter cut-off frequency = 5 Hz); and (iv) normalized to the maximum revealed within the calibration trials.

STEP 1.2: Mean and onset identification

The EMG onsets were identified with a local minima algorithm, based on the following procedure: (i) mean of the five-channel EMG preprocessed signals; (ii) first order low-pass filtering; (iii) local minima identification that corresponded to the sought movement onset. With this approach, the onset was identified based on the mean of the five EMG signals, and therefore it will be the same for all channels. Given that the local minima algorithm was based on a first order low-pass filter, it required one previous sample beside the one that was being evaluated. In other words, movement onset was evaluated using two-samples overlapping windows. The critical parameter was the low-pass filter cut-off frequency that was identified as described in the ‘Local minima algorithm frequency definition’ section described below under ‘Parameter optimization’.

STEP 1.3: Vectorialization

For each trial of each hand grasp task, an EMG signal vector was defined as input for the task-classifier. In particular, a single window of 100 ms (i.e. 205 samples) was identified on each of the five channels after the movement onset, which is common to all channels for construction. A new vector, the pattern vector, containing 205 samples per channel for a total of 1025 samples was then created as input for the subsequent steps. An example of vectoralization for a single pinching trial is presented in Figure 5.

Figure 5.

Vectorialization exemplification on a single pinching trial for study subject S01 (i.e. STEP 1.3). Grey shaded area represents the 100 ms window.

STEP 2: Calibration

Graphical representation of the calibration procedure is presented in Figure 6 and described below.

Figure 6.

Graphical outline of the calibration procedure. Input: pattern vectors; STEP 2.1: subject-specific clustering; STEP 2.2: cascade artificial neural network (ANN) training. The colour version of this figure is available at: http://imr.sagepub.com.

STEP 2.1: Subject specific clustering – unsupervised clustering

The pattern vectors of each subject were scanned to find similarities between different hand grasp functional tasks with an on-purpose developed algorithm. The base of the clustering algorithm was the k-means unsupervised clustering approach.²⁶ The best clustering solution was adopted, as evaluated by the Silhouette coefficient that can be calculated for each sample i as follows:²⁷

s (i) = {1 - \frac{a (i)}{b (i)} se a (i) < b (i) 0 se a (i) = b (i) \frac{b (i)}{a (i)} - 1 se a (i) > b (i)

Where a(i) is the mean distance of the i-th sample with samples within the same cluster, and b(i) is the minimum distance between the i-th sample and samples outside its proper cluster. Among the same dataset, the clustering is better when the lower is a(i), and the higher is b(i). The Silhouette coefficient is bounded between –1 and 1, where 1 indicates the best clustering for the given sample. The subject-specific clustering algorithm was performed as follows: (i) k-means algorithm execution with 1000 different initial weights configuration; (ii) clustering evaluation by means of the Silhouette coefficients mean taken as a global marker of the clustering performance; (iii) selection of the best number of clusters based on mean Silhouette coefficients. The clustering algorithm was completely blind with respect to the type of hand grasp functional task the pattern represented.

STEP 2.1: Subject-specific clustering – cluster association

Once the unsupervised clustering was completed, which was blind to the association between EMG patterns and tasks, each hand grasp functional task was assigned to the cluster where it is most represented.

STEP 2.2: Cascade ANN training

A small number of trials per hand grasp functional task were used to train the cascade ANN, formed by two ANNs. The optimal number of trials to be selected was a parameter that needed to be optimized. The two ANNs were feedforward one layer networks. The first ANN has as inputs the 1025 EMG samples of each task (i.e. the pattern vector), a hidden layer with a number of neurons that is a parameter that needed to be optimized, and an output layer with a number of neurons equal to the number of identified clusters. For each cluster that contains more than one hand grasp functional task, a further feedforward ANN will be trained to classify the grouped hand grasp functional tasks. This second layer ANN will have the same 1025 input neurons as the first layer ANN (i.e. the pattern vector), a hidden layer as in the first layer ANN, and an output layer with a number of neurons equal to the number of hand grasp functional tasks classified within the cluster. For each ANN, the training algorithm was set to scaled conjugate gradient backpropagation with 1000 maximum iterations, and six maximum validation failures. Training trials were partitioned as follows: 60% of the trials per hand grasp functional task as the training set, 20% of the trials per hand grasp functional task as the validation set, and 20% of the trials per hand grasp functional task as the test set. Hidden layer neuron activation function and the training algorithm were parameters that needed to be optimized. Since a classifier was being trained, output layer neuron activation function was set to softmax function:

f (x_{i}) = \frac{e^{x_{i}}}{\sum_{j = 1}^{N} e^{x_{j}}}

Where i represents the i-th neuron in the output layer, and N the total neuron number. Softmax is a probabilistic function that assigns to every output neuron the probability that the input pertains to the class represented by the neuron. In a postprocessing step, the input was assigned to the neuron that has the higher probability. The overall cascade ANN performance was evaluated by means of correctly classified hand grasp functional tasks within trials used for the cascade ANN training.

STEP 3: Testing

Trials not used for cascade ANN training were used for cascade ANN testing, and the percentage of correctly classified hand grasp functional tasks was considered as the outcome measure.

Parameter optimization

Local minima algorithm frequency definition

The accuracy of the algorithm to identify EMG onsets was evaluated with respect to EMG onset visually as determined by an experienced examiner (M.G.).²⁸ Both methods have been run on the mean of five channels EMG preprocessed signals (i.e. where the specific application requires to calculate the onsets).

With regard to the visual onset determination, the examiner calculated the onset times for all traces on two separate days, 3 days apart, to evaluate the repeatability of the visual recordings. To evaluate the repeatability of the visually determined onset times between days the mean among the two visually determined onset times was determined, along with the Interclass Correlation Coefficient (ICC). The mean of the visually determined onset times between days was used for the evaluation of the computer-based methods, as previously described.²⁸

The local minima algorithm cut-off frequencies for the low-pass filter was optimized in the range 0.01–0.2Hz by maximizing the F1-score, defined as the harmonic mean between the precision and recall indices.²⁹ Precision and recall indices were calculated as follows, where the closer to 1 the index was, the better the performance was:

precision = \frac{true positive}{true positive + false positive} recall = \frac{true positive}{true positive + false positive}

F1-score results therefore to be:

F 1 - score = 2 * \frac{precision * recall}{precision + recall}

Further, so to identify the degree to which the computer-determined values deviated from those determined visually, linear regression equations were calculated on true positive onsets using the mean of the visually determined onset times between days as the independent variable.²⁸

Cascade ANN parameter definition

Different cascade ANN parameters needed to be defined as follows: (i) number of trials considered for cascade ANN definition; (ii) number of neurons in the hidden layer; (iii) hidden layer neuron activation function; and (iv) learning rate. For each parameter, a range of values were selected as outlined in Table 1.

Table 1.

Different cascade artificial neural network (ANN) parameters that were tested as part of the evaluation of the electromyography-based task classifier developed in this study.

Parameter	Range tested
Number of trials considered for cascade ANN definition	4, 5, 6, 7, 8
Number of neurons in the hidden layer	10, 15, 20, 25, 30
Hidden layer neuron activation function	Sigmoid, hyperbolic tangent
Learning rate	0.01, 0.1

In order to define the best parameter mix, the study proceeded as follows. The number of trials considered for cascade ANN definition was set at six (i.e. the mid value), and then all possible combinations of the other parameter values were considered. Therefore, 20 cascade ANNs were trained and tested with the same dataset. The best parameter mix was determined as the one with the highest percentage of correctly classified hand grasp functional tasks during cascade ANN testing (i.e. STEP 3). Once the best parameter mix was determined, the number of trials considered for cascade ANN definition was modified according to the range tested to determine the minimum number of trials required to get a proper testing performance for the present application.

Neurological patient pilot test procedure

Neurological patients followed the same experimental protocol as healthy control subjects, but the number of trials per block was set to 15, and the auditory cue that was increased to be every 15 s. EMG electrodes were positioned on the affected side.

In order to obtain the effective movement execution to further motivate the patients, they were supported in the correct execution of the movement with the Gloreha rehabilitation glove (GLOve REhabilitation Hand). Gloreha is a device for neuromotor rehabilitation of the hand, developed and produced by Idrogenet Srl (Lumezzane, Italy). It is composed by two main elements: a comfortable and light glove, and a chassis containing electromechanical actuators and an electronic board. The device allows the execution of all of the combinations of joint flexion-extension. Specifically, finger movement is performed thanks to five electromechanical actuators and an electronic board, placed in the chassis, not accessible to the operator. Each actuator is linked to a wire. In a compartment of the chassis, the operator can adjust the length of the five cables that generate the finger movement to set the starting position of the hand, which is also the maximum level of extension the glove will reach during the physical therapy. Gloreha was subject-specifically set in order to obtain the selected hand grasp functional tasks (i.e., pinching, grasp an object, grasping). Thanks to a trigger myocontrol previously detailed,³⁰ it was possible to be sure that the movement was patient-initiated.

Results

Participants

Eight healthy subjects (four females, four males; age range 25–26 years) with no neurological or orthopaedic impairment volunteered for this study and all of them succeeded in completing the experimental procedure. Two neurological post-stroke patients were recruited to test the proposed approach. PZ01 was a 48-year-old female with a lesion located in the left hemisphere in the capsular striatum obtained in April 2014. PZ02 was 50-year-old male with a lesion located in the left hemisphere, in the temporal, paraventricular, and parietal cortico-subcortical areas, obtained in April 2003. Both of them had impairment to the upper limb contralateral to the lesion; PZ01 had a Medical Research Council index of 3 for both the wrist and elbow flexors; and PZ02 had a Medical Research Council index of 4 for both the wrist and elbow flexors.³¹

Local minima algorithm performance

The repeatability of the visually determined EMG onset times between days was found to be high with a mean ± SD difference of 43 ± 110 ms. The ICC was 0.999. The mean F1-scores and offsets of the regression lines obtained are shown in Figure 7. The best F1-score was obtained for 0.09 Hz (0.9655), while the offset that was closer to zero was in correspondence with 0.08 Hz (6 ms). The study selected 0.09 Hz as the optimal cut-off frequency for the local minima algorithm, as maximizing the F1-score. The correspondent offset with respect to visually determined onsets was 36 ms, in line with results obtained in the literature.²⁸

Figure 7.

Local minima algorithm frequency definition. (a) F1-scores obtained for cut-off frequencies in the 0.01–0.2 Hz range; (b) offsets of the regression lines obtained comparing onsets obtained by the local minima algorithm with cut-off frequencies in the 0.01–0.2 Hz range, and visually determined onsets. Grey shaded area represent the selected cut-off frequency, i.e. 0.09 Hz.

Cascade ANN parameter definition

The number of trials considered for cascade ANN definition was set at six (i.e. the mid value), and then all possible combinations of the other parameter values were considered. Twenty cascade ANNs were trained and tested considering six trials for task-selection classifier definition, and 14 for testing on both repetitions of each healthy control, resulting in 16 testing datasets (Table 2). The parameter mix that obtained the higher percentage of correctly classified hand grasp functional tasks during task-selection classifier testing (i.e. STEP 3) was the number of neurons in the hidden layer equal to 25, sigmoid as the hidden layer neuron activation function, and 0.01 as the learning rate, which lead to a mean ± SD testing performance of 76% ± 14% of correctly classified hand grasp functional tasks. With these parameters, the number of trials considered for cascade ANN definition was explored (Figure 8). The best testing and calibration performances were obtained with six trials, with a mean ± SD of 76% ± 14% of correct classifications during testing; and a mean ±SD of 93 ± 7% during calibration. Using fewer trials did not allow the classifier to correctly explore the dataset, while using more trials resulted in overfitting of the calibration dataset.

Table 2.

Testing performance of the task-selection classifier in terms of percentage of correctly classified hand functional tasks while exploring different parameter values with the number of trials considered for the cascade artificial neural network definition set at six in eight healthy subjects.^a

Number of neurons	Activation function	LR	S01		S02		S03		S04		S05		S06		S07		S08		Mean, %	SD, %
Number of neurons	Activation function	LR	REP1	REP2	REP1	REP2	REP1	REP2	REP1	REP2	REP1	REP2	REP1	REP2	REP1	REP2	REP1	REP2	Mean, %	SD, %
10	Sig	0.01	98	74	63	59	57	64	60	81	81	50	44	45	83	81	90	93	70.19	17.20
15	Sig	0.01	100	69	61	59	64	73	69	81	81	52	24	40	86	76	88	83	69.13	19.16
20	Sig	0.01	98	83	63	56	72	64	67	81	76	48	76	53	88	76	93	90	74.00	14.71
25	Sig	0.01	100	83	66	67	72	70	69	81	76	52	88	45	83	88	88	86	75.88	14.26
30	Sig	0.01	98	74	61	62	64	68	74	79	79	50	60	45	88	76	88	88	72.13	14.66
10	Sig	0.1	98	69	61	59	57	57	50	62	81	52	8	45	83	74	88	74	63.63	20.94
15	Sig	0.1	98	57	55	62	60	57	62	62	79	45	64	48	55	55	80	83	63.88	14.10
20	Sig	0.1	98	71	58	56	64	70	67	55	79	50	68	58	76	67	71	74	67.63	11.53
25	Sig	0.1	100	57	61	56	64	59	62	81	67	50	48	43	79	67	68	57	63.69	13.99
30	Sig	0.1	98	74	61	64	57	66	45	79	67	55	12	55	74	69	80	52	63.00	18.78
10	Hyptg	0.01	98	60	66	62	66	66	67	76	79	52	88	48	88	69	93	90	73.00	14.98
15	Hyptg	0.01	98	55	66	59	70	75	64	79	81	55	76	38	88	76	88	93	72.56	15.92
20	Hyptg	0.01	100	64	63	56	66	61	62	86	86	55	64	38	86	79	90	95	71.94	17.21
25	Hyptg	0.01	98	86	61	56	68	77	74	86	81	50	60	48	90	79	88	90	74.50	15.55
30	Hyptg	0.01	95	83	61	64	66	70	71	83	81	57	80	43	86	86	90	88	75.25	14.17
10	Hyptg	0.1	98	52	55	59	60	61	57	52	79	57	28	48	88	71	85	69	63.69	17.40
15	Hyptg	0.1	83	74	39	67	62	61	64	71	38	52	8	55	81	36	80	57	58.00	19.99
20	Hyptg	0.1	98	67	47	64	64	61	60	52	79	50	84	53	83	62	76	69	66.81	14.08
25	Hyptg	0.1	98	69	63	56	57	43	64	74	17	52	56	43	76	71	71	64	60.88	17.85
30	Hyptg	0.1	98	33	45	26	64	70	62	76	33	52	48	48	76	55	56	50	55.75	18.47

Data presented as the percentage (%) of correctly classified hand functional tasks.

^aThe different parameters that were explored: (i) number of neurons in the hidden layer; (ii) hidden layer neuron activation function, either sigmoid (Sig) or hyperbolic tangent (Hyptg); and iii) learning rate (LR), either 0.01 or 0.1.

REP, repetition; S, healthy control subject.

Figure 8.

Number of trials considered for cascade artificial neural network exploration. (a) Testing performance; (b) calibration performance. Data presented as mean ± SD of the healthy control group.

Task-selection controller performance

Final parameters were set as 25 neurons in the hidden layer, sigmoid as the hidden layer neuron activation function, 0.01 as the learning rate, and six trials for cascade ANN calibration, which resulted in a mean performance of 93% during calibration, and 76% during testing in healthy control subjects. The mean ± SD difference in testing performance between the two repetitions executed by the same participant after electrode repositioning was 13 ± 14%, with four subjects having a testing performance difference < 5%.

Task-selection controller tests on the two patients PZ01 and PZ02 resulted in a calibration performance of 83% and 100%, and a testing performance of 79% and 100%, respectively, which were in line with the results obtained for the healthy control subjects.

Discussion

When dealing with neurological rehabilitation, patients usually ask if they will functionally benefit from the treatment offered to them. Indeed, patients attribute great value to being able to return to simple functional activities of daily life such as pouring water from a bottle, which leads to improved self-esteem and better recovery, as highlighted by the International Classification of Functioning, Disability and Health, which mentions body functions, activities, and social context rather than specific characteristics such as muscle strength or range of motion.³² Daily life functional tasks, especially those directly involving the hand, always take advantage of the simultaneous involvement of multiple degrees-of-freedom. These considerations lead to the present study choosing to use the multiple degrees-of-freedom functional movements that could be detected though the controller. Indeed, the goal of the present study was to design and implement a task-selection controller characterized by: (i) recording of the intention of the user to perform the movement, which was captured through the myoelectric signal; (ii) detection of the movement to be executed, which was implemented though a cascade ANN controller based on 100 ms of EMG signal rather than the whole recorded signal; (iii) rehabilitation of targeted functional movements involving multiple degrees-of-freedom at the same time; (iv) easy-to-use approach, so that the movement was triggered, and recognized on the same limb that would benefit from the mobilization; and (v) the possibility of being implemented in real-time.

The superficial EMG signal opens a door to the patient's residual ability, even if only mild movements can be performed, and it can be immediately recorded via the self-adhesive superficial electrodes. The present study set the number of recorded channels to five. This was due to technical reasons and the need to have an easy set-up procedure, for which the electrode positioning did not need to target any specific muscle. Electrodes were spaced equally around the lower arm 2–3 cm from the elbow and could be placed by nonexpert personnel (e.g. a caregiver). Moreover, nonspecific electrode positioning removes the crosstalk problem, which usually affects EMG-based controllers.³³

In order to recognize the intended movement to be executed, only about 5% of the movement EMG was used, roughly corresponding to the time delay between muscle activation and effective kinematic movement. This was in contrast with previous research, where all EMG signals were used, from the beginning to the end of the contraction or for a high percentage of the signal (i.e. 50%–70%).²³ To our knowledge, only one previous study developed an EMG-based classifier with a support vector machine approach to predict goal-directed movements in the horizontal plane with a 200 ms window, but the classifier failed when tested on neurological patients.¹⁸ Moreover, the authors used muscle activity recorded between −100 ms and 100 ms with respect to the movement onset, which makes this approach not suitable for real-time use.¹⁸ In this context, EMG onset detection is therefore crucial. This current study proposed a rather computationally simple approach based on first order filter at 0.09 Hz which lead to very good results, as indicated by an F1-score of 0.9655.

The core of the proposed task-selection classifier was a cascade ANN that was optimized with respect to the number of trials considered for its definition, the number of neurons in the hidden layer, the hidden layer neuron activation function, and the learning rate, which were set at six, 25, sigmoid, and 0.01, respectively. The number of trials used for the calibration was set to the minimum possible, without affecting classifier performance under testing conditions. Increasing the number of neurons in the hidden layer has been shown to be linked to a better ANN performance.²³ In this present study, the best performances were recorded with 20 or 25 neurons in the hidden layer. The neuron activation function did not appear to be crucial in affecting performance, while a learning rate of 0.01 overall resulted in better performance with respect to a learning rate of 0.1.

With the final architecture, the task-selection controller performance was demonstrated to be 93 ± 7% for calibration and 76 ± 14% for testing in the present study, which was comparable or superior to other published studies with multiple degrees-of-freedom.^23,34,35 Intra-subject repeatability was assessed, demonstrating the validity of the approach for longitudinal rehabilitation sessions. Two neurological patients volunteered to test the controller. They obtained very encouraging results on testing performance, 79% and 100%, confirming the validity of the proposed approach. It has been shown that moderately impaired neurological patients have mean classification accuracy of 71.3%, and severely impaired patients 37.9%, using a linear discriminant analysis.³⁴ Research findings strongly suggest that the EMG pattern classification system for stroke survivors should be designed specifically for each subject.³⁴ The selection of target tasks (number and complexity), for example, should reflect the functional impairment level of each subject.³⁴ Although the approach proposed in this present study works with only 100 ms of EMG signal and with three functional tasks, it was a subject-specific EMG classification paradigm, and therefore it would be expected to work specifically on patient features.

It has to be noted that task-classifier performance depends on having good quality EMG signals on all recorded channels. Recorded noise on individual channels can be due, for example, to incomplete relaxation between different movements induced by the open-hand resting position that is detected by an electrode pair or to degraded electrode–skin impedance. Identification of the onset is not affected by a single noisy channel because it is based on the mean of the five channels. However, the pattern vector input to the task-selection classifier includes the 100 ms window for all channels, so a noisy channel might result in a lower classifier performance, as can be observed by the overall lower performance obtained by S05-REP 2 and S06-REP 2 (Table 2). A possible improvement includes the removal of noisy channels via software before task-selection classifier calibration and use. Moreover, particular care has to be devoted to electrode placement by carefully cleaning the skin. With a view to the clinical application of this task-classifier, the major limitation of the present approach is the need to calibrate the task-classifier at each use. Indeed, the use of superficial electrodes has the advantage of being noninvasive and easily set-up, but it presents the drawback of being strongly dependent on electrode positioning and electrode–skin impedance. Moreover, when a subject's motion intention is misclassified, the task-classifier is designed so that a movement would be executed, even if it is not the planned one. This approach has the advantage of providing in any case a rehabilitation exercise, but it might be disturbing for some patients. In contrast, in the case of missed onsets (i.e. the algorithm doesn't identify a true onset of motion intent), the task-classifier performance might be influenced. This is the case for example of S06-REP1, where poor onset identification for the pinching task resulted in performances as low as 8% with some parameter settings (Table 2). Further tests on control subjects and end-users are required to demonstrate the repeatability, robustness, and validity of the current approach.

In conclusion, this present study designed and implemented an EMG-based task-selection controller that was able to estimate with a 100-ms signal window the intended movement to be performed. As far as we know, this is the only task-classifier designed to predict grasping functional tasks using an EMG signal corresponding to the electromechanical delay latency that has been successfully tested on neurological patients. The task-selection controller dealt with multiple degrees-of-freedom functional movements, and it was based on a cascade ANN that demonstrated an accuracy of 76 ± 14% under testing conditions. Two pilot post-stroke neurological patients obtained 79% and 100% under testing conditions, confirming the validity of the proposed approach.

Footnotes

Declaration of conflicting interests

The authors declare that there are no conflicts of interest.

Funding

The work was supported by the Think and Go project funded by Lombardy Region (POR FSE 2007/2013) and Fondazione Cariplo.

References

Pedrocchi

Ferrante

Ambrosini

et al.

MUNDUS project: MUltimodal neuroprosthesis for daily upper limb support. J Neuroeng Rehabil 2013; 10: 66.

Lawrence

Coshall

Dundas

et al.

Estimates of the prevalence of acute stroke impairments and disability in a multiethnic population. Stroke 2001; 32: 1279–1284.

Bernhardt

Dewey

Thrift

et al.

Inactive and alone: physical activity within the first 14 days of acute stroke unit care. Stroke 2004; 35: 1005–1009.

Schabowsky

Godfrey

Holley

et al.

Development and pilot testing of HEXORR: hand EXOskeleton rehabilitation robot. J Neuroeng Rehabil 2010; 7: 36.

Langhorne

Coupar

Pollock

. Motor recovery after stroke: a systematic review. Lancet Neurol 2009; 8: 741–754.

Gandolla

Ferrante

Molteni

et al.

Re-thinking the role of motor cortex: context-sensitive motor outputs?

Neuroimage 2014; 91: 366–374.

Gandolla

Ward

Molteni

et al.

The neural correlates of long-term carryover following functional electrical stimulation for stroke. Neural Plast 2016; 2016: 4192718, doi:10.1155/2016/4192718.

Gandolla

Molteni

Ward

et al.

Validation of a quantitative single-subject based evaluation for rehabilitation-induced improvement assessment. Ann Biomed Eng 2015; 43: 2686–2698.

Jang

Kim

Lee

. Effects of brain-computer interface-controlled functional electrical stimulation training on shoulder subluxation for patients with stroke: a randomized controlled trial. Occup Ther Int 2016; 23: 175–185.

10.

Murphy

Guggenmos

Bundy

et al.

Current challenges facing the translation of brain computer interfaces from preclinical trials to use in human patients. Front Cell Neurosci 2016; 9: 497.

11.

Casellato

Pedrocchi

Zorzi

et al.

EMG-based visual-haptic biofeedback: a tool to improve motor control in children with primary dystonia. IEEE Trans Neural Syst Rehabil Eng 2013; 21: 474–480.

12.

Singh

Chatterji

. Trends and challenges in EMG based control scheme of exoskeleton robots - a review. Int J Sci Eng Res 2012; 3: 933–940.

13.

Ho NSK, Tong KY, Hu XL, et al. An EMG-driven exoskeleton hand robotic training device on chronic stroke subjects: Task training system for stroke rehabilitation. In: 2011 IEEE International Conference on Rehabilitation Robotics (ICORR). ETH Zurich Science City, Switzerland, 29 June – 1 July 2011, p. 1–5.

14.

Lang

Macdonald

Reisman

et al.

Observation of amounts of movement practice provided during stroke rehabilitation. Arch Phys Med Rehabil 2009; 90: 1692–1698.

15.

Castellini

van der Smagt

. Evidence of muscle synergies during human grasping. Biol Cybern 2013; 107: 233–245.

16.

Geethanjali

. Comparative study of PCA in classification of multichannel EMG signals. Australas Phys Eng Sci Med 2015; 38: 331–343.

17.

Fligge

Urbanek

van der Smagt

. Relation between object properties and EMG during reaching to grasp. J Electromyogr Kinesiol 2013; 23: 402–410.

18.

Cesqui

Tropea

Micera

et al.

EMG-based pattern recognition approach in post stroke robot-aided rehabilitation: a feasibility study. J Neuroeng Rehabil 2013; 10: 75.

19.

Alkan

Gunay

. Identification of EMG signals using discriminant analysis and SVM classifier. Expert Syst Appl 2012; 39: 44–47.

20.

Wojtczak

Amaral

Dias

et al.

Hand movement recognition based on biosignal analysis. Eng Appl Artif Intell 2009; 22: 608–615.

21.

Zecca

Micera

Carrozza

et al.

Control of multifunctional prosthetic hands by processing the electromyographic signal. Crit Rev Biomed Eng 2002; 30: 459–485.

22.

Gandolla M, Ferrante S, Baldassini D, et al. Artificial Neural-Network EMG Classifier for Hand Movements Prediction. In: Kyriacou E, Christofides S, Pattichis CS, editors. XIV Mediterranean Conference on Medical and Biological Engineering and Computing 2016. Springer International Publishing; 2016 p. 634–637. (IFMBE Proceedings).

23.

Muceli

Farina

. Simultaneous and proportional estimation of hand kinematics from EMG during mirrored movements at multiple degrees-of-freedom. IEEE Trans Neural Syst Rehabil Eng 2012; 20: 371–378.

24.

Cavanagh

Komi

. Electromechanical delay in human skeletal muscle under concentric and eccentric contractions. Eur J Appl Physiol Occup Physiol 1979; 42: 159–163.

25.

Gijsberts

Atzori

Castellini

et al.

Movement error rate for evaluation of machine learning methods for sEMG-based hand movement classification. IEEE Trans Neural Syst Rehabil Eng 2014; 22: 735–744.

26.

Jain

. Data clustering: 50 years beyond K-means. Pattern Recognit Lett 2010; 31: 651–666.

27.

Rousseeuw

. Silhouettes: A graphical aid to the interpretation and validation of cluster analysis. J Comput Appl Math 1987; 20: 53–65.

28.

Hodges

Bui

. A comparison of computer-based methods for the determination of onset of muscle contraction using electromyography. Electroencephalogr Clin Neurophysiol 1996; 101: 511–519.

29.

Powers

DMW

. Evaluation: from precision, recall and F-measure to ROC, informedness, markedness & correlation. International Journal of Machine Learning Technology 2011; 2: 37–63.

30.

Gandolla M, Ferrante S, Baldassini D, et al. EMG-controlled robotic hand rehabilitation device for domestic training. In: Kyriacou E, Christofides, S Pattichis, CS, editors. XIV Mediterranean Conference on Medical and Biological Engineering and Computing 2016. Springer International Publishing; 2016 p. 638–642. (IFMBE Proceedings).

31.

Medical Research Council/Guarantors of Brain. Aids to the Examination of the Peripheral Nervous System, London: Bailliere Tindall, 1986.

32.

Geyh

Cieza

Schouten

et al.

ICF core sets for stroke. J Rehabil Med 2004; (44 supl): 135–141.

33.

Jiang

Englehart

Parker

. Extracting simultaneous and proportional neural control information for multiple-DOF prostheses from the surface electromyographic signal. IEEE Trans Biomed Eng 2009; 56: 1070–1080.

34.

Lee

Wilson

Lock

et al.

Subject-specific myoelectric pattern classification of functional hand movements for stroke survivors. IEEE Trans Neural Syst Rehabil Eng 2011; 19: 558–566.

35.

Ortiz-Catalan

. Cardinality as a highly descriptive feature in myoelectric pattern recognition for decoding motor volition. Front Neurosci 2015; 9: 416.