Active rehabilitation training system for upper limb based on virtual reality

Abstract

In this article, an active rehabilitation training system based on the virtual reality technology is designed for patients with the upper-limb hemiparesis. The six-axis inertial measurement unit sensors are used to acquire the range of motion of both shoulder and elbow joints. In order to enhance the effect of rehabilitation training, several virtual rehabilitation training games based on the Unity3D engine are designed to complete different tasks from simple level to complicated level. The purpose is to increase the patients’ interest during the rehabilitation training. The basic functions of the virtual rehabilitation task scenes are tested and verified through the single-joint training and the multi-joint compounding training experiments. The experimental results show that the ranges of motion of both shoulder and elbow joints can reach the required ranges of a normal human in the rehabilitation training games. Therefore, the system which is easy to wear, low cost, wireless communication, real-time data acquisition, and interesting virtual rehabilitation task games can provide an effective rehabilitation training process for the upper-limb hemiparesis at home.

Keywords

Upper limb hemiparesis six-axis inertial measurement unit sensor virtual reality rehabilitation training

Introduction

The upper limb has many degrees of freedom, and it is also a complex part of the human body by which people can accomplish fine movements during their activities in daily life. With the intensification of the aging problem in the world, the amount of stroke hemiparesis has shown a growing trend, especially in China, which has an enormous population.¹ Approximately 30%–50% of these stroke survivors will suffer from chronic hemiparesis, especially involving their hands. In addition, spinal cord injury (SCI) and traffic accident survivors may also find limb movements’ disorder. Injury within the cervical region of the cord leads to tetraplegia, which leads to impairment of all four limbs. An estimated result shows that 55% of new cases will result in tetraplegia, while the other 45% will experience paraplegia due to injury below the cervical level.² Limb hemiparesis which is caused by stroke, SCI, or traffic accidents not only gives the patient’s daily life a great deal of inconvenience and even more makes the patient suffer from great mental pain but also brings a heavy stress and medical burden for the patient’s family and society. Technology has been developed in an effort to facilitate rehabilitation for the patient. Upper-limb rehabilitation is one of the fastest growing areas in modern neurorehabilitation. Quality of life can be improved with efficient therapy.³ At present, rehabilitation therapy of upper limb with traditional rehabilitation therapy is commonly used, that is, rehabilitation therapists perform rehabilitation trainings on individuals. Now with the development of robot technology, the rehabilitation of robot-assisted training is also rising up. The MIT-Manus⁴ is an example of end-effector-based and arm-rehabilitation robotic device, while the ARMin device⁵ is an example of arm-rehabilitation exoskeletons which also allows pronation/supination of the lower arm and wrist flexion/extension. It could be operated in three modes: passive mobilization, active game-supported arm therapy, and active training of activities of daily living (ADLs). The end-effector-based robots have practical advantages (usability, simplicity, and cost-effectiveness), and exoskeleton robots have biomechanical advantages (better guidance). Currently, the automatic rehabilitation devices on market as mentioned above are mostly complex and expensive, which are often used in the hospitals and clinics are not affordable to ordinary patients. Therefore, one of the research objectives aims to develop the upper-limb rehabilitation training system with minimal structure and low cost and can be used in patient’s home. But in China, it can be seen that patients with upper-limb orthosis in home is only for fixing the arm and just move autonomously according to the setting angle. The researches on intelligent domestic rehabilitation device just begins, most of which are in the experimental stage and not yet market oriented.^6,7

Another problem is that the patients are treated with low initiative and dull training process which does not motivate them, while the treatment effect is not obvious.^8,9 Computer games based on virtual reality (VR) are a good way to mobilize the patients’ initiative in the training, so the rehabilitation effect on a particular movement task will be greatly improved.¹⁰ VR environments provide an excellent method to manipulate task conditions in training. The effects and the intensity of training can be enhanced and designed more challenging, since the implementation of VR can build a channel both visual and haptic communication can be involved in. The research on VR system which is applied to rehabilitation training was initiated a few years ago. Mazzone et al.¹¹ made a study on the effect of rehabilitation training for patients with shoulder joints training using VR technology. This study aimed to determine whether performance of shoulder exercises in virtual reality gaming (VRG) results in similar muscle activation as non-VRG exercise. The conclusion was drawn that exercise with VRG should be effective to reduce shoulder pain caused by spinal injury. Fischer et al.¹² conducted a preliminary study claim that stroke patients could assist themselves in training their hands in the virtual environment. The purpose of this pilot study was to investigate the impact of assisted motor training in a virtual environment on hand function for the stroke survivors. Participants had 6 weeks of training in reach-to-grasp of both virtual and actual objects. After the training period, participants in all three groups demonstrated a decrease in time to perform some of the functional tasks. These designs based on VR have achieved some success and then the second research objective is to add the VR technology to the intelligent domestic rehabilitation device. These studies are mainly designed for the single joint of the upper-limb rehabilitation training. Therefore, it is necessary to carry out the research on multi-joint combined training device for patients who can just stay home by training with VR tasks of adjustable game levels.

Another important element which needs to be considered as an ultimate success using at home is its ease of use. Therefore, simple active rehabilitation device should be developed. The setup time of such device should be fast, besides measurement, treatment approaches, and incorporating gaming, and should provide intuitive interfaces that can be directly utilized by the individuals. This study will introduce an active rehabilitation training system for upper limb based on VR technology, which has some advantages such as simple structure, easy to manipulate, and portable for household. It also mobilizes patients’ initiative with adjustable difficulty level of VR tasks so that the individuals’ rehabilitation effect of the upper limb is obviously improved.

Overall design of the system

The active rehabilitation training system for upper limb based on VR is designed for the pronation/supination and flexion movement trainings of the elbow joint and the extension/flexion and abduction exercises of the shoulder joint. By adding the games in training processes, the patients may actively participate in rehabilitation trainings, while the efficiency will be greatly improved. The portable and easy-to-use design of this system can effectively reduce the problem of the medical resources shortage in the rehabilitation field.

Overall scheme of the system

The system is composed of two parts: the upper-limb posture detection system and the virtual rehabilitation training task scene, as shown in Figure 1.

Figure 1.

Schematic diagram of an active rehabilitation training system for upper limb based on VR.

The upper-limb posture detection system based on two six-axis inertial measurement unit (IMU) sensors is wearing on the patient’s arm. The joint angles are measured by IMU sensors following the positions of the patient’s arm in training and then the sampled data are transmitted to personal computer (PC) by the wireless Bluetooth. The visual studio programming platform is used to develop a detection system and realize the collection of the range of motion (ROM) of the forearm and the upper arm. The virtual rehabilitation training scene is designed with the game engine of Unity3D software to build interaction between human and computer. The rehabilitation games are designed with adjustable difficulty level of VR tasks, so that the patients may take more active rehabilitation trainings in order to meet the requirements. Furthermore, it can display the changes of the training parameters in the scene and show the state of rehabilitation training of the individuals more clearly, which forms the basis of the training effect assessment for the physical therapist. Compared with the LabVIEW software, which is used to develop the training task scenes, the characters in Unity3D are designed with three-dimensional (3D) models, and the scenes are more complex, interesting, and diverse. Unity3D is a comprehensive multi-platform development tool with which video games can be easily created, such as 3D architectural visualization, real-time 3D animation, and other types of interactive content games.

Design and analysis of upper-limb posture detection system

According to the analysis of the human upper-limb movement, the training ranges of the shoulder and elbow joints should be met with the required ROMs defined in The Physiology of the Joints written by Dr Kapandji,¹³ so the detection system is designed to acquire the postures of the upper limb.

There are two six-axis IMU sensors attached to the patient’s shoulder and elbow joints by bonds, as shown in Figure 2. When the arm stretches parallel to the ground, the Y-axis is defined as the direction along the arm, and the X-axis is vertical to the arm in the horizontal plane and then the Z-axis is perpendicular to the horizontal plane and conforms to the right-hand rule. The positive directions of the coordinate axes are shown in Figure 2. Wherein, the pronation/supination and flexion angle of the elbow joint are collected by the sensors on the forearm. Therefore, for the elbow joint, the corresponding value of the pronation/supination angle is detected by the sensor rotating around the Y₁-axis, while the flexion angle is that of the rotation of the Z₁-axis. Similarly, the extension/flexion angle and the abduction angle of shoulder joint are collected by the sensor on the upper arm. The corresponding value of the extension/flexion angle is the rotation of the Z₂-axis, and the abduction angle is that of the X₂-axis.

Figure 2.

Schematic diagram of two six-axis IMU sensors on the upper limb.

The single-joint training action is shown in Figure 3 (the coordinates definition in Figure 3 is same as the definitions in Figure 2). If the training is for the right arm of the patient, the left picture in Figure 3(a) is the zero position for the elbow pronation/supination training action, as well as the right picture in Figure 3(a) is the zero position for the shoulder training action and the elbow flexion training action. The standard ranges¹³ of the elbow joint’s pronation/supination angle and the flexion angle are (−85° to 0° to +90°) and (0°–+145°), respectively, as shown in Figure 3(b). The standard ranges of the shoulder joint’s extension/flexion angle and the abduction angle¹³ are (−50° to 0° to 180°) and (0°–180°), respectively, as shown in Figure 3(c). The VR game scenes should be designed to meet with the requirements of these joint training ranges.

Figure 3.

Schematic diagrams of the upper-limb single-joint training: (a) zero-position definition of the elbow and shoulder joint training actions, (b) schematic diagrams of the elbow joint’s single training action, and (c) schematic diagrams of the shoulder joint’s single training action.

Design and analysis of virtual rehabilitation training scene

The scenes of virtual rehabilitation task based on Unity3D engine are designed and integrated in this system. It can provide patients a real interactive experience so that they may immerse in training unknowingly. At the same time, the ROMs of the patient’s joints in all directions are displayed in time, so it is convenient for the patients to correct the upper-limb posture during the rehabilitation training and then they can achieve the requirements of rehabilitation treatment while playing the games.¹⁴ The software includes three parts: (1) the 2D rehabilitation training scenes for the pronation/supination and flexion movement of the elbow joint, (2) the 2D rehabilitation training scenes for the extension/flexion and abduction of the shoulder joint, and (3) the 3D rehabilitation training scenes for the compound training movements of both the shoulder joint and the elbow joint.

Elbow joint rehabilitation training scene

The elbow joint rehabilitation scene is designed as a game of an angry bird is trying to rescue its partner. According to the patient’s condition and rehabilitation of different, the game is developed for six game scenes. The functionality and difficulty of each scene is different, and the patient can be trained through adjustable levels from simple to complex. The game is full of comfortable music under which patients may feel physically and mentally relaxed, so they are more conducive to the rehabilitation processes. Six different elbow joint rehabilitation game scenes are shown in Figure 4.

Figure 4.

Game scenes for the elbow joint rehabilitation.

As shown in Figure 5, the rehabilitation training for the patient’s elbow joint is divided into three major parts. The scenes a and b in Figure 4 are aimed at training of the elbow joint’s pronation/supination actions. With the patient’s elbow joint’s pronation/supination actions, the red bird moves around to rescue the bird in the cage. Similarly, the scenes c and d in Figure 4 are of the elbow joint’s flexion movement. With the patient’s arm swings up and down, the red bird moves up and down to rescue the caged bird. The scenes e and f in Figure 4 are the compound training of the elbow joint’s pronation/supination and the flexion movement. In these two scenarios, the patient needs to combine the two rehabilitation exercises to control the red bird flying away from the stationary wall and the rotating windmill to save the caged bird.

Figure 5.

Analysis of the elbow joint’s rehabilitation training scene.

Shoulder joint rehabilitation training scene

The rehabilitation of the shoulder joint in the virtual game scene is developed as a fishing game for big fish eating small fish. In order to meet the requirements of the different severities of the patient, eight game scenes are designed. Each game has different scene features and difficulty levels, and patients will be trained from simple to complex level. These movement angles of the patient are displayed in real time in the scenes, which are convenient for the medical staff to observe and correct the patient’s rehabilitation. Thus, the patients can feel relaxed, and the effect of rehabilitation training will be more obvious. Eight shoulder joint rehabilitation game scenes are shown in Figure 6.

Figure 6.

Game scenes of the shoulder joint’s rehabilitation.

As shown in Figure 7, the rehabilitation training for the patient’s arm is mainly divided into three actions. The scenes a and b in Figure 6 are mainly aimed at the shoulder joint’s extension/flexion training. With the swing of the patient’s shoulder joint, the red fish will move around to eat the still fish. The scenes c and d in Figure 6 are mainly aimed at the shoulder joint’s abduction training. With the patient’s shoulder joint swing up and down, the red fish will move up and down to eat the still fish. The scenes e–h in Figure 6 are mainly aimed at the compound training action of the shoulder joint’s extension/flexion and the abduction. In the scenes e and f in Figure 6, the patient needs to combine the two rehabilitation exercises to control the red fish eating different static fish in different directions. In the scenes g and h in Figure 6, the patient also needs to combine two rehabilitation training actions, but the game scene difficulty upgrades. The patient needs to control the red fish through the movement of the shoulder joint to eat fish in different directions within the setting time.

Figure 7.

Analysis of the shoulder joint’s rehabilitation training scene.

The training processes of the elbow joint with VR scenes are shown in Figure 4, and the shoulder joint with VR scenes is shown in Figure 6. The detection system sampled the ROMs of the joints and transmitted the data to the controller. Patients can adjust their motion according to the display values in the scene in real time and see the scores after the rehabilitation immediately. The physiotherapist evaluates the rehabilitation training effect by the data stored in the controller and provides helpful instructions for the patient.

Rehabilitation training scene for the compound movement of the shoulder joint and the elbow joint

When the single-joint trainings of the shoulder or the elbow achieve the desired effect, it is necessary to train multiple-joint coordination for the extremities exact and flexible action. As such, the virtual rehabilitation training scenes with a 3D character model realize the objective. The scene interactive functions and programs were achieved by the scripts developed with C# programming language. The 3D human body model can follow the movement of the patient in real time; thus, the individuals can be immersed in the rehabilitation training. The completed 3D models were imported into Unity3D project file with a file format of .FBX. The joints of the human body model are displayed directly in the form of “hierarchical tree,” which is for the control of the joints by the script, and a 3D human body model is shown in Figure 8.

Figure 8.

Joint hierarchical tree and 3D human body model.

During each training session, each group repeated 25 times. The training parameters are displayed in the virtual environment, including the shoulder and elbow joint detected by the sensor in three directions, the time of completing a set of actions, and the training times completed. The system is provided to the patient who has basically recovered at home for the later rehabilitation training, and in order to simplify the system and decrease the cost, there are no biological signal feedback sensors. The patients can test their heart rate, respiratory rate, pulse, and other biological signals with their own heart rate meters or other household equipment after training by themselves.

Experiment and data analysis

Before the clinical trial, the performance evaluation has been carried out on the healthy subjects. In order to verify data detection system, the experiments are carried out to sample the ROMs of shoulder and elbow joints in rehabilitation training. And the experiments of the shoulder and elbow joint movements which drive the 2D games and 3D character model in VR scenes are completed to test the transmission interface design and the effectiveness of the rehabilitation task scenes. In order to make patients more comfortable, during training with the rehabilitation system, individuals are allowed to sit in front of the virtual scene. The two sensors and the controller are integrated as a whole and worn on the patient’s shoulder and elbow joints by bonds, respectively, as shown in Figure 9.

Figure 9.

Upper-limb rehabilitation training prototype.

Data acquisition and processing method

The rehabilitation training experiments for the elbow joint’s pronation/supination and flexion and the shoulder joint’s extension/flexion and abduction movement were carried out. The initial zero coordinate of two six-axis IMU sensors is fixed in the absolute coordinate system. If every time the patient wears a sensor, he need to turn to a particular direction in order to seek the zero position of the sensor, which will be very troublesome. Moreover, a slight change in the sensor’s position on the patient’s upper limb can cause the error of the angle. In this case, the initial value of the sensor is cleared in the programming of each game scene so that the synchronization and accuracy of the patient’s movement and the virtual scene can still be guaranteed whenever the patient wears in any direction or not accurate enough. Therefore, the preparation time is short and easy to wear for patients to finish at home.

The sample time of each angle is 100 ms to follow the patients’ action. In order to eliminate the signal noise, the moving average filtering algorithm is adopted to avoid the random interference to the measurement results. The data sequence length is 10 for the moving average filtering algorithm.

The healthy subjects are taken part in the experiments to verify the stability and accuracy of the data acquisition. Each experimenter completed four single-joint rehabilitation actions and repeated 25 times with their right arms in the experiments to complete the training tasks in VR scenes, as described in section “Overall design of the system.” The training data of the four experimenters are listed in Tables 1 and 2, in which the two sets of data sampled in each training action are listed here due to space limitations.

Table 1.

Experiment of the elbow joint rehabilitation training.

Name	Age (years)	Sex	Pronation/supination		Flexion movement
			1	2	1	2
Zhang	21	Female	−83°/88°	−85°/90°	1°/143°	−1°/144°
Song	22	Male	−84°/90°	−84°/89°	0°/145°	1°/145°
Yang	23	Female	−83°/90°	−84°/90°	−1°/144°	0°/144°
Ding	23	Male	−84°/89°	−83°/89°	1°/145°	0°/143°

Table 2.

Experiment of the shoulder joint rehabilitation training.

Name	Age (years)	Sex	extension / flexion		abduction movement
			1	2	1	2
Zhang	21	Female	−49°/178°	−50°/181°	1°/179°	1°/180°
Song	22	Male	−48°/179°	−47°/179°	−1°/178°	1°/179°
Yang	23	Female	−49°/180°	−48°/179°	0°/180°	0°/178°
Ding	23	Male	−50°/179°	−49°/178°	−1°/179°	0°/180°

As shown in Tables 1 and 2, it can be seen that the training data of the single joint reach the required range. Then, the detailed analyses are given in the following two sections.

Data analysis of the elbow joint rehabilitation training experiments

Taking the first experimenter (name Zhang, age 21 years, female) in Table 1 as an example, based on the collection of the angles of the elbow joint’s pronation/supination and flexion movement in virtual rehabilitation training, as described in the scenes a, b, c, and d, respectively in Figure 4, the relationship between the angle and the training time was drawn in Figure 10. According to the data curves in Figure 10(a) and (b), it can be seen that the actual angles of the elbow joint’s pronation/supination actions are within the range of (−83° to 0° to 88°) and the angle of the elbow joint’s flexion movement swings within the range of (1°–143°), which meet the training requirements of the range of (−85° to 0° to 90°) and (0°–145°), respectively, as mentioned in section “Design and analysis of upper-limb posture detection system.” Because the actual sample data are within the allowable error for the rehabilitation training, the training actions of the elbow joint meet the requirements of the corresponding standard angles.

Figure 10.

Curves of the movement angle and time in the rehabilitation training of the elbow joint: (a) angle curve of the pronation/supination action (experimenter 1) and (b) angle curve of the flexion action (experimenter 1).

We introduce the average value $(\bar{X})$ , the root mean square error (RMSE), and the standard deviation (S) to verify the validity and the reliability of the angle sampled by the sensors.¹⁵ The average value is calculated by equation (1) as follows

\bar{X} = \frac{\sum_{i = 1}^{n} X_{i}}{n}

(1)

where X_i is the sampled data and n is the data numbers.

The RMSE is defined by equation (2)

RMSE = \sqrt{\frac{\sum_{i = 1}^{n} {(X_{obs, i} - X_{model, i})}^{2}}{n}}

(2)

where X_obs is the sampled data, X_model is the truth-value, and n is the data numbers.

In practical measurement, the numbers of data are always limited, and the truth-value is replaced by the standard setting value as mentioned in section “Design and analysis of upper-limb posture detection system.” The RMSE is sensitive to the error in a set of measurement. Therefore, the RMSE can reflect the precision of measurement very well.

Standard deviation is the arithmetic square root of the variance, also called mean square deviation, as described in equation (3). It can reflect the discrete degree of a dataset

S = \sqrt{\frac{\sum_{i = 1}^{n} {(X_{i} - \bar{X})}^{2}}{n}}

(3)

In order to analyze the accuracy and dispersion of the joint angles sampled by the sensors, 10 sets of experimental data were randomly drawn out from the 25 sets of the first experimenter. And the number of data collected from each rehabilitation movement (supination, pronation, and flexion action) is 100. Then, the $\bar{X}$ , RMSE, and S are calculated and listed in Table 3. It can be seen that the average angle is stable within the allowable range of error. In the case of limited data acquisition, the RMSE and S are also stable within the range of (0.10–0.20), which verifies the stability and accuracy of the sensor angle outputs.

Table 3.

Statistical analysis for the elbow joint angles.

Supination action (experimenter 1)
${\bar{Y}}_{1 \min}$	−84.99°	−85.01°	−85.00°	−84.98°	−84.01°	−84.98°	−83.99°	−84.99°	−85.01°	−83.99°
$RMSE$	0.10	0.10	0.14	0.14	0.10	0.14	0.10	0.10	0.10	0.10
$S$	0.10	0.10	0.14	0.14	0.10	0.14	0.10	0.10	0.10	0.10
Pronation action (experimenter 1)
${\bar{Y}}_{1 \max}$	88.01°	89.01°	88.00°	88.99°	89.99°	89.00°	88.00°	89.99°	90.00°	89.01°
$RMSE$	0.10	0.10	0.14	0.10	0.10	0.14	0.14	0.10	0.14	0.10
$S$	0.10	0.10	0.14	0.10	0.10	0.14	0.14	0.10	0.14	0.10
Flexion action (experimenter 1)
${\bar{Z}}_{1 \max}$	144.98°	144.01°	143.99°	145.01°	144.02°	145.00°	143.98°	144.99°	143.98°	144.01°
$RMSE$	0.14	0.10	0.10	0.17	0.14	0.14	0.14	0.10	0.14	0.17
$S$	0.14	0.10	0.10	0.17	0.14	0.14	0.14	0.10	0.14	0.17

Data analysis of the shoulder joint rehabilitation training experiments

Taking the first experimenter (name Zhang, age 21 years, female) in Table 2 as an example, as shown in Figure 11(a) and (b), the curves indicate the relationship between time and movement angles when the shoulder joint is trained in the extension/flexion and abduction actions in the virtual environments which was described in scenes a, b, c, and d, respectively, in Figure 6. The actual measurement data of the extension/flexion action of the shoulder joint are within the range of (−49° to 0° to 178°) and the abduction angle is within the range of (1°–179°), which meet the training required ranges of (−50° to 0° to 180°) and (0°–180°), respectively. The arm extension/flexion and abduction angles also achieve the required ROMs of normal human, which further proves that the system is suitable for upper-limb rehabilitation training.

Figure 11.

Curves of the movement angle and time in the rehabilitation training of the shoulder joint: (a) angle curve of the extension/flexion action (experimenter 1) and (b) angle curve of the abduction action (experimenter 1).

A similar method for the elbow joint angle analysis is presented to examine the accuracy and dispersion of shoulder angles acquired by the system sensors. Taking the angles of the shoulder training of the first experimenter as the example, the statistical analysis results are listed in Table 4, which also verifies the stability and accuracy of the sensor angle outputs.

Table 4.

Statistical analysis for the shoulder joint angles.

Extension action (experimenter 1)
${\bar{Z}}_{2 \min}$	−49.98°	−49.99°	−50.01°	−50.00°	−48.98°	−48.99°	−49.00°	−48.02°	−49.99°	−49.01°
$RMSE$	0.14	0.10	0.17	0.14	0.14	0.10	0.14	0.14	0.10	0.17
$S$	0.14	0.10	0.17	0.14	0.14	0.10	0.14	0.14	0.10	0.17
Flexion action (experimenter 1)
${\bar{Z}}_{2 \max}$	180.00°	179.99°	178.99°	178.01°	179.99°	178.00°	178.99°	179.00°	180.01°	180.00°
$RMSE$	0.00	0.10	0.10	0.10	0.10	0.14	0.10	0.14	0.10	0.14
$S$	0.00	0.10	0.10	0.10	0.10	0.14	0.10	0.14	0.10	0.14
Abduction action (experimenter 1)
${\bar{X}}_{2 \max}$	179.99°	179.01°	178.99°	179.00°	180.01°	180.00°	179.99°	179.01°	178.00°	180.01°
$RMSE$	0.10	0.10	0.10	0.14	0.10	0.14	0.10	0.10	0.14	0.10
$S$	0.10	0.10	0.10	0.14	0.10	0.14	0.10	0.10	0.14	0.10

Model-driven experiment and analysis of the compound training of multi-joints

In order to prove the effect of the compound training of the multi-joints, that is, the shoulder and elbow joints, the whole system including the 3D training scenes, the two six-axis IMU sensors for data detection, the wireless Bluetooth communication module, and the control system are tested simultaneously. The right shoulder and elbow joints of the 3D character model are controlled by the data transmitted from the detection system in real time. Therefore, the 3D human body model in the scene follows up the human movement at the same time. In scripting, the “Find (GameObject)” method is used to control the right shoulder and elbow of the human model to synchronize with patient motions. The following action of the 3D character model with the real action of the human is shown in Figure 12. The two IMUs are worn on the right arm of the human. When the human arm moves from action a to action b as in the right pictures of Figure 12, the 3D character in scene follows simultaneously, as shown in the left pictures of Figure 12. And the human right arm action is the compound action, which needs the shoulder joint bending and the elbow joint spinning and lifting. It is illustrated that the 3D VR training scenes run smoothly and follow the human action through the data sampled and transmitted by the detection system.

Figure 12.

Compound training experiments: (a) impression drawing of the bending the shoulder joint and the spinning the elbow joint and (b) impression drawing of the curved shoulder joint and the lifted elbow joint.

In a word, many experiments were conducted to evaluate the performance of the system with healthy subjects. Through the above experiments, including the single-joint training and the compound training of the multi-joints, we can summarize that the training ranges of the shoulder joint and the elbow joint have reached the required ranges of the corresponding joint in the reference book of The Physiology of the Joints. During the experimental processes, it can be seen that the stability and accuracy of the training data acquisition are relatively high in the rehabilitation training. In addition, it can be transmitted to the VR scenes in real time and with visual feedback.

Conclusion and outlook

In this study, a wearable rehabilitation training system for the upper limb with VR training scenes was designed for the patients with the upper-limb hemiparesis caused by stroke, SCI, or traffic accident. The system combines VR technology with rehabilitation medicine effectively and develops with different difficult levels of training tasks for patients, so it increases the patients’ interests in the rehabilitation training. At the same time, the parameters of rehabilitation training tasks can be sampled and recorded accurately and timely in the training process, which is convenient for the medical staff or the patients himself to correct the rehabilitation training posture immediately. The overall structure of the system has the advantages of simple structure, convenient wearing, and lower cost.

Compared with the current rehabilitation devices for the upper-limb rehabilitation, it is appropriate for the majority of patients with mild upper extremity hemiparesis in a family. The system has some limitations and shortages as described below. Unlike the exoskeleton rehabilitation device, there is no actuator in our rehabilitation system, so it is only for the patients to do training exercises by their own active actions. And the system we designed at present is the training for two joints, that is, the shoulder and the elbow joints. The system will be extended with the design of the wrist and hand training tasks in the later stage in order to make the upper-limb active rehabilitation training system more perfect. We will also design a simple mechanism to provide some damping forces for the patients to overcome, in order to increase the strength of rehabilitation training gradually. Clinical trials for patients will be planned to investigate the performance of the system. And with recent technical innovations, in-home tele-therapy (IHT) system supervised over the Internet will be developed and added in the system in the next step.

Footnotes

Handling Editor: Francesco Aggogeri

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by Henan Scientific and Technological Research Program (172102210036) and Natural Science Foundation of Henan Province (162300410082).

References

Liang

Dou

Wang

. Application of virtual reality technique in rehabilitation of hemiplegic upper extremities function of stroke patients. Chinese J Rehabil Med 2013; 2: 114–118.

Kamper

. Restoration of hand function in stroke and spinal cord injury. In: Dietz

Nef

Rymer

(eds) Neurorehabilitation technology. London: Springer, 2016, pp.175–190.

Valencia

Cardoso

Frizera

et al . Serious game for post-stroke upper limb rehabilitation. In: Ibáñez

González-Vargas

Azorín

et al . (eds) Converging clinical and engineering research on neurorehabilitation II. Berlin: Springer, 2017, pp.1445–1450.

Krebs

Ferraro

Buerger

et al . Rehabilitation robotics: pilot trial of a spatial extension for MITManus. J Neuroeng Rehabil 2004; 1: 5–9.

Nef

Guidali

Riener

. ARMin III—arm therapy exoskeleton with an ergonomic shoulder actuation. Appl Bionics Biomech 2009; 6: 127–142.

Zhang

. Research of finger rehabilitation system based on virtual reality technology. Wuhan, China: Huazhong University of Science and Technology, 2012.

Mei

et al . Design and test of a portable exoskeleton elbow rehabilitation training device. Chinese J Rehabil Med 2015; 11: 1155–1157.

Lange

Requejo

Flynn

et al . The potential of virtual reality and gaming to assist successful aging with disability. Phys Med Rehabil Clin N Am 2010; 21: 339–356.

Peng

Song

. Upper limb rehabilitation robot based on motor imagery EEG. Robot 2011; 33: 307–313.

10.

Wang

. Status of application of virtual reality technique in motor rehabilitation in stroke. Chinese J Rehabil Theory Pract 2014; 10: 911–915.

11.

Mazzone

Haubert

Mulroy

et al . Intensity of shoulder muscle activation during resistive exercises performed with and without virtual reality games. In: International conference on virtual rehabilitation, Philadelphia, PA, 26–29 August 2013, pp.127–133. New York: IEEE.

12.

Fischer

Stubblefield

Kline

. Hand rehabilitation following stroke: a pilot study of assisted finger extension training in a virtual environment. Top Stroke Rehabil 2014; 14: 1–12.

13.

Kapandji

. The physiology of the joints. 6th ed. Beijing, China: People’s Military Medical Press, 2011.

14.

Tian

Chen

. Application of virtual reality training system and rehabilitation therapy in the treatment in hemiplegic patients with upper limb dysfunction. Chinese J Rehabil Med 2011; 26: 579–581.

15.

Xincheng

. Mathematical statistics. Xian, China: Northwestern Polytechnical University Press, 2002, pp.6–25.