Abstract
BACKGROUND:
Reduced coordination of precise small movements of the hand, wrist and fingers in Parkinson’s disease (PD) has been mostly solved by medications and deep brain stimulation. The effects have been evaluated by clinical tests only.
OBJECTIVE:
Virtual reality-based exergaming may enhance fine movements, decrease the medications dosage and provide an additional non-subjective evaluation.
METHODS:
3D pick-and-place task (10Cubes) has been developed in a virtual world. The person placed the virtual cubes by the virtual hand, an avatar of the real hand tracked by a Leap Motion Controller (LMC). The system computed the time of manipulating the cube, the total time, the average time, the speed, and the distance. It counted and managed the number of cubes touched, and calculated the hand shake level, i.e. the average tremor index. A pilot test was carried out in a healthy neurologically intact person and a patient with PD using a 3D head-mounted device (HMD) or LCD screen.
RESULTS:
The results indicate that substantial and also statistically significant (
CONCLUSIONS:
The evaluation system of 10Cubes has proved applicable at an unchanged medication plan, but its clinical effectiveness could be confirmed with a randomized study.
Introduction
Parkinson’s disease (PD) reduces coordination of precise movements, especially fingers, hand and wrist movements [1]. As the disease is progressive, it affects motion in general and often causes slow movements or even stiffness. The symptoms may get worse over time, the barely noticeable tremor often becomes shaking of hands or fingers. Persons with PD usually cannot control the pinch grip very well, and experience difficulties in grabbing small objects due to the tremor at rest. Additionally, slow motion (bradykinesia) also hinders the movements, and consequently most of the simple tasks become time consuming and difficult. Impairments of precise movements can be a consequence of PD, also of an essential tremor (ET), a different neurological disorder. Yu et al. [2] presented a study of using a simple drawing task to distinguish between the impaired fine motor movements resulting from ET and PD. Both groups suffer from a slower reaction time and movement velocity of a wrist flexion and extension task as shown by Montgomery et al. [3]. Individuals with ET also exhibit impairments in rapid repetitive finger movements and visual reaction time tasks [4] suggesting that patients with ET also suffer from bradykinesia.
For the time being, PD cannot be cured; therefore, the symptoms are reduced and controlled by medications and rarely a surgery, implanting electrodes for deep brain stimulation. Tremor can be efficiently controlled by deep brain stimulation [5]. Other treatment includes nutrition, daily living activities and exercising. Exergaming also seems to be feasible for PD, but only if games/exercises are specifically tailored [6]. Although there is a lack of evidence that exergaming can be used as a rehabilitation tool, the studies [7] reported that physiotherapy or controlled physical exercise of upper extremities had positive effects on physical capacities in 67%, motor control in 52%, a range of motion in 47%, and activities of daily living 59%. Positive effects (71%) in clinical symptoms were also reported for the Unified Parkinson’s Disease Rating Scale (UPDRS) [8] part III in a review [7]. Mateos-Toset [9] reported on the improvement of manual dexterity and strength in persons with PD exercising single-handedly. Hand and finger forces were reported [10] significantly lower in patients with PD than the healthy control group. This study also demonstrated that elusive changes in motor coordination in patients at the early stage of PD could be objectively evaluated. Additionally, kinematics and propulsion of hand and eye movements can be tracked [11]. Various equipment like accelerometer and gyroscope-based devices (inertial measurement systems) and strain-gauges were used to assess hand movement. Laboratory measurements include 3D marker-based optical measurements (Optotrak, NDI, Ontario, Canada or Vicon, Vicon Motion Systems Ltd UK) that can provide accurate measurements, but often their clinical applicability is low. Outpatient’s setting may require an easy-to-use and, an easy to don/doff equipment with measuring equipment integrated in PowerGlove [12] that can measure kinematics and the position of the 3D hand and fingers.
Furthermore, conventional physiotherapeutic approaches have significantly changed in the past two decades, and virtual reality technology has become a new rehabilitation tool with an added value [13]. Although there have been numerous authors [13] reporting on effectiveness, large-scale randomized control trials are needed to provide clinical evidence. Our clinical experience and observation suggests that VR technology enables difficulty level control and easy-to-change targeted tasks, and can also be used for the evaluation of movements. However, the PD population is rather unsatisfied with wearing additional equipment, therefore our preference was a camera-based kinematic tracking (Microsoft Kinect
To develop a specific interactive environment that enables kinematic tracking. To find and test objective parameters for the evaluation of neuro-motor deficiencies.
We designed a case-control study to identify whether the developed objective evaluation parameters can distinguish the differences in treatments and would be suitable for a future prospective cohort study or randomized pilot study.
The virtual reality (VR) supported game was designed to increase and control the intensity, difficulty and data of pick-and-place tasks in persons with upper extremity motor disorders. The idea for the game was based on the validated Box & Blocks clinical test [14]. Ten virtual cubes of the same size, virtual weight and material with bouncing ability (Rigidbody, BoxCollider) were spread on the virtual lawn. The size of the virtual cubes was approximately the same as that of the real cubes used in occupational therapy, and enabled a comfortable pinch. Invisible virtual walls were implemented around the center court to keep the cubes inside the playing area. Each cube had a unique color and an identifier number to enable tracking. The user was expected to pick the cubes with a virtual hand by 2- or 3-finger pinch, one by one, and place them in an open chest (Fig. 1). The virtual hand was a computer model of a real hand. The cube could be dropped or thrown, but could bounce back or away from the chest, thus making the task moderately difficult. For the purpose of testing, only one difficulty level was considered without an option to change any characteristics of the cubes. The time foreseen to finish the entire task was 120 s. The task required visual motor integration, an ability to combine visual and hand coordination [15].
10Cubes exergame for training and evaluation of precise movements with upper extremities.
The position and orientation of the hand (palm and fingers) were recorded and transformed into the base (virtual world) coordinate system: RxO
The tracked kinematics targeted major motor symptoms that are also well known in persons with PD; e.g. tremor is the most common motor type symptom and appears in the early stage of the disease. The tremor noticeable in hands is often present in hibernation or resting, and occasionally disappears during movement or sleeping. The tremor frequency ranges from 3 to 6 Hz [16]. We focused on movements, game efficiency and tremor assessment, and evaluated these parameters through the kinematics.
The data were stored as .csv text files during the game sessions, and imported into the Matlab (MathWorks, Natick, MA, USA) environment. The following parameters were calculated: the number of trials and successfully inserted cubes, a total and an average time of the trial, an average velocity and a distance of manipulation, and an average tremor indicator.
Number of trials/cubes moved
The number of trials (N) was identified from the cube ID (each cube has a unique identifier); how many times in a row does the same cube ID for an individual trial appear in the stored data (Table 1)? For example, the user held a specific cube (grabbedCubeID 8) for 2–5 s, which was recorded in the data, followed by the data entry grabbedCubeID 0, meaning that no cubes were held. If the next data entry is the cube ID 8, it means that the user held the same cube again. Such action was considered as an additional trial. If the user picked another cube (e.g. ID 4), each movement of the cubes was considered as another trial. The number of trials was a sum of all partial attempts to move the cubes. If a cube was successfully placed in the chest, an additional record was made in the stored data. The sum of successfully placed cubes in the chest in the last entered data (NrOfInsertedCubes) presented the number of cubes moved.
Data record. The user previously placed cube nr 4 into the chest, then picked cube nr 8 and lost it on the way (grabbedCubeID 0). Then she finally placed cube nr 8 in the chest and raised the counter NrOfInsertedCubes to 2
Data record. The user previously placed cube nr 4 into the chest, then picked cube nr 8 and lost it on the way (grabbedCubeID 0). Then she finally placed cube nr 8 in the chest and raised the counter NrOfInsertedCubes to 2
The traces of the 10 cubes when held by the person (lines of various styles and colors), and trajectories of the empty hand (solid blue line). The base coordinate system (O
The total time of a cube manipulation was calculated by a C# function using the stored data. The function calculated for how long the user held a specific cube (grabbedCubeID). If the user meanwhile released the cube (grabbedCubeID changed), the calculation for that specific cube continued later. The algorithm checked whether the grabbedCubeID was equal to the analyzed cube ID, and calculated the manipulation time between the last and the first touch of the cube. The total cube manipulation time was a sum of all the 10 cubes’ manipulation times.
The average cube manipulation time was calculated by dividing the total manipulation time by the number of trials.
The total time of the trial was defined by the time from the first touch of a cube until the last cube was placed in the chest unless the user finished the task. Then, the total time was equal to the upper limit, i.e. 120 s.
Average velocity of manipulation
The average velocity of manipulation was calculated as a sum of average cube manipulation times divided by the number of trials.
Average distance of manipulation
The average distance of manipulation was a sum of all cube moves divided by the number of trials. The sum of all cube moves was defined by the algorithm that consecutively calculated the distances (Fig. 3C) made by each cube and each trial (Fig. 2). These distances were a sum of small Euclidian distances between the two consecutive positions of the hand in the time series data. The average shortest distance of manipulation was defined as the average of all Euclidian distances between the picking and the dropping point of the cube (Fig. 3B). The average longer distance of manipulation defined how much longer the actual average distance was than the average shortest distance of manipulation.
The ways of moving the cube into the open chest: B – straight, ideal and the shortest distance line, C – the most convenient trajectory for the pick-and-place task taken by the participants, and A – the movement of the cube when tremor is present. Each significant change of the direction adds 1 point to the tremor indicator index.
Information about the tremor required an accurate hand motion trajectory analysis during the cube manipulation in 3D space. Tremor could be defined as shaking of the hand with an increased frequency around the selected trajectory (Fig. 3A). We assumed that the fluctuation of the hand position in directions x, y and z was irregular, uneven. Each sudden change of the direction of movement increased the parameter by 1 for the specific direction. Ideally, the user would carry the cube straightforward taking the shortest way and no sudden changes would appear. In such case, the tremor indicator parameter would have a value of 3 (1 for each axis) as the position of the hand would gradually increase or decrease. The obtained tremor indicator parameter was then divided by the number of axes, which is 3 (for x, y, z), providing an ideal overall value of the average tremor indicator (AVTI) parameter 1. Furthermore, the average tremor indicator was defined by the sum of all parameters calculated for each cube divided by the number of trials N.
Figure 3 demonstrates three ways of moving the cube into the open chest. The shortest path from the pick-up place to the release place may be path B, but most of the users would take path C and lift the cube higher to be on the safe side, and would therefore move the cube along an arc. In such cases, the tremor indicator would be equal to 1 as only a single change of the direction was required. However, if the user demonstrated a hand tremor and changed the direction of movement several times (A), then the tremor indicator would rise, e.g. to 33. The tremor index with values
Experiment
Method
Participants
So far, the method has been tested on 2 people participating voluntarily in the study; a healthy individual IC (43y, male, right handed) without any neuromotor disorders and normal visual capabilities, and a participant RT with PD for 5 years (49y, female, right handed), Mini-Mental Status Exam score
The study was approved by a local ethics committee, and the participants provided informed written consent.
Materials
The 10Cubes
Procedure
The neurologically intact participant (IC) was seated in a comfortable chair, and the LMC was placed on the desk in front of him. He exercised with the 10Cubes game in the TV mode. Participant RT with Parkinson’s disease performed an identical task in the VR mode, also in the seating position.
Both participants took 10 consecutive sessions in 14 days. Within each session they performed 5 virtual pick-and-place exercises. Each trial lasted up to 120 s unless the participants managed to finish the task earlier. The participants raised their dominant/more affected hand above the virtual cubes and started to pick and place the cubes in the open chest one by one. The task was considered successfully accomplished when the participant put all the 10 cubes in the wooden chest.
The participant with PD additionally performed the Box and Blocks clinical test [14], and the UPDRS [8] test prior to and after the sessions.
Results and discussion
Two case studies were analyzed for the above presented kinematic data; the total time of manipulation, the average time of manipulation, the number of inserted cubes and the total number of attempts, the average tremor indicator, the average speed and the distance of manipulation. The assessed data were averaged for each day for both cases, and the two types of visual feedback modes were compared. More insight into the details between the two approaches and the difference between the two cases was revealed by a two-way analysis of variances (ANOVA). Preliminary data were tested for homogeneity of mean variances with the Levene’s test [18]. If the null hypothesis is rejected, we apply a two-way Friedman’s non-parametric test. Additionally, we also carried out multiple comparison of group means to see if the data of participant RT is much different than the healthy participant’s data. A significance level of 0.05 was used for statistical testing. We used Matlab (MathWorks, Natick, MA, USA), and the Matlab Statistical Toolbox (MathWorks, Natick, MA, USA) for data processing.
Kinematic analysis
Figure 4 presents the outcomes of the healthy neuromuscularly intact person that were rather predictable; the person had finished the task by putting all the 10 virtual cubes into the chest before the time elapsed. Mostly 11 or even 12 trials were needed, but fewer with the 3D HMD (IC VR). The average time of manipulating a single cube varied between 0.6 s and 1.1 s, slightly less with the 3D equipment. However, the total time of manipulation ranged between 5 and 12 s. The top speed of manipulating an individual cube was 325 cm/s, while the average speed was 267 cm/s. Consequently, the average performed distance was around 200 cm. Differences between the trials with different equipment were minimal in average distance or average longer distance parameters. Nevertheless, the average tremor indicator was rather low (
Analysis of kinematic parameters for the healthy neurologically intact participant for 10 days, and two types of equipment (TV – LCD screen, VR – Oculus Rift 3D HMD).
Analysis of kinematic parameters for the patient with Parkinson’s disease for 10 days, and two types of equipment (TV – LCD screen, VR – Oculus Rift 3D HMD).
The objective evaluation parameters obtained from the kinematic data of participant RT with Parkinson’s disease (Fig. 5) demonstrated significantly different values than the data of the neurologically intact person (Fig. 4). The total time of manipulation was over 30 s, or 20 s with the 3D equipment, and the average time of manipulation was even twice as long as with the neurologically intact person. The average speed of performance was evidently lower, but higher with the 3D equipment, while maintaining the same amount of tremor (average tremor indicator 10–20). Surprisingly, participant RT failed to finish the task in the designated time only three times, and not even once when using the 3D equipment (RT VR). Participant RT needed an additional trial only twice (in the 3
Both participants started the task with a self-selected speed; the healthy neurologically intact participant almost twice as fast as the participant with PD. They were gradually increasing the speed from session to session until they realized that the faster they moved, the more likely they would lose the cube on the way and would have to try again. However, in the last two sessions, they were already fast enough to spend less time for manipulation even with additional attempts, which was particularly true for the participant with PD. On the other hand, their performance while using the 3D HMD was even superior with fewer additional attempts. Despite the person’s good performance, the average tremor index was significantly (5x) smaller in the neurologically intact person, indicating that the parameter has a potential for detecting neuromuscular diseases. Besides, the tremor index was very similar at the end of the sessions regardless of the feedback equipment used in the trials. Another kinematic parameter that should get more attention was the average time of manipulation – such a parameter may significantly change with the improvement of neuromotor control or in progressive diseases. Similar results, but with the haptic device Phantom (SensAble Technologies, USA), were presented by Snider et al. [19]. They found reduced peak speed and impaired motor coordination in PD patients on/off medication. Furthermore, dopaminergic therapy increased the peak speed, but no improvement of motor coordination was shown. The patients still had problems with precise coordination, which we noticed through the tremor indicator.
The results indicate that the choice of the feedback equipment, an LCD screen or a 3D HMD, should not play a crucial role in neuromotor rehabilitation. We have found statistically significant differences in certain kinematic parameters that may also appear in the studies with larger groups. However, it is less likely that the neuromuscular injured or persons with a neuromuscular disease would easily adopt and accept an additional head mounted device unless the rehabilitation outcomes are really superior.
Considering the results from Figs 4 and 5 we could conclude that the application of 3D equipment would help both persons to improve their results. The differences between the subjects suggest that the kinematic parameters may be applicable for rehabilitation progress evaluation as well as for the prediction of indications for neuromuscular diseases or disorders. The data were tested for equality of variances with the Levene’s test. The data for both cases and both gear types confirmed the hypothesis for the average speed (Levene’s test
Statistical differences between the two cases and the types of visual feedback
Physiotherapy could play an important role in PD patients [22]. Besides gait and balance it also improves small-scale movements of upper extremities, e.g. writing [23]. The occupational and physiotherapeutic activities and their outcomes are often evaluated by functional clinical tests, such as Jebsen test [24] for hand dexterity or a precision grip-and-lift task [25]. The latter reasonably well correlates with the UPDRS [8]. However, these tests require qualified and trained experts to carry out the assessments. Nevertheless, participant RT achieved 73 and 20 points at the Box and Blocks [14] and the UPDRS III tests, respectively, before the treatment, and 79 and 18 points after the treatment. When the results are being compared to large databases and evaluated, one must also consider that the data are subjectively assessed by different experts not taking into account that their assessment may even vary with daily wellbeing. The objective computerized assessment during the therapy with modern technologies shall not reduce the value of the clinical tests, but rather demonstrate correlation and thus present a valuable modern tool for the monitoring of rehabilitation or disease progress with specific parameters; e.g. tremor indicator, speed at the specific successful rate (number of trials vs inserted cubes). This data would most likely correlate [26] with the clinical tests, e.g. Box and Blocks [27, 28], and the intermediate results would be the most helpful information for dosing medications and planning further treatments. In persons with ET the tremor may go away on its own, but this is not the case for persons with PD, where dosing of medications is important for the control of the tremor. Nevertheless, resistance to medications may cause an ineffective treatment. The proposed system has been designed to evaluate the progress of persons with PD, and with the current design would not be able to differentiate between ET and PD. The system could be upgraded with artificial intelligence algorithms and by implementing the knowledge of graphical tasks [2] into extended virtual reality exercises. However, patients always come to physiotherapy with an already diagnosed condition.
Limitations
The kinematic parameters were assessed in controlled laboratory settings. The game was designed intentionally as a pick-and-place task, where finger dexterity was also important. The 3D equipment and the LMC were calibrated, and the reference did not change the position. However, the data of the participant with PD were analyzed offline, but assessed under the same conditions as for the neurologically intact person. Obviously, the small amount of data cannot provide power to the statistical analysis, but a case-control study can present a selection of kinematic parameters for a larger-scale controlled study. Nevertheless, we cannot state with confidence whether the proposed method can be an effective rehabilitation tool until larger randomized trials in the clinical settings are carried out.
Conclusion
The 10Cubes interactive game uses Leap Motion Controller and analyzes kinematic parameters, of which the tremor indicator and the average time of manipulation of a single cube may present future parameters for the evaluation of neurophysiologic motor control. We followed the recommendations for game design for PD that suggested easier games than commercial, with no negative feedback, clear goals and low cognitive demands [6]. We also found significant differences in all assessed kinematic parameters between the neurologically intact healthy person and the participant with PD, but cannot confirm that the 3D visual feedback demonstrated substantial advantages over the feedback on the conventional LCD.
Footnotes
Acknowledgments
The authors would like to thank Jan Vidic, M.Sc. for technical support. The authors also acknowledge the financial support from the Slovenian Research Agency (research core funding No. P2-0228), and the European Regional Development Fund (EkoSmart). The authors also acknowledge that the project (ID J2-7357) was in part financially supported by the Slovenian Research Agency.
Conflict of interest
The authors declare that they have no competing interests.
