Determination of Repeatability of Kinect Sensor

Introduction

Repeatability is the degree of agreement between measurements conducted by the same observer. Repeatability is one of the major issues when developing new devices. The recent availability of a low-cost single markerless camera system (MLS) for three-dimensional vision provides opportunities for low-cost applications in biomechanics and revalidation sciences. The currently available Kinect™ (Microsoft™, Redmond, WA) software provides a very simple skeleton (a stick figure model composed of 20 points) and has sufficient accuracy to track movements when gaming. Several studies have analyzed the precision of the Kinect, ^1,2 although no information can be found about interday repeatability of measurements performed with this kind of device. Marker-based systems (MBSs) are commonly used in clinics. ³ The precision and reproducibility of an MBS are highly sensitive to marker placement ⁴ : if markers are not placed in exactly the same place as landmarks, there will be errors. ⁵

The aim of this work was to evaluate the repeatability of a low-cost MLS (fully automated method based on advanced pattern recognition methods ⁶ ) in comparison with the gold standard MBS (requiring manual palpation to set markers on the subject prior to motion analysis) to evaluate if this type of device can be used for patients' evaluation and follow-up during rehabilitation. In a previous study, the accuracy and repeatability of manual palpation for the shoulder complex were evaluated; results were 4 mm for repeatability (measurements performed by the same observers during two sessions) and 8 mm for interobserver measurements. ⁷

Materials and Methods

Forty healthy adults (25±7 years old; 174±8 cm tall; weight, 70±11 kg; body mass index, 23±3 kg/m²; 16 women) were recruited to participate in this study. This study was approved (protocol approval number CCB B406201111989) by the Ethics Committee of Erasme Hospital (Brussels, Belgium), and written informed consent was obtained from all subjects prior to participation in the study.

The MLS used was the Kinect camera (3D sensor from PrimeSense, Tel Aviv, Israel). A skeleton model (stick figure) to estimate segment lengths was obtained using Microsoft's Software Development Kit Kinect software. The camera was placed on a tripod at 1.5 m from the ground. No calibration was required for the Kinect camera.

In order to compare repeatability, a stereophotogrammetric system (8 MXT40s cameras, Vicon Nexus software; Vicon, Oxford, United Kingdom) was used (MBS). Subjects were equipped with a set of 31 markers in order to reconstruct a skeleton as similar as possible to the one obtained from MLS. Palpation and marker placement were always performed by the same trained physiotherapist.

Subjects were placed at a distance of 2 m from the camera; this distance is recommended for the best use of the device (at least for gaming purposes). ⁸ Subjects were asked to remain motionless in front of the camera. Data were recorded (5 s) simultaneously with MLS and MBS.

To assess intrasession repeatability, five trials were recorded during one session. The same protocol was repeated after 1 week to assess intersession repeatability.

Segment lengths were computed as follows: arm, distance between the shoulder center and elbow center; forearm, distance between the elbow and wrist center; hand, distance between the wrist center and head of the third metacarpus; thigh, distance between the hip and knee center; shank, distance between the knee and ankle center; foot distance between the ankle center and head of the second metatarsus; and total height.

The intraclass correlations coefficient (ICC) values were computed to assess intra- and intersession repeatability for both devices. The coefficient of variation (CV) (=[standard deviation/mean]×100) was also computed.

Results and Discussion

ICC results for intersession repeatability are presented in Table 1, ICC for intrasession repeatability in Table 2, and CV for sessions 1 and 2 in Table 3.

For both devices, all ICC values for intersession repeatability are higher than 0.8 (mean ICC, 0.88 and 0.87 for MLS and MBS, respectively), indicating an almost perfect agreement between measurements performed during sessions 1 and 2.

For trial to trial repeatability (five trials), all ICC values were higher than 0.89 (mean ICC for sessions 1 and 2, 0.94) for MLS and higher than 0.95 (mean ICC for sessions 1 and 2, 0.98) for MBS.

Concerning CV, the mean CV was 8.9% and 9.5% for MLS during sessions 1 and 2, respectively. For MBS, the mean CV was 7% and 7.4% during sessions 1 and 2 respectively.

Because of high accuracy with respect to the position of the markers in three-dimensional space, ⁹ intrasession repeatability is almost perfect for the MBS (mean ICC, 0.98). However, the MLS also shows a very high intrasession repeatability (mean ICC, 0.94).

Because MBSs are highly sensitive to marker placement, ^4,5,7 intersession repeatability was of the most interest when comparing repeatability of both devices. ICC values were the same (0.88 for MLS and 0.87 for MBS) for both devices.

CVs were higher for MLS; these differences are due to relatively poor results obtained for the hand and thigh.

Conclusions

Both devices demonstrate an excellent repeatability. Results of this study suggest that a cheap, easy-to-use, and portable single markerless camera offers the same repeatability during posture assessment as an expensive, time-consuming, and nontransportable marker-based device. This study underlines the fact that results of measurements obtained with Kinect sensors are reliable. Future studies are still needed to further assess accuracy and to have a better understanding of the future possibility of this kind of device in the medical field. ^1,2 This device could open up new perspectives in motion assessment and patient follow-up during rehabilitation exercises performed at home. A running study is currently assessing the MLS reproducibility (i.e., the degree of agreement between measurements conducted by different observers).