Robotic Light Touch Assists Human Balance Control During Maximum Forward Reaching

Participants

We tested 10 healthy young adults (average age = 28.5, SD 3.35 years, 3 females and 7 males) as CR performing a MFR task. Participants were not affected by any neurological or orthopedic indications. Participants were recruited as an opportunity sample from students of the university. The study was approved by the ethical committee of the medical faculty of the TU Munich and all participants gave written informed consent.

Equipment and Experimental Procedure

CRs stood blindfolded on a force plate (Bertec 4060, Columbus, OH, USA; 500 Hz) in normal bipedal stance (lateral distance between feet was 24 cm) performing the MFR task. CR’s body sway was determined in terms of the anteroposterior (AP) and mediolateral (ML) components of the Center of Pressure (CoP), as derived from the six components of the ground reaction forces and moments. Before the start of a trial, CRs stood in a relaxed manner, the right arm extended at shoulder height to reach horizontally above a height-adjusted table. The table provided emergency mechanical support in case of a balance loss but, apart from that, touching the tabletop’s surface was not allowed. It also served as a lower boundary constraint keeping participants’ hand movements on the same height and preventing drastic changes in the postural strategy, such as increased knee bend, to better enable contact provision in the rIPT conditions. Any explicit instructions for a specific movement strategy were not provided, but CRs had to remain static for at least 5 s (baseline) until an auditory signal cued the start of an MFR trial and then to reach quickly but safely as far forward as possible by bending the torso (Figure 1a).

OPEN IN VIEWER

Figure 1

Experimental setup. (a) Execution of the maximum forward reach task with human interpersonal touch (hIPT) support. (b) Robotic IPT without mechanical coupling in hybrid force-position control.

One healthy adult, male CP applied light IPT to the CR’s ulnar side of the wrist with the right extended arm. During IPT, the CP stood in bipedal stance between the CR’s force plate and the table, parallel to the reaching direction facing the CR orthogonally. The CP kept the eyes open to receive visual cues of the CR’s motion as would the robotic systems by optical motion tracking and the CP did wear a thin rubber glove to provide a tactile sensation for the CR similar to rIPT where the end effector of the robot had a rubber surface.

In a pilot experiment, 12 participants were tested in a similar experimental setup but with a force-torque transducer (ATI Nano 17, Apex, NC, USA; 500 Hz) embedded in a wrist bracelet of the extended arm. It was used to acquire the forces and moment in three directions at the contact location during hIPT. Force recordings indicated an average absolute normal interaction force of .15 N (SD 0.14) between the CR and the CP, which is lower than the .3 N applied in the rIPT conditions. By the CP being required to grasp a rod mounted onto the force-torque transducer, the wrist bracelet created an unnatural interpersonal link so that it was not used in the hIPT condition of the present study.

During the robotic IPT conditions, a single KUKA LWR4 +manipulator (Augsburg, Germany) served as the CP. The CR’s wrist was tracked by the end effector of the robotic system without any mechanical coupling keeping the relative orthogonal distance constant (Figure 1b). The robotic system provided contact via a hemispherical rubber pad attached to a force sensor (OptoForce 3D OMD, OnRobot. Odense, Denmark; 500 Hz) at the end of an “artificial finger.” The CR’s wrist position was tracked by an optoelectronic motion capture system (OptiTrack, NaturalPoint, Corvallis, OR, USA; 100 Hz) by placing three reflective markers on the CR’s right hand (one on the caput ulnae/processus styloideus radii/basis, and two on the ossa metacarpi). The robotic control scheme required high control frequencies to avoid unstable behaviors (Siciliano et al., 2009) and therefore was also controlled at 500 Hz. Hence, motion-tracking data were up-sampled in real time to match the robot control frequency.

Three modes of IPT provision were contrasted: hIPT, rIPT with reactively following the participant’s movements (rIPTfollow), rIPT with anticipation of the participant’s movements (rIPTanticip). The three IPT conditions were assessed in blocks of five trials. The order of the blocked conditions was fully randomized, and each single trial lasted 20 s. Out of a total of 200 trials, 13 trials failed to track the CR’s hand and are excluded from the analysis.

Data Reduction

All data post-processing was conducted in Matlab (2016b) (Mathworks, Natick, MA, USA). Kinematic and force-torque sensor data were spline-interpolated to 500 Hz and subsequently merged with the force plate recordings. The data were smoothed using a generic dual-pass, 4th order Butterworth low-pass filter with a cut-off frequency of 10 Hz. CoP and marker data were differentiated to yield velocity. Each trial was segmented into three phases of the MFR (baseline phase, reaching phase, and MFR end-state; Figure 2 ) based on the AP position of the CR’s wrist marker. Reach onset was determined as the first frame that exceeded four standard deviations of AP wrist position within the initial 3 s baseline. Stop of forward reaching was determined as the velocity zero-crossing closest to 95% of the absolute maximum reach distance. For the description of the motion dynamics in the MFR end-state, time series data until the end of a trial was used (>10 s).

OPEN IN VIEWER

Figure 2

Typical profiles of kinematic and dynamic variables illustrated by data of a single trial for a single participant. Maximum forward reaching (MFR) of the hand marker divided into three phases and the corresponding Centre-of-Pressure (CoP) position, CoP velocity (dCoP), and horizontal shear force in the anteroposterior (AP) direction. Especially, dCoP and the shear force panels well demonstrate the balance challenge imposed by holding a static but unstable posture in the MFR end-state.

Several performance measures were selected to characterize participants’ movement patterns. Reaching performance was analyzed in the horizontal plane only with two parameters: amplitude and directional angle. Maximum amplitude was determined as the difference between the wrist’s average position in the baseline phase and in the MFR end-state. As additional characteristics, curvature in terms of the normalized path length (path length/amplitude), the average and standard deviation of reaching velocity were extracted. In order to quantify the efficiency of balance control, we determined the horizontal CoP amplitude and variability in the MFR end-state as well as calculated the standard deviation of CoP velocity (SD dCoP) as a variability measure for both directions in each reaching phase, as velocity information is predominant for body sway control (Delignières et al., 2011, Jeka et al., 2004; Masani et al., 2003, 2014).

In order to evaluate a potential speed-accuracy tradeoff, we calculated an Index of Performance (IoP) for the control of CoP in the AP direction based on a modification of Fitts and Peterson’s IoP (Bootsma et al., 2004; Fitts & Peterson, 1964). Duarte et al. (Danion et al., 1999; Duarte & Freitas, 2005) applied Fitts’ law to the balance domain. The unit of the IoP is bit/s (bitrate) and expresses the informational “throughput” of a participant during the movement. An increased IoP resembles greater processing “bandwidth.” The IoP was derived from the Index of Difficulty (IoD) over movement time (MT). The IoD is equal to the base 2 logarithm of double the CoP amplitude (ACoP) over the effective dispersion of CoP (WCoP) in the MFR end-state. The averaged standard deviation of CoP position in the AP direction was used as a measure of CoP dispersion. An increased IoD would indicate greater amplitude for a given CoP variability or reduced CoP variability at a given amplitude.

I n d e x o f D i f f i c u l t y ( I o D ) = log ⁡ 2 ( 2 ∗ A C o P W C o P )

I n d e x o f P e r f o r m a n c e ( I o P ) = I o D M T

Statistical Analysis

The statistical analysis was conducted in R 3.6.1 (RStudio v1.1.456). All performance parameters were log-linearized before statistical analysis to approximate normal distribution. A linear mixed model with IPT condition as four-leveled within-subject factor including participant as random effect was applied using maximum likelihood estimation (lmer function of the lme4 package v1.1–21). For each performance parameter, an α level of .05 was used to test for statistical significance of the main effect of IPT condition. In case that error probability fell below the alpha level, three additional post-hoc comparisons were computed (Helmert contrasts): (1) contrasting the two robotic IPT conditions (rIPTanticip vs. rIPTfollow), (2) contrasting human against robotic IPT (hIPT vs. both rIPT combined), (3) and all IPT combined against No IPT. We applied a corrected α level of .017 to evaluate the statistical significance of the individual contrasts. Effect sizes (d) for mixed-effects models were calculated for the pairwise comparisons (Brysbaert & Stevens, 2018; Westfall et al., 2014).

Robotic Control

Both the robot end-effector position and the interaction force were actively controlled using a hybrid force-position controller based on the prediction of the CR’s wrist motion. A Linear Kalman Filter (LKF; Kalman, 1960) with a constant velocity model was exploited to generate a reference for the participant’s wrist trajectory. A constant velocity LKF assumes that the motion is generated by the discrete linear system

s ( t + 1 ) = [ p K F ( t + 1 ) v K F ( t + 1 ) ] = [ I Δ t I 0 I ] [ p K F ( t ) v K F ( t ) ] + η = F s ( t ) + η ,

where the state vector s ( t ) contains the Kalman-estimated wrist position p K F ( t ) and velocity v K F ( t ) , I is an identity matrix, Δ t is the sampling time, and η is an additive Gaussian noise. The LKF predicts the next state s K F ( t ) = [ p K F ( t ) T v K F ( t ) T ] T = F s ( t − 1 ) + y ( t ) , where the correction term y ( t ) is computed as in Kalman (1960) and it depends on the measured wrist position. In our setup, the correction term was set to y ( t ) = 0 until a new measure of the wrist position was available. In this way, the predicted position p K F ( t ) controlled the robotic system and realized the two different robotic modes. To implement the rIPTfollow mode, the position p K F ( t ) (position error: AP – .010218 m, ML – .004994 m; Figure 3b) predicted by the LFK at the actual time instant t was used to generate the control command described above. In this way, the robotic system followed the wrist position with one sample delay (10 ms). To generate the rIPTanticip mode, the LKF was exploited to make a one step prediction of the wrist position. In particular, the predicted future position p K F ( t + 1 ) = F P K F ( t ) (position error: AP – 0.012256, ML – .007164 m; Figure 3a) was used to generate the control command. In this way, the robot was anticipating the human motion by one sample (10 ms), thereby leading the movement execution.

OPEN IN VIEWER

Figure 3

Kalman filtered hand position during maximum forward reaching (MFR). (a) Predicted and measured hand position during MFR for anticipatory robotic interpersonal touch (rIPT) in the anteroposterior (AP) direction. (b) Estimated and measured hand position during MFR for rIPT in follower mode in the AP direction.

The robotic system was controlled to exert a maximum of 1 N force along the ML and vertical directions (force-controlled directions), while tracking the hand motion along the AP axis (position-controlled direction). The force f m = [ f m , x f m , y f m , z ] T measured at the contact point and the CR’s Kalman-estimated wrist position p K F = [ p K F , x p K F , y p K F , z ] T were used to define the desired position of the robot end-effector as p x = p K F , x + k f ( f m , x − f d e s ) and p z = p K F , z + k f ( f m , z − f d e s ) . The desired contact force f d e s was set to .3 N and the gain k_f was set to .00004 m/N, thus regulating the robot motion at the speed of 2.5 mm/s for f m , i – f d e s = 1 N at the update cycle. For the AP direction, the desired robot position was p y = p K F , y . Roughly speaking, the presented controller was adding a delta of position k_f ( f m – f d e s ) to ML and vertical directions if the measured force was different than f d e s = .3 N. If the measured force was larger than .3 N, the delta of position was negative and the robot moved slightly back to reduce the force. If the measured force was smaller than .3 N, the delta of the position was positive and the robot pushed slightly against the CR’s wrist to remain in contact. In this way, the end-effector kept in contact with the user’s wrist while maintaining low interaction forces. The forces were not different between the two rIPT modes. As expected, the average contact force was only slightly higher than the prespecified value of .3N (mean force = .32 N, SD 0.05; rIPTfollow: mean = .31, SD 0.04; rIPTanticip: mean = .32 N, SD 0.05).

Reaching Performance and Body Sway

Augmentation of perceived self-motion relative to the environment by light, mechanically nonsupportive tactile contact with an earth-fixed reference improves body equilibrium and postural control (Holden et al., 1994; Jeka & Lackner, 1994). Touch can also be utilized to stabilize body sway when the tactile contact is received passively (Krishnamoorthy et al., 2002; Rogers et al., 2001). Lightly touching an oscillating contact shows strong effects in terms of body sway entrainment and can be used to drive an individual’s body sway in quiet standing (Jeka et al., 1997, 1998; Verite et al., 2013; Wing et al., 2011). Therefore, it may be possible that even in a more dynamic postural context such as the MFR task in the present study, motion of the light touch contact “attracts” swaying motion of the body. For example, subtle forward motion of the contact could be wrongly interpreted as backward sway so that a forward adjustment would follow the contact’s lead. This effect could have been more pronounced in the rIPTanticip than the rIPTfollow condition. Although, we have not found any evidence to support this assumption.

An increased MFR amplitude would demonstrate improved confidence in the ability of keeping own body balance stable while approaching one’s forward limits of stability (Duncan et al., 1990; Maki & McIlroy, 2006). As we did not observe any difference in reaching amplitude between all four conditions, it also means that IPT provided by a robotic system or a human did neither disrupt nor distract the human CR. This observation corresponds to the previous study, in which hIPT also did not affect reaching distance (Steinl & Johannsen, 2017). On the other hand, a general reduction in MFR velocity and its variability was an obvious change in their behavior when IPT was provided by the human partner or the robotic system. As body sway was reduced in these situations too, these adjustments could reflect a trade-off between speed and accuracy (Fitts, 1954). Participants may have effectively controlled sway variability more carefully to fulfill the task goal of MFR with IPT support in the face of either “hardware” constraints imposed by technical limitations of the robotic system or social constraints imposed by the human partner (Bardy et al., 1999; Scholz & Schöner, 1999). The fact that the IoP indicated narrower informational throughput during the reaching movement in the three IPT conditions compared to No IPT, however, could mean that all IPT conditions were burdened with an additional processing load. Possibly due to a shift in participants from less to more reactive, feedback-dependent postural control, CRs increased their movement time to adjust their motion more precisely to the current position of the robotic end-effector or the human partner and/or to allow the same to stay in better contact with their own wrist.

Human–Robotic Movement Coordination

Haptic interactions between caregiver and patient play an prominent role in cooperative and collaborative human-human sensorimotor interactions in physical rehabilitation (Sawers & Ting, 2014). More recently, Haarman et al. (Haarman et al., 2017) investigated the balance-assistive forces applied by therapists to the pelvis of patients during gait training. Using force-torques sensors, they quantified the predominant corrective forces applied by the therapists in the mediolateral direction to both sides of the hips at about 9N, amounting to approximately 2% of participants’ body weight. Compared to the forces imposed by the robotic systems in our current study, the forces applied by the therapists are still by magnitudes greater.

In a cooperative physical human–human interactions, the relationship between interaction forces and movement kinematics is important for communicating intended movement direction (Mojtahedi et al., 2017; Sawers et al., 2017; Takagi et al., 2018). Gentry and Murray-Smith (2003) described the influence of haptic signals used for coordination and synchronization in human dancing. Hoelldampf et al., 2010 used interaction forces to adjust and optimize the robot’s motion in a system designed for human–robot interactive dancing. Similarly, Chen et al. (2015, 2017) developed a mobile robotic system responsive to interaction forces to practice dance stepping with a human partner. Response gain and compliance of the robot’s effectors altered human upper body posture and human–robot coordination. Interestingly, the majority of human partners perceived the robots as following their movements (Chen et al., 2015).

“Assist-as-needed” (Cai et al., 2006) robotic devices will provide corrective forces only if a participant’s limb movement kinematics hit the walls of a predefined “virtual tunnel” (Duschau-Wicke et al., 2010) aiming to keep an individual’s body or limbs within an initially defined “normal” range. In contrast to this kind of “positive” force feedback, our deliberately light interpersonal touch paradigm could be considered to act with “negative” force feedback. This means that if participants stray from a reaching trajectory, they will perceive a reduction in touch, which might cue them to perform a subtle correction, such as moving toward the contact, with the intention to keep a constant force and to minimize contact force variability. In this sense, the robotic system in our study was controlled according to a similar principle and we believe it imitated the behavior of the CR and CP more naturally. The reaching trajectories were not prespecified within the robotic system but emerged as a compromise between the CR and the respective CP so that the CR’s movements remained unconstrained physically.

In this context, it is remarkable that rIPT led to straighter forward reaching trajectories with least amount of medial drift. This could mean that a robotic system is a better haptic “communicator” in the sense that it made participants “listen” more closely to the haptic feedback they received. The dynamics of the robotic system were not independent but a direct consequence of CR’s movements. Despite the lack of any real “social cognitive” capabilities of the robotic system, this fact can nevertheless be interpreted as highly precise responsiveness, which a human CP could never match. Possibly, participants interpreted rIPT as a more reliable spatial reference and therefore adjusted their reaching movements more in a feedback-driven manner.

Influence of Visual Feedback for CP

The provision of hIPT involved visual feedback or optical tracking of CR’s body and movements. In human pairs, the presence of visual feedback with habitual visual dominance is likely to turn the CP into a follower of CR’s movement (Steinl & Johannsen, 2017). Assessing human–human as well as human–robot interactions in a single degree of freedom object manipulation task, Groten et al. (2009a, 2009b) characterized inter-agent dominance as a function of the interaction force with dominance between both partners varying flexibly. Generally speaking, in most physical interactions between 2 human individuals leader-follower relationships are not necessarily fixed. It seems to be the case, however, that the more adaptive individual, for example the person on whom fewer requirements to fulfill specific movement constraints are imposed, is more likely to take a follower role (Skewes et al., 2015). This interpretation implies that in hIPT the CP coordinated the movements in a reactive fashion as well, potentially in follower mode due to visual dominance.

Limitations

The results of our study are subject to limitations, such as small sample size limiting not only the possibility to generalize our findings to a wider population of older adults or patients with disturbed body balance. Similarly, our experimental setup and task represent a specific laboratory situation that imposed specific constraints onto participants. As a consequence, the generalizability of our findings to other postural tasks and daily life activities is restricted too. Another limitation is the lack of force recordings in the hIPT condition. As we do not know the absolute interaction forces applied between the CP and CR, it could mean that IPT had not been applied in a light fashion and therefore potentially influenced the CR’s movements in some way. We believe, however, that touch had being applied lightly in our hIPT condition as the overall movement pattern observed in hIPT was not dramatically different from either the No IPT or the rIPT conditions in the present study as well as hIPT in a pilot experiment, where the interaction forces and torques were being recorded. Usually, hIPT tended to fall in between No IPT and rIPT, which implies that the mechanical coupling between both partners was not much stronger than in the rIPT conditions. The possibility remains that phases occurred during which contact between the CP and CR was not present. One way to evaluate the movement coupling between both partners would be the recording of both partners’ movement dynamics. Unfortunately, our setup was limited to the acquisition of only CR’s motion for the lack of a second force plate and more extensive motion capture coverage.