Multisensory control of a straight locomotor trajectory

1 Experimental procedures

1.3 Experimental set-up and protocol

Anthropometric measurements were recorded and thirty-nine reflective markers were placed on specific body landmarks in order to create a full body marker set, as specified by the Vicon Plug-in Gait model. Subjects were fitted with the HMD, which allowed them to view the VE consisting of a large, rich-textured (40 m×25 m) virtual room with pillars located at its end [25]. Head coordinates were fed real-time through the Tarsus engine from Vicon to the VR (Caren-2) system that controlled the VE.

Subjects were first habituated and familiarized to walking in the VE before any data were collected. The subjects were randomly exposed to rotational flows with pseudo-FOEs shifting from 0° to 40° to the left (–40°) or right (+40°), or no flow rotation (0°). The rotational flows were triggered either externally (visually simulated head turn: SHT) or by having the subject rotate their head (active head turn: AHT). In the SHT condition, the virtual environment was rotated while the subject was walking and fixating on a red stationary target located 2 m straight ahead, at eye level (Fig. 1). Note that in this condition, the target remained fixed and centered with respect to the HMD screen. In the AHT condition, subjects were walking while fixating a target of similar characteristics rotating from 0° to 40° (Fig. 1). In order to verify whether the presence of a target had an effect on the steering behaviour, the simulated condition was also performed without a target (SHT_NT).

OPEN IN VIEWER

Fig.1

Schematic bird’s eye view of an active (left panel) and a visually simulated (right panel) head turn to the right. Subjects were walking in a virtual environment (represented by rectangular box) while fixating on a round target (represented by black dot) located at 2 m in front of them. At 1.5 m of forward walking, the target (active head turn condition) or the scene (simulated head turn) started to rotate, until it reached a final location of 40° at 5 m of forward walking. Note that the virtual room (40 m×25 m) and target location represented are not drawn to scale. The simulated head turn condition with no target is not shown.

Subjects were instructed to walk straight in the physical world, as opposed to the virtual world being viewed in the HMD, and to maintain the red target in the centre of the HMD screen. After 1.5 m of forward walking, the optic flow began to be rotated due to the AHT or SHT. That rotation lasted over 3.5 m, such that after 5 m of forward walking, the rotational flow experienced by the subjects was±40° or remained at 0°. Each direction (–40°, 0°, +40°) was repeated four times across the three conditions (AHT, SHT, SHT_NT) for a total of 36 trials that were block-randomized for each subject. The main outcome measure retained for analysis was heading orientation, defined as the mean instantaneous angular deviation of CoM trajectory in the horizontal plane. Secondary outcome measures included CoM mediolateral deviation as well as horizontal orientation of the head, thorax, pelvis and left foot. Data were examined in the physical world, which represented the coordinate system of interest, and averaged over the distance comprised between 4.75 m and 5 m of forward walking. Virtual head rotation was also examined to ensure that the perceived head rotation (stimulus) was identical across conditions. Data were thereafter averaged across trial conditions and across subjects.

2 Results

In all three head turn conditions, the instruction was to walk straight in the physical world while being presented with a rotational flow pattern, whether simulated or actively induced. A perfect maintenance of a straight heading direction would result in no deviation of CoM trajectory, as well as no segment reorientation in the horizontal plane in the physical world (other than when an active head turn was present). Figure 2 illustrates examples of CoM trajectories and head reorientation patterns across head turn conditions and pseudo-FOE locations. Subtle changes in CoM trajectory were observed when the flow was rotated from the neutral (0°) condition. In the example illustrated, the subject veered slightly in the direction of the pseudo-FOE shift in the AHT condition, but away from the pseudo-FOE location in SHT and SHT_NT conditions. CoM trajectories were also identical between the physical and virtual worlds, indicating a rotation around the subject’s head without any translation. For head rotations, the difference across conditions in the physical world accounts for the fact that an active head turn was performed in the AHT condition, while it was visually simulated in the SHT and SHT_NT conditions. In the virtual world, head orientation traces were similar across optic flow conditions, indicating that subjects were exposed to similar amounts of perceived head rotation.

OPEN IN VIEWER

Fig.2

Traces of individual trials for all three final locations (40° left, 0°, 40° right) of focus expansion (pseudo-FOE) for the (A) active head turn (AHT), (B) visually simulated head turn (SHT) and (C) visually simulated head turn with no target (SHT_NT) conditions. For each condition, mediolateral (ML) displacement of body centre of mass (CoM) (left panels) and head yaw (right panels) are illustrated as a function of CoM anteroposterior (AP) displacement, both in the physical (upper graphs) and virtual (lower graphs) worlds.

In the physical world, significant main effects due to pseudo-FOE shifts and interaction effects between head turn conditions and pseudo-FOE shifts were found for CoM heading orientation (Fig. 3), as well as most body segment horizontal orientations (Fig. 4). On average, CoM heading orientation deviated away from the pseudo-FOE location for the SHT and SHT_NT conditions, but not the AHT condition. While no significant differences in CoM heading orientation emerged for any directions of rotational flow between the different head turn conditions, a differential effect of pseudo-FOE location was observed in the simulated head turn conditions only, as CoM heading orientation was found to differ between the left (–40°) vs. right (+40°) pseudo-FOE locations (p < 0.01 to p < 0.05). Similar results were obtained for CoM mediolateral deviation.

OPEN IN VIEWER

Fig.3

Heading errors and CoM mediolateral (ML) displacement for all three final locations (40° left, 0°, 40° right) of focus expansion (pseudo-FOE) and for the active head turn (AHT), the visually simulated head turn (SHT) and the visually simulated head turn with no target (SHT_NT) conditions. Mean (±1SE) values are represented in the physical world coordinates. Statistically significant pairwise comparisons are indicated, with p < 0.05 (*), p < 0.01 (**).

OPEN IN VIEWER

Fig.4

Mean (±1SE) values of body segment orientations for all three final locations (40° left, 0°, 40° right) of focus expansion (pseudo-FOE) for the active head turn (AHT), the visually simulated head turn (SHT) and the visually simulated head turn with no target (SHT_NT) conditions. Data are represented in the physical world. Statistically significant pairwise comparisons are indicated, with p < 0.05 (*), p < 0.01 (**), p < 0.001 (***).

The head was reoriented in the direction of the active head turn in the AHT condition while a small reorientation (typically <8°) in the direction of CoM heading reorientation was systematically observed for the SHT and SHT_NT conditions (Fig. 4). The head reorientation pattern differed across head turn conditions when the pseudo-FOE was shifted (p < 0.001). Furthermore, head reorientation differed across all pseudo-FOE locations for the AHT condition (p < 0.0001), and between the right vs. left pseudo-FOE locations (p < 0.01 to p < 0.05) for the visually simulated head turns (SHT, SHT_NT). Thorax, pelvis and foot orientation displayed reorientation patterns similar to the head, although the magnitude of this reorientation became attenuated in more distal (lower) segments (e.g. the foot) (see detailed pairwise comparisons, Fig. 4). Pelvis and foot segment orientations remained similar across pseudo-FOE locations for the AHT condition and were only found to differ (p < 0.05) between left (–40°) vs. right (+40°) pseudo-FOE shifts for the SHT condition.

In order to verify whether the perceived head turn rotation (stimulus) was comparable across conditions, the amplitude and rate of virtual head rotations were also analyzed. As expected, there was no significant difference in virtual head rotation amplitude (p > 0.88) regardless of active or simulated head turn conditions. Note that the average head turn amplitude did not reach the full 40°, by 5° to 8°. Mean rotational optic flow rate induced by the head rotations, which was dependent on gait speed (1.0±0.1 m/s), was found to be 11.0±1.3°/s (mean±1SD, for all conditions and pseudo FOE¬ =0). Similar optic flow rates were observed for the AHT (mean±1SD: 10.5±1.6 deg/sec), SHT_NT (11.3±1.6 deg/sec) and SHT conditions (11.1±1.3 deg/sec).

We have created a unique paradigm which includes active and visually simulated head turns to investigate the multisensory control of straight locomotor trajectory by comparing the heading and body segment orientations in response to rotational optic flow generated internally vs. externally during walking. Our results revealed an interaction effect between the direction of pseudo-FOE shifts and head turn condition for heading orientation, such that the rotational optic flow caused the subjects to veer in the direction opposite to the head turns only when those head turns were simulated, i.e. externally generated rotational flow in the absence of actual head motions. Likewise, head, thorax and pelvis displayed small but systematic reorientations in the direction opposite to the pseudo-FOE shifts during the visually simulated head turns, which is consistent with the presence of a steering response [25]. When a horizontal rotational flow is combined with an expanding translational flow, as in the present study, the resulting flow can be perceived either as a head rotation while walking or as walking along a curved path [1, 28], the latter implying a change of trajectory. In the present study, the externally generated rotation flow generated by the visually simulated head turns shifted the perceived pseudo-FOE locations in the direction of the head rotations, therefore resulting in a perceived locomotor heading in the head rotation direction. As subjects were instructed to walk straight ahead, they corrected their perceived heading trajectory by veering in the direction opposite to the pseudo-FOE location or head turn direction. Such explanation is consistent with our previous findings showing that healthy subjects veer, in the physical world coordinates, in the direction opposite to perceived pseudo-FOE location when exposed to rotational flows [25]. The presence of walking trajectory deviations contralateral to the pseudo-FOE location in the simulated but not the active head turns in the present study suggest that non-visual cues are contributing to flow decomposition and hence to heading identification and steering, as suggested by previous psychophysical experiments on heading identification [15, 30]. An alternate explanation could be that non-visual cues contribute to the assessment and correction of an asymmetry in the optic flow field created by the head rotation, with no need to decompose directional components of the flow and identify heading from the optic flow itself. Regardless of whether the CNS correct for an asymmetry or a perceived change in FOE location, the fact that a straight locomotor trajectory was maintained when exposed to internally but not externally generated rotational flows (i.e. active as opposed to simulated head turn) suggests that non-visual cues do contribute to the control of locomotor heading when executing head turns while walking. Conversely, the fact that a trajectory deviation did occur in presence of an externally-generated rotational optic flow suggests that optic flow information is a powerful source of visual information that can impact on the control of locomotor heading. The multi-sensory fusion model supports the integration of both visual and non-visual elements in heading estimates [6].

There are multiple sources of heading perception during voluntary head turns. Crowell and colleague [3] demonstrated that vestibular information, neck proprioception and the efference copy of the head motor command were non-visual cues contributing to a proper identification of heading direction during active head rotations performed while seated. Eye movement information was also shown to contribute during active gaze shifts [1, 23]. In the present experiment, active head turns were also compared to visually simulated head turns while the subjects were fixating on a visual target. A fine eye/head coordination is normally present while changing direction during walking [8–10], as well as during voluntary gaze (eye/head) shifts [19]. It would have been ideal to collect eye movements during the present experiment, such that the relative contribution of eye movement information could be appreciated. Unfortunately, valid eye movements could not be collected due to the technical difficulty of stabilizing the combined HMD/eye-tracking system with respect to the subjects’ pupil while walking. A closer examination of head rotation amplitudes shows that the perceived virtual head rotations were similar across conditions but did not reach the full 40°, despite the fact that subjects were required to keep the target centered in the HMD screen. This suggests that some degrees of eye movement most likely contributed to gaze fixation on target, at least in the active head turn condition. In the simulated conditions, the virtual head rotation was slightly reduced by the opposite physical head rotation which, as expected, accompanied the steering response. Therefore, while non-visual information generated from the head movement have certainly contributed to control locomotor heading while actively turning the head, oculomotor responses most likely played a role, which warrants further investigation.

Only small changes in heading direction were provoked by relatively large magnitudes of visually simulated head turns. The heading changes averaged only ∼4° in the SHT condition, corresponding to ∼0.33 m of CoM mediolateral deviation after 5 m of forward walking. There may be several explanations for this. Firstly, it was demonstrated that richly textured scenes with geometrical information that contains parallax information, as in the present experiment, resulted in a strong source of retinal flow that yielded more accurate heading judgments [15, 32]. Furthermore, our average optic flow rotation rate reached approximately 11°/sec. While some studies report accurate heading judgment from retinal flow field, ie. without the contribution of any extra-retinal cues, for optic flow rotation rates up to ∼ 1°/sec [1, 23], others have identified good performances for rotation rates as high as 16°/s to 20°/s [16, 26]. Due to parallax information and relatively slow rates of rotational flow in this experiment, retinal flow itself might, to some extent, have contributed to minimizing heading errors in the simulated head turn conditions. As voluntary head turns while walking can reach speeds as high as 400°/s [13, 20], however, it appears reasonable to conclude that non-visual cues would normally be contributing to the control of a straight trajectory when turning the head while walking. Secondly, the fact that gaze, although required to fixate onto a target, was free to shift and scan the travel path while walking, might have further contributed to minimize the heading errors [33]. Thirdly, instructions and expectations as to whether the individuals will be travelling along a straight or curved path also makes a difference in heading judgment [14]. In the present study, instructions were similar across conditions. The order of presentation of the active vs. simulated head turns condition were block randomized, and head turn directions were randomly presented and interspersed with catch trials (FOE at 0°) within each block. Expectations, if any, would have been minimized and counterbalanced among subjects. Moreover, other powerful sources of sensorimotor information associated with locomotion were present in this study, which was at variance with previous optic flow studies conducted in stationary sitting. The CNS has the ability to re-weight and up-regulate accurate sources of sensory information when subjected to incongruent sensory information [17]. It is likely that the small steering changes observed in the simulated head turn condition also reflects the ability of the CNS to re-weight the sensory information in favour of eye, neck, and leg proprioceptive information, as well as vestibular information, which all signalled that no change in heading orientation occurred. Finally, previous studies have shown that an interaction in steering strategies (optic flow vs. egocentric) was present when both optic flow and target elements were utilized during locomotion [2, 32]. Results from our study, however, show that the presence or absence of a target does not affect CoM heading direction and body segment orientation in the simulated head turn conditions. These results are consistent with the fact that the target element in the present study, which remained still and centered with respect to the HMD screen in the simulated head turn condition (SHT_T), could not be used as a goal to walk to.

In conclusion, this current study provides further insight on the role of visual and non-visual cues in the control of locomotor heading during horizontal head rotations. While results indicates that optic flow information is powerful and influences locomor heading, they also suggest that non-visual information is needed for an appropriate interpretation of complex optic flows such as those induced during head turns while walking in order to head in the desired direction. Through the understanding of the role and interaction of retinal flow and non-visual information, we can better understand the deficits in locomotor steering with vestibulopathy, and design virtual environments that will allow them to use or restore this ability.