The Effectiveness of Simulator Motion in the Transfer of Performance on a Tracking Task Is Influenced by Vision and Motion Disturbance Cues

Abstract

Objective:

To examine the importance of platform motion to the transfer of performance in motion simulators.

Background:

The importance of platform motion in simulators for pilot training is strongly debated. We hypothesized that the type of motion (e.g., disturbance) contributes significantly to performance differences.

Methods:

Participants used a joystick to perform a target tracking task in a pod on top of a MOOG Stewart motion platform. Five conditions compared training without motion, with correlated motion, with disturbance motion, with disturbance motion isolated to the visual display, and with both correlated and disturbance motion. The test condition involved the full motion model with both correlated and disturbance motion. We analyzed speed and accuracy across training and test as well as strategic differences in joystick control.

Results:

Training with disturbance cues produced critical behavioral differences compared to training without disturbance; motion itself was less important.

Conclusion:

Incorporation of disturbance cues is a potentially important source of variance between studies that do or do not show a benefit of motion platforms in the transfer of performance in simulators.

Application:

Potential applications of this research include the assessment of the importance of motion platforms in flight simulators, with a focus on the efficacy of incorporating disturbance cues during training.

Keywords

self-motion learning transfer of training practice attention

Introduction

The use of flight simulators has become an increasingly common aspect of pilot training over the past few decades because trainees can accumulate experience flying without risk of human injury and vehicular loss. Self-motion cues provided by expensive motion platforms are often key components of these simulated environments, and simulator designers tend to focus on realism and technical advances over examination of which specific factors support learning (Bowen, Oakley, & Barnett, 2006; Roessingh, 2005; Salas, Bowers, & Rhodenizer, 1998). However, the advantage (if any) to adding full-body motion to these training sessions is still unclear (Bürki-Cohen, Sparko, & Bellman, 2011; Caro, 1979; McCauley, 2006; Valverde, 1973). In motion simulators, two types of motion are presented: motion that is correlated with pilot maneuvering (correlated motion) and motion that is related to environmental changes (disturbance motion due to wind shears, turbulence, or engine failure). Both sources of motion can provide feedback that is then used to adjust flight control. Here we are interested in whether disturbance motion in particular improves performance relative to correlated motion and/or no motion.

Empirical evidence to support the claim that motion platform simulators are superior to non-motion platform simulators is mixed. Some studies have shown that self-motion is a critical component during training (Lee & Bussolari, 1989; McDaniel, Scott, & Browning, 1983; Proctor, Bauer, & Lucario, 2007; Van der Pal, 1999), and Johnston and Catano (2013) found that performance in a motion simulator predicted later success in pilot training. However, other studies have found self-motion to be less important (Jacobs & Roscoe, 1975; Koonce, 1979; Woodruff, Smith, Fuller, & Weyer, 1976). A series of four studies conducted by the Volpe center under the FAA-sponsored Flight Simulator Human Factors Program found minimal differences between performance with and without motion and no transfer effects (Bürki-Cohen & Go, 2005; Go, Bürki-Cohen, & Soja, 2000; Go et al., 2003; Sparko, Bürki-Cohen, & Go, 2010). This is just a small sample of the vast literature on flight simulation, but a thorough treatment of previous results and conclusions can be found in Bürki-Coken et al. (2011).

A critical issue in evaluating these studies is that there is considerable variation in how motion is implemented and what tasks were evaluated. We believe these differences have significantly contributed to the current confusion in the literature. The next logical step is to find and isolate contextual factors that support transfer of training. Of course there are many contextual factors to choose from (e.g., task type, motion type, high vs. low fidelity, etc.), and we do not propose to address them all in one paper. Here we will start with an idea investigated by de Winter, Dodou, and Mulder (2012) in a meta-analysis. Across 24 studies, they found a significant effect in favor of training with motion, but this benefit was largely contingent on studies that included disturbance motion. They concluded that the efficacy of training with motion may depend on the presence of disturbance motion during training.

Why might disturbance motion matter? In general, humans rely more heavily on their visual system than vestibular system during navigation (Lishman & Lee, 1973); however, the vestibular system is faster (although not to reach conscious awareness; Barnett-Cowan & Harris, 2009) and does not require directed attention for an alerting response. Thus, when the aircraft is stable, visual input may provide sufficient sensory feedback to maintain a stable flight path. In contrast, when there is a disturbance, vestibular input may be faster at eliciting a corrective response because directed attention is not required. During training, this vestibular input might provide an advantage in learning how to effectively keep the aircraft on course (Bürki-Cohen, Soja, & Longridge, 1998; Caro, 1979; de Winter et al., 2012; Gundry, 1976). There are two critical questions then: (a) Does the presence of motion disturbance improve performance over visual disturbances, and (b) does this result in better training?

In the present study, we examine the disturbance motion hypothesis by training participants on a compensatory tracking task using a MOOG Stewart motion platform with novice undergraduates. We chose to use undergraduates following evidence that the effect of motion input is larger for novices relative to experienced pilots (de Winter et al., 2012); therefore, using these participants should provide the best chance to find significant differences. The training conditions were designed to compare two key factors: training with or without correlated motion (e.g., a moving platform vs. a stationary platform) and training with or without disturbance motion. We use the term turbulence hereafter to refer to the presence of wind gusts or other such forces that affect the system and to distinguish between turbulence and mechanical vibration noise. This resulted in five distinct training conditions: (1) no correlated motion and no turbulence, (2) correlated motion but no turbulence, (3) correlated motion and turbulence, (4) no correlated motion but turbulence motion, and (5) turbulence but no motion (visual turbulence only). The fourth and fifth training conditions were included to isolate the various contributions of disturbance forces. Condition 4 enables us to determine if turbulence motion alone (without any correlated motion) influences performance at test. In Condition 5 (visual turbulence only), the motion platform was stationary, but the progress of the crosshair was affected by turbulence forces. From the perspective of the participant, the only evidence of the turbulence was visual. This enables us to determine if any turbulence (even in the absence of motion) influences performance at test. This condition is important because if turbulence motion during training improves transfer of training effects, we can then ask whether the motion platform is necessary or whether visual turbulence alone may provide training benefits.

Following training, participants in all five conditions were tested on the tracking task under the full simulation that included both correlated motion and turbulence motion (i.e., Condition 3 described previously). We predicted that training with correlated motion would be more beneficial than training without correlated motion only when turbulence cues were present. Our expectations for the turbulence conditions that did not also involve correlated motion were less certain. It is possible that turbulence cues alone (via the visual and/or vestibular/proprioceptive senses) during training are enough for participants to adopt different joystick control strategies than those that train without turbulence cues.

Methods

Participants

Seventy-nine students (42 males) from McMaster University participated as volunteers or in exchange for course credit. Data from 4 participants were excluded due to hardware errors; the remaining 75 individuals were distributed across the five conditions (15 participants per condition). All participants reported normal or corrected to normal vision.

Apparatus

The pod is supported by a MOOG platform with 6 degrees of freedom motion (Moog series 6DOF2000E). All stimuli were controlled by a program coded in C++ using the Vega Prime (Presagis) library. The program was synchronized to a 60 Hz signal. Target stimuli were presented on a 42” (diagonal) LCD panel with a resolution of 1,920 × 1,080 pixels and refresh rate of 60 Hz. Participants sat in an automobile style bucket seat bolted to the floor at center of mass inside the simulator pod; this position maintained an approximate distance of 120 cm between the participants’ eyes and the LCD display screen. Participant responses were recorded as continuous data at 60 Hz with a USB-connected Logitech joystick mounted on a T-shaped apparatus that allowed the joystick to rest on the participants’ lap, with the vertical portion of the T-shaped apparatus resting between participants’ legs, effectively preventing the joystick from shifting and equating joystick stability across conditions. Auditory noise from simulator motors and mechanics was reduced with earplugs and masked with white audio noise. The only source of light came from the LCD display screen, resulting in a dimly lit environment. Two cameras were mounted inside the pod for monitoring the screen and the participant.

Stimuli and Procedure

The experimental session was approximately one hour in duration for each participant. Participants were randomly assigned to one of five training conditions (described in the following). Two training blocks (200 trials per block) were followed by two test blocks (200 trials per block). Short breaks were provided between each block. In the figures and analyses, the blocks are referred to as practice blocks P1 and P2 and test blocks T1 and T2.

Each trial was 3 seconds in duration and began with a background of sky and clouds, superposed by a crosshair and a target (Figure 1, Panel A). The crosshair was a white fixation cross inside a red square (5.6 cm × 5.6 cm, subtending a visual angle of 2.67°), fixed at the center of the screen for the duration of the trial. The target location was indicated by a blue circle (diameter 5.6 cm, visual angle 2.67°) presented in a random location along the horizontal meridian of the screen between 9.3 cm (4.4°) and 23.2 cm (10.9°) from center of crosshair to center of target circle. We used a compensatory tracking task in which a joystick was used to move the vehicle so that the position of the central crosshair corresponded with the position of the blue target circle (Figure 1, Panel B). At the end of each trial, the display was replaced by a grey screen for 334 milliseconds. The vehicle position parameters were reset to center prior to the onset of the next trial.

Figure 1.

The visual display consisted of a blue background with clouds, a crosshair (white cross at the center of a red square), and a target (blue circle). (A) At the beginning of each trial, the crosshair was displayed at the center of the screen while the target was positioned to the left or right of the crosshair. (B) The task was to move the crosshair (which remained fixed to the center of the screen) to the target circle.

Motion Control

The motion system was a second-order model with 1 degree of freedom (yaw: left and right lateral movements). The visual and platform motion stimuli were coordinated by the use of position and force models that integrated information from the input sources (joystick control or noise in the form of vibration and/or turbulence) and translated this information into a change of position of the vehicle, which was presented to the subject via the two output sources: the stimulus display and the motion platform (see Figure 2). Acceleration cues were presented within the physical limitations of the platform; constant velocity was simulated with washout motion (i.e., the platform returned to the neutral position at a rate below threshold for motion detection).

Figure 2.

The input to the motion platform and to the visual display screen is filtered through the position model (which determines the position of the crosshair relative to the target) and the force model (which determines how the motion platform moves). Vibration noise (1–5 Hz) mimics engine and vehicle vibration. Turbulence represents medium to large forces that move the vehicle off course. Each of switches A, B, and C could be “off” or “on,” resulting in the five practice conditions.

Joystick movements were translated as control input to a second-order system, realized as a second-order low pass filter with a cutoff frequency of .7 Hz and damping of 4. The filter output, together with added noise, represents the force, which is integrated and converted to motion by a high pass filter (washout filter) with cutoff frequency .4 Hz and a damping of 20 to compute the position of the motion platform. The filters are implemented digitally by transforming them from Laplace to the Z domain, processing 60 Hz sampled data.

Low-frequency turbulence was added to the force and position models to mimic random wind forces that move the vehicle slightly off target. The turbulence took the form of random left and right lateral forces at 20% of the maximum force produced by joystick movements, with a random duration but always lasting less than 1.5 seconds. The wind bursts were modeled by a sine wave with a phase of 3 seconds, modulated so that more than one wind burst during a trial was possible (approximately 2 per trial).

Medium-frequency vibration noise (1–5 Hz), which did not contribute to the position model, was added to the force model to mimic engine and vehicle vibration typically experienced when operating a motor vehicle. This also helped to dampen the sound generated by the movement of the simulator platform. The vibration noise was present whenever the simulator platform was active.

Post-experiment debriefing revealed that participants were clearly able to detect and differentiate between the different types of motion (i.e., correlated, uncorrelated turbulence, uncorrelated vibration).

Training Conditions

The training conditions differed in terms of whether platform motion was turned on or off, which could be done independently for correlated and turbulence motion. Table 1 summarizes the type of motion in each condition, and Figure 2 illustrates control in the motion model to achieve the following five conditions.

Table 1:

The Five Training Conditions Summarized with Respect to Type of Motion

Training Condition	Visual Correlated Motion	Platform Correlated Motion	Visual Turbulence Motion	Platform Turbulence Motion
V	
V-CM		
V-TV			
V-TM				
V-CTM				

Note. V = visual tracking task only; V-CM = visual tracking task with correlated motion; V-TV = visual tracking task with visual turbulence; V-TM = visual tracking task with turbulence motion; V-CTM = visual tracking task with correlated and turbulence motion (this also defines the condition at test).

V: visual tracking task only condition

Changes in the position of the vehicle were represented as changes in the visual display in response to correlated motion (i.e., left and right movements in response to joystick movements).

V-CM: visual tracking task with correlated motion condition

Changes in the position of the vehicle were represented as changes in the visual display and movement of the platform in response to correlated motion.

V-TV: visual tracking task with visual turbulence condition

Changes in the position of the vehicle were represented as changes in the visual display only, in response to both correlated and turbulence motion.

V-TM: visual tracking task with turbulence motion condition

Changes in the position of the vehicle were represented as changes in the visual display in response to both correlated and turbulence motion and movement of the platform in response to turbulence motion only.

V-CTM: visual tracking task with correlated and turbulence motion condition

Changes in the position of the vehicle were represented as changes in both the visual display and movement of the platform in response to both correlated and turbulence motion. This training condition was also used as the test condition.

Results

Performance was evaluated based on accuracy, track time, error integral, and joystick control (magnitude of movement and control reversals). Accuracy reflects the proportion of trials successfully tracked: The center of the crosshair did not deviate more than 10% from the center of the target circle for at least 500 milliseconds. Track time was evaluated for successfully tracked targets only and reflects the elapsed time from the start of the trial to the start of the 500 millisecond time window used to classify the target as successfully tracked. To illustrate the relative positions of the target, crosshair, and joystick over the duration of a trial, we show five examples in Figure 3. The crosshair represents changes in the position of the vehicle in response to joystick movements and is reset to center position at the beginning of the next trial. To compute the error integral on each trial, the distances were normalized (1 unit = distance between the starting position of the crosshair and the center of the target), and the error was calculated as area under the curve that defined the deviation of the center of the crosshair from the center of the target position. The integral of joystick movement was calculated as the area under the curve that maps the position of the joystick relative to the neutral joystick position, integrated over the 3-second trial. Joystick directional corrections or control reversals were a direct count of the number of times the joystick crossed the central neutral position of the controller within each trial, executed (a) to correct a movement in the wrong direction, (b) as a strategy to slow or stop vehicle movement, or (c) to return to the target after overshooting its location.

Figure 3.

A depiction of five example trials showing the relative position of the target, the joystick, and the crosshair. The crosshair is reset to the starting position at the beginning of each trial (depicted by vertical grey bars). The initial movement of the joystick triggers the movement of the crosshair toward the target. Trial 1: Not tracked, the crosshair does not reach the position of the target for the required duration (500 milliseconds). Trials 2 and 3: Tracked, the crosshair successfully reaches the location of the target for the required duration. Trial 4: Not tracked, the crosshair overshoots the target. On each trial, the error integral is calculated over the area between crosshair and target (e.g., shaded area). Trial 5: Not tracked, the initial joystick movement is away from the target; this is corrected but not in time to successfully track the target.

Each measure (accuracy, tracking time, error integral, magnitude of joystick movement, and control reversals) was statistically examined by performing a mixed-model ANOVA crossing the five-factor between-subject variable training condition (V, V-CM, V-CTM, V-TM, V-TV) with the four-factor within-subject variable block (P1, P2, T1, T2). Linear trend analyses were used to examine practice effects within each of the training conditions. Multiple comparisons were subjected to Holm’s sequential Bonferroni correction to maintain a familywise alpha of .05 (Holm, 1979). Presentation of the results is organized according to whether the training blocks P1 and P2 involved turbulence motion (V and V-CM) or not (V-CMT, V-TM, and V-TV).

Accuracy

Overall, accuracy improved across blocks, F(3, 70) = 30.88, p < .01, η_p² = .31. Although training condition was not significant, F(4, 70) = 1.63, p = .176, η_p² = .179, there was an interaction between block and training condition, F(12, 70) = 2.50, p = .04, η_p² = .13, which was examined by looking at the linear trends across blocks (Figure 4).

Figure 4.

Accuracy for each training condition across practice and test blocks. NT = no turbulence during training; T = turbulence during training; P1 = practice block 1; P2 = practice block 2; T1 = test block 1; T2 = test block 2; training conditions: V = visual tracking task only; V-CM = visual tracking task with correlated motion; V-CTM = visual tracking task with correlated motion and turbulence motion; V-TM = visual tracking task with turbulence motion; V-TV = visual tracking task with visual turbulence. Test blocks presented the full motion model (V-CTM). Error bars represent standard error.

No turbulence conditions

The linear trend was not significant, V: F(1, 14) = .315, p > .1; V-CM: F(1, 14) = .054, p > .1. This is explained by the finding that accuracy decreased from practice to test, t(29) = 2.68, p = .01.

Turbulence conditions

The linear trend was significant across blocks, V-CTM: F(1, 14) = 13.130, p < .01; V-TM: F(1, 14) = 24.314, p < .01; V-TV: F(1, 14) = 4.95, p < .05, reflecting an increase in accuracy that was maintained from practice to test blocks, in contrast to the no turbulence conditions.

Even though this pattern may seem suggestive of a transfer of training effect only when the training conditions included turbulence motion, when we compare the training conditions within each test block, we find no significant differences on accuracy between the five training conditions on either block (T1 and T2; all ps > .05). In other words, participants trained without turbulence motion show a decrease in accuracy when they encounter turbulence motion at test (T1 and T2); however, their overall accuracy at test is just as good as the groups who trained with turbulence motion. From that perspective, it makes no difference whether training incorporates turbulence motion or not. We will revisit this issue when we look at measures of joystick control.

Target Track Time

The time required to successfully track the target decreased across blocks, F(3, 70) = 17.23, p < .01, η_p² = .20; this was further supported by the linear trend for block, F(1, 70) = 26.514, p < .001. However, there was no effect of training condition, F(3, 70) = 0.46, p = .76, and no interaction, F(3, 70) = 0.97, p = .50, suggesting that target track time is not a sensitive measure for revealing effects of training with or without motion (Figure 5).

Figure 5.

Mean target track time for each training condition across practice and test blocks. NT = no turbulence during training; T = turbulence during training; P1 = practice block 1; P2 = practice block 2; T1 = test block 1; T2 = test block 2; training conditions: V = visual tracking task only; V-CM = visual tracking task with correlated motion; V-CTM = visual tracking task with correlated motion and turbulence motion; V-TM = visual tracking task with turbulence motion; V-TV = visual tracking task with visual turbulence. Test blocks presented the full motion model (V-CTM). Error bars represent standard error.

Error Integral

The error integral is affected by target track time in that the faster the target is tracked, the smaller the measure of spatial deviation between the crosshair and the target circle. The error integral also captures variance related to magnitude of joystick movements and the extent to which the crosshair may overshoot the target position (see illustration in Figure 3). We observed a practice effect in that the magnitude of the error integral was reduced across blocks, F(3, 70) = 11.59, p < .01, η_p² = .14; this was supported by a significant linear trend for blocks, F(1, 70) = 22.57, p = .001. Consistent with target track time, the effect of training condition, F(4, 70) = 0.69, p = .60, and the interaction between block and training condition, F(12, 70) = 1.50, p = .14, did not reach significance, providing further evidence that these particular measures are not sensitive to training with or without motion (Figure 6).

Figure 6.

Error integral is calculated as area under the curve that defines the deviation of the center of the crosshair from the center of the target position, as illustrated in Figure 3. P1 = practice block 1; P2 = practice block 2; T1 = test block 1; T2 = test block 2; training conditions: V = visual tracking task only; V-CM = visual tracking task with correlated motion; V-CTM = visual tracking task with correlated motion and turbulence motion; V-TM = visual tracking task with turbulence motion; V-TV = visual tracking task with visual turbulence. Test blocks presented the full motion model (V-CTM). Error bars represent standard error.

Joystick Control (Extent of Movement)

The extent of joystick movements is measured as deviation of the joystick from center neutral position, integrated over the 3-second trial. This provides a performance measure that may index differences in strategy, as some conditions may encourage larger movements (Figure 7). Differences in the extent of joystick movements were supported by a significant interaction between block and training condition, F(12, 70) = 23.90, p < .001, η_p² = .60, which is important in interpreting the main effects across blocks, F(3, 70) = 75.70, p < .001, η_p² = .52, and training conditions, F(4, 70) = 15.44, p < .001, η_p² = .50.

Figure 7.

Average amount of joystick movements made (as measured by the joystick movement index) as a function of training condition across practice and test blocks. P1 = practice block 1; P2 = practice block 2; T1 = test block 1; T2 = test block 2; training conditions: V = visual tracking task only; V-CM = visual tracking task with correlated motion; V-CTM = visual tracking task with correlated motion and turbulence motion; V-TM = visual tracking task with turbulence motion; V-TV = visual tracking task with visual turbulence. Test blocks presented the full motion model (V-CTM). Error bars represent standard error.

No turbulence conditions

There was a significant linear trend across blocks, V: F(1, 14) = 193.4 p < .01; V-CM: F(1, 14) = 360.7, p < .01, driven primarily by the contrast between practice blocks and test blocks. Participants who trained without turbulence appeared to employ qualitatively different strategies from practice to test in which they learned to reduce the extent of joystick movements during the training blocks, then expanded the extent of their movements during test when turbulence motion was introduced. Moreover, training without turbulence motion appeared to have a long-term effect on joystick movements such that even the extra practice during the two test blocks with turbulence was not enough to increase the extent of joystick movements to the level of the groups who trained with turbulence motion. To test this, we compared the turbulence groups to the no turbulence groups at test; participants in the no turbulence groups made smaller joystick movements during the test blocks compared to the groups who trained with turbulence motion, t(73) = −2.75, p = .008.

Turbulence conditions

Participants who trained with turbulence motion learned to use larger movements during training, and this strategy did not change with practice or between practice and test. No linear trend across the four blocks is consistent with no reduction (or increase) in the extent of joystick movements (all ps > .1).

Joystick Control (Control Reversals)

The number of times the joystick crossed the central neutral position of the controller within each trial is a measure of strategic joystick control; these movements are executed to correct direction, slow or stop the vehicle, or compensate for overshooting the target position. The first movement at the beginning of the trial was not included in the count. Overall, the number of control reversals increased across blocks, F(3, 70) = 86.54, p < .001, η_p² = .55. Training condition and the interaction were also significant, F(4, 70) = 40.48, p < .001, η_p² = .70; F(12, 70) = 46.06, p < .001, η_p² = .73.

No turbulence conditions

A significant linear trend across blocks, V: F(1, 14) = 181.8, p < .001; V-CM: F(1, 14) = 77.3, p < .001, was driven by a large increase in the number of control reversals between practice and test blocks (Figure 8). At test, both of the no turbulence training groups increased control reversals to compensate for the introduction of turbulence motion; however, pairwise comparisons revealed that this did not reach the level of groups who trained with turbulence motion. In particular, there were fewer control reversals for the no turbulence conditions compared to the turbulence conditions, t(73) = −2.70, p = .009.

Figure 8.

Average number of control reversals as a function of training condition across practice and test blocks. P1 = practice block 1; P2 = practice block 2; T1 = test block 1; T2 = test block 2; training conditions: V = visual tracking task only; V-CM = visual tracking task with correlated motion; V-CTM = visual tracking task with correlated motion and turbulence motion; V-TM = visual tracking task with turbulence motion; V-TV = visual tracking task with visual turbulence. Test blocks presented the full motion model (V-CTM). Error bars represent standard error.

Turbulence conditions

Training with turbulence produced linear trends in the opposite direction to training without turbulence, suggesting that practice leads to fewer control reversals, V-CTM: F(1, 14) = 5.31, p < .05; V-TM: F(1, 14) = 8.26, p < .05. Notably, the V-TV condition did not reach significance, V-TV: F(1, 14) = 2.79, p > .1, suggesting that turbulence in the motion platform itself (compared to turbulence in the visual display only) may be important to look at more closely in future studies.

Discussion

There is considerable debate over whether or not it is beneficial to train with platform motion within simulators (Bürki-Cohen et al., 2000; Bürki-Cohen, Go, & Longbridge, 2001; Jacobs & Roscoe, 1975; Koonce, 1979; Lee & Bussolari, 1989; McDaniel et al., 1983; Van der Pal, 1999; Woodruff et al., 1976). We explored a potential variable contributing to this controversy by using a quasi-transfer experiment in which we hypothesized that disturbance motion (e.g., turbulence) is an important component contributing to the effects of training with self-motion in a target tacking task (Caro, 1979; de Winter et al., 2012). Participants performed a compensatory tracking task within a simulated environment and were trained in one of five conditions. Two conditions involved training without turbulence motion, and three conditions involved training with turbulence motion. All participants were then tested on the target tracking task with the full motion model including correlated motion and turbulence motion.

Our results indicated that accuracy dropped at test for the non-turbulence training groups but did not drop at test for the turbulence groups. However, this observation was complicated by the fact that at test, none of the groups differed in accuracy performance. There are two potential interpretations here. One interpretation is that the turbulence groups displayed more transfer of training because they were able to maintain accuracy from training to test without a drop in performance. Another interpretation is that the non-turbulence groups were showing higher accuracy than the turbulence training groups during practice because that version of the task is easier and the addition of turbulence brought them down to the same level as the other groups at test. According to the accuracy observations then, no strong conclusions can be made, and it may not matter whether training involves turbulence motion or not. However, we also found that those who trained without turbulence motion made considerably fewer joystick movements and control reversals than those who trained with turbulence, especially during training. At test, the no turbulence groups increased the extent of their joystick movements and control reversals, but this did not quite reach the level of those that trained with turbulence. Thus, training with turbulence motion led to qualitatively different joystick control strategies, which persisted from training into test. Given that this did not seem to affect accuracy, it is unclear if this strategy provides a benefit or not. Indeed, one might conclude that because accuracy is not affected, this is not particularly important. However, in flight control, how you maneuver the aircraft is critical (Lone & Cooke, 2013), and the control behavior of the pilot is a skill-based behavior (Hosman & Stassen, 1999). For example, joystick control strategies may result in better compensation for large, unexpected vehicle movements that were not tested here. Alternatively, if you imagine a continuous flight task, an increase in joystick movements might be less efficient and result in a “bumpier” flight. While it is not immediately clear if the strategy employed by people trained in the turbulence conditions is more beneficial (further tasks will be needed to explore this issue), it is clear that this impacts how they are moving the vehicle, which is critical to flight (Hosman & Stassen, 1999; Lone & Cooke, 2013).

It is not yet clear what mechanism participants use to compensate for turbulence; they may learn to ignore turbulence, or they may learn to actively compensate for motion turbulence. Ignoring turbulence motion may involve a more relaxed approach to joystick control, using a less tight grip and providing more slack. A more active approach to compensation may impose more control over the joystick by attending to turbulence and retroactively compensating for them in order to stay on track. Another possibility is that the turbulence groups are provided with more opportunity to manipulate and understand the behavior of the control system and as such, more opportunity to understand the dynamics of the system for continuous manual control. Future studies will examine these possibilities by examining more closely the dynamics of control and compensation.

Types of Turbulence

We used three different turbulence conditions. The first, correlated motion with turbulence motion, is the condition that we used at test. One might expect that this condition would show the greatest benefit of practice as subjects are performing the same task across four blocks (i.e., this is a pure practice condition). However, this was not observed. The other two turbulence motion training conditions (V-TM and V-TV) both produced comparable performance across all measures.

Most interesting to us was the effectiveness of the isolated visual turbulence. This finding is particularly important because this training condition does not require a motion platform. This condition has never been examined in isolation before. If visual turbulence is enough to elicit changes that are similar to the other turbulence conditions in the test phase of our task, then it is possible that this condition can be used for training, reducing the amount of time needed in the full motion simulator. We recognize that it is unlikely pilots will move away from using motion simulators entirely: They are generally preferred by pilots (Bürki-Cohen et al., 2001) and may have advantages outside of what is tested here (e.g., reduced motion sickness, better transfer for more complex tasks). However, just reducing the amount of time required in a motion simulator (replacing it with a stationary simulator) would provide a considerable cost advantage in training pilots (Nieuwenhuizen, Mulder, van Paassen, & Bülthoff, 2013).

Limitations and Future Directions

In the present experiment, we used a tracking task in which each trial lasted 3 seconds, and the motion platform only moved laterally; thus, it is possible that the transfer effects observed here do not extend to simulations in which a three-dimensional continuous flight task is used. We opted to first use the two-dimensional tracking task specifically because of its relative simplicity. This allows us to more easily isolate the independent contributions of visual and motion cues to performance; complex environments are confounded by both perceptual and cognitive load factors that might interact with our variables of interest. However, work is currently underway to see if similar findings can be obtained using a three-dimensional environment in which participants are subjected to a continuous flight path. Preliminary results are similar to what we report here, indicating that this pattern holds across at least two tasks.

Another limitation of the present study is that we did not include any measures of motion sickness or sopite syndrome, and it is possible that these factors influenced performance between groups. Sopite syndrome is a neurological disorder that is characterized by drowsiness after one experiences prolonged periods of motion (Graybiel & Knepton, 1976; Lawson & Mead, 1998). Participants in our study were randomly assigned to groups, and no participants produced self-reports of motion sickness; future studies should ensure that these factors do not influence critical performance differences. It is also possible that the different conditions lead to different levels of fatigue by the end of the task. For example, it could be that the inclusion of turbulence leads to more physical or visual fatigue. Given that we do not find differences in accuracy or reaction time between groups in the final block of trials, this seems unlikely; however, in the future, directly testing fatigue would be useful.

A final limitation is that here we chose to use novice undergraduate subjects rather than skilled pilots. This was done in part for practical reasons and in part because we are interested in how these factors affect novices. Experienced pilots are likely to perform at ceiling on the simplified task used here and so would not show much improvement regardless of training condition. The purpose of the present experiment was to demonstrate if motion and/or turbulence can affect training in novices; future research is needed to extend findings to pilots.

Conclusion

The current study focused on how different types of motion presented during training (no motion, correlated motion, and turbulence motion) affect later performance when both correlated and turbulence motion were present. We found that training with turbulence motion influences transfer of training to test conditions relative to training without turbulence, specifically in the strategies used for joystick control. Interestingly, a visual representation of turbulence that does not use the motion platform may function as effectively as turbulence applied to the entire motion platform with respect to transfer of training effects. This research goes some way to resolving the confusion in the literature regarding the benefit of motion during transfer of training studies (i.e., that the presence of turbulence can be beneficial), but it is by no means the final word. This represents a first step in the systematic investigation of contextual factors that support learning in a simulated environment.

Key Points

Motion platforms used during pilot simulator training are extremely costly, yet their practical usefulness is controversial.

We examined the possibility that the type of motion cues implemented during training contributes to this controversy.

Results revealed that turbulence cues but not necessarily platform motion contributed significantly to the transfer of performance on a tracking task.

Footnotes

Acknowledgements

This research was funded by a Canada Foundation for Innovation (CFI) grant to JMS and MvM and a Natural Sciences and Engineering Research Council of Canada (NSERC) grant to JMS. SN and JGG contributed equally to this paper. We thank Dennis Beringer for his helpful comments on an earlier version of this paper.

John G. Grundy completed his PhD in psychology at McMaster University in Hamilton, ON, Canada, in 2013. He now holds a postdoctoral research position in the Department of Psychology at York University, Toronto, ON, Canada.

Stefan Nazar completed his master’s in psychology at McMaster University in Hamilton, ON, Canada, in 2014. He is now doing marketing research.

Shannon O’Malley completed her PhD in psychology at the University of Waterloo, Waterloo, ON, Canada, in 2013. She now holds a postdoctoral research position in the Department of Psychology, Neuroscience & Behaviour at McMaster University, Hamilton, ON, Canada.

Martin v. Mohrenshildt obtained his PhD in mathematics from ETH Zuerich in 1994 and is now an associate professor in the computing and software department within the Faculty of Engineering at McMaster University, ON, Canada.

Judith M. Shedden obtained her PhD in psychology from the University of Pittsburgh in 1995 and is now an associate professor in the Department of Psychology, Neuroscience & Behaviour within the Faculty of Science at McMaster University, ON, Canada.

References

Barnett-Cowan

Harris

L. R.

(2009). Perceived timing of vestibular stimulation relative to touch, light and sound. Experimental Brain Research, 198, 221–231.

Bowen

S. A.

Oakley

B. P.

Barnett

J. S.

(2006). Effects of motion on skill acquisition in future simulators (ARI Study Report 2006–7). Arlington, VA: U.S. Army Research Institute for the Behavioral and Social Sciences.

Bürki-Cohen

Boothe

E. M.

Soja

N. N.

DiSario

R. D.

Longridge

(2000, May). Simulator fidelity: The effect of platform motion. In Proceedings of the Royal Aeronautical Society Conference on Flight Simulation—The Next Decade. London: Royal Aeronautical Society.

Bürki-Cohen

T. H.

(2005). The effect of simulator motion cues on initial training of airline pilots. In Proceedings of the AIAA Modeling and Simulation Technologies Conference and Exhibit (AIAA 2005–6109). San Francisco, CA: AIAA.

Bürki-Cohen

T. H.

Longbridge

(2001, August). Flight simulator fidelity considerations for total air line pilot training and evaluation (AIAA Paper 2001–4425). Paper presented at the AIAA Modeling and Simulation Technologies Conference, Montreal, Quebec, Canada.

Bürki-Cohen

Soja

N. N.

Longridge

(1998). Simulator platform motion—The need revisited. The International Journal of Aviation Psychology, 8(3), 293–317.

Bürki-Cohen

Sparko

A. L.

Bellman

(2011). Flight simulator motion literature pertinent to airline-pilot recurrent training and evaluation. In Proceedings of the AIAA Modeling and Simulation Technologies Conference and Exhibit. San Francisco, CA: AIAA.

Caro

P. W.

(1979). The relationship between flight simulator motion and training requirements. Human Factors: The Journal of the Human Factors and Ergonomics Society, 21(4), 493–501.

de Winter

J. C. F.

Dodou

Mulder

(2012). Training effectiveness of whole body flight simulator motion: A comprehensive meta-analysis. The International Journal of Aviation Psychology, 22(2), 164–183. doi:10.1080/10508414.2012.663247

10.

T. H.

Bürki-Cohen

Chung

W. W.

Schroeder

Saillant

Jacobs

Longridge

(2003, August). The effects of enhanced hexapod motion on airline pilot recurrent training and evaluation. In Proceedings of the AIAA Modeling and Simulation Technologies Conference. San Francisco, CA: AIAA.

11.

T. H.

Bürki-Cohen

Soja

N. N.

(2000, August). The effect of simulator motion on pilot training and evaluation. In Proceedings of the AIAA Modeling and Simulation Technologies Conference (pp. 14–17). San Francisco, CA: AIAA

12.

Graybiel

Knepton

(1976). Sopite syndrome: A sometimes sole manifestation of motion sickness. Aviation, Space, and Environmental Medicine, 47(8), 873–882.

13.

Gundry

(1976, April). Man and motion cues. In Third Flight Simulation Symposium. London: Royal Aeronautical Society.

14.

Holm

(1979). A simple sequential rejective multiple test procedure. Scandinavian Journal of Statistics, 6, 65–70.

15.

Hosman

Stassen

(1999). Pilot’s perception in the control of aircraft motions. Control Engineering Practice, 7(11), 1421–1428.

16.

Jacobs

R. S.

Roscoe

S. N.

(1975, January). Simulator cockpit motion and the transfer of initial flight training. In The Interservice/Industry Training, Simulation & Education Conference (I/ITSEC). Arlington, VA: National Training Systems Association.

17.

Johnston

P. J.

Catano

V. M.

(2013). Investigating the validity of previous flying experience, both actual and simulated, in predicting initial and advanced military pilot training performance. The International Journal of Aviation Psychology, 23(3), 227–244.

18.

Koonce

J. M.

(1979). Predictive validity of flight simulators as a function of simulator motion. Human Factors: The Journal of the Human Factors and Ergonomics Society, 21, 215–223.

19.

Lawson

B. D.

Mead

A. M.

(1998). The sopite syndrome revisited: Drowsiness and mood changes during real or apparent motion. Acta Astronautica, 43(3), 181–192.

20.

Lee

A. T.

Bussolari

S. R.

(1989). Flight simulator platform motion and air transport pilot training. Aviation, Space, and Environmental Medicine, 60, 136–140.

21.

Lishman

J. R.

Lee

D. N.

(1973). The autonomy of visual kinaesthesis. Perception, 2, 287–294.

22.

Lone

M. M.

Cooke

A. K.

(2013). Pilot-model-in-the-loop simulation environment to study large aircraft dynamics. Proceedings of the Institution of Mechanical Engineers, Part G: Journal of Aerospace Engineering, 227, 555–568.

23.

McCauley

M. E.

(2006). Do Army helicopter training simulators need motion bases? (Technical Report 1179). Arlington, VA: U.S. Army Research Institute for the Behavioral and Social Sciences.

24.

McDaniel

W. C.

Scott

P. G.

Browning

R. F.

(1983). Contribution of platform motion simulation in SH-3 helicopter pilot training (No. TAEG-TR-153). Orlando, FL: Training Analysis and Evaluation Group (Navy).

25.

Nieuwenhuizen

F. M.

Mulder

van Paassen

M. M.

Bülthoff

H. H.

(2013). Influences of simulator motion system characteristics on pilot control behavior. Journal of Guidance, Control, and Dynamics, 36(3), 667–676.

26.

Proctor

M. D.

Bauer

Lucario

(2007). Helicopter flight training through serious aviation gaming. The Journal of Defense Modeling and Simulation: Applications, Methodology, Technology, 4(3), 277–294.

27.

Roessingh

J. J.

(2005). Transfer of manual flying skills from PC-based simulation to actual flight-comparison of in-flight measured data and instructor ratings. The International Journal of Aviation Psychology, 15(1), 67–90.

28.

Salas

Bowers

C. A.

Rhodenizer

(1998). It is not how much you have but how you use it: Toward a rational use of simulation to support aviation training. The International Journal of Aviation Psychology, 8(3), 197–208.

29.

Sparko

A. L.

Bürki-Cohen

T. H.

(2010). Transfer of training from a full-flight simulator vs. a high level flight training device with a dynamic seat. In Proceedings of the AIAA Modeling and Simulation Technologies Conference and Exhibit (Vol. 8218). San Francisco, CA: AIAA.

30.

Valverde

H. H.

(1973). A review of flight simulator transfer of training studies. Human Factors: The Journal of the Human Factors and Ergonomics Society, 15, 510–522.

31.

Van der Pal

(1999). The effect of simulator motion on parameter training for F–16 pilots. Engineering psychology and cognitive ergonomics: Vol. 3. Transportation systems, medical ergonomics and training. In Proceedings of the 1st International Conference on Engineering Psychology and Cognitive Ergonomics (pp. 267–275). Oxford, UK: Ashgate.

32.

Woodruff

R. R.

Smith

J. F.

Fuller

J. R.

Weyer

D. C.

(1976). Full mission simulation in undergraduate pilot training: An exploratory study (AFHRL–TR–76–84). Brooks Air Force Base, TX: Air Force Human Research Laboratory.