Eject,Eject,Eject! Conducting a Cognitive Task Analysis to Assess Parachute Descent Training Simulators

Abstract

We conducted a cognitive task analysis to provide input on the curriculum and technology currently used to train parachute descent procedures in the Naval Aviation Survival Training Program. Subject matter experts provided insight into the correct decision-making thought processes used in parachute descent. Techniques for analysis included focus groups, interviews, and a think-aloud procedure involving a parachute descent simulator at a Naval Aviation Survival Training Center. Results and findings inform the instructional design process to improve the training pipeline’s development, implementation, and evaluation.

Keywords

virtual environment aircraft ejection military training aviation survival cognitive task analysis parachute

A pilot’s ability to safely operate a parachute to descend and land following emergency egress from an aircraft may be the difference between life and death. Accordingly, the use of parachute simulation devices for training date back to World War II (Hogue & Pelz, 2008).

Emergency descent training is covered in all military branches within the U.S. and other countries (e.g., Singapore) and has applications beyond the armed forces (e.g., smokejumping; U.S. Department of Agriculture, 2018). Due to the hazardous nature of emergency bailouts and ejections, the psychological fidelity of the training is critical (e.g., Ragan et al., 2010; Stone, 2008).

In this article, we provide a brief overview of the process followed to obtain subject matter expert (SME) information about the tasks and decision-making involved in parachute descent procedures (PDP) in the Naval Aviation Survival Training Program (NASTP), with a focus on the use of virtual environments to train for high-risk events. We also discuss approaches used to assess the training’s psychological relevance to real-world applications and how the U.S. Navy can use the results to develop strategies to train PDP.

Virtual Environments as a Training Tool

Virtual training environments (VTE) have gained popularity in a wide array of work areas, from motor-skills training in medical settings (Gallagher et al., 1999) to soft skills training for teachers (Stavroulia et al., 2018). VTEs allow trainees to learn necessary skills in a safe environment where the cost or dangers of real-world training are high. The use of VTEs has increased significantly for several reasons: (1) simulations allow the environment to be reproduced under the control of the training designer/developer, (2) behaviors are too complex to be safely executed by a trainee, (3) instructor-led controlled environments allow trainees to exercise gained knowledge and conduct scenario-based training while receiving feedback from the instructor and simulator in real-time, and (4) simulation efforts are often cost-effective when compared to using on-the-job equipment (Goldstein & Ford, 2002).

Virtual Training Environments and Skills Transfer

The effectiveness of a virtual environment’s ability to train complex decision-making skills depends on the system’s psychological fidelity. Psychological fidelity requires the trainees to practice the mental processes and behaviors observed in the operational setting (Goldstein & Ford, 2002). When psychological fidelity is high, positive transfer is more likely to occur and facilitate correct behavior patterns similar to real-world performance. Several studies have demonstrated the transfer of spatial skills and procedural learning from a virtual environment to a real environment (e.g., Aurich et al., 2009; Brooks, 1999).

The necessity of psychological fidelity underscores the importance of human factors input in designing VTEs, as advanced technological capabilities alone are insufficient for learning and skills transfer (Stone, 2008). In fact, International Standard ISO 9241–210, Human-Centered Design Processes for Interactive Systems (2010), outlines how the application of human factors and ergonomics methods to interactive systems enhances the system’s effectiveness and efficiency. Stone (2008) suggested that even a robust training program can be completely undermined if content and fidelity are not adequately considered when creating interactive technologies.

The present paper reports on our use of a cognitive task analysis (CTA) to document the mental processes required by Navy pilots and aircrew to perform PDP successfully. These protocols can be used for training and incorporated into the development phase of VTEs.

Background on Cognitive Task Analysis

Task analysis is the most widely used approach to identifying worker behavior (Cannon-Bowers et al., 2013; Klein, 1995). The overarching goals of task analysis are to detail interactions between individuals and their working environment and to decompose jobs into tasks and subtasks (Cannon-Bowers et al., 2013; Stone, 2008). A CTA provides more comprehensive results by asking questions about the interrelations amongst tasks, knowledge, and skills in situations that involve less visible behaviors (e.g., decision-making; Goldstein & Ford, 2002). CTA includes a family of techniques used to elicit the cognitive processes that distinguish expert from novice performance on complex tasks (Clark et al., 2008; Klein, 1995) with a significant focus on goals, strategies, and decisions (Crandall et al., 2006; DuBois et al., 1998). CTA has been used successfully in various settings for training purposes. For example, in the context of medical procedures, Cannon-Bowers et al. (2013) used CTA to generate training requirements, performance metrics, training scenarios, and simulator requirements for combat casualty care techniques. Similarly, Johnson et al. (2006) used CTA to identify and describe the decision-making and physical actions required in core interventional radiology procedures. Given researchers’ prior success in implementing CTA in various domains, we expected the same methodologies to successfully identify the cognitive processes executed during PDP.

The Application Domain

Pilots view ejection as a last-ditch strategy. Nevertheless, pilots must be prepared to act when things go horribly wrong with the aircraft, such as combat damage or total engine failure. Of course, ejection is not as simple as pulling a lever and gliding peacefully to the earth. Pilots face possible turbulence, canopy malfunctions, and serious bodily injury, among other hazards.

The U.S. Navy requires pilots and aircrew to complete Aviation Survival Training before flying in a naval aircraft and every 4 years thereafter. PDP training is a component of the curriculum intended to teach aircrew and pilots the processes and decision-making strategies necessary to survive an ejection. To properly evaluate this training, it is crucial to accurately and completely describe the tasks and decision-making involved in PDP. Thus, a CTA was implemented to (1) identify the tasks, knowledge, skills, and abilities (KSAs) required for successful PDP training, (2) create performance metrics, and (3) evaluate parachute descent training simulators on their ability to train the required KSAs (Cannon-Bowers et al., 2013). This information was subsequently used to inform an iterative design process for an emerging training simulator and improve training design and implementation.

Context of Simulated Training

Navy pilot and aircrew training focuses on teaching the KSAs required for safely ejecting from an aircraft to touching down on land or water. Prior to practicing PDP in the simulators, trainees are required to complete a 30-minute classroom lecture. Once in the simulator, trainees must successfully demonstrate all components of PDP training. Navy PDP training involves four major categories: canopy malfunctions, IROK (Inflate, Release, Options, Koch Fittings), steering, and parachute landing fall (PLF) position. The IROK acronym reminds trainees to inflate the life preserver unit (LPU), release the survival kit or raft, decide on options (i.e., removal of gloves, oxygen mask, visor), and disconnect canopy release fittings. In simulated scenarios, trainees are required to perform corrective procedures for two canopy malfunctions and IROK procedures. Trainees must also understand parachute canopy steering mechanisms and operate the parachute canopy steering system. Finally, trainees must successfully identify the correct body position and procedures for PLF.

The Simulator Assessment

Participants and Procedure

Before participating in the study, SMEs completed an informed consent form. An Institutional Review Board approved the study protocol. We began the CTA by conducting a document analysis and observing Navy PDP training. Additional information was collected from Martin-Baker, a company that manufactures ejection seats for the Department of Defense. Martin-Baker’s Web site contains a section labeled Ejection Tie Club, which describes scenarios from real-world aircraft ejection experiences. This initial stage identified emergency bailout KSAs and provided the foundational knowledge required for protocol design in subsequent CTA steps (Clark et al., 2008).

Numerous knowledge elicitation methods exist for CTA, and when multiple methods are employed, researchers may gain significantly more insight (Crandall et al., 2006). Accordingly, this study utilized a multi-method approach to overcome the elicitation challenges inherent in emergency responses (e.g., limited SMEs, non-typical circumstances). First, we conducted interviews and focus groups with 10 SMEs, including Navy parachute descent instructors (n = 7), professional skydivers (n = 2), and a former pilot who had experienced an emergency ejection. Core questions, asked of all SMEs, revolved around standard descent procedures. We asked additional interview questions to leverage differences in SME expertise. For example, instructors described Navy PDP action sequences and essential KSAs, while skydivers provided unique perspectives on parachute malfunctions and protocols in atypical circumstances. The pilot with emergency ejection experience provided invaluable insight into the judgments and decisions required in a real-world event.

Next, nine SMEs participated in a three-phase think-aloud procedure while completing simulated descents, three of whom participated in the focus groups and interviews. SMEs included Navy PDP instructors (n = 6), pilots with experience in emergency bailouts (n = 2), and a free-fall jump master (n = 1). Instructions encouraged SMEs to verbalize everything on their mind as they completed tasks. While no formal practice on think-aloud protocols was conducted during this study, the first run-through was open-ended, with researchers providing minimal prompting to help guide SMEs through this process. This session allowed SMEs to familiarize themselves with the simulator and thinking aloud. In the second run-through, SMEs were instructed to complete the simulated descent while responding to questions about critical cues and decision points. In the third and final run-through, SMEs were asked to think aloud about differences between experts and novices in task completion, consequences of common errors, and observable trainee behaviors. Note takers and video recordings captured all relevant data (Figure 1).

Figure 1.

Parachute descent training simulator used for CTA.

During the second and third run-throughs, the simulation was paused at specific points to ask questions and repeat critical steps to elicit as much information as possible from SMEs. The pauses were essential because the task is not self-paced and, therefore, could prevent SMEs from verbalizing all aspects in real-time. Further, pauses also helped minimize cognitive overload that may result from completing a complex task while simultaneously providing verbal evaluation (Koch et al., 2019).

Throughout the simulated descents, each SME experienced various task/environmental conditions, including ejection altitudes, times of day, wind and weather conditions, terrain, and canopy malfunctions. Analogous to the interviews and focus groups, we asked all SMEs a core set of questions in rounds two and three of the think-aloud procedure. We asked additional questions depending on variable settings and SME insights. For example, in the second run-through, we prompted SMEs with parachute experience to consider both real-world and simulator cues; in the third run-through, we asked instructor SMEs questions related to determinants of successful task completion.

Results

Cognitive task analysis results are reported separately for each of PDP’s four main objectives: Correcting canopy malfunctions, IROK, steering, and PLF. Following the design outlined by Cannon-Bowers et al. (2013), we sorted responses and observations for each training objective into tables with the following columns: (1) tasks comprising the training objective, (2) cues used to perform the tasks, (3) the current simulator’s deficiencies or additional requirements, (4) observable trainee behaviors, (5) typical trainee/novice errors, and (6) decision-making demands.

Correcting Canopy Malfunctions Results

Results from the correcting canopy malfunctions analysis are shown in Table 1. Trainees must distinguish between various canopy malfunctions and a normal functioning parachute using visual, audio, and haptic feedback. At the most basic level, simulators should include accurate visual depictions of the malfunctions. Several SMEs suggested that different canopy malfunctions cause distinctly different auditory cues for each unique malfunction. As such, training could be improved by providing auditory feedback to trainees. Results indicate that trainees often incorrectly identify the canopy malfunction and recall erroneous corrective procedures. Failure to correct a canopy malfunction in a real-world situation can result in severe injury or death. Consequently, training should provide correct cues for canopy malfunction identification and corrective procedures.

Table 1.

Correct Canopy Malfunction.

Task	Cues Used to Perform	Simulator (Skyfall) Deficiencies or Requirements	Observable Trainee Behaviors	Typical Errors	Decision-Making Demands
1.1 Inspect canopy for malfunctions	Visual cues from parachute canopy and auditory cues (e.g., rushing wind)	Does not simulate opening shock	Trainee looks up	Forgetting to inspect canopy for malfunction	Interpretation of visual cues
	Haptic feedback	Does not simulate temporary loss of consciousness, possible injuries, and emotional stress	Trainee verbally states the malfunction	Waiting too long to inspect canopy for malfunctions
		Does not provide auditory cues or haptic feedback
1.2 Determine if corrective procedures are necessary	Visual cues from parachute canopy and auditory cues (e.g., rushing wind)	Visual depiction of streamer does not look like a streamer; there is no visual depiction of partial inversion or full inversion	Trainee verbally indicates if corrective procedures are necessary	Incorrect recognition of malfunction	Interpretation of visual and auditory cues
	Haptic feedback	There is no haptic feedback from the canopy or from pulling on the risers or lines		Incorrect corrective procedures for the identified malfunction	Identifying the appropriate corrective procedures for the identified malfunction
		There are fewer lines in the simulation connected to the parachute than in the real-world		Assuming canopy is functional when a malfunction is present
		There are no auditory cues
1.3 Perform corrective procedures for the identified malfunction	Visual cues from parachute canopy and auditory cues (e.g., rushing wind)	Does not simulate the corrective procedures for a full line over	Trainee pulls on risers	Incorrect recall of corrective procedures	Interpretation of visual and auditory cues
	Haptic feedback	Does not simulate the corrective procedures for a partial inversion	Trainee bicycle kicks	Pulling on the wrong component of the simulator to correct the malfunction (e.g., steering toggles instead of risers)	Interpretation of haptic feedback
		Does not simulate the corrective procedures for a full inversion	Trainee turns their head away while pulling on the risers	No corrective procedures attempted
		Calibration issues with the amount of force needed to correct malfunctions

IROK Results

In Table 2, six tasks constitute the training objective of performing IROK procedures. The most critical cues include the horizon line and visual feedback from the surrounding terrain. Such cues aid in determining altitude, which dictates when specific IROK procedures should be completed. For example, when experts' eyes were even with the horizon, they released their seat kit¹ and prepared for landing. The simulator deficiencies most regularly noted were horizon inconsistencies and the perception of a slower than real-world rate of descent. For instance, some SMEs questioned whether the simulator’s environment is mathematically correct. In order to solidify trust in the system’s accuracy, simulators should include various environmental cues to support determination of altitude, such as terrain features (e.g., waves, mountains, parachute shadows). According to SMEs, these features allow for object-size comparisons that aid in decision-making. Typical trainee errors include forgetting components of IROK procedures and the inability to interpret cues when estimating altitude, which could indicate that the VE provides insufficient psychological fidelity. Performance metrics should perhaps include an accuracy measure of student estimates of altitude and a measure of whether the trainee has performed specific IROK procedures by a specified altitude.

Table 2.

Perform IROK Procedures.

Task	Cues used to perform	Simulator deficiencies or requirements	Observable trainee behaviors	Typical errors	Decision-making demands
2.1 Prioritize parachute descent procedures based on altitude and terrain	Visual scan of terrain	Auditory cues (e.g., sounds of water)	Trainee looks at the displays	Trainee incorrectly identifies the scenario location	Interpretation of visual and auditory cues
	Auditory cues (e.g., sounds of water)	Horizon line may disappear	Trainee verbally indicates if they are overland or overwater	Trainee incorrectly judges their altitude	Knowledge of which procedures to prioritize
	Known flight information (e.g., location of plane during ejection or bailout)		Trainee verbally indicates if they have time to go through all parachute descent procedures	Trainee does not look at the horizon
	Visual scan of the horizon		Trainee looks at the horizon	Trainee does not look at objects in the environment (e.g., trees, waves)
	Object-size comparisons (e.g., trees, parachute shadow, ships, waves)		Trainee looks at objects in the environment (e.g., trees, waves)
2.2 Monitor rate of descent	Object-size comparisons (e.g., trees, parachute shadow, ships, waves)	The gap between the monitors makes it difficult for some to establish a horizon line	Trainee keeps eyes on the horizon line	Trainee does not look at the horizon	Interpretation of visual cues
	Visual scan of the horizon	The rate of descent in the simulation is slower than the rate of descent in a real-world scenario	Trainee looks around for changes in object sizes	Trainee does not look at objects in the environment (e.g., trees, waves)
		The simulated waves do not look realistic enough
2.3 Inflate LPU	Altitude	LPU does not limit field of view as it would in real-life	Trainee pulls LPU handle	Trainee does not pull LPU handle	Interpretation of visual cues
2.3 Inflate LPU	Visual scan of surrounding environment		Trainee verbally states they are inflating the LPU	Trainee does not verbalize that they are inflating their LPU	Interpretation of physical cues (e.g., correct force to use when pulling handle)
2.4 Activate seat kit to deploy raft (if overwater) or Discard seat kit unit (if overland)	Altitude	Motions to activate seat kit and release seat kit are not differentiated	Trainee unhinges seat kit from their harness	Trainee does not release seat kit (overland)	Interpretation of visual cues
	Visual scan of terrain	Trainee verbally states they are releasing their seat kit	Trainee does not activate seat kit (overwater)		Interpretation of physical cues (e.g., seat kit disconnecting)
	Object-size comparisons (e.g., trees, parachute shadow, ships, waves)	Trainee verbally states they are activating their seat kit	Trainee activates seat kit (overland)
			Trainee releases seat kit (overwater)
			Trainee releases or activates seat kit prior to 250 ft altitude
2.5 Raise visor, remove oxygen mask, remove gloves (if overwater) or loosen oxygen mask (if overland)	Visual scan of terrain	Visor, mask, and gloves are not simulated	Trainee does or does not verbally state that they are taking off their mask, visor, or gloves	Trainee does not verbalize any portion of the Options step of IROK	Interpretation of visual cues
		Mask is not included in the simulator checklist		Trainee does not raise oxygen mask (overwater)
				Trainee does not remove gloves (overwater)
				Trainee does not raise their visor (overwater)
				Trainee does not loosen their oxygen mask (overland)
				Trainee removes their gloves (overland)
				Trainee removes their visor (overland)
2.6 Release koch fittings	Altitude	Koch fittings are not included in the simulator checklist	Trainee locates koch fittings	Trainee does not locate koch fittings	Interpretation of visual cues
2.6 Release koch fittings	Visual scan of terrain		Trainee verbalizes releasing koch fittings	Trainee releases koch fittings too soon

Steering Results

Table 3 reveals the five tasks that make up the training objective of steering. According to SMEs, steering should only be utilized if time permits. In the case of a low-altitude ejection, steering will not typically be useful. In fact, SMEs advised that activating steering in such situations may be fatal, as this takes precious time away from completing other tasks that should be prioritized (i.e., inflating LPU, preparing for PLF). Furthermore, emergency bailout parachutes are often round in shape and do not steer as well as parachutes used for skydiving. Training programs should emphasize this decision-making process involved with steering so trainees can quickly assess and prioritize descent procedures if a real-world emergency bailout occurs.

Table 3.

Steer Parachute.

Task	Cues used to perform	Simulator deficiencies or requirements	Observable trainee behaviors	Typical errors	Decision-making points
3.1 Activate steering	Visual cues - do not activate if ejection is at very low altitude	N/A	Trainee pulls down sharply on toggles—located on inside of risers	Activating steering too quickly instead of going through other IROK steps	If and when to activate
	Tactile cues from steering toggles—grasp with hands and pull down to waist to fully activate		Letting go of steering lines		Interpretation of physical cues
			Not pulling down far enough to activate steering
3.2 Identify wind direction	Sensory cues (e.g., wind in face) and visual cues from smoke, rain, snow, waves, clouds, and surrounding terrain	More clouds are needed to cue wind direction and wind speed	Trainee verbally states wind direction	Does not look for wind cues or misinterprets wind cues	Interpretation of visual and sensory cues
		Does not simulate white caps on waves
		Does not simulate snow
		No sensory cues
3.3 Turn parachute into the wind	Tactile cues from steering toggles and visual cues from environment (e.g., field of view changes)	Steering slow to respond to user actions	Trainee steers into wind (e.g., look for trainees to steer in the opposite direction of waves)	Does not turn parachute into wind or turns in the wrong direction	Interpretation of tactile and visual cues
3.3 Turn parachute into the wind					Amount of force to apply to steering toggles
3.4 Scan terrain to avoid nonoptimal landing areas	Visual cues from surrounding environment	N/A	Trainee visually scans surrounding terrain and verbally states landing areas to avoid	Does not consider unsafe landing areas	Interpretation of visual cues
3.5 Steer away from hazardous landings	Visual cues from surrounding environment and tactile cues from steering toggles	Limited obstacles that must be avoided	Trainee steers around any hazardous areas and attempts to land in a safe landing zone	Trainees try to land on aircraft carrier instead of steering away from it or do not steer away from trees	Interpretation of visual and tactile cues
3.5 Steer away from hazardous landings					Determining safest landing environment

If it is necessary to activate steering, the most significant cue available to pilots is wind. This cue is essential because pilots and aircrew are trained to steer into the wind to reduce their descent rate. Correspondingly, simulators should provide sufficient wind cues such as movement in the surrounding terrain and sensory feedback. Results also revealed the importance of steering away from obstacles (e.g., trees) to land safely. Trainees will not meet this objective without a functional understanding of steering components. Therefore, training programs and simulators should create realistic cues regarding steering capabilities and limitations. Observable trainee behaviors include trainees sharply pulling down on loops inside the risers, visually scanning the surrounding terrain, and steering around hazardous areas. Typical trainee errors include forgetting to activate, activating pre-emptively, and not steering into the wind. Metrics could include activating steering by a specific altitude, steering into the wind, and knowledge of steering behaviors.

Parachute Landing Fall Position Results

Table 4 details the two tasks comprising the PLF Position training objective. Though trainees do not perform an entire PLF in a training simulator, results determined that they should display specific behaviors such as relaxing the body, slightly bending their knees, keeping their feet and knees together, and keeping their chin up. To successfully prepare for landing, SMEs use the horizon, parachute shadow, and an indication of forward or backward movement to assume the PLF positioning at the right moment. Some SMEs highlighted the lack of ground references to indicate distance from the ground as an area for recommended simulator improvements. Ground references are necessary for trainees to identify their current altitude so that they may prepare for PLF at 250 feet above ground level. Typical trainee errors include preparing for landing too late or displaying improper body position. Metrics could include the display of correct body positioning at a specific altitude.

Table 4.

PLF Position.

Task	Cues used to perform	Simulator deficiencies or requirements	Observable trainee behaviors	Typical errors	Decision-making demands
4.1 Recognize ground rush if overland	Visual cues from horizon and parachute shadow	Ground speed is too slow	Bracing for landing by tightly grasping risers	Misjudges ground rush or begins bracing for landing too late	Interpretation of visual and proprioceptive cues
4.1 Recognize ground rush if overland	Proprioceptive cues (e.g., forward or backward motion)	Not enough ground references to indicate distance from ground			Knowledge of when to brace for landing
4.2 Get in proper body position for a PLF	Visual cues from environment and kinesthetic cues (e.g., knees slightly bent)	Does not provide physical feedback of touchdown	Prepares for PLF at approximately 200 ft. Altitude	Improper body positioning or does not get into PLF position at correct altitude	Interpretation of visual and kinesthetic cues
4.2 Get in proper body position for a PLF			Trainee keeps chin up, feet together, knees slightly bent, arms at chest, body relaxed		Knowledge of when to prep for PLF

Significance of Improving Virtual Environment Training Systems

While methods such as analysis of incident and accident reports can yield recommendations for training (e.g., Taber, 2014), leveraging a CTA produces more comprehensive results that can inform various training-related domains (e.g., training needs, performance metrics, simulator capability requirements). In the realm of instructional design, particularly instructional design for high-risk training, CTA is nearly essential. Training programs are most efficient when the mental models, strategies, and skills required for successful task performance are considered early in the design phase.

Several findings of this study were implemented in the design and development of a novel parachute descent training simulator. Specifically, the simulator developers augmented graphical images to include various depth cues that aid in malfunction identification and altitude determination and added additional automated performance metrics. Further, a virtual reality (VR) headset was integrated to allow for an evaluation of monitor versus VR display in a follow-up study. In addition, the CTA identified essential training elements, including typical errors, decision-making demands, and performance metrics. Resulting improvements in the psychological fidelity of critical cues in the simulator are likely to contribute to more effective learning of PDP in the United States Navy.

Like prior studies involving CTA for high-level skills (e.g., Cannon-Bowers et al., 2013; Koch et al., 2019), very few individuals have experienced a real-world emergency bailout with a canopy malfunction. Due to this limited expert pool, recruiting individuals with real-world experience to serve as SMEs proved challenging. Illustrating this issue, Newman (2013) found that out of 562 low-level ejections, 274 were fatalities, resulting in a survival rate of 51.2%. To address the sample size restrictions, PDP training instructors served as SMEs, per the strategy of Cannon-Bowers et al. (2013), which brought the total sample to 16 SMEs. In such a specialized sample, potential outside influences could exist yet remain undetected. For example, there may be slight variations in the ejection process depending on the aircraft and parachute system. Although we sought a diverse set of opinions and used triangulation techniques to improve the validity of our results, a follow-up study could explore such issues in greater depth. With access to a larger sample, researchers could stratify by aircraft platform and participant experience to improve generalizability and coverage. Additionally, leveraging a think-aloud protocol with a pre-phase to increase familiarization with the process might increase the robustness of data responses.

To our knowledge, CTA has not previously been utilized in this domain. Results of the current CTA contributed to a more thorough understanding of the tasks comprising PDP training and how current trainers can be improved by adding more realistic task-related cues. Should the individuals that undergo PDP training experience a real-world ejection and canopy malfunction in the future, the improved psychological fidelity provided by CTAs will potentially result in transfer of training, which may mean the difference between life and death.

Footnotes

Note

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Small Business Innovation (Navy Phase III SBIR N161-007, Subcontract No.: SC-).

Cayla Hartley is a Personnel Psychologist at the National Science Foundation. She received her M.A. in I-O Psychology from the University of West Florida in 2021. She has been involved in consultancy projects in the areas of training, assessment, and organizational change. Her primary research interest is the relationship between individual attributes and performance in work settings.

Kenzie Hurley is a current Industrial-Organizational (I-O) Psychology PhD Candidate at Clemson University, with her M.A. in I-O Psychology from the University of West Florida. As a consultant, Ms. Hurley has worked with the Department of Defense and Navy for 3 years, as well as for other large-scale organizations. Her work has shaped training for the Navy and has assisted the Department of Defense with understanding factors that can influence workplace equality in military units.

Valerie J. Morganson is an Associate Professor of Psychology at the University of West Florida. She earned her PhD in Industrial-Organizational Psychology at Old Dominion University. Dr Morganson teaches undergraduate and graduate courses related to workplace psychology and research methods.

Steve Kass earned a PhD in human factors psychology from the University of Central Florida (1993) and M.A. in industrial-organizational psychology from the University of West Florida (1989). He currently holds the rank of Distinguished University Professor at the University of West Florida where he teaches General Psychology, Psychology of Learning, Training & Development, and Human Factors and conducts research on topics related to attention, including situation awareness, distracted driving, boredom, training, and virtual reality.

Beth F. Wheeler Atkinson is a Senior Research Psychologist at Naval Air Warfare Center Training Systems Division and a NAVAIR Associate Fellow. She oversees a portfolio of research and development efforts related to aircrew and maintenance training, aviation survival training, human performance data analytics, automation and decision support tools, and instructional technology improvements. She holds an M.A. in Psychology, Applied Experimental Concentration, from the University of West Florida.

Matthew Pierce is a Research Psychologist at the Naval Air Warfare Center Training Systems Division (NAWCTSD). He received a Bachelor of Science degree in psychology from Indiana University Purdue University of Indianapolis and a Master’s degree in Human Factors Psychology from Embry-Riddle Aeronautical University. His current field is developing training that meets fleet needs with a primary focus on aviation training. His research interests are in mixed reality, training effectiveness, and aviation survival training.

The authors would like to thank Dean Jones for his early contributions to the project.

Presentation of this material does not constitute or imply its endorsement, recommendation, or favoring by the U.S. Navy or the Department of Defense (DoD). The views expressed in this paper are solely those of the authors, and do not necessarily reflect the opinions of the Naval Air Warfare Center Training Systems Division or any other Department of Defense agency, unless stated in official directives.

Small Business Innovation Research (Navy Phase III SBIR N161-007, Subcontract No.: SC-).

References

Aurich

J. C.

Ostermayer

Wagenknecht

C. H.

(2009). Improvement of manufacturing processes with virtual reality-based CIP workshops. International Journal of Production Research, 47(19), 5297–5309. https://doi.org/10.1080/00207540701816569

Brooks

B. M

. (1999). The specificity of memory enhancement during interaction with a virtual environment. Memory, 7(1), 65–78. https://doi.org/10.1080/741943713

Cannon-Bowers

Bowers

Stout

Ricci

Hildabrand

(2013). Using cognitive task analysis to develop simulation-based training for medical tasks. Military Medicine, 178(10), 15–21. https://doi.org/10.7205/MILMED-D-13-00211

Clark

R. E.

Feldon

vanMerriënboer

J. J. G.

Yates

Early

(2008). Cognitive task analysis. In Spector

J. M.

Merrill

M. D.

van Merriënboer

J. J. G.

Driscoll

M. P.

(Eds.), Handbook of research on educational communications and technology (3rd ed.). Lawrence Erlbaum Associates.

Crandall

Klein

Hoffman

R. R.

(2006). Working minds. A practitioners guide to cognitive task analysis. MIT Press.

DuBois

D. A.

Shalin

V. L.

Levi

K. R.

Borman

W. C.

(1998). A cognitively oriented approach to task analysis and test development. https://corescholar.libraries.wright.edu/psychology/455

Gallagher

A. G.

McClure

McGuigan

Crothers

Browning

(1999). Virtual reality training in laparoscopic surgery: A preliminary assessment of minimally invasive surgical trainer virtual reality (MIST VR). Endoscopy, 31(04), 310–313. https://doi.org/10.1055/s-1999-15

Goldstein

Ford

(2002). Training in organizations (4th ed.). Wadsworth/Thomson Learning.

Hogue

J. R.

Pelz

C. A.

(2008). Enhanced safety and effectiveness for parachute training and mission performance through virtual reality simulation (Paper No. 705). Systems Technology, Incorporated. http://parasim.com/wp-content/uploads/2016/04/STI-P-705.pdf

10.

International Organization for Standardization . (2010). Ergonomics of human-system interaction – Part 210: Human-centered design for interactive systems: International Organization for Standardization. (ISO Standard No. 9241-210:2010). https://www.iso.org/standard/52075

11.

Johnson

Healey

Evans

Murphy

Crawshaw

Gould

(2006). Physical and cognitive task analysis in interventional radiology. Clinical Radiology, 61(1), 97–103. https://doi.org/10.1016/j.crad.2005.09.003

12.

Klein

(1995). The value added by cognitive task analysis. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 39(9), 530–533. https://doi.org/10.1177/154193129503900920

13.

Koch

Pfandler

Stefan

Wucherer

Lazarovici

Navab

Stumph

Schmidmaier

Glaser

Weigl

(2019). Say, what is on your mind? Surgeons’ evaluations of realism and usability of a virtual reality vertebroplasty simulator. Surgical Innovation, 26(2), 234–243. https://doi.org/10.1177/1553350618822869

14.

Newman

D. G.

(2013). Survival outcomes in low-level ejections from high performance aircraft. Aviation, Space, and Environmental Medicine, 84(10), 1061–1065. https://doi.org/10.3357/ASEM.3626.2013

15.

Ragan

E. D.

Sowndararajan

Kopper

Bowman

D. A.

(2010). The effects of higher levels of immersion on procedure memorization performance and implications for educational virtual environments. Presence: Teleoperators and Virtual Environments, 19(6), 527–543. https://doi.org/10.1162/pres_a_00016

16.

Stavroulia

K. E.

Baka

Lanitis

Magnenat-Thalmann

(2018). Designing a virtual environment for teacher training: Enhancing presence and empathy. Proceedings of Computer Graphics International 2018, 273–282. Association for Computing Machinery. https://doi.org/10.1145/3208159.3208177

17.

Stone

R. J.

(2008). Human factors guidelines for interactive 3D and games-based training systems design. Human Factors Integration Defence Technology Centre Publication, 6(1), 5–86. https://www.researchgate.net/profile/Robert-Stone-12/publication/228615549_Human_factors_guidelines_for_interactive_3D_and_games-based_training_systems_design/links/00b495253e767b3855000000/Human-factors-guidelines-for-interactive-3D-and-games-based-training-systems-design.pdf

18.

Taber

M. J.

(2014). Simulation fidelity and contextual interference in helicopter underwater egress training: An analysis of training and retention of egress skills. Safety Science, 62(1), 271–278. https://doi.org/10.1016/j.ssci.2013.08.019

19.

U.S. Department of Agriculture, Forest Service . (2018). Ram-air parachute system transition operations plan. U.S. Department of Agriculture, Forest Service. https://www.fs.usda.gov/sites/default/files/media_wysiwyg/2018_ram-air_operations_plan_final_31january2018signed_1.pdf