Subjective Performance Measures in the Test and Evaluation of Simulator Training Scenarios

Scenarios were developed to provide A-10 student pilots training in low level navigation (LLN) and elementary basic attack maneuvers (BAM). The primary objective of this effort was to evaluate the effectiveness of these scenarios using the A-10 Advanced Simulator for Pilot Training (ASPT) as the training medium. Training effectiveness was assessed using simulator automated performance measures and instructor pilot (IP) evaluations of student aircraft performances in a transfer of training paradigm. Forty-two subjects participated in the BAM phase of the experiment: thirty-six subjects were used in the LLN phase. Simulator measures showed enhanced performance as a function of training for three of the five BAM tasks trained: LLN simulator performance data were not analyzed due to difficulty with data collection apparatus. Power values, reliability tests and correlation results for aircraft performance measures made questionable the validity of using these measures for test purposes as was done in this experiment. Discussion focusses on the need for more objective performance metrics in future evaluation programs and reviews alternative solutions to fulfill this requirement.