The question of simulation fidelity and validity is of concern in many driving applications. This paper reviews the general issues associated with fidelity and validity. For training applications, fidelity and validity relate to training effectiveness. Training effectiveness is typically measured according to real-world driver behavior and performance. This paper describes a novice driver training experiment that used simulators with three levels of fidelity. Driver simulator performance was measured for complex driving scenarios containing repeated exposures to hazardous situations. The driving records of the participants were also analyzed for a period of 40 months after they were trained. Inferences about simulator fidelity and validity are drawn on the basis of the participants' simulator performance and real-world driving records. Confounding effects in the database due to several uncontrolled variables are also discussed. These confounding effects are relevant to the design of future transfer-of-training studies.
Get full access to this article
View all access options for this article.
References
1.
FildesB. N., GodleyS., TriggsT. J., and JarvisJ.Perceptual Countermeasures: Simulator Validation Study. Report FORS CR-169. Australian Federal Office of Road Safety, Canberra, April 1997.
2.
WadeM. G., and HammondC.Simulation Validation: Evaluating Driver Performance in Simulation and the Real World. Final report MN/RC-1998–28. University of Minnesota, Minneapolis, July 1998.
3.
LeonardJ. J., and WierwilleW. W.Human Performance Validation of Simulators; Theory and Experimental Verification. Presented at 19th Annual Meeting of the Human Factors Society, Dallas, Tex., 1975.
4.
KapteinN. A., TheeuwesJ., and Van Der HorstR.Driving Simulator Validity: Some Considerations. In Transportation Research Record 1550, TRB, National Research Council, Washington, D.C., 1996, pp. 30–36.
5.
Technical Recommendations for Psychological Tests and Diagnostic Techniques.American Psychological Association, Washington, D.C., 1954.
EbelR. L.Must All Tests Be Valid?American Psychologist, Vol. 16, No. 10, 1961, pp. 640–647.
8.
McCoyW. K.Problems of Validity of Measures Used in Investigating Man–Machine System. Human Factors, Vol. 5, 1963, pp. 373–377.
9.
GoodwinL. D., and LeechN. L.The Meaning of Validity in the New Standards for Educational and Psychological Testing: Implications for Measurement Courses. Measurement and Evaluation in Counseling and Development, Vol. 36, Oct. 2003.
10.
BellaF.Validation of a Driving Simulator for Work Zone Design. In Transportation Research Record: Journal of the Transportation Research Board, No. 1937, Transportation Research Board of the National Academies, Washington, D.C., 2005, pp. 136–144.
11.
AllenR. W., RosenthalT. J., and AponsoB. L.Performance Measurement Paradigms for Driving Simulator Research. Advances in Transportation Studies, an International Journal, Special Issue, 2005, pp. 45–58.
12.
AllenR. W., ChrstosJ. P., HoweJ. G., KlydeD. H., and RosenthalT. J.Validation of a Non-Linear Vehicle Dynamics Simulation for Limit Handling. Journal of Automobile Engineering Special Issue Paper, Proc. Institution of Mechanical Engineers, Vol. 216, Part D, 2002, pp. 319–327.
13.
BlaiwesA. S., PuigJ. A., and ReganJ. J.Transfer of Training and the Measurement of Training Effectiveness. Human Factors, Vol. 15, No. 6, 1973, pp. 523–533.
14.
RoseA., EvansR., and WheatonG.Methodological Approaches for Simulator Evaluations. In Transfer of Learning: Contemporary Research and Applications (CormierS. M., and HagmanJ. D., eds.), Academic Press, San Diego, Calif., 1987.
15.
AllenR. W., MitchellD. G., SteinA. C., and HogueJ. R.Validation of Real-Time Man-in-the-Loop Simulation. VTI Report 372A, Part 4. Swedish Road and Traffic Research Institute, 1991.
16.
SieglerI., ReymondG., KemenyA., and BerthozA.Sensorimotor Integration in a Driving Simulator: Contributions of Motion Cueing in Elementary Driving Tasks. Proc. Driving Simulation Conference, Sophia-Antipolis, France, 2001.
17.
SargentR.Verification and Validation of Simulation Models. Proc. 1998 Winter Simulation Conference. Special Interest Group on Simulation, Association of Computing Machinery, New York, 1998.
18.
BalciO.Verification, Validation and Accreditation of Simulation Models. Proc. 1997 Winter Simulation Conference. Special Interest Group on Simulation, Association of Computing Machinery, New York, 1997.
19.
Clinical Data Interchange Standards Consortium Submission Data Standards Team.Study Data Tabulation Model Implementation Guide: Human Clinical Trials.http://www.cdisc.org/. Accessed July 9, 2009.
20.
AllenR. W., RosenthalT. J., ParkG. D., CookM. L., FiorentinoD. D., and ViirreE.Experience with a Low Cost, PC Based System for Young Driver Training. In Driver Behaviour and Training, Vol. I (DornL., ed.), Ashgate Publishing Limited, Hampshire, United Kingdom, 2003.
21.
AllenR. W., CookM. L., and ParkG. D.Novice Driver Performance Improvement with Simulator Training. In Driver Behaviour and Training, Vol. II (DornL., ed.), Ashgate Publishing Limited, Hampshire, United Kingdom, 2005, pp. 81–91.
22.
ParkG. D., CookM. L., AllenR. W., and FiorentinoD. D.Automated Assessment and Training of Novice Drivers. Advances in Transportation Studies, an International Journal, Special Issue, 2006, pp. 87–96.
23.
AllenR. W., RosenthalT. J., AponsoB. L., HarmsenA., and MarkhamS.A Low Cost Platform for Training and Evaluating Driver Behavior. Presented at 7th Driving Simulator Conference and Exhibition, Paris, 2002.
24.
ParkG. D., RosenthalT. J., and AponsoB. L.Developing Driving Scenarios for Research, Training and Clinical Applications. Advances in Transportation Studies, an International Journal, Special Issue, 2004, pp. 19–28.
25.
IvancicK., and HeskethB.Learning from Errors in a Driving Simulation: Effects on Driving Skill and Self-Confidence. Ergonomics, Vol. 43, No. 12, 2000, pp. 1966–1984.
26.
JoungW., HeskethB., and NealA.Using “War Stories” to Train for Adaptive Performance: Is It Better to Learn from Error or Success?Applied Psychology, Vol. 55, No. 2, 2006, pp. 282–302.
27.
AllenR. W., ParkG. D., and CookM. L.The Effect of Simulation Training on Novice Driver Accident Rates. In Driver Behaviour and Training, Vol. III (DornL., ed.), Ashgate Publishing Limited, Burlington, Vt., 2008.
28.
ParkG. D., AllenR. W., RosenthalT. J., and FiorentinoD. D.Training Effectiveness: How Does Driving Simulator Fidelity Influence Driver Performance? Presented at Human Factors Ergonomics Society 49th Annual Meeting, Orlando, Fla., 2005.
29.
JankeM. K., MastenS. V., McKenzieD. M., GebersM. A., and KelseyS. L.Teen and Senior Drivers. Report CAL-DMV-RSS-03-194. California Department of Motor Vehicles, Sacramento, July 2003.
30.
MayhewD., SimpsonH., and PakA.Changes in Collision Rates Among Novice Drivers During the First Months of Driving. Accident, Analysis and Prevention, Vol. 35, No. 5, 2003, pp. 683–691.
31.
AllenR. W., ParkG. D., CookM. L., and FiorentinoD. D.The Effect of Driving Simulator Fidelity on Training Effectiveness. Presented at Driving Simulation Conference 2007 North America, Iowa City, Iowa, 2007.
32.
WestraD. P., SheppardD. J., JonesS. A., and HettingerL. J.Simulator Design Features for Helicopter Shipboard Landings. II. Performance Experiments. Final Report NTSC TR87-041. Essex Corporation, Orlando, Fla., July 9, 1987.
