OT&E Service Report Classification

Many problems encountered during operational test and evaluation of Air Force systems are described in service reports, an important element in the communication of test findings to the system acquisition community. Since many service reports cite deficiencies in the man-machine interface, service report data are an important source of human factors information. However, they are not readily integrated with other test data because of their qualitative character. This paper describes attempts to classify service reports from two complex command, control and communications type systems in order to make the data more usable. Classification schemes emphasizing (1) likely engineering solutions to the problems and (2) human activities affected by the problems were applied. Classification results were consistent with known characteristics of the system, and revealed interesting trends in the data. Some support was obtained for both the reliability and validity of the classification schemes as well. Implications of the results for service report analysis are discussed.