Checking Out the Checklist: Evaluation of a Job Performance Aid for Assessing Organizational Resource Management

As a job performance aid, the checklist has been applied to a wide variety of organizational requirements such as task analysis, survey questionnaire development, office automation, and maintenance. Although there are many examples of checklists that have been developed for different organizational requirements, there has been relatively little research performed to evaluate the checklists. One reason for this may be the lack of an acceptable methodology and evaluation criteria. This paper (a) describes a methodology and the criteria developed to evaluate the checklist used to assess the management of aviation resources by Army Reserve Component training facilities and units and (b) summarizes the results of the evaluation. Army National Guard and Army Reserve aviators and aviation technicians used 5-point rating scales to evaluate the Detectability, Importance, and Criticality of the checklist items. A data base was developed to summarize the Detectability, Importance, and Criticality information. In addition, a graphic decision aid was designed to help military managers decide whether to retain, revise, or delete individual checklist items. The methodology and criteria developed for this research may be applicable in other organizational contexts, especially those in which there is a requirement to evaluate an existing checklist and a full-scale validation effort is not feasible.