Workload Measurement in System Design and Evaluation

Abstract

Because of its central role in system development, workload measurement has been extensively researched. These efforts have produced a variety of workload assessment techniques, many of which can be classified as either subjective, physiological, or behavioral measures. These categories of measures can vary along several dimensions that can be used as criteria in selection of a technique for a particular application. The proposed selection criteria include the sensitivity, diagnosticity, and intrusiveness associated with a technique. Different stages of system design can require techniques that differ on the noted dimensions. Since no technique is capable of meeting all of the applicable criteria, a comprehensive approach to workload assessment will require a battery of subjective, physiological, and behavioral measures. Future research dealing with comparative evaluation of the various assessment techniques along the noted dimensions will be required in order to refine workload metric selection criteria.

Get full access to this article

View all access options for this article.

References

Beatty

(1982). Task-evoked pupillary responses, processing load, and the structure of processing resources. Psychological Bulletin, 91, 276–292.

Eggemeier

F.T.

(1984). Workload metrics for system evaluation. Proceedings of the Defense Research Group Panel VIII Workshop “Application of System Ergonomics to Weapon System Development“, Shrivenham, England, pp. c/5–c/20.

Eggemeier

F.T.

Crabtree

M.S.

LaPoint

P.A.

(1983). The effect of delayed report ou subjective ratings of mental workload. Proceedings of the Human Factors Society Twenty-Seventh Annual Meeting, pp. 139–143.

Isreal

J.B.

Chesney

G.L.

Wickens

CD.

Donchin

(1980). P300 and tracking difficulty: Evidence for multiple resources in dual-task performance. Psychophysiology, 17, 259–273.

O'Donnell

R.D.

Eggemeier

F.T.

(1985). Workload assessment methodology. In Kaufman

Thomas

Boff

(Eds.), Handbook of Perception and Performance, New York: Wiley, in press.

Reid

G.B.

(1985). Systematic development of a subjective measure of workload. Proceedings of the Ninth Congress of the International Ergonomics Association, in press.

Schifflet

S.G.

Linton

P.M.

Spicuzza

R.J.

(1982). Evaluation of a pilot workload assessment device to test alternative display formats and control handling qualities. Proceedings of the AIAA Workshop on Flight Testing to Identify Pilot Workload and Pilot Dynamics, pp. 222–233.

Shingledecker

C.A.

(1983). Behavioral and subjective workload metrics for operational enviroilments. Proceedings of the AGARD Symposium on Sustained Intensive Air Operations, Physiological and Performance Aspects, (AGARD-CP-338), pp. 6/1-6/10.

Shingledecker

C.A.

Acton

W.H.

Crabtree

M.S.

Development and application of a criterion task set for workload metric evaluation. (Paper No. 831419). Warrendale, Pennsylvania: Society of Automotive Engineers Technical Paper.

10.

Wickens

CD.

(1984). Processing resources in attention. In Parasuraman

Davies

D.R.

(Eds.), Varieties of Attention, New York: Academic Press, 1984.

11.

Wickens

CD.

Derrick

(1981). Workload measurement and multiple resources. Proceedings of the IEEE Conference on Cybernetics and Society, pp. 600–603.

12.

Wickens

CD.

Kessel

(1980). The processing resource demands of failure detection in dynamic systems. Journal of Experimental Psychology: Human Perception and Performance, 6, 564–577.

13.

Wickens

CD.

Yeh

Y.Y.

(1983). The dissociation of subjective ratings and performance: A multiple resources approach. Proceedings of the Human Factors Society Twenty-Seventh Annual Meeting, pp. 244–248.

14.

Wierwille

W.W.

Casali

J.G.

(1983). The sensitivity and intrusion of mental workload estimation techniques in piloting tasks. Blacksburg, Virginia: IEOR Department Report No: 8309, Department of Industrial Engineering and Operations Research, Virginia Polytechnic Institute and State University.

15.

Wierwille

W.W.

Williges

R.C.

(1978). Survey and analysis of operator workload assessment techniques. Blacksburg, Virginia: Report No. S-78-101, Systemetrics, Inc.