Book Review: Workload Assessment: How to Diagnose Workload Issues and Enhance Performance

This users’ guide definitely lives up to that designation. Although the authors say in the preface that they did not want to produce a simple cookbook, it is a very good cookbook indeed. Like all good cookbooks, it goes beyond a mere listing of ingredients; it provides useful how-to guidance, including discussions of things to look out for and things to avoid. Chapters cover theories, uses and applications of workload measurement, the relation of mental workload to attention and vigilance, the relation of mental workload measurement to performance measurement, and team workload. It describes many individual methods, pointing out strengths and limitations of each. The book covers psychometric approaches (e.g., validity, reliability, scaling issues). Case studies detail the application of mental workload measurement, and these include useful roadmaps in the form of steps and checklists.

Workload Assessment is an excellent example of scientific scholarship at its very best. The authors show a firm grasp of the literature and effectively encapsulate that literature, which spans decades. Abundant references point readers to key papers and publications that go into depth about particular topics and applications. This is commendable.

An aspect of the Zeitgeist that plays a big role in the evaluation of mental workload is the distinction between so-called subjective and so-called objective measures. It is commonly – or almost invariably – assumed that subjective measures (e.g., ratings, verbal reports) are limited, biased, untrustworthy, and so on and that the primary purpose of physiological measures is to make the research “robust” (that is, make sure it is Genuine Science). The authors of this book try to avoid endorsing this stance. The distinction between subjective and objective measures was dynamited in philosophy of science decades ago, even in the pages of Human Factors. To be balanced, Matthews and Reinerman-Jones point out the limitations of physiological measures and performance measures as surrogates for mental workload measures.

Although the book is judicious regarding the strengths and limitations of the methods, and includes many useful cautionary tales, I do have a few quibbles that speak to the philosophical or methodological mind-set that seems a part of the Zeitgeist. I offer one of these as a footnote to this excellent book.

The book’s subtitle includes the word diagnose. It is commonly − or almost invariably − assumed that high mental workload is a bad thing and that the primary reason for building new technologies is to reduce operator mental workload. Although the automated workplace can sometimes enslave operators, and off-nominal events can be overwhelming, we also know that genuine expertise is achieved only after long experience working hard on hard problems. As technology advances, we need more experts, not fewer operators. On the assumption that surprise is always likely and the workplace is a moving target, we certainly want people to experience high workload, to actually practice at it.

I emphasize that mental workload measurement is about mental workload. Any evaluation must somehow get inside the heads of workers, even if it relies on physiological measurement. Just because a method − any method − has limitations and potential data interpretation issues does not mean one should not apply the method and see where the results lead. As Neville Moray once told me, reflecting on his decades of work creating and evaluating numerous measurement methods, “If someone says they feel mentally overloaded, they probably are.”

But I find myself now perhaps overplaying some methodological or philosophical issues. I advise those who are new to this area of human factors to absorb, and not just read, this excellent book. And also read a good treatment of the history of cognitive task analysis. I wish above all to emphasize the value of Workload Assessment. I intend to keep it very handy, and I most highly recommend that it be embraced by cognitive systems engineers and utilized to its fullest by anyone who is engaged in human factors evaluation of the human–machine workplace.