Proceedings of the Human Factors and Ergonomics Society Annual Meeting

Cognitive Engineering and Naturalistic Decision Making are presented as two related fields of endeavor that seek to understand how people process information and perform within complex systems and to develop ways of applying this knowledge within the design and training process This panel presents an overview of the current state of the art in this research domain and charts paths for needed developments in the field in the near future.

The Tactical Decision Making Under Stress program is being conducted to apply recent developments in decision theory and human-system interaction technology to the design of a decision support system for enhancing tactical decision making under highly complex conditions. Topics to be discussed include: (1) a description of the difficult tasks identified for analysis; (2) the general methodological approach; (3) development of the performance measures and issues related to their development; (4) discussion of the modification and extension of the TapRoot® Incident Investigation System; and (5) discussion of the types of errors made by decision makers and interpretations for the cause of these errors based in the cognitive psychology literature.

It has often been noted that today's human operators of complex industrial systems must occasionally deal with multiple component failures, but that they are trained instead to think in terms of single faults. This can lead to the minor inconvenience of simply taking longer to troubleshoot, or to the major hazards of a fundamental misunderstanding of system state. In two studies we examined these influences on subjects' ability to diagnose a multiple fault in a simulated electronic circuit: (1) objective multiple fault difficulty, (2) prior practice with multiple faults, and (3) expectancy or mental set for multiple faults. Previous research had confounded the latter two variables. Experiment 1 showed that the difficulty of multiple faults varied as our model predicted, but that the difficulty of certain faults interacted with verbalization. Experiment 2 showed that both prior practice and expectancy influence how effectively subjects deal with a difficult multiple fault, but not quite as expected. We conclude that the ability to diagnose multiple faults is multiply determined, depending on degree of practice, mental set, and the difficulty of the multiple fault itself. These results will help us define the requirements for decision support tools and they have also led us to perform investigations in the field, as reported in Reising and Sanderson (1995).

Recent experimental research has indicated that different multiple faults impose differing levels of objective and subjective difficulty on human troubleshooters. Technological advances suggest that systems are becoming more complex and integrated, in which case multiple components will fail. Operators will have to be able to deal with these more complex failures. In this paper we report field work conducted in order to build and substantiate a model of the factors influencing fault diagnosis in the field. By conducting field observations and by constructing concept maps, we investigated how expert troubleshooters handle the difficulty associated with diagnosing multiple faults. The troubleshooters were expert electronic technicians in departmental repair shops on a large university campus. The end product of the research is a model of fault diagnosis that is grounded in field data. Our results suggest that diagnostic difficulty arises from several factors: (1) organizational structure, (2) technicians' strategies for fault diagnosis, and (3) equipment design. The field observations and concept maps indicate that technicians approach the diagnostic task with standard, ritualistic methods that they have developed over years of experience. They generally go through two phases of troubleshooting: (1) the problem definition phase and (2) what we call the At-the-Equipment-TroubleShooting (AETS) phase. Technicians also reason about multiple failures in series, considering one simple explanation at a time. Our principal conclusion is that in real-world settings the three previously mentioned factors have evolved to avoid situations in which technicians must engage in prolonged functional reasoning. These findings will be used (1) to develop further the model of fault diagnosis, and (2) to guide future experimental investigations studying the influences of fault diagnosis.

One of the goals of naturalistic studies of human decision making is to reveal the cognitive loads or task difficulties imposed on the decision maker in real work environments. Fixation errors or cognitive lockups have been reported as a unique type of performance failure in dynamic work environments, and are thus particularly valuable to the understanding of the challenges and difficulties confronting practitioners in dynamic environments. In this paper, we present the analysis of fixation errors during real-life trauma patient resuscitation. The analysis elicits two factors, both rooted in the inherent complexity of the domain, that contributed to the occurrence of fixation errors: unreliable monitoring devices and delayed feedback. The former induces the behavior of preferring confirmatory information, partly for redundancy checks. The latter may create a false sense of system stability and divert attention away from the correct diagnosis.

One way to examine the progress of the Human Factors discipline is to perform a content analysis of the programs of the Society's early annual meetings. The period covered is 1959-1972, after which proceedings were published. The results of the analysis shows that interest in some research topics ebbs and flows, whereas other interests remain relatively stable.

In this paper, the results of a first study into the use of virtual reality for human factor studies and design of simple and complex models of control systems, components, and processes are described. The objective was to design a model in a virtual environment that would reflect more characteristics of the user's mental model of a system and fewer of the designer's. The technology of a CAVE™ virtual environment and the methodology of Neuro Linguistic Programming were employed in this study.

Designing ScienceSpace, a series of virtual realities for teaching difficult science concepts and skills, has implications for designing sensorily immersive educational virtual realities. Through the design and evaluation of the worlds in ScienceSpace we are gaining insights into the general utility of sensorial immersion, as well as virtual reality's potential and limitations for enhancing learning. This paper focuses on the learner-centered design and evaluation of NewtonWorld, one of the virtual worlds in ScienceSpace. NewtonWorld is a sensorily immersive virtual learning environment in which students can challenge their intuitions about Newton's laws and the conservation of energy and momentum through game-like inquiry activities. We discuss how usability and learning issues have shaped the design and refinement of NewtonWorld. Additionally, we discuss implications of our work for designing sensorily immersive virtual reality interfaces that are usable and facilitate learning.

A set of radiation overexposure event reports were reviewed as part of a program to examine human performance in industrial radiography for the U.S. Nuclear Regulatory Commission. Incident records for a seven year period were retrieved from an event database. Ninety-five exposure events were initially categorized and sorted for further analysis. Descriptive models were applied to a subset of severe overexposure events. Modeling included: (1) operational sequence tables to outline the key human actions and interactions with equipment, (2) human reliability event trees, (3) an application of an information processing failures model, and (4) an extrapolated use of the error influences and effects diagram. Results of the modeling analyses provided insights into the industrial radiography task and suggested areas for further action and study to decrease overexposures.

This paper presents a six-month longitudinal study of the effects of ecological interface design (EID) on fault management performance. The research was conducted in the context of DURESS II, a real-time, interactive thermal-hydraulic process control simulation that was designed to be representative of industrial systems. Subjects' performance on two interfaces was compared, one based on the principles of EID and another based on a more traditional piping and instrumentation diagram (P&ID) format. Subjects were required to perform several control tasks, including startup, tuning, shutdown, and fault management on both routine and non-routine faults. At the end of the experiment, subjects used the interface that the other group had been using to control the system. The results indicate that there are substantial individual differences in performance, but that overall, the EID interface led to faster fault detection, more accurate fault diagnosis, and faster fault compensation.

A motivation in the design of Ecological Interfaces for process control is to augment the strengths of the human operator with the strengths of the automation in the computer system. This is especially important when building interfaces for managing the use of Wide Area Networks (WANs) in enterprise computing. This domain requires network operators to be able to control the allocation of resources and to monitor the occurrence of structural and functional failures. The relationship between the network architecture, infrastructure, traffic characteristics, and pricing scheme is dynamic and highly inter-related. A “Tetris-like”, bin-packing interface is designed based on principles drawn from an ecological user model and a situation theoretic representation of the problem space and the user's task. The effectiveness of this display is based on a number of ecological features whose design is automatically generated through the determination and selection of the interface context.

Emergency dispatchers must make complex life or death decisions under extreme time pressure. Using Ecological Task Analysis (ETA), a technique normally applied to aerospace human factors problems, a new display was designed that would better assist their decision making task. The major design constraints were identified to be the beat number and priority of incidents, available units, and the spatial relationship of the those units to the incident. Using these and other less formal factors, a GUI interface was designed and an evaluation was conducted at the Richmond, CA police dispatch center. The results suggest that the GUI display may reduce training times and increase situational awareness.

This paper outlines the need for better conceptual and methodological tools for performing observational data analysis in support of cognitive engineering research and practice and presents a tool, MacSHAPA, that has been designed to support such work. MacSHAPA is particularly suited for cognitive engineering studies of complex real-world decisionmaking. MacSHAPA lets users (1) enter or import data into a spreadsheet-like viewing medium, (2) annotate, manipulate, and visualize data in various ways, (3) carry out statistical analyses of various kinds, and (4) export data and results to other applications. MacSHAPA controls video devices, capturing timecode and inserting it into the database, and using timestamps in the database to locate events of interest on videotape. MacSHAPA's statistical routines include content and duration analysis, transition analysis (with some Markov statistics), lag sequential analysis, cycles reports, and some kinds of sequential pattern matching. The paper concludes with several examples of how MacSHAPA has been used to obtain useful results from observational data collected in laboratory and field settings.

Computer-based human modelling technology has been in existence since the early 1980s. However, most earlier human models were either hard to use or lacked appealing graphics. With rapid developments in 3D computer graphics, it is now possible to interactively manipulate and analyze human models in a virtual environment. This coupled with growing user interest has spurred rapid development and use of human modelling and simulation.

Cognitive task analysis (CTA) is increasingly being used to effectively address a wide variety of human factors problems. However, different researchers are using significantly different methods. In many cases, a particular method is used solely by its originators. Therefore, there are significant issues that must be worked through before CTA becomes a widely accepted and easily transferable human factors tool. The objectives of this symposium are to: bring CTA to the attention of a wider audience; develop a better understanding of the differences and similarities between different CTA methods; and demonstrate the practical advantages of CTA.

Cognitive task analysis (CTA) methods have grown out of the need to explicitly consider cognitive processing requirements of complex tasks. A number of approaches to CTA have been developed that vary in goals, the tools they bring to bear, and their data requirements. We present a particular CTA technique that we are utilizing in the design of new person-machine interfaces for first-of-a-kind advanced process control plants. The methodology has its roots in the formal analytic goal-means decomposition method pioneered by Rasmussen (1986). It contrasts with other approaches in that it is intended: (1) for design of first-of-a-kind systems for which there are no close existing analogues, precluding the use of CTA techniques that rely on empirical analysis of expert performance; (2) to define person-machine interface requirements to support operator problem-solving and decision-making in unanticipated situations; and (3) to be a pragmatic, codified, tool that can be used reliably by person-machine interface designers.

Cognitive task analysis is accomplished using a wide variety of methodologies, and we have previously argued that different methods will tend to elicit qualitatively different types of knowledge and skills. Because of this, many practitioners use complementary methods for a given project. We have developed such a complementary package of knowledge elicitation techniques, along with a specific representational method, which together are termed conceptual graph analysis. Conceptual graph analysis is domain-independent and can be used to evaluate complex cognitive tasks or subtasks. It relies on the successive use of document analysis, interviews, task observation, and induction based on review of task performance. The information from these elicitation techniques is represented as a set of interrelated conceptual graphs, but can be represented in other formats also. There are several issues relevant to cognitive task analysis that are currently being faced, including when to perform this type of analysis, and what methods to use. One answer is to perform cognitive task analysis when the task has an inherently high degree of cognitive complexity.

Cognitive Task Analysis (CTA) attempts to describe how people perform tasks: the cues and patterns they use, their inferences and strategies, mental models, and other related topics. It differs from behavioral task analyses that seek to enumerate the steps that must be followed without examining the expertise needed to perform critical steps. Therefore, CTA provides a more in-depth picture, which complements the broader and more comprehensive behavioral task analysis.

A CTA usually consists of five steps: Preparation, Knowledge Elicitation, Data Analysis, Knowledge Representation, and Application. The applications of CTA can take a number of forms, such as training, system design, personnel selection, and market research.

The term cognitive task analysis (CTA) has been appearing in the human factors literature with increasing frequency. Others have used the term cognitive work analysis (CWA). Is there a difference? Do either of these methods differ from traditional task analysis (TA)? If so, what advantages can CTA/CWA provide human factors engineers? To address these issues, the history of work analysis methods and the evolution of work are reviewed. Work method analyses of the 19th century were suited to manual labor. As job demands progressed beyond the physical, traditional TA was introduced to provide a broader perspective. CTA has since been introduced to increase the emphasis on cognitive task demands. However, CTA, like TA, is incapable of dealing with unanticipated task demands. CWA has been introduced to deal with complex systems whose demands include unanticipated events. The initial evidence available indicates that CWA can be applied to industry-scale problems, leading to innovative designs.