Abstract
We describe a structured method for decreasing the gap between cognitive work analysis (CWA) and sociotechnical systems design within industrial contexts. Key to closing this gap is reducing the volume of data collected during initial CWA stages so as to focus further development effort on the key determinants of system performance. The goal of the current study was to develop a method to accompany control task analysis (ConTA; and associated contextual activity templates [CATs]). The method developed, “activity prioritization,” involved a multifaceted procedure for prioritizing activities to identify each activity’s overall impact on system performance. Using three key values, each activity receives a priority score that is ordered to create an activity priority list. For the prioritization, the functions described in the CAT were the purpose-related functions identified via a work domain analysis of long haulage trucking. The method described was used to guide development of highly automated trucks within a recently completed industrial project (methods for designing future autonomous systems [MODAS]).
Keywords
The goal of the current study is to develop a systematic process for focusing the analysis effort and in that way decreasing the gap between analysis and design, within the cognitive work analysis (CWA) framework (Rasmussen, Pejtersen, & Goodstein, 1994; Vicente, 1999). In the current paper, we report a method for focusing the control task analysis (ConTA; the second phase of a CWA) on the activities within the domain that have the greatest potential influence on system performance—we call this method “activity prioritization.” The aim with the method is to focus analysis and design effort to ensure a more efficient use of available resources at later CWA stages of an industrial project.
CWA is a formative framework for analyzing and modeling the complexity of the real world and the relationship between human and technical components. The framework supports understanding of the constraints that influence how work could be completed “if the appropriate tools were available” (Vicente, 1999, p. 340). There are five stages included in CWA: (a) work domain analysis (WDA), (b) ConTA, (c) strategies analysis, (d) social organization and cooperation analysis, and (e) worker competencies analysis. The work described in the current paper builds upon the WDA and belongs to ConTA, and therefore, these only two CWA stages are defined in more detail below.
Within WDA, the structure of the system is defined independent of workers, events, and tasks (Vicente, 1999). The results of a WDA can be modeled using an abstraction hierarchy (AH). The AH consists typically of five abstraction levels, showing how physical and intentional system constraints influence how the purpose of the system can be achieved (see Table 1 for a summary). Each level consists of a number of nodes, and mean-end links connect the nodes between the levels. The fifth (bottom) level of the AH presents the physical objects contained within the system. The nodes on the fourth level are called object-related processes and represent the functional capabilities and limitations of the physical objects. The third level contains purpose-related functions and presents the functional constraints of a work system and maps the possibilities of the physical objects to the values and priority measures of the system (presented on the second level). The first (top) level of the AH presents the functional purpose of the system—the rationale for the existence of the system. The names of the AH levels used in the current paper are developed by Rasmussen, as reported by Reising (2000).
Summary of Abstraction Hierarchy Levels, Adapted From Reising (2000)
To validate the AH in a situated context, Burns, Bryant, and Chalmers (2001) suggested that the AH be examined in relation to scenarios that were not included in the modeling process. By this, Burns et al. (2001) aimed to ensure that the AH captured all possible domain constraints. They argued that performing a WDA for a large system can be difficult and time-consuming, and it is therefore reasonable to try and confirm that the analysis is “on track” during the process and validate the analysis before entering a design phase.
Whereas the WDA delivers an activity-independent analysis of the work, the ConTA complements this analysis by defining requirements for known situations. Note, however, that the ConTA has limited value in identifying support needed in unanticipated (and therefore often critical) events (Vicente, 1999). The ConTA describes what needs to be achieved in the (identified) situations—that is, the control tasks. The development of control task models can be conducted with decision ladders (Rasmussen, 1974, 1976; see also Jenkins, Stanton, Walker, & Salmon, 2009).
Naikar, Moylan, and Pearce (2006) present an approach for completing a ConTA where the first activity is decomposed into a set of recurring situations and/or a set of required functions. A contextual activity template (CAT; Naikar et al., 2006) is a product of the ConTA and models activity in systems defined by both functions and situations involving temporal or spatial components (see Figure 1 for a summary of the CAT). Each unique cell on the CAT, corresponding to a specific function and a specific situation, is here referred to as an activity. According to Naikar et al. (2006), functions either “typically occur,” “can occur,” or “do not occur” in the situations. A detailed analysis of any complex system would likely identify many activities, where each activity could be decomposed into the required control tasks (for example via decision ladders) and thus be used to support system and user interface/interaction design.
Summary of a contextual activity template (CAT), adapted from Naikar et al. (2006).
The Use of CWA Results to Support Development and Design
There are several methods by which the CWA results can be used as input for design and thus encourage product development. In a review of how CWA results have been used in the design process, Read, Salmon, and Lenné (2012) identified four broad categories of application: (a) “direct contribution,” where the results from the CWA were directly mapped to design; (b) “restructure of CWA outputs,” where the design requirements were identified by iterative use of CWA methods; (c) “additional guidelines, principles, criteria,” where the results from the CWA were used in conjunction with human factors guidelines or principles for interface or systems design; and (d) “additional method/process,” where the CWA was used in another, wider, design process. Whereas the conclusions made by Read et al. (2012) sufficiently describe broad categories for how CWA results have been used for design, detailed descriptions for how to proceed within each category are missing.
In a more concrete example of how to use CWA results to influence system design, Salmon, Regan, Lenné, Stanton, and Young (2007) created a WDA for the road transport domain, which was modeled using an abstraction decomposition space (ADS). An ADS is an AH extended with levels of system decomposition, ranging from the whole system to components (Vicente, 1999). The information in the ADS was used by Salmon et al. (2007) to conduct an information requirements analysis for drivers. The information requirements were then compared with the current technology available to identify the needed design improvements.
Another method used to go from CWA analysis to design was developed by Lamoureux and Chalmers (2009), who analyzed the operational environment for military naval command and control operators. Lamoureux and Chalmers’ (2009) method was used during the second stage of CWA analysis, the ConTA. Possible design improvements, called “design seeds,” were constructed by examining the decision ladders and mapping the steps in the ladder to the four stages of human information-processing (Wickens, 1984): (a) perceptual encoding, (b) working memory, (c) central processing, and (d) responding. According to Lamoureux and Chalmers (2009), the selection of the design seeds should focus on how to assist the operators complete the different steps of information processing.
Prioritization of Analyzing Effort
The time and resource constraints associated with research and development within financially competitive industries introduce new constraints on the thoroughness of CWA of large systems (considering that such analyses are typically highly resource consuming). Unfortunately, methods for supporting commercial industry manage such large volumes of data generated from CWAs in an efficient way are missing. Arguably, this remiss can result in at least two outcomes: the failure to use the CWA method or an arbitrary selection of a subset of data for further analysis. In the current article, we aim to deliver a structured approach for refining data (from the ConTA stage) for future analysis that should support more efficient follow-up analysis and increase industry update of CWA. Key to this is prioritizing data collected during the ConTA stage of analysis. How the prioritizing process should look like is an underestimated problem that needs to be addressed to meet the needs from industry and practice. Therefore, a part of decreasing the gap between analysis and design is to prioritize the analysis results and allow analysts or designers to move towards defining design requirements for the more critical system components. We argue that the process of choosing what to focus the analysis on should be deliberate and explicit.
The need for prioritization was recognized and addressed by Birrell, Young, Jenkins, and Stanton (2011), who suggested a heuristic method for prioritizing user requirements by assessing the priority of lower level nodes in an AH and their contributions to high-level functions or values of the system. Their method was developed within, and for, the project “Foot-LITE,” which aimed to encourage drivers to adopt a more environmentally friendly and safer driving. To prioritize nodes in an AH, Birrell et al. (2011) proposed a method by which the nodes in the AH were categorized as having low, medium, or high priority. To complete this process, first, the nodes on the three highest levels in the AH are scored by the research team, using knowledge gained through completion of the previous stages of the CWA (Birrell et al., 2011). The priority scores for the nodes on the fourth and fifth level in the AH are then calculated according to how many mean-end links each node had to high-, medium-, and low-priority nodes at the level above. A consistent heuristic was used for setting the scores; a link to a high-priority node, medium-priority node, and a low-priority node gave all a certain amount of points. The number of points decided the priority of the particular node according to number ranges defined differently for object-related processes and physical objects. Birrell et al. also suggested a similar approach for identifying the relative contributions of the purpose-related functions and object-related processes to the functional purposes described in their AH. This was done to enable the analyst to focus on functions and processes that contribute the most to the desired functional purpose.
Birrell et al. (2011) argue that the main benefit of the priority calculations is that they provide a high-level summary highlighting the highly connected nodes that may be more important for future consideration within the project. However, the authors also acknowledge that their method might be oversimplified and is not without limitations, most notably because the importance of different nodes will likely change in different contexts.
An additional problem with the method suggested by Birrell et al. (2011) is that it does not consider if an object-related process is the only node connected to a purpose-related function, or if there is redundancy. It could be that an object-related process is critical to the system performance even if it is only related to one purpose-related function—that is, if that purpose-related function is critical to system performance and only realized through the one object-related process.
Hassall, Sanderson, and Cameron (2010) developed a process for identifying human factors hazards by prioritizing activities for hazard analysis, called the HumHID process. The process combines methods from the CWA framework and a methodology for identifying potential hazards associated with human activity. First, the work context and activity is modeled in a CAT and includes decision ladders for each work context/activity. The prioritization is conducted with a high-level risk assessment process based on the likelihood and consequence of the following: (a) “not doing the activity,” (b) “not completing the activity,” (c) “doing the activity incorrectly,” (d) “doing the activity out of order,” and (e) “doing the activity steps out of order.” The activities with high prioritization are included in a strategies analysis, and the identified strategies are investigated with the same high-level risk assessment procedure. Decision ladders are then conducted for the strategy steps associated with significant risk, and the hazard analysis is conducted based on information from the decision ladders. The high-level risk assessment is a simple method of ensuring efficiency of the analysis work by focusing the HumHID process on activities possessing significant risk (Hassall et al., 2010). Additionally, Hassall et al. argue that their case study was focused on anticipated situations, whereas hazard could also arise from unanticipated situations.
The Methods Developed
This section describes the activity prioritization method. The method focuses the ConTA onto the activities with the greatest potential influence on system performance and by doing so ensures a more efficient use of available resources at later CWA stages of an industrial project. The method is in one way a continuation of the method by Birrell et al. (2011), but the objective is to identify the activities on which to focus the ConTA instead of making prioritizations of what to support in the work domain. Birrell et al.’s approach of counting mean-end links to find the priorities are thus not used because the functions included in the CAT for the current study are from the purpose-related level in the AH.
Activity Prioritization Method
To create a prioritized list of activities using the activity prioritization method, a priority score was calculated for all activities identified in the CAT. To calculate the activity priority score, the following data were obtained;
function priority: the priorities of the functions from the AH;
situation frequency: the proportion of working time spent in each situation; and
contextual activity frequency: how frequently each function occurred in each situation.
The latter two scores, the situation frequency and contextual activity frequency, were combined to create an activity frequency score (i.e., the overall frequency of an activity). Combining the activity frequency with the function priority resulted in a score for activity prioritization, the activity priority score (i.e., the overall priority of an activity).
The method by which the raw data were obtained and the composite scores calculated is described in the following sections and is modeled in Figure 2.
The different subscores contributing to the activity priority score and their scales. All scales except the situation frequency score consist of whole numbers.
Function priority
To prioritize nodes in an AH, Birrell et al. (2011) described a method by which the nodes in the higher levels of the AH were categorized as having low, medium, or high priority. Here the same process can be used by the research team to give scores to the nodes on the three highest levels in the AH, using knowledge gained through completion of the previous stages of the CWA (Birrell et al., 2011).
Situation frequency
The situation frequency represents how often each situation occurs and was obtained by the mean percentage of the work time the workers spend in the situations. The percentage value was then transferred to a scale from 1 to 9 such that the highest ranked situation scored 9 and the lowest ranked situation scored 1 (the rationale for this scaling will be described in the “Activity Frequency” section but in brief was intended to provide equal weighting to the final prioritization scores arising from situation frequency and contextual activity frequency).
Situation frequency was calculated because the overall importance of an activity depends in part on the frequency of the situation in which the activity occurs. For example, although park stability (afforded by the park brake) is always used during parking (the situation), parking is a relatively rare event in long haul driving, occurring only twice or three times during a standard 8-hr shift. To maximize research and development resources, we must invest effort in frequent situations.
Contextual activity frequency
The contextual activity frequency is a measure of how frequently a function occurs, or is needed, in a situation and is the information usually modeled in a CAT (Naikar et al., 2006). The functions that never occur in the situation (or alternatively, activities that never occurred; e.g., loading or unloading during driving) were given a contextual activity frequency score of 0. Activities that occur infrequently (but could occur; e.g., emergency braking during driving) were given a contextual activity frequency of 1. Finally, activities that occurred frequently (for example, maintaining vehicle stability when driving on a country road) scored 2.
Contextual activity frequency was calculated because the degrees to which a function is used is heavily dependent on the situation. For example, fuel efficiency is not relevant during parking but is usually relevant during driving.
Activity frequency
The activity frequency is the frequency at which an activity occurs and was calculated to distinguish between more and less often occurring activities. A more frequently occurring activity was seen as having high priority for further development. If the product and system performance is enhanced for a frequently occurring activity, there should be greater benefits than if optimized for rare activities.
The activity frequency was calculated by multiplying contextual activity frequency with situation frequency (see Figure 2). Recall that the conditional activity frequencies were 0, 1, or 2 and that situation frequencies were scaled (from percentages) to whole numbers between 1 and 9, which would give a scale from 0 to 18 for the activity frequency. Each activity frequency score was multiplied by a weighting of 0.5 to create an activity frequency scale from 0 to 9. This was completed to give equal influence to function priority (with the highest score 9) and activity frequency on the overall activity priority.
Activity priority
The activity priority score is the combination of the function priority score and activity frequency score. This was calculated to create a priority list of the activities to focus further analysis and development. The activity priority score considers both how frequently an activity occurred and how much the activity contributed to system performance. To obtain the activity priority, the activity frequency was multiplied with the function priority (see Figures 2 and 3).
An excerpt of the CAT showing activity priority scores and the component scores.
The Long Haul Truck Project—Case Study
In what follows, we explain how the method defined above was applied in an industrial use case involving the development of long haul commercial heavy vehicles (hereafter, “trucks”). It was necessary that the developed method support the “long-horizon” systems development of next generation, highly automated trucks. This type of systems development would typically involve multiple organizational units attempting to (re)define the scope of their products within the to-be-developed system. Long-horizon development would thus also involve identifying the work to be conducted in these “future systems” and the information needed by operators to conduct tasks and supervise systems. Consistent with Scania’s systems approach to cab development, the method was meant to fit within the CWA framework as part of (what was then) ongoing work conducted within the Methods for Designing Future Autonomous Systems (MODAS) project (Krupenia et al., 2014).
As part of the development work within the project, a WDA was completed (Bodin, 2013) that contained a high number of nodes (approximately 325) on the resultant AH. The subsequent CAT similarly resulted in a (very) large amount of data (629 activities). Given this large amount of data, an exhaustive follow-up analysis (for example, via decision ladders) for all activities was unreasonable. Therefore, a need was identified to focus the analysis early to ensure a more efficient use of available resources at later stages of the project. In other words, if we were to directly use Lamoureux and Chalmers’ (2009) method and construct decision ladders for all activities identified, then the available industrial resources would be quickly exhausted. What was needed was a method for utilizing the available resources more efficiently. To achieve this, an attempt was made to use the activity prioritization method to structure and prioritize the activities within the CAT.
The WDA and CAT for the Current Project
To achieve an understanding of the work domain, a total of 17 hr of interviews were conducted with an expert professional driver and a subject matter expert (separately). Additionally, an observation study was conducted during which three video cameras were used to capture 16 hr of footage from 35 hr (2,500 km) of observation (one driver/day over a 4-day period). During the observation study, drivers were also asked to talk out loud during short sessions, and a few weeks later verbalization interviews were conducted (a total of 5 hr for the four drivers). The AH was iteratively modeled between the interviews. See Figure 4 for a simplified version of the top four levels.
Top four levels of the MODAS abstraction hierarchy.
A CAT was modeled by describing the activities in the work domain by both functions and situations (see Figure 1 for a representative example). The situations were identified through reviewing the video data obtained from the observation study. The functions were adopted from the purpose-related function level of the AH, which was modeled for the current work domain. The higher levels of the AH (including purpose-related functions) are closely connected to the higher level purposes and values within the domain and are therefore relevant also in the future and for long-horizon system development. When focusing on a future system that is not yet developed, it is arguably better to look into the constraints imposed by the purpose and values of the system, rather than into the system itself. The procedure for completing the WDA for long haulage truck drivers and identifying the situations in the CAT is described in Bodin (2013).
Implementation
This section provides a description of the data collection on which the activity prioritization score and component subscores are based.
Calculation of the function priority score
The nodes on the purpose-related function level in the AH were assigned high or medium priority as per Birrell at al. (2011). This was completed by reviewing the material from the data collection for the AH and assigning priorities to the nodes on the purpose-related functions level in the AH. A node with high priority received 9 points and a medium priority node received 3 points. Recall that the purpose-related functions from the AH (third level) are presented as functions on the CAT.
Data collection for the situation frequency score
The situation frequency was obtained via one-on-one interviews with 10 professional long haulage truck drivers from various transport companies. The drivers were asked how much working time was spent in each situation with responses given either in terms of time (minutes or hours) or as a percent of total working time. The responses given in terms of time were transformed into a percentage together with the driver. The situation frequency score is then calculated by the mean percentages of working time workers spend in the situations, transferred to a scale from 1 to 9.
For example, in response to the situation “highway driving,” a driver may say that this situation consumes 3 hr of an average working day. If the driver works 15 hr each day (including driving time and other working time), then the situation “highway driving” occupies 20% of working time. Assuming that the most frequently occurring situation occupied 35% of working time and the minimally occurring situation occurred 0% of working time, then the value of 20% translates to a situation frequency score of 6.
It is important to note that for the current study, a decision was made to measure situation frequency in terms of time and not number of occurrences because for long haul driving, time corresponds better with the work system. If the alternate approach was taken, then situations that occurred a few times but took several hours each time (for example, driving) would receive a low situation frequency score even though the situation consumed a large part of the working time.
The data collection for the contextual activity frequency score
In the current project, the information used to assign the contextual activity frequency scores was obtained through an observation study and follow-up interviews with five professional long haul truck drivers (for a full description of the observation study, see Bodin, 2013). Given the large number of functions and situations identified, two drivers gave a rating of 0, 1, or 2 to half of the identified activities and the other two drivers rated the other half of the activities. If the information from the two informants were inconsistent or if the information did not seem to match the results from the observation study, follow-up interviews were conducted and the fifth driver (with 39 years of experience) was consulted.
The questions presented during the interviews were extracted from the CAT and included reference to both the function and the situation—for example, “How frequently does road holding (function) occur when driving on highways (situation)? Usually/can occur/never occurs?” The responses therefore gave information about the function frequency in the given situation but not the overall activity frequency. For example, the function “assistability” usually occurred in the situation “truck breakdown,” but the overall frequency for the activity “assistability during truck breakdown” depends also on the frequency of the situation “truck breakdown” (which is arguably very low).
The data collection was conducted for a CAT modeled with functions from the object-related processes level in the AH. This object-related CAT was later truncated to a higher abstraction level—that is, to a purpose-related functions CAT. In most cases, there were several contextual activity frequency scores for object-related processes truncated into one contextual activity frequency score for a purpose-related function. The contextual activity frequency score for each activity in the CAT was determined by truncating the scores from the activities in the CAT modeled with object-related processes (for the same situations and by using the mean-end links in the AH), to determine which object-related processes and purpose-related functions were connected.
The frequency and prioritization of the activities
The collected data and calculated situation frequency and contextual activity frequency score were multiplied to provide a score describing the frequency at which activities occur, called the activity frequency score. These scores were multiplied with the function priority scores to provide the activity priority scores. On the basis of these values, a prioritized list of all activities was created (the “priority list”). The priority list consisted of six columns: (a) the situation, (b) the function (defines the activity together with the situation), (c) the contextual activity frequency score, (d) the situation frequency score, (e) the function priority score, and finally (f) the activity priority score (Table 2).
The Beginning of the Priority List With the Highest Prioritized Activities (Described as Situations and Functions From the Purpose-Related Function Level of the AH) and Their Activity Priority Scores
The Results Described in the Priority List
The (six) activities with the top score (81) described in the priority list are (a) “vehicle control (driving),” (b) “relocation,” (c) “cargo transport,” (d) “visual observation,” (e) “transparency of driving intent,” and (f) “short-term planability,” all during “country road driving.” The next function to be found on the priority list is “logistical planning” (activity priority score of 54), during the situation “start and end of shift.” The situations associated with activities on the upper part of the priority list are (a) “country road (including low traffic),” (b) “slippery road,” (c) “highway,” (d) “start and end of shift,” and (e) “hills.” Table 2 presents an excerpt of the full priority list and shows only those activities with a priority score higher than 40 (on a scale from 0 to 81). The situations that contained the highest ranking activities were identified as areas of potential research and development resource investment. These were thus proposed as design opportunities within the MODAS project.
Discussion
This section begins with a discussion about the function priority score and continues with a section about the activity prioritization method, including its limitations, generalizability, advantages, and disadvantages. In this final section, we also discuss the disadvantages with prioritization methods and the limitations regarding safety in critical systems when analyzing tasks or situations.
Function Priority Scores Without Consideration of Situations
The function priority scores are obtained from the event independent WDA. That different nodes in the AH are more or less important depending on the current situation is well illustrated in prior work validating AHs by mapping them to specific scenarios (Burns et al., 2001). Therefore, the function priority scores have to be seen as the potential contributions to the values and priorities and functional purposes of the system. This means that the scores are based on how the functions can support the higher level nodes without consideration of the current situation. The contextual activity frequency scores then take into account if the functions are actually occurring in the situations. It would therefore not be appropriate to use the function priority scores without the contextual activity frequency scores when considering how important functions are in any situation or scenario. For a singular situation, the function priorities alone would never reflect the truth.
When presenting their method, Birrell et al. (2011) encounter a very similar problem. In response, they argued that their approach was limited and that it is a simplification. Whereas the activity prioritization method uses information from the WDA, the CAT, and the frequencies of the situations to prioritize the activities for further analysis of control tasks, the method suggested by Birrell et al. (2011) prioritizes the WDA nodes to highlight those nodes that are more important for supporting design. Their approach seems especially problematic because their priorities are extracted from the event-independent AH directly, without adding any information about the different situations. However, in defense of Birrell et al.’s method, even if the implication is that their priorities are not correct in most (if any) situations, this does necessarily imply a problem with the resultant system. Interfaces and systems should not be designed to address only known or predicted situations but must support operators preemptively and resolve the unknown or unexpected. Therefore, Birrell et al.’s arbitrarily assigned priorities (for the nodes on the three top levels of the AH—where scores were set by researchers based on collected data) could be appropriate to use for design considerations.
A problem with the activity prioritization method is that there is the potential for overly high-rated activities in some situations. For example, if one high-priority function that is usually occurring in a situation but is in that particular situation less important than in general, then this would result in an undeservingly high activity priority score. The prioritization from the activity prioritization method is, even with the contextual activity frequency score, still a simplification. It is a simplification in two ways. First, numbers are added to a model, which is, as per the definition of model, a simplification of reality and thus not reality itself. Second, the method for prioritization is limited and (necessarily) adds to the simplification when achieving its purpose of being useable in a practical industrial context.
Advantages and Disadvantages of Prioritization
During completion of the CWA within the automotive industry, it was noticed that constructing decision ladders and completing the strategies analysis for all activities would involve a significant resources investment (and thus unlikely to be undertaken by the organization). However, a structured approach for defining which activities are worth investigating and which may be more peripheral to system efficiency was lacking. The activity prioritization method thus arose as an industry need, or constraint, on the application of CWA. Consistent with CWA, the next step after using the activity prioritization method would be to investigate the tasks conducted during the high-priority activities. The activity prioritization method is not intended to replace other methods used to translate analysis results into design requirements at later stages of the analyzing process. Instead, the proposed method is a way to focus the analysis to support the more efficient allocation of research and development resources for systems that are too complex or impractical to analyze in full. The activity prioritization method is thus intended to support the earlier focusing of effort in the analysis process, which is the method’s main advantage.
It is important to consider that using the activity prioritization method, as described in the current study, results in excluding some activities from further analysis. The best possible case would be to complete the analysis for all activities before using the knowledge as input for design, but when that is not possible to do, a deliberate choice regarding what to focus on is to be preferred (as opposed to either an ad hoc, subjective, or unstructured approach, or indeed the abandonment of further analysis effort). An advantage achieved when using the activity prioritizing method is the explicit and deliberate process of deciding what is included in further task analyses.
The activity prioritization method was used to find which activities the control task analyses should be focused on to improve the future system. Therefore, the frequency and the contribution to system values was the basis for activity prioritization. In other projects, the activities could be prioritized based on other criteria. Because of this, we argue that the activity prioritization method is generalizable and could include subscores other than the ones used in the current project. Consider, for example, that because workers are better adapted (trained, prepared, or supported) to solving the more common events, it could be of greater interest to investigate further the least common situations instead. To do this, the frequency scales could be inverted to prioritize infrequently occurring functions. Alternatively, a method for ranking the situations according to criticality rather than frequency, or using data from incident reports, could be used. A significant disadvantage with activity prioritization is the methods’ failure to address the critical situations and unanticipated events. This is discussed in the following section.
The HumHID process developed by Hassall et al. (2010) was made for identifying human factor hazards and therefore focuses on the activities with a high risk (defined as likelihood and consequences) of not being completed in a correct way. A key difference between the activity prioritization method and that of Hassall et al. is that activity prioritization applies at an earlier stage of analysis—that is, before conducting decision ladders. Further, the function priority scores are based on the WDA and the AH, but a higher score would imply that there is a larger consequence if the function is not completed when needed. Alternatively, the high-level risk assessment process by Hassall et al. is based on the CAT and decision ladders. In the activity prioritization method, the frequency of an activity is included, instead of frequency of not completing an activity correctly (as per Hassall et al., 2010). We recognize that the approach by Hassall et al. could be an alternative to provide the prioritization another focus. In response, we would suggest that hazardous outcomes can occur from situations not involving incorrect task performance and that these activities may also require further design support. For example, consider when an automation or a subsystem breakdown requires the worker to perform a task that is not associated with an activity that was previously identified as having a high risk of not being completed in the intended way. The issue of unexpected events is considered further in the following section.
Safety critical situations and unanticipated events
Regardless of how the criteria for the prioritization are defined, or if all known situations and functions described in the CAT are investigated, an analysis focusing on tasks will never cover all critical events (given that there may be an infinite number of situations). Furthermore, arguably the largest threats to system safety for complex sociotechnical systems are the events that are unfamiliar to workers and for which the system has not been designed (Vicente, 1999). The problem with failing to uncover unanticipated events is not a product of the prioritization process but rather is from focusing on tasks and situations when it is impossible to analyze an unknown task and impossible to know if the “unknown” is sufficiently known. The “failure” to exhaustively describe the unknown is itself not unsurprising; indeed, Vicente (1999) says in the beginning of his chapter on ConTA that “although an analysis of control tasks does not identify the support requirements to deal with unanticipated events, it does allow us to identify the requirements associated with known, recurring classes of situations” (p. 181).
Our aim with conducting a ConTA in the MODAS project and, specifically, the subsequent activity prioritization method was to identify the activities for further analysis. The goal was not to identify and avoid all safety risks nor to accommodate for all unanticipated events. The key was to understand more about some central and anticipated parts of the truck drivers’ work. The WDA was conducted and modeled in an AH prior to the start of the task analysis to give an understanding of the information requirements, functions, and priorities in the system independent of situations (Bodin, 2013). Even when the task analysis was focused to a number of situations, we assumed unknown hazardous events will occur during system use. The design philosophy used was intended to support the development of an interface (or set of interfaces) to support the driver maintain supervision and control their vehicle and to have the ability to take control of the system regardless of the situation—including importantly, the unforeseen situations (Jansson, Stensson, Bodin, Axelsson, & Tschirner, 2014).
It could be argued that the focus on control tasks, situations, and activities is risky because it is based on known recurring classes of situations rather than unanticipated events and that this problem will be exacerbated when prioritizing the activities as opposed to covering all identified activities or prioritizing with respect to safety criticality. We suggest that this becomes problematic if the design philosophy used in the project is to (just) make sure the system works in the situations that are analyzed and tested. Therefore, we argue that a design philosophy focusing on supporting the worker given any situation is needed beyond a task analysis of prioritized activities. The importance of both a (control) task analysis and a WDA cannot be understated. The two together should support the development of systems that deliver increased transparency rather than hiding system (or domain) complexity (Andersson, Jansson, Sandblad, & Tschirner, 2014).
Conclusion
We have presented a key method that can be used to facilitate the transfer and uptake of CWA from the more academic or theoretical domains into commercial industry. The goal of the method is to provide the analyst with a structured process for decreasing the gap between analysis and design. A key step to closing this gap is reducing the volume of data collected during initial CWA stages so as to focus future development on the key determinants of system performance—a method we call activity prioritization. The method and results described are of relevance and value to all analysts using CWA but should be particularly appealing to industrial organizations where the efficient use of resources is necessary for profit and organizational survival. Finally, although we acknowledge that the method is not without its flaws, we argue that it extends beyond the capability of existing methods. Further work should seek to define how additional determinants of systems success (e.g., risk or hazard severity) can be included into a multidimensional prioritization process.
Footnotes
Acknowledgements
The authors would like to thank Dr. Anette Karltun for feedback and support throughout the development and documentation process. Additionally, we thank Dr. Tania Xiao, Mr. Anton Axelsson, Dr. Anders Jansson, and three anonymous reviewers for their valuable feedback on earlier versions of this manuscript. Finally, we thank the Scania Transport Laboratory and their drivers for their ongoing collaboration on this work. This work was funded, in part, by the Strategic Vehicle Research and Innovation (FFI) funding scheme to the MODAS project (2012-03678).
Ida Bodin is a PhD student in computer science specializing in human computer interaction at Uppsala University, Sweden. In 2013, she received her MSc in engineering and a master’s degree in ergonomics and human-technology-organization from the Royal Institute of Technology in Stockholm, Sweden.
Stas S. Krupenia obtained his PhD in human factors and cognitive engineering in 2007 from the University of Queensland, St. Lucia, Australia. From 2007 to 2009, he completed a postdoctoral fellowship at the Israel Institute of Technology. Since 2009, he has been working as a cognitive engineer within the defense (Thales Research and Technology; 2009-2012) and automotive (Scania CV AB; 2012-present) industries.