Utility of Expert Cue Exposure as a Mechanism to Improve Decision-Making Performance Among Novice Criminal Investigators

Abstract

The research tested whether systematic exposure to expert-identified cues would improve novice criminal investigators’ cue recognition and, in turn, decision making. Two studies are reported, the first of which was a pre- to postexposure assessment of 20 novices’ cue recognition. This involved testing novices’ recognition (accuracy and latency) of pairings of text-based labels (elicited via cognitive task analyses with subject matter experts) prior to and following an exposure phase. The results revealed statistically significant improvements in comparison with a control group. In the second study, an assessment of 36 novices’ decision-making performance was undertaken prior to and following cue-based exposure (either expert or control cues). Participants engaged one of two decision tasks, which varied in the level of decision support offered: high (i.e., most pertinent features were highlighted for users) or low (i.e., features were naturally “embedded” in the task environment). Although participants receiving expert cue exposure demonstrated improvements in decision-making efficiency, advances in accuracy could be established only where a high level of support was offered. It was concluded that expert cue exposure can offer opportunities for learner development; however, a combination of exposure programs and decision support systems offers the greatest potential in improving the situation assessment skills of less experienced investigators.

Keywords

situation assessment decision making cues cue utilization decision support systems

Introduction

The initial assessment of a situation is paramount to the formulation of a successful decision and represents a major component of Klein’s (1993) model of expert decision making. Here, the decision maker invariably engages a process of matching features in the environment to feature-event associations resident in memory. This associative process is consistent with Anderson and Matessa’s (1997) rule-based conception of if-then productions in the “adaptive control of thought–rational” model and the feature-based relationships described in Brunswik’s (1956) lens model. It is a repertoire of feature-event associations, or cues, in memory that enables skilled decision makers to recognize a situation as familiar and generate an appropriate response.

Cue-Based Learning

The development of cue-based associations in memory is largely dependent on the accumulation of experience in an environment (Wiggins, 2014, 2015a). Presumably then, the effectiveness with which cues are employed by experts is likely to be moderated in part by their level of exposure to a given domain. Consistent with this proposition, differences in expert-novice cue processing are evident in a number of domains that clearly span intuitive and reasoning processing styles (Kahneman, 2011)—including firefighting (Klein, Calderwood, & Clinton-Cirocco, 1986; Perry, Wiggins, Childs, & Fogarty, 2012), medical diagnoses and intensive care (McCormack, Wiggins, Loveday, & Festa, 2014), courtroom judgments (Ebbesen & Konecni, 1975), aviation (Stokes, Kemper, & Marsh, 1992), chess (de Groot, 1966), finance (Searle & Rooney, 2015), driving (Fisher & Pollatsek, 2007), nursing (Shanteau, 1991), criminal investigation (Morrison, Wiggins, Bond, & Tyler, 2013), and rugby league (Johnston & Morrison, 2016). In comparison with their novice counterparts, expert decision makers appear to use a relatively limited number of cues, use different cues, and engage different strategies to acquire cue-based information (Jackson, Warren, & Abernethy, 2006; Johnston & Morrison, 2016; Loveday & Wiggins, 2014; Shanteau, 1992).

Of most interest to the present study is the observation that expert decision makers tend to target cues that embody relatively greater levels of diagnosticity (Schriver, Morrow, Wickens, & Talleur, 2008). Diagnosticity refers to the relative power of a cue to test a hypothesis—that is, its predictive validity (Wickens, Hollands, Banbury, & Parasuraman, 2013). Such validity is thought to be established through repeated interactions with these productions (Anderson & Matessa, 1997; Boreham, 1995; Kirschenbaum, 1992; Morrow et al., 2009; Schriver et al., 2008; Stokes et al., 1992). Moreover, the absence of these cues is associated with degradations in performance, particularly among experts, presumably because they develop a significant reliance on these features for problem resolution (Beilock, Wierenga, & Carr, 2002). This effect has been demonstrated regularly in occlusion studies in sports domains (Abernethy & Russell, 1987; Buckolz, Prapavessis, & Fairs, 1988; Jones & Miles, 1978), in which the occlusion of key features significantly hampers experts’ execution of refined skills, with little or no impact on the performance of nonexperts (e.g., hitting a cricket delivery).

As attention toward appropriate cues appears central to skilled situation assessment across a number of domains, there are significant implications for performance, including failures in decision making. Moreover, because expert behavior is generally associated with superior performance, attempts to improve the performance of less experienced decision makers are likely to be assisted through the elicitation of those cues that are associated with expertise in a given domain.

Approaches to Cue-Based Exposure

By identifying an expert cue inventory within a specific domain, it may be possible to direct less experienced operators to target a limited number of features that they would not otherwise employ. Such cue-based exposure involves teaching nonexperts to attend to those features employed regularly by experts when making decisions (Wiggins & O’Hare, 2003). This method is based on the notion of guided discovery (Smeeton, Williams, Hodges, & Ward, 2005), and it typically involves the acquisition of selected features during engagement with the environment. Through repeated associations between features and events or objects, there is an assumption that users will develop cue-based relationships in memory and that their acquisition in context will enable the subsequent generalization of the cue to the broader environment.

The difficulties associated with an unguided process of cue discovery include the extended exposure necessary to enable the identification of those features that are predictive of events or objects, the concurrent costs associated with this extended exposure, the potential for errors as inappropriate features are activated, and the potential loss of motivation. The presentation of cues systematically allows their acquisition to be monitored together with the rate at which they are acquired. However, the disadvantage associated with this approach is the potential lack of generalizability, given that individual cue-based relationships may be acquired in isolation with a lack of understanding of the broader relationships between cues.

The cue-based learning approach adopted in the current research was designed to capitalize on the efficiency associated with a systematic approach to cue acquisition and the generalizability afforded by the cue discovery approach. It involved the presentation of a limited number of preidentified cue-based associations used frequently by experts. In doing so, a limited context for the application of the cue is provided, with a framework that enables the integration of additional cues, thereby potentially increasing the rate at which subsequent cues are acquired (Chater & Oaksford, 2006; Perry et al., 2013). Such instructional techniques that help learners to create long-term memory structures are thought to be most effective for naïve or novice learners, who typically approach a learning situation in the absence of these knowledge structures (Spanjers, Wouters, van Gog, & van Merrienboer, 2011).

Like most learning initiatives, establishing the validity of a cue-based exposure initiative requires an assessment of the application of the outcomes within the operational context (e.g., Perry et al., 2013; Wiggins & O’Hare, 2003). However, in the case of safety-related and high-consequence environments, the contexts within which the outcomes of exposure courses are evaluated are normally simulated renditions. This reduces the risk of system failures in the event that the appropriate knowledge and/or skills were not acquired.

Although simulated renditions of the operational environment can embody very high levels of visual and functional fidelity, there are difficulties associated with their application for the purposes of cue-based exposure evaluation. Specifically, they may not present the nuanced features that are necessary to initiate the application of a cue. Therefore, the failure of a learner to initiate an expected response may be due to either the absence of the cue in memory (i.e., the failure of the exposure initiative) or the failure to activate the cue due to the absence of an appropriate trigger within the simulated environment.

Given the difficulties in assessing performance in some operational environments, with the limitations of simulation, the outcomes of cue-based exposure initiatives can be especially difficult to establish because poor performance might be related to the failure to acquire and therefore implement cues and/or the failure to provide an environment that presents the features in a form appropriate to initiate a response. Therefore, in the case of cue-based exposure initiatives, an intermediate process of evaluation is necessary that will clearly establish whether cues have been acquired as part of an exposure process.

Measuring Cue Acquisition

Although the acquisition of cues can be difficult to establish, paired association tasks have been successfully used to differentiate the recognition of cues among experts and novices (Morrison et al., 2013). This involves the generation and presentation of pairs of features and events that relate to the concepts under consideration. In the context of cue-based associations, respondents are asked to decide, as quickly as possible, the extent to which a feature and event are related. Previous research has indicated that, in the absence of exposure and experience, shorter mean response latencies are recorded for pairings that are considered most related among expert practitioners (Morrison et al., 2013). Therefore, improving the performance of novices involves the capacity to recognize the relationships between pairings and to do so within a relatively limited period. From the perspective of cue-based exposure, such techniques offer an opportunity to evaluate the extent to which a cue has been retained in memory, prior to an evaluation of the application of the cue in context.

Current Research

The current study employed a guided cue exposure approach that was designed to expose novice criminal investigators to a limited number of cue-based associations typically used by expert investigators. In doing so, we hypothesized that novices would significantly improve their recognition of expert associations, as assessed with a paired association task methodology (Morrison et al., 2013). Having established improvements in cue recognition, we hypothesized that improvements would also be evident in the accuracy and efficiency of novice decision making during a simulated criminal investigation task.

Criminal investigation was selected as the domain of interest because it constitutes at least two capabilities: (1) the identification of key features associated with a crime scene and (2) the association between these features and the features/events commonly associated with persons of interest (also known as suspects). The study targeted the second stage of this process and controlled for the preceding stage through greater and lesser levels of decision support. In the case of lesser decision support, the key features pertaining to the crime scene were presented in the absence of any prioritization. In the case of greater decision support, the key features associated with the crime scene were highlighted for participants. This is consistent with the approach to cue-based decision support employed successfully by Perry et al. (2012) in the context of firefighting.

Two studies are reported. Study 1 was designed to test whether exposure to a set of expert-identified cue-based associations in the absence of contextual variables could improve participants’ cue recognition. This study was designed to ensure that any improvement in participants’ cue recognition performance could be attributed to the cue-based exposure adopted here and not to their exposure to the outcome measures employed (i.e., the paired association task or the decision-making task used in Study 2).

Study 2 was designed to test whether (1) a significant improvement in novices’ cue recognition established in Study 1 (and replicated in Study 2) would be associated with improvements in decision-making performance (accuracy, time, and information acquisition) in a criminal investigation task and (2) decision-making performance was moderated by the degree of decision support available during the simulated criminal investigation task.

Study 1

Method

Design

The study comprised a pretest-posttest control group design, which centered on a cue exposure intervention. The independent variables were the time of testing (pre- or postexposure) and the type of intervention (exposure or no exposure). The dependent variable was participants’ recognition of expert cue-based associations, which was assessed with response accuracy and latency on a paired association task. The expert associations were originally identified by Morrison et al. (2013), who developed a method for decomposing expert cue-based relationships among criminal investigators. Their intention was to develop a mixed methods approach to identifying and validating cue-based relationships that overcame the inherent limitations of a purely descriptive approach. Using a cognitive task analysis approach (Militello & Hutton, 1998), feature and event descriptions were collected from practitioners from a number of law enforcement agencies (e.g., the Federal Bureau of Investigation, the Royal Canadian Mounted Police) and across multiple countries (e.g., Australia, the United States, and Canada). Pairings of these descriptions were then presented to expert practitioners in a paired association task, which required them to respond as to whether an association existed between each pairing. High levels of agreement and relatively rapid recognition among experts was used to infer the validity of a cue-based relationship (i.e., see Anderson’s [1983] discussion regarding production “strength” in memory), and, presumably, the utility of the relationship in practice (i.e., often termed cue diagnosticity; Wickens et al., 2013).

Participants

The participants comprised 20 novice criminal investigators (12 male and 8 female, M_age = 25.3 years). They were drawn from a pool of undergraduate university students studying forensic science and policing. In the context of the present study, a novice was defined as an individual who had declarative knowledge pertaining to the domain but little or no task-oriented experience. This research complied with the American Psychological Association code of ethics and was approved by the Institutional Review Board at Western Sydney University, Australia. Informed consent was obtained from each participant.

Materials

Forty one crime-related and 28 offender-related features previously identified by Morrison et al. (2013) were used as the paired association task stimuli. These features were selected because they yielded the highest degree of agreement and relatively rapid recognition among experts in the study. The task presented 168 trials consisting of 46 feature-event pairings and 122 randomly selected distractor pairings (drawn from the 1,102 remaining possible pairings).

The task was presented to participants on a laptop computer with DMDX software (Forster & Forster, 2003). The software recorded the responses selected (i.e., whether participants considered an association between the items to exist) and the participants’ response latency. As stimulus durations were based on a multiple of the laptop’s specific screen refresh time of 16.46 ms, the same laptop was used for all participants. Two markers were posted at the top of the computer screen differentiating the crime-related label (left of screen) from the offender-related label (right of screen) to assist the orientation of participants. The right shift key was labeled Yes, and the left shift key was labeled No. The crimerelated event was displayed briefly (left of screen) prior to the potentially associated offender feature (right of screen). Several arrow marks were presented at the center of screen to aid participants’ tracking of information. The exposure program adopted a guided cue exposure method where feature-event pairs were presented to participants with DMDX software (Forster & Forster, 2003). Each pair was presented for 5,764.5 ms, three times, to facilitate retention. There was no requirement for a response during the exposure program.

Procedure

The participants were seated in front of the laptop computer. Prior to the paired association task, participants were asked to engage familiarization trials, each consisting of a feature-event pairing that was either largely associated (e.g., sky + blue) or not (e.g., sky + crocodile). Participants were instructed that the pairs to be used in the test condition would all be drawn from the domain of criminal investigation. Prior to commencing the test condition, participants were provided the list of crime scene feature-event labels to ensure that they were familiar with the terminology used. After it was confirmed that all terminology was understood correctly, participants began the test condition.

For each test trial, participants were briefly presented with a crime-related label at the center of the screen (3,292 ms). Subsequently, this screen was replaced with the presentation of the same label on the left side of screen for 1,646 ms. After the label disappeared from the screen, several arrow marks (i.e., >>>>>>>) were presented at the center of screen for 1,234.5 ms to prompt the user’s attention to the right of screen. After the arrows disappeared, an offender-related feature label was presented on the right side of the screen for 1,646 ms (consistent with the presentation durations used by Morrison et al., 2013). An illustration of the paired association task is shown in Figure 1.

Figure 1.

An example of the sequence of screen presentations within the paired association task.

For each presentation, participants were instructed to decide whether each pairing was associated by striking either the right (Yes) or left (No) shift key on the computer provided, as quickly as possible. The test phase consisted of two blocks of presentations. Each block comprised 84 trials that constituted the combination of 12 crime-related events paired with seven offender-related features, a total of 168 trials in the experimental session. Participants were given the opportunity to rest between blocks, if desired.

Subsequent to a brief rest period, participants engaged one of two conditions: exposure or no exposure (control). This was designed to ensure that any observed improvement in cue recognition could be attributed primarily to the participants’ engagement in exposure and was not the result of experience with the paired association task.

In the exposure condition, participants were again seated in front of the laptop. Prior to commencing the exposure trials, participants were informed that they would be shown a number of paired labels that were reported as being strongly associated by a number of expert criminal investigators. Furthermore, participants were asked to commit these pairings to memory.

For each trial in the exposure condition, participants were briefly and simultaneously presented with a crime-related label on the left side of screen and an offender-related label on the right. As the aim of the exposure condition was not to promote recognition memory, as intended within the paired association task, but instead to promote acquisition, presentation time was increased to 5,764.5 ms (approximately 3.5 times the presentation duration in the paired association task). Participants were not required to respond to any of the presentations. The exposure program consisted of four blocks of trials. Each block had a number of trials (12, 12, 11, and 11, for a total of 46) and was repeated three times to promote long-term acquisition. Blocks maintained a consistent ordering of pairings to promote rapid acquisition. Participants were given the opportunity to rest between blocks, if desired.

For the no-exposure group (control), participants did not receive exposure to the expert sample of feature-event pairings. Instead, they received a filler task of a similar duration to the exposure condition.

Finally, participants from both groups again engaged in cue recognition testing that was consistent with the preexposure phase (i.e., paired association task), with the exception of the familiarization protocol and the order in which the feature-event pairs were presented (to control for practice effects).

Results

Analyses

The aim of these analyses was to determine whether there was a difference in cue recognition accuracy and latency between pre- and postexposure. An accuracy rate was calculated for each participant as the ratio of correct to incorrect responses. The mean ratio of accurate to inaccurate responses was then compared across pre- and postexposure conditions with a mixed repeated analysis of variance (ANOVA), incorporating the time of testing as a within-groups factor with two levels (pre- and postexposure) and intervention type as a between-groups factor with two levels (exposure or no exposure). With parametric assumptions met, the results revealed a statistically significant main effect for time of testing, F(1, 18) = 12.28, p = .003, η² = .41, revealing an overall increase in recognition accuracy from preexposure (M = 1.03, SD = 0.32) to postexposure (M = 2.38, SD = 2.38). A Time of Testing × Intervention Type interaction was significant, F(1, 18) = 12.43, p = .002, η_p² = .41, revealing a statistically significant increase in accuracy from pre- to postexposure for the participants who received the expert cue set (M_Diff = 2.70, SE = 0.77), F(1, 9) = 12.37, p = .007, η_p² = .58, but not for those who received the control set (M_Diff = 0.01, SE = 0.03; p > .05).

A mixed repeated ANOVA was used to assess for differences in response latency, incorporating the time of testing as a within-groups factor with two levels (pre- or postexposure) and intervention type as a between-groups factor with two levels (exposure or no exposure). With parametric assumptions met, the results revealed a statistically significant main effect for time of testing, F(1, 18) = 11.09, p = .004, η² = .38, indicating an overall decrease in response latency from preexposure (M = 1,090.27, SD = 143.53) to postexposure (M = 974.72, SD = 200.64). A statistically significant main effect was also evident for the intervention type, F(1, 18) = 11.57, p = .030, η_p² = .39, indicating that the participants in the exposure group demonstrated significantly shorter response latencies (M = 948.85, SD = 105.95) than the participants in the no-exposure group (M = 1,116.13, SD = 150.62). A Time of Testing × Intervention Type interaction was significant, F(1, 18) = 18.64, p < .001, η_p² = .51, revealing a statistically significant decrease in response latency from pre- to postexposure for the participants who received the expert cue set (M_Diff = −265.39, SE = 53.50), F(1, 9) = 24.61, p = .001, η_p² = .73, but not for those who received the control set (M_Diff = −34.28, SE = 44.23; p > .05).

These results suggest that those who received expert cue exposure subsequently improved their recognition of the target feature-event pairings beyond those in the control group. Ultimately, these results suggest that the observed improvement of participants who received the exposure intervention was primarily attributable to the exposure strategy.

Conclusion

The aim of Study 1 was to test whether exposure to a set of expert cue-based associations in the absence of contextual variables could improve participants’ subsequent recognition of these pairs. The study was devised to ensure that any observed improvement in participants’ performance could be attributed to the cue-based exposure employed and not to their exposure to the outcome measure itself.

Although the findings of Study 1 suggest that the recognition of expert cue-based associations may be improved through exposure, the value of such improvements remains unclear in the absence of an evaluation in which the application of these associations is applied in context. Therefore, Study 2 incorporated an additional stage of testing, which added a criminal investigation decision-making task to determine whether the outcomes of expert-cue exposure translate to improvements in decision-making performance.

Study 2

Method

Design

The study comprised a pretest-posttest control group design that centered on a cue exposure intervention (see Figure 2). The independent variables were the time of testing (pre- or postexposure), the type of intervention (expert cue set or control), and the degree of decision support within the assessment interface (low or high; detailed in Materials). The dependent variables were participants’ recognition of expert cue-based associations (response accuracy and latency on a paired association task), decision scenario accuracy (correct identification of person of interest), the time to complete decision scenarios (in seconds), and information acquisition efficiency during decision scenarios (the number of features accessed).

Figure 2.

A flowchart of the experimental design of Study 2.

Participants

The participants included 36 novice criminal investigators (12 male and 24 female, M_age = 23.45 years) who did not participate in Study 1. Consistent with Study 1, they were drawn from a pool of undergraduate university students studying forensic science and policing. This research complied with the American Psychological Association’s code of ethics and was approved by the Institutional Review Board at Western Sydney University, Australia. Informed consent was obtained from each participant.

Materials

Study 2 adopted the same materials used in the paired association task and exposure program in Study 1. To assess decision-making performance, the study involved an online criminal investigation task designed by the research team. The task was designed to operate as a surrogate for the operational environment—in this case, the criminal investigation process. The interface provided an opportunity for controlled manipulation of a naturalistic decision scenario, which, for many domains (including criminal investigation), is impractical to observe in situ. The task had an online interface that presented users with several screens that were intended to reflect a decision task typically encountered by criminal investigators. A similar interface was used by Perry et al. (2013) in investigating the use of reduced processing decision support systems in a firefighting context.

The interface initially presented the user with a brief outline of a scenario, which described the details of a serious crime (e.g., homicide) with case-specific details provided by experienced investigators with access to real-life homicide case files from the United States and Australia. These cases were “low profile” to reduce the possibility that they would be familiar to participants.

After completion of the introduction screens, a more expansive representation of the scenario was described, presented in one of two forms, each differing in the extent to which it reflected the naturalistic environment. First, within the low decision support interface, crime-related features were embedded within a summary of the case. No attempt was made by the programmer to highlight or extract the most pertinent features of interest. In contrast, the high decision support interface, outlined the features of interest to the user. Here, the information from the naturalistic scenario was reduced to the features in a way that was largely consistent with the format of the exposure technique used and did not require extraction or generalization by the user.

Having acquired information from the scenario, users were presented with descriptions of three persons of interest. They were asked to formulate a decision regarding which person of interest was most likely to be the offender in the given case. To formulate this decision, users were provided with information regarding each person, presented in a list of clickable icons. These icons represent the offender-related feature labels provided during exposure. When “clicked,” the case-specific value of the feature was displayed. For example, when the feature label degree of intelligence was clicked, the case-specific value of this feature (e.g., above average) was displayed.

Four scenarios based on actual cases were included in the study. Although low profile, these cases were relatively complex investigations that required the use of forensic investigative techniques. The cases had all been previously solved and included a detailed profile of each offender. Feature labels identified by Morrison et al. (2013) were assigned case-specific values based on the information present within the real-life cases. However, only one person of interest was assigned features consistent with the known offender from the real-life case. The two remaining persons were assigned feature values consistent with persons of interest from the case and alternative case files.

Procedure

The participants were randomly assigned to one of two groups: one that had the high decision support interface format and one that had the low decision support interface format as part of the assessment strategy. Using a hyperlink to the online interfaces, participants were asked to complete two scenarios and determine which of three persons of interest was most likely to be the offender based on the features available. The presentation order of the scenarios was counterbalanced across both conditions for all participants.

Following the completion of the preintervention scenarios, participants were invited to complete a paired association task (consistent with the process outlined in Study 1). On completion of the task and a brief rest period, participants were randomly assigned to complete one of two cue exposure activities, which were consistent with the process outlined in Study 1; however, one featured the expert cue set and the other a control cue set (comprising cues deemed irrelevant to the criminal investigation context).

Following the exposure activity, participants completed the same version of the paired association task initially employed, although the presentation order was randomized. On completion of the second paired association task, they were asked to complete two further criminal investigation scenarios, the presentation of which was counterbalanced across the high and low support interface conditions. In each case, participants’ decisions, time to respond, and the sequence in which information was acquired were recorded.

Results

The analyses were divided into two sections, each relating to different performance variables examined: cue recognition (paired association task; i.e., response accuracy and latency) and decision making (criminal investigation task; i.e., decision accuracy, decision time, and information acquisition).

Cue recognition

The aim of these analyses was to determine whether a difference existed in participants’ response accuracy and latency in responding to associations from the expert target sample across the pre- and postexposure conditions. The approach to calculating response accuracy and latency was consistent with the approach used in Study 1.

Cue recognition: Response accuracy. A mixed repeated measures ANOVA was used to test for differences in recognition accuracy from pre- to postexposure conditions. Time of testing (pre- and postexposure) was the within-subjects variable, whereas intervention type (expert cue set or control) and interface (low or high decision support) were the between-subjects variables. With alpha set at .05 and parametric assumptions met, results revealed a statistically significant main effect for time of testing, F(1, 32) = 72.07, p < .001, η_p² = .69, revealing a significant increase in recognition accuracy between preexposure (M = 1.35, SD = 0.51) and postexposure (M = 3.28, SD = 1.45). A Time of Testing × Intervention Type interaction was significant, F(1, 32) = 74.50, p < .001, η_p² = .70, revealing that the increase in accuracy occurred for the participants who received the expert cue set exposure activity (M_Diff = 2.92, SE = 0.20), F(1, 22) = 205.44, p < .001, η_p² = .90, but not for those who received the control exercise (M_Diff = 0.02, SE = 0.26; p > .05). No main effects or interactions were found for interface type (p > .05).

Cue recognition: Response latency. A mixed repeated measures ANOVA was used to test for differences in response latency from pre- to postexposure conditions. Time of testing (pre- and postexposure) was the within-subjects variable, whereas intervention type (expert cue set or control) and interface (low or high decision support) were the between-subjects variables. With alpha set at .05 and parametric assumptions met, results revealed a statistically significant main effect for time of testing, F(1, 32) = 72.07, p < .001, η_p² = .11, indicating a significant reduction in mean response latency between preexposure (M = 1,156.96, SD = 313.29) and postexposure (M = 1,053.78, SD = 183.73). A Time of Testing × Intervention Type interaction was significant, F(1, 32) = 7.24, p = .011, η_p² = .18, revealing that a statistically significant reduction in response latencies occurred for the participants who received the expert cue set (M_Diff = −167.44, SE = 37.80), F(1, 34) = 7.57, p = .009, η_p² = .18, but not for those who received the control (M_Diff = −25.33, SE = 68.32; p > .05). No main effects or interactions were found for interface type (p > .05).

Decision making: Decision accuracy

To establish the accuracy of participants’ responses to the scenarios, their choice of the person of interest (of the three possible options) was coded as either 1 (for a correct response) or 0 (for an incorrect response) for each of the two scenarios, at each time of testing (pre- and postexposure). Therefore, participants could attain 0, 1, or 2 correct responses in each of the pre- and postexposure conditions. Given a nonnormal distribution, two Wilcoxon signed-rank nonparametric tests examined differences in accuracy for those participants receiving the expert cue set exposure: one analysis for each of the interface styles (high and low decision support). With alpha set at .05, the results for the low decision support interface failed to reveal a significant difference between the median rank for the preexposure condition (Mdn = 1, range = 2) and that for the postexposure condition (Mdn = 1, range = 1), z (n = 11) = 1.61, p = .111. However, a significant effect was evident for the high decision support interface, in which the median rank for the postexposure condition (Mdn = 2, range = 1) was significantly greater than the median rank for the preexposure condition (Mdn = 1, range = 2), z (n = 12) = 2.13, p = .030, r = .22. Wilcoxon signed-rank nonparametric tests were also used to examine differences in accuracy for those participants receiving the control cue set exposure; however, no significant differences were found (p > .05). These results indicated that the improvement in decision accuracy was restricted to those participants who received the expert cue exposure and engaged the high decision support interface.

Decision making: Decision time

A mixed repeated measures ANOVA was used to determine whether a difference in decision time existed from pre- to postexposure conditions. Time of testing (pre- and postexposure) was the within-subjects variable, whereas intervention type (expert cue set or control) and interface (low or high decision support) were the between-subjects variables. With alpha set at .05 and parametric assumptions met, the results revealed a statistically significant main effect for the mean decision time, F(1, 31) = 4.28, p = .047, η_p² = .12, revealing a significant reduction in decision time from preexposure (M = 595.73, SD = 242.38) to postexposure (M = 493.94, SD = 182.86). A Time of Testing × Intervention Type interaction was significant, F(1, 31) = 5.02, p = .032, η_p² = .14, revealing that a statistically significant reduction in decision time occurred for the participants who received the expert cue set exposure (M_Diff = −156.33, SE = 50.48), F(1, 21) = 9.59, p = .005, η_p² = .31, but not for those who received the control set (M_Diff = 6.205, SE = 21.75; p > .05). No main effects or interactions were found for interface type (p > .05). These results indicated a statistically significant reduction in the time taken to complete the scenarios following expert cue exposure, regardless of the interface engaged.

Decision making: Information acquisition

A mixed repeated measures ANOVA was used to determine whether a difference in the number of features accessed existed from pre- to postexposure conditions. Time of testing (pre- and postexposure) was the within-subjects variable, whereas intervention type (expert cue set or control) and interface (low or high decision support) were the between-subjects variables. With alpha set at .05 and parametric assumptions met, the results failed to reveal a statistically significant main effect for the mean number of features accessed between pre- and postexposure (p > .05). However, a Time of Testing × Intervention Type interaction was significant, F(1, 31) = 4.88, p = .035, η_p² = .14, revealing that a statistically significant reduction in features accessed occurred for the participants who received the expert cue set (M_Diff = −17.19, SE = 5.89), F(1, 21) = 8.51, p = .008, η_p² = .29, but not for those who received the control set (M_Diff = 6.205, SE = 21.75; p > .05). No main effects or interactions were found for interface type (p > .05). These results indicated a significant decrease in the number of features accessed when scenarios are engaged subsequent to expert cue set exposure, regardless of interface type.

Discussion

The current research tested whether (1) a significant improvement occurred in novice recognition (accuracy and latency) of expert identified cue-based associations following cue exposure, (2) a significant improvement in novices’ cue recognition was matched by improvements in decision-making performance (accuracy, time to complete, and information acquisition) during simulated scenarios, and (3) decision-making performance was moderated by the degree of decision support available during a criminal investigation decision-making task.

The results revealed a statistically significant improvement in participants’ recognition of the expert cue-based associations following cue exposure. Furthermore, exposure to expert feature-event pairs was associated with a reduction in decision time and the amount of information acquired during simulated scenarios and with an increase in decision accuracy where the scenarios were presented with a high decision support interface. This suggests that although exposure to cue-based associations was associated with improvements in their recognition of said associations, the successful application of these associations was restricted to scenarios that explicitly embodied the cues as they were presented during exposure. This is consistent with the study by Perry et al. (2013), who reported that the cue-based exposure for novice incident command personnel was effective in improving decision performance only when learners were provided with an interface that restricted information access to the most critical cues. The results emphasize the importance of acquiring cue-based associations in context, above and beyond the initial conditioning stages of cue acquisition. This is largely consistent with Anderson and Matessa’s (1997) concept of production development during the progression toward expertise.

Importantly, generalized transfer may have been hampered by the block ordering of exposure in the acquisition phase. Although variable order (i.e., random practice) has been shown to result in poorer performance during initial acquisition as compared with block ordering, it typically yields superior performance for retention and generalization (see the contextual interference effect; Merbah & Meulemans, 2011). Furthermore, there appeared greater separation among participants on process measures (e.g., information acquisition) than on outcome measures (e.g., decision accuracy), which may be partly explained by the restricted measurement range in the latter. Future studies may consider expanding data collection to include alternative process measures (e.g., eye tracking).

Applied Implications

In the context of decision making, criminal investigation is a relatively unusual domain because the aim is to identify a potential suspect on the basis of a series of features that are evident as part of a crime scene (i.e., events). It requires the capacity to identify key features associated with a crime scene and the capacity to associate these features and the features of suspects. The present study targeted the final stage of this process, while controlling for the preceding stages through the use of high and low decision support interfaces. However, in doing so, it revealed the potential utility of decision support mechanisms for novices in developing the capability to identify the key features associated with a crime scene and associate these features with the events.

In essence, the high decision support interface constituted a reduced processing decision support system (Morrison, Wiggins, & Porter, 2010; Perry et al., 2013), the intention of which was to guide users toward a limited set of key features that might be considered a part of the decision-making process. Reduced processing decision support systems are particularly useful for less experienced operators, given that such systems expose learners to specific cue-based relationships and provide a safeguard against the acquisition of inappropriate information and a resultant loss of decision accuracy. The utility of reduced processing decision support systems as a learning tool has been demonstrated in aviation (Wiggins & Bollwerk, 2006) and firefighting (Perry et al., 2012) and might offer opportunities within the context of criminal investigation.

At a theoretical level, the outcomes confirm the utility of cue acquisition as a mechanism to improve decision accuracy. However, the results also demonstrate that the utility of cue-based exposure in the context of criminal investigation is dependent on a preceding skill base. Previous approaches to cue-based exposure have typically involved at least competent practitioners, reflecting the importance of a “mental model” (an internal mental explanation of the external world; Staggers & Norcio, 1993) and a context within which to “situate” the cues that are acquired. For example, in the case of the cue-based approach to in-flight weather conditions, Wiggins and O’Hare (2003) trained pilots with at least a private pilot’s license, recognizing that these pilots would have acquired some experience in responding to changing weather conditions and would have the capacity to contextualize the cue-based associations to which they were exposed.

Conclusion

The outcomes of this study offer insight into the utility of expert cue exposure as a mechanism to improve the decision-making performance among novice criminal investigators. Although the participants involved in the current research developed some level of awareness of the key associations involved in the domain—who, what, when, and where—their capacity to recognize meaning within a bounded context, as well as the inherent strategies and limitations of an investigation, appears to have remained limited. Accordingly, it would appear prudent that cue-based exposure programs be administered to operators who have reached a level of competency and who, on the basis of their exposure to previous cases, may be able to construct mental models that can guide their selection and interpretation of cue-based information.

Overall, the current findings suggest that although the acquisition of valid cue-based associations is vital to the development of expertise, a cognitive gap remains between expert and novice performance that cue use alone cannot account for. Ultimately, the key to closing this gap appears to hinge on a number of additional cognitive skills (e.g., sense making), which many decision makers will most likely acquire as a result of extensive domain-specific operational experience (Klein, 1998; Wiggins, 2015b).

Footnotes

Ben W. Morrison is a registered psychologist and senior lecturer in psychological sciences at the Australian College of Applied Psychology. He gained his PhD in psychology from the University of Western Sydney, Australia.

Mark W. Wiggins is a registered organizational psychologist and professor of psychology at Macquarie University, Australia. He gained his PhD in psychology from the University of Otago, New Zealand.

Natalie M. V. Morrison is a registered psychologist and senior lecturer in mental health at the University of Western Sydney. Natalie completed her PhD at the University of Western Sydney, Australia.

References

Abernethy

Russell

D. G.

(1987). The relationship between expertise and visual search strategy in a racquet sport. Human Movement Science, 6, 283–319.

Anderson

J. R.

(1983). The architecture of cognition. Cambridge, MA: Harvard University Press.

Anderson

J. R.

Matessa

M. P.

(1997). A production system theory of serial memory. Psychological Review, 104, 728–748.

Beilock

S. L.

Wierenga

S. A.

Carr

T. H.

(2002). Expertise, attention, and memory in sensorimotor skill execution: Impact of novel task constraints on dual-task performance and episodic memory. Quarterly Journal of Experimental Psychology: Section A, 55, 1211–1240.

Boreham

(1995). Error analysis and expert-novice differences in medical diagnosis. In Hoc

J. M.

Cacciabue

P. C.

Hollnagel

(Eds.), Expertise and technology (pp. 93–105). Hillsdale, NJ: Erlbaum.

Brunswik

(1956). Perception and the representative design of psychological experiments. Berkeley, CA: University of California Press.

Buckolz

Prapavessis

Fairs

(1988). Advance cues and their use in predicting tennis passing shots. Canadian Journal of Sport Science, 13(1), 20–30.

Chater

Oaksford

(2006). Mental mechanisms: Speculations on human causal learning and reasoning. In Fiedler

Juslin

(Eds.), Information sampling and adaptive cognition (pp. 309–330). Cambridge, UK: Cambridge University Press.

de Groot

A. D

. (1966). Perception and memory versus thought: Some old ideas and recent findings. In Kleinmuntz

(Eds.), Problem-solving: Research, method, and theory (pp. 19–50). New York, NY: Wiley.

10.

Ebbesen

Konecni

(1975). Decision making and information integration in the courts: The setting of bail. Journal of Personality and Social Psychology, 32, 805–821.

11.

Fisher

Pollatsek

(2007). Novice driver crashes: Failure to divide attention or failure to recognize risks. In Kramer

A. F.

Wiegmann

D. A.

Kirlik

(Eds.), Attention: From theory to practice (pp. 134–156). Oxford, UK: Oxford University Press.

12.

Forster

K. I.

Forster

J. C.

(2003). DMDX: A Windows display program with millisecond accuracy. Behavior Research Methods, Instruments, and Computers, 35, 116–124.

13.

Jackson

R. C.

Warren

Abernethy

(2006). Anticipation skill and susceptibility to deceptive movement. Acta Psychologica, 123(3), 355–371.

14.

Johnston

Morrison

B. W.

(2016). Exploring cue use in rugby league playmakers to inform training initiatives. Journal of Cognitive Engineering and Decision Making, 10(4), 391–410.

15.

Jones

Miles

(1978). Use of advance cues in predicting the flight of a lawn tennis ball. Journal of Human Movement Studies, 4, 231–235.

16.

Kahneman

(2011). Thinking, fast and slow. New York, NY: Farrar, Straus & Giroux.

17.

Kirschenbaum

S. S.

(1992). Influence of experience on information-search strategies. Journal of Applied Psychology, 77, 343–352.

18.

Klein

G. A.

(1993). A recognition primed decision (RPD) model of rapid decision making. In Klein

G. A.

Orasanu

Calderwood

Zsambok

C. E.

(Eds.), Decision making in action: Models and methods (pp. 138–147). Norwood, NJ: Ablex.

19.

Klein

G. A.

(1998). Sources of power: How people make decisions. Cambridge, MA: MIT Press.

20.

Klein

G. A.

Calderwood

Clinton-Cirocco

(1986). Rapid decision-making on the fire ground. Proceedings of the Human Factors and Ergonomics Society 30th Annual Meeting, 1, 576–580.

21.

Loveday

Wiggins

M. W.

(2014). Cue utilization and broad indicators of workplace expertise. Journal of Cognitive Engineering and Decision-Making, 8, 98–113.

22.

McCormack

Wiggins

M. W.

Loveday

Festa

(2014). Expert and competent non-expert visual cues during simulated diagnosis in intensive care. Frontiers in Cognition, 5, 949.

23.

Merbah

Meulemans

(2011). Learning a motor skill: Effects of blocked versus random practice—a review. Psychologica Belgica, 51(1), 15–48.

24.

Militello

L. G.

Hutton

J. R.

(1998). Applied cognitive task analysis (ACTA): A practitioner’s toolkit for understanding cognitive task demands. Ergonomics, 41(11), 1618–1641.

25.

Morrison

Wiggins

Porter

(2010). User preference for a control-based reduced processing decision support interface. International Journal of Human-Computer Interaction, 26, 297–316.

26.

Morrison

Wiggins

M. W.

Bond

Tyler

(2013). Measuring cue strength as a means of identifying an inventory of expert offender profiling cues. Journal of Cognitive Engineering and Decision-Making, 7, 211–226.

27.

Morrow

D. G.

Miller

L. M. S.

Ridolfo

H. E.

Magnor

Fischer

U. M.

Kokayeff

N. K.

Stine-Morrow

E. A.

(2009). Expertise and age differences in pilot decision making. Aging, Neuropsychology, and Cognition, 16, 33–55.

28.

Perry

Wiggins

M.W.

Childs

Fogarty

(2012). Can reduced processing decision support interfaces improve the decision-making of less-experienced incident commanders? Decision Support Systems, 52, 497–504.

29.

Perry

N. C

Wiggins

M. W.

Childs

Fogarty

(2013). The application of reduced processing decision support systems to facilitate the acquisition of decision-making skills. Human Factors, 55, 535–544.

30.

Schriver

A. T.

Morrow

D. G.

Wickens

C. D.

Talleur

D. A.

(2008). Expertise differences in attentional strategies related to pilot decision making. Human Factors, 50(6), 864–878.

31.

Searle

B. J.

Rooney

(2015). Diagnostic cues in finance. In Wiggins

M. W.

Loveday

(Eds.), Diagnostic expertise in organisational environments (pp. 101–116). Aldershot, UK: Ashgate.

32.

Shanteau

(1991). Psychological characteristics and strategies of experts. In Wright

Bolger

(Eds.), Expertise and decision support (pp. 55–76). New York, NY: Plenum.

33.

Shanteau

(1992). How much information does an expert use? Is it relevant? Acta Psychologica, 81(1), 75–86.

34.

Smeeton

N. J.

Williams

A. M.

Hodges

N. J.

Ward

(2005). The relative effectiveness of various instructional approaches in developing anticipation skill. Journal of Experimental Psychology: Applied, 11(2), 98–110.

35.

Spanjers

I. A. E.

Wouters

van Gog

van Merrienboer

J. J. G.

(2011). An expertise reversal effect of segmentation in learning from animated worked-out examples. Computers in Human Behavior, 27(1), 46–52.

36.

Staggers

Norcio

A. F.

(1993). Mental models: Concepts for human-computer interaction research. International Journal of Man-Machine Studies, 38(4), 587–605.

37.

Stokes

A. F.

Kemper

K. L.

Marsh

(1992). Time-stressed flight decision making: A study of expert and novice aviators (Technical Report No. ARL-93-1/INEL-93-1). Savoy, IL: Aviation Research Laboratory.

38.

Wickens

C. D.

Hollands

J. G.

Banbury

Parasuraman

(2013). Engineering psychology and human performance (4th ed.). Upper Saddle River, NJ: Pearson.

39.

Wiggins

M. W.

(2014). The role of cue utilisation and adaptive interface design in the management of skilled performance in operations control. Theoretical Issues in Ergonomics Science, 15(3), 283–292.

40.

Wiggins

M. W.

(2015a). Cues in diagnostic reasoning. In Wiggins

M. W.

Loveday

(Eds.), Diagnostic expertise in organizational environments (pp. 1–13). Aldershot, UK: Ashgate.

41.

Wiggins

M. W.

(2015b). Diagnosis and instructional systems design. In Wiggins

M. W.

Loveday

(Eds.), Diagnostic expertise in organizational environments (pp. 76–90). Aldershot, UK: Ashgate.

42.

Wiggins

M. W.

Bollwerk

(2006). Information acquisition and decision-making amongst pilots. Human Factors, 48, 734–746.

43.

Wiggins

O’Hare

(2003). Expert and novice pilot perceptions of static in-flight images of weather. International Journal of Aviation Psychology, 13(2), 173–187.