The Role of Human Factors/Ergonomics in the Science of Security

Science of Security

In 2011, the National Science and Technology Council (NSTC) published a strategic plan for the federal cybersecurity research and development program. One thrust in the plan is Developing Scientific Foundations. The plan describes the current state of research as a patchwork of solutions to specific vulnerabilities and states, “A more fruitful way to ground research efforts, and to nurture and sustain progress in the kinds of improved cybersecurity solutions that benefit society, is to develop a science of security” (p. 10). The idea is to develop scientific laws, principles, and models that can be applied to a range of cybersecurity issues as they arise. “Sound methods for integrating humans in the system: usability and security” (p. 11) is among seven areas specified in which research is needed.

As part of the science of security research effort, the National Security Agency (NSA) has fostered development of a research community for which the goal is “to bring scientific rigor to research in the cybersecurity domain” (Science of Security, 2014). Closely following the NSTC strategic plan, one of the “hard problems” targeted by NSA is “understanding and accounting for human behavior.” Interest exists in methods to model adversaries and for integrating humans in the system, with an emphasis on usability (Networking and Information Technology Research and Development Program, 2014). More specifically, a central concern is to bring scientific knowledge of human cognition to bear on understanding perceptions of security risks, security-related decisions, and the choices among alternative actions that humans make that may protect or imperil system security.

Scientific principles of risk perception, decision making, action selection, and training have been developed in basic and applied cognitive research conducted from the “cognitive revolution” in the 1950s to the present (Healy, Schneider, & Bourne, 2012; Proctor & Vu, 2010; Wickens, Hollands, Banbury, & Parasuraman, 2012), and these can provide a principled basis for analyzing the factors affecting security-related human decisions and choices. We demonstrate the value of taking an approach to cybersecurity based on scientific principles using two vulnerabilities, phishing attacks and malicious applications (apps), as examples. These vulnerabilities were selected because HF/E scholars should be familiar with them and because they have been identified as the primary cyberthreats to mobile devices (Ruggiero & Foote, 2011), which now exceed PCs in Internet usage.

Detection of Phishing Attacks

Phishing is defined by the Merriam-Webster Dictionary (2014) as “a scam by which an e-mail user is duped into revealing personal or confidential information which the scammer can use illicitly.” Phishing attacks, like most violations of information privacy and security, rely on deception. An interchange between a deceiver and a target receiver occurs, but the decisions made by the receiver ultimately determine whether the deception is successful. Phishing is of concern because sophisticated phishing messages arrive regularly in e-mail, and people are not accurate at distinguishing well-designed phishing messages and sites from genuine ones (Dhamija, Tygar, & Hearst, 2006; Downs, Holbrook, & Cranor, 2006).

To reduce susceptibility to phishing, it is essential to understand the processes by which users assess the risks associated with phishing, detect that a message may be a phishing attack, and decide whether to submit the requested information (Downs, Barbagallo, & Acquisti, 2015). A variety of software tools have been developed to display a warning when a possible phishing message or site is detected. However, these have not been very successful, in part because they miss as many as 50% of phishing sites (Cranor, Egelman, Hong, & Zhang, 2006). Because user decisions still must be made even with an automatic phishing detector, how the user decides to trust particular messages and makes decisions whether to act on them must be considered (e.g., Shahriar & Zulkernine, 2011).

A human information-processing theory that has been applied to detection of deception in computer-mediated environments is the theory of deception detection (Grazioli, 2004; Johnson, Grazioli, Jamal, & Berryman, 2001). This theory specifies four processing stages that affect whether a receiver will detect the deception: activation, hypothesis generation, hypothesis evaluation, and global assessment (see Figure 1 for descriptions). The different stages in the model enable detailed examination for the determinants of successfully detecting an Internet deception such as a phishing attack. Specifically, Grazioli (2004) found that the major factor differentiating persons who were able to detect a deceptive Web site was in their ability to evaluate cues (hypothesis evaluation), from which he concluded, “Domain-specific knowledge about how to evaluate individual cues is a strong performance differentiator” (p. 164).

OPEN IN VIEWER

Figure 1.

Model of deception detection (adapted from Grazioli, 2004) as applied to detecting a phishing email. In the activation stage, the user attends to cues that mismatch with expectations, suggesting “something may be wrong.” At the stage of hypothesis generation, the user generates hypotheses to explain the mismatch with expectations, at least one hypothesis of which must include deception if the deception is to be detected. Hypothesis evaluation, which follows, involves evaluation of the generated deception hypotheses, with the user deciding whether to accept the deception hypothesis for each cue. In the fourth, global assessment stage, the user assigns different weights to the individual hypotheses, along with information cues relating to trust and assurance, in an overall assessment of deception. The decision that there has been deception can be based on a single, strong deception hypothesis or the sum of several weaker ones.

A model by Vishwanath, Herath, Chen, Wang, and Rao (2011) places more emphasis on attention and memory. It includes an initial attention stage that comprises whether the user attends to the cues. Greater perceived relevance of the message to the user’s needs is assumed to lead to greater attention to its details. Detection, hypothesis generation, and evaluation are attributed to an elaboration process, which, in agreement with Grazioli (2004), requires comparison of the cues with domain-specific knowledge stored in memory. E-mail load—the number of e-mails that the individual receives daily—is considered to be critical. As with mental workload in general (Wickens, 2008), the increase in e-mail load is presumed to reduce the amount of effort the user can devote to specific messages and their details, increasing the likelihood of being phished.

The models of Grazioli (2004) and Vishwanath et al. (2011) illustrate how the processes in which a user engages when deciding whether to respond to a phishing message can be analyzed and evaluated systematically. However, the models address only explicitly perceived deception and not the implicit processes that many decision-making models consider to also play an important role (Evans & Stanovich, 2013; Hommel, 2013). In Kahneman’s (2011) words, whereas explicit processes allocate “attention to the effortful mental activities that demand it,” implicit processes operate “automatically and quickly, with little or no effort” (pp. 20–21).

Downs et al. (2015) showed that it is essential to take implicit as well as explicit processes into account by analyzing people’s responses to phishing attacks. Participants role-played being an office worker performing an e-mail management task and chose among actions for each message that were not mutually exclusive. Participants whose stated intentions were to use stronger strategies to prevent being phished, and who evidenced explicit knowledge of phishing, showed a more conservative decision criterion of responding less frequently to both phishing and legitimate messages but not greater sensitivity at distinguishing them. But participants who exhibited implicit procedural knowledge (ability to ascertain legitimacy of a URL) yielded a decreased frequency of responding to phishing attacks without an increase in false alarms for legitimate e-mails (i.e., greater sensitivity). These studies demonstrate a need for researchers to measure actual choices among actions in studies of phishing and to ground the research in contemporary knowledge of decision-making processes and information processing, both of which fall within the purview of human factors specialists.

Selection of Mobile Apps

Another way in which malicious agents collect users’ privacy data without users’ awareness is through mobile apps. With the advent of mobile devices, online stores have developed that enable users to download apps for various purposes. For example, in the Google Play store for Android apps, an icon for the app is accompanied by an average user rating score (one to five filled stars) and the number of users on which it is based. A user can judge from these values whether an app is perceived by other users as useful and appropriate for carrying out its intended function. However, privacy risks are more difficult to evaluate. The risks associated with a specific app are identified in lengthy verbal permissions that are available to the user only after making the initial decision to download that app.

Because a preliminary decision to download the app has already been made, if the permissions indicate that the risks associated with the app are high, the user must now make a contrary decision that requires returning to the list of relevant apps and beginning the evaluation process anew. Based on the fact that people tend to minimize effort and rely on heuristics to simplify decisions (e.g., Kahneman, 2011; Wickens, 2014), it can be predicted that users will be reluctant to take such action that requires additional effort and movement away from the goal of installing an app. Indeed, Kelley, Cranor, and Sadeh (2013) showed that when permissions were displayed in simplified form on the main app screen prior to the initial decision, users selected apps that requested fewer permissions.

Even when permissions are presented earlier in the decision process, users will likely have difficulty comprehending them and may even ignore them. Felt et al. (2012) conducted an Internet survey of Android users, as well as a laboratory study. Only 17% of participants attended to the permissions when installing an app, and only 3% could correctly answer all of three permission-comprehension questions. Felt et al. concluded, “This indicates that current Android permission warnings do not help most users make correct security decisions” (p. 1).

We have been conducting interdisciplinary research on risk communication in app selection intended to remedy this problem (e.g., Chen, Gates, Li, & Proctor, in press). The human factors component is based on principles of decision making, including that information framing influences decisions (Kahneman & Tversky, 1979) and that people often rely on gist representations rather than on detailed verbatim representations (Brust-Renck, Royer, & Reyna, 2013). Central to our approach is an assumption that most users will prefer less detailed displays of risk information, such as a summary risk score (Chen et al., in press; Gates, Chen, Li, & Proctor, 2014) and indication of risk categories (Jorgensen et al., 2015).

With regard to summary risk scores, progress is being made toward developing techniques that can provide a reliable measure of overall risk for an app (Gates, Li, et al., 2014). Summary risk information has been shown to be effective in online app-selection studies and more tightly controlled laboratory experiments. When verbal risk indexes (low, medium, high) accompany two apps, the risky app is chosen less often and the users indicate that they paid more attention to the apps’ permissions and the associated risks (Gates, Chen, et al., 2014). Presentation of the risk information in the form of filled circles (more filled circles indicates greater risk), much like the user ratings, is similarly effective to the verbal indexes at influencing users’ app-selection choices.

When the filled circles are framed as amount of safety (more filled circles indicates greater safety) rather than risk, the influence is greater, and the users show less confusion about what the symbols indicate when subsequently asked in a survey question (Chen et al., in press; Gates, Chen, et al., 2014; see also Choe, Jung, Lee, & Fisher, 2013). A similar idea that positive framing of privacy information is beneficial is incorporated into assigning privacy grades (A, B, C, D) to Android apps (http://privacygrade.org/).

In addition to a summary risk (or safety) index, research has also identified the major risk categories (personal information privacy, monetary, and device availability/stability) that can be utilized by users who want more detailed information (Jorgensen et al., 2015). Principles of display design (e.g., Bennett & Flach, 2011) can form the basis for research to determine the most effective way to convey the risks associated with these categories to users when choosing among apps.

Conclusion

We have illustrated the applicability of scientifically based analyses of decision making to the domain of cybersecurity in the areas of detection of phishing attacks and mobile app-selection decisions. In each area, understanding the users and how they process information is key to developing tools that are effective in yielding actions that support cybersecurity. Both areas can be characterized as requiring users to make choices under conditions involving risk. In these conditions, (a) users’ knowledge of the risks is often limited, (b) the specific risks are difficult to comprehend, (c) attention needs to be directed to appropriate cues, and (d) decisions are likely impacted by both explicit and implicit processes.

Although we focused on only two areas, decision making and choice are also central to other areas in the domain of cybersecurity: Individual users must make decisions about passwords to use (Das, Bonneau, Caesar, Borisov, & Wang, 2014) and the privacy offered by Web sites (Vu, Chambers, Creekmur, Cho, & Proctor, 2010), which involve processes similar to those of phishing detection and app selection. System administrators must configure access control policies, detect intrusions, and take appropriate actions (Corchado & Herrero, 2011), and security analysts need to anticipate adversaries’ reactions to particular security actions (Proctor, Vu, & Schultz, 2009; Schultz, 2012). There is a need for research to integrate these areas of cybersecurity using scientifically based principles of perception, decision making, and action. The knowledge of human information processing in applied contexts possessed by human factors specialists and cognitive scientists, particularly when coupled with the knowledge of technology possessed by computer scientists and cybersecurity experts, should facilitate transformative advances in the science of security.

Key Points