The Triad of Risk-Related Behaviors (TriRB): A Three-Dimensional Model of Cyber Risk Taking

Abstract

Objective:

We identify three risk-related behaviors in coping with cyber threats—the exposure to risk a person chooses, use of security features, and responses to security indications. The combinations of behaviors that users choose determine how well they cope with threats and the severity of adverse events they experience.

Background:

End users’ coping with risks is a major factor in cybersecurity. This behavior results from a combination of risk-related behaviors rather than from a single risk-taking tendency.

Method:

In two experiments, participants played a Tetris-like game, attempting to maximize their gains, while exogenous occasional attacks could diminish earnings. An alerting system provided indications about possible attacks, and participants could take protective actions to limit the losses from attacks.

Results:

Variables such as the costs of protective actions, reliability of the alerting system, and attack severity affected the three behaviors differently. Also, users dynamically adjusted each of the three risk-related behaviors after gaining experience with the system.

Conclusion:

The results demonstrate that users’ risk taking is the complex combination of three behaviors rather than the expression of a general risk-taking tendency. The use of security features, exposure to risk, and responses to security indications reflect long-term strategy, short-term tactical decisions, and immediate maneuvering in coping with risks in dynamic environments.

Application:

The results have implications for the analysis of cybersecurity-related decisions and actions as well as for the evaluation and design of systems and targeted interventions in other domains.

Keywords

cybersecurity risk taking alerts adaptive behavior modeling

Introduction

Cybersecurity is one of today’s major technological and societal challenges (Whitman & Mattord, 2011). It involves technical aspects, such as encryption, access control, firewalls, and malware detection, which mostly relate to attackers’ and defenders’ interaction. However, successful mitigation of threats also requires an understanding of end users’ behavior (Proctor & Chen, 2015). Users may provide attackers with entry points to a system, for example, by choosing weak passwords or poorly protecting passwords (Stobert & Biddle, 2014). At times, they may open attachments with malicious payload (Canfield, Fischoff, & Davis, 2016). Even experienced end users or network administrators can fail to install software patches on time or can set security configurations and settings incorrectly (Pfleeger, Sasse, & Furnham, 2014).

The growing complexity of the cyber infrastructure, the interdependencies between its components, and the effects of user behavior on security make it necessary to study human interactions with cybersecurity risks (Lange et al., 2017). This research aims to develop models of user behavior and interaction that can support the successful development and implementation of security mechanisms, advise cybersecurity training, and guide policy decisions.

Many of the behaviors that affect security derive from users’ judgment about whether a risk is acceptable and their choice of whether to engage in risky behavior. The science of risk is complex and multifaceted (Loewenstein, Weber, Hsee, & Welch, 2001). We still lack a full understanding of human risk-related behavior, even in the limited domain of information systems (Kott & Arnold, 2013; Sasse, Brostoff, & Weirich, 2001).

An important question in this context is whether coping with risks is a single behavior or a combination of several behaviors. The answer to this question has important implications. If there is a single, measurable, risk-taking tendency, one may be able to use it in employee selection, training, and evaluation. Potentially, it can serve as the basis for adjusting system properties to individual users’ risk-taking tendencies. In contrast, if risk taking results from a combination of behaviors, no single measure will suffice, and no single intervention will be effective. Instead, we need to understand the different behaviors and their interdependencies to assess and possibly change users’ risk-related behaviors.

We present here a model for users’ coping with risk, named the triad of risk-related behaviors (TriRB). As seen in Figure 1, TriRB identifies three risk-related behaviors: (a) exposure to risk, (b) use of security features, and (c) responses to security indications, such as alerts and other information. The model assumes that the user’s interaction with the system expresses the choice of a particular combination of the three behaviors. The choice depends on normative factors (such as the likelihood and severity of threats or the effectiveness of a security mechanism), task factors (such as characteristics of the user’s activity and the momentary mental workload), and user characteristics (such as personality, general knowledge, and prior experience with the system) (Meyer, 2004). Although we developed the model in the context of cybersecurity, it is also applicable to risk-related behaviors in other dynamic environments (e.g., driving safety or health care).

Figure 1.

The triad of risk-related behaviors.

The Three Behaviors

Exposure to risk is the extent to which a person exposes herself/himself to the possibility of experiencing undesired outcomes. It can be intentional and result from a deliberate action or inaction, or it can be unintentional. Examples of behaviors that determine the exposure to cyber risk include the tendency to access sensitive services (e.g., cloud services and online banking) using open public Wi-Fi networks, the frequency of data backups, and the installation of software from unreliable sources. Users often have some control over their exposure to risk, but unintentional risk exposure is also common, possibly due to a lack of attention, lack of knowledge, and limited understanding of the complex implications of actions (see Olmstead & Smith, 2017, for a report on users’ limited knowledge about cybersecurity).

The extent to which users expose themselves to risk can depend on situational determinants, such as workload, fatigue, and arousal (Albrechtsen, 2007; Ng, Kankanhalli, & Xu, 2009). Individual characteristics (e.g., sensation seeking) also can affect the exposure to risk, and some individuals seek certain kinds of risk that others try to avoid (Bromiley & Curley, 1992). These individual differences affect the use of computers (Vance, Anderson, Kirwan, & Eargle, 2014) and the likelihood that computers will actually be infected (Herrero, Urueña, Torres, & Hidalgo, 2016).

Use of security features is the extent to which a person installs security features and the settings the person chooses for these features. Awareness of cyber risks can lead to the more extensive use of tools such as antivirus and firewalls. It also can lead to using more strict security settings for the Web browser. Some systems and applications include security features and default settings. Nevertheless, users often need to activate and configure security features or install dedicated security software (Ho, Dearman, & Truong, 2010; Kainda, Flechais, & Roscoe, 2010).

The most widely used security features are monitoring and alerting mechanisms. Users often can change the alerting rule or threshold depending on factors related to the task they are involved in or the environment in which they operate (Botzer, Meyer, Bak, & Parmet, 2010). Users may be willing to accept high false alert rates when they perceive the expected damage from missing detections exceeds the cost of unnecessary protective actions following false alerts. In contrast, if alerts disturb the user’s workflow, and the perceived risk of experiencing an attack is low, the user may prefer a low rate of alerts, even if this lowers the probability of detecting a threat (Buchanan, D′Amico, & Kirkpatrick, 2016; Schechter, Dhamija, Ozment, & Fischer, 2007). Previous studies on user adjustments of alerting thresholds show that users are sensitive to the quality of the alerting system but that they still tend to set nonoptimal alerting thresholds. For instance, users avoid extreme threshold values even if these are optimal (Botzer et al., 2010). Also, an analysis of the available information on which users can base their adjustment of alerting thresholds shows that users often do not have sufficient information for selecting the correct system settings (Meyer & Sheridan, 2017).

Response to indications is the degree to which a person responds to information from the security system. More specifically, response to indications refers to the tendency to engage in a security-related activity when receiving an indication from a security system about a possible risk. Examples include the user’s decision about whether to access a website, the user’s following alerts regarding software that might compromise the user’s privacy or damage the computer, and the users’ actions following alerts indicating the need to install patches or updates. Much research dealt with the design and evaluation of different security-related communications (e.g., Cranor, 2008; Laughery, 2006; Schechter et al., 2007; Wogalter, 2006). In the context of cybersecurity, Egelman, Cranor, and Hong (2008) identified design flaws in Web browsers’ phishing alerts and proposed changes to create more effective phishing alerts. Bravo-Lillo, Cranor, Downs, and Komanduri (2011) analyzed users’ mental models of computer security warnings and the implications these models have on users’ risk-taking behavior. Sunshine, Egelman, Almuhimedi, Atri, and Cranor (2009) showed how modified phishing alerts facilitated a more secure response to the alerts. However, there is evidence that users tend to ignore many alerts related to cybersecurity (Akhawe & Felt, 2013; Bahr & Ford, 2011), a decision that may be considered rational given the possible costs of security actions (Herley, 2009). Modic and Anderson (2014) address steps one can take to raise users’ tendency to respond to alerts and to lower the tendency to turn alerts off when possible.

A system with high detection accuracy elicits greater trust in the system’s outputs, and appropriate responses to its alerts become more likely (Cranor, 2008; Maltz & Meyer, 2001). Low-accuracy alerts, with frequent false alerts or missed events that interfere with the user’s main task, may require cognitive resources and can have a negative effect on a user’s attitude toward security (Sasse et al., 2001). The user may perceive such a system as untrustworthy and annoying. This can lead to ignoring the alerts, habituating responses, and eventually ceasing to use the system (Bliss, Gilson, & Deaton, 1995; Sunshine et al., 2009). On the other hand, users can develop very high levels of trust that will lead them to rely entirely on the alerts, a phenomenon known as “automation bias” and “complacency” (Mosier & Skitka, 1996; Parasuraman, 2000; Parasuraman & Manzey, 2010).

Influencing Factors

As indicated in the model (see Figure 1), normative, task, and user factors (Meyer, 2004) can influence each of the three behaviors as well as the relations between them. Normative factors include the probability and severity of threats, the availability of information for identifying risks, and the diagnostic properties of the alerting system. Task factors are related to the primary task the user is currently involved in and include the characteristics of the user interface, the sensitivity of the task, its urgency, the workload the user experiences, and the organizational security culture (Albrechtsen, 2007; Kainda et al., 2010; Workman, Bommer, & Straub, 2008).

User factors include the user’s knowledge about and experience with the specific system as well as cybersecurity expertise (Ben-Asher & Gonzalez, 2015). User factors also include personality characteristics such as risk attitudes, locus of control, and self-efficacy. Other user factors are the perceived susceptibility to threats and the perceived severity of the consequences if a threat is realized (Cranor, 2008; Meyer, 2004; Ng, Kankanhalli, & Xu, 2009; Workman et al., 2008).

Overall Risk Taking and the Triad of Behaviors

Figure 1 depicts the relations between the three behaviors in TriRB. Behaviors are connected, but each behavior can change independently due to exogenous environmental or user-related factors. The overall level of risk may not change much, because greater risk taking in one behavior may be accompanied by more caution in other behaviors, eliminating the effect on the overall level of risk.

For example, intensive use of security features, such as setting a high security level for a system, can raise the frequency of alerts and security-related communications or even directly affect the usability of the system (Möller, Ben-Asher, Engelbrecht, Englert, & Meyer, 2011). This, in turn, can alter the user’s response to these indications. Depending on the reliability of the system, if most alerts are false, the user may ignore the alerts or may cease using the system entirely. Similarly, if the user engages in behavior that exposes the system to threats, the frequency of alerts is likely to increase (Meyer & Bitan, 2002). By paying attention to the alerts and responding to them appropriately, the user can maintain an acceptable level of risk, even when threats become more likely.

TriRB can be considered a three-dimensional space (see Figure 2). For each behavior, the values can range on a continuum from cautious to risky. Users position themselves at some point in this space, and their position determines their overall level of risk. If they are at the cautious end of all three dimensions (i.e., α in Figure 2), they act very cautiously. Similarly, they can be at the risky end of all three dimensions (i.e., β in Figure 2), exposing themselves to high risks. There also are many intermediate positions. For instance, one user may choose a relatively high level of exposure to risk but can compensate the high exposure by using security features and responding cautiously to indications about risks (i.e., γ in Figure 2). Another user may ignore indications but may be relatively safe by choosing only very limited exposure to risks. The positions in the three-dimensional space are not static but rather can change dynamically in response to changes in task requirements and the environment. These dynamic changes can correspond with predictions from the theory of risk homeostasis (Wilde, 1982), where users tend to adjust the level of risk dynamically to some comfortable level.

Figure 2.

The triad of risk-related behaviors as a three-dimensional space with examples for extremely cautious (α), extremely risky (β), and intermediate (γ) risk-taking behaviors.

We conducted two experiments to assess the independence of the three behaviors and to evaluate the relations between the TriRB components. In Experiment 1, we looked at the effects of the severity of the damage from an attack and the costs of performing protective actions (protective actions costs, or PA costs) on the three behaviors. Experiment 2 evaluated the effects of the reliability of the alerting system and PA cost on the three behaviors.

The Experimental System

We developed a research platform to study the three behaviors in TriRB (see Ben-Asher, Meyer, Parmet, Moeller, & Engler, 2010; Möller et al., 2011, for earlier descriptions of the system). The experimental system is a variant of the Tetris game in which players steer descending objects, each consisting of four squares, to positions on the screen, rotating the objects and moving them laterally. We chose the Tetris game because it is a simple, popular game that requires little prior knowledge. The interaction with the game-like system resembles normal, prolonged, and enjoyable computer use.

Unlike the original Tetris game, in our system (see Figure 3), completed rows did not automatically disappear. Instead, the player could press a “Clear Rows” button to remove the completed rows. Also, in our game a “virus” could attack and randomly delete some of the squares on the screen that had not been cleared by pressing the “Clear Rows” button, turning completed rows to incomplete ones and thereby diminishing the player’s gains. The player could initiate a protective action at any time by clicking on the “Clear Rows” button and thereby save unprotected gains. However, the clear rows action took some time, during which the game stopped, leaving the player with less time to play and accumulate gains. Thus, there was an inverse relation between protective actions and productivity.

Figure 3.

Screen capture of the experimental system.

A security system provided indications about possible attacks, and the player decided whether and how to react to these indications. Based on signal detection theory (e.g., Green & Swets, 1966; Macmillan & Creelman, 2005), with attacks designated as signals, the experimenter set the reliability (d′) of the security system. Players could adjust the setting of the security system by pressing the “Change Security Level” button and choosing one of seven possible security levels, ranging from “Very Low” to “Very High” security, thereby determining the rate of true positive (TP) and false positive (FP) alerts. Higher security levels raised the number of both correct and incorrect alerts and lowered the number of missed detections.

This security system is a passive system that provides only information rather than an active system that blocks detected threats (Yue & Çakanyıldırım, 2010). As such, the system issued alerts 10 s before attacks occurred, allowing the player to respond to the alert with a protective action (i.e., press the “Clear Rows” button) or ignore it. Normatively, this decision should be based on PA cost, which is how long the game stopped when rows were saved; the selected security level; the player’s previous experience with the security system; and the possible damage from an attack (the amount of unprotected gains). We paid players according to the number of saved squares, so they had an incentive to maximize gains and to minimize losses.

Experiment 1: PA Cost and Damage From an Attack

In the first experiment, we examined the effects of PA cost and the severity of the damage from an attack on the three behaviors in the triad. The two variables should affect risk taking and the tendency to carry out protective actions in opposite directions. The more costly the protective actions are, the less they should be performed, and the more severe the damage caused by an attack, the more cautious players should be, carrying out protective actions more frequently. When users receive alerts regarding possible threats, they should assess the possible damage, depending on the current state (e.g., how many rows are on the screen) and the cost of prevention.

We predict that higher PA costs will lead to a decrease in the frequency of protective actions. Thus, participants who face higher PA costs will accumulate more completed rows before saving them and will be less likely to save completed rows following an alert, compared to participants who use a security system with lower PA costs. Also, higher PA costs may lower the tendency to initiate protective actions without alerts and increase the tendency to carry out protective actions following alerts.

Method

Participants

We recruited 40 students (mean age 25 years, SD = 1.90; 15 females) from the university participant pool. Their payment depended on the number of completed rows (i.e., fully filled with squares) they managed to save, with a 0.5 Israeli Shekels (about $0.12) payment for each saved row.

Design and procedure

In this experiment, we aimed to assess the separate and combined effects of the damage attacks can cause and the PA cost on participants’ behavior. The damage caused by an attack could be high or low, with an attack deleting either 20% or 5% of the squares on the display. The two levels of the PA cost were high (a delay of 22 s) and low (a delay of 7 s). Thus, the experimental design was a 2 × 2 design with four conditions.

The experiment began with a 3-min session without alerts or attacks to familiarize players with the game. It was followed by three 20-min experimental sessions, held on three days. The 20 min consisted of sixty 20-s intervals. In each interval, the system determined randomly whether an attack occurred with .1 probability for an attack (i.e., an attack occurred on average six times during an experimental session). The system also determined whether to issue an alert given the occurrence (or nonoccurrence) of an attack, based on the probabilities of TP and FP alerts. These depended on the selected setting of the security level, which was controlled by the participant, and on the reliability of the alerting system (set to d′ = 2) (see Table 1).

Table 1:

Probability of True Positive (p[TP]) and Probability of False Positive (p[FP]) for the Seven Security Levels (Reliability = 2)

	Security Level
	1	2	3	4	5	6	7
p(TP)	.31	.50	.69	.84	.93	.98	.99
p(FP)	.01	.02	.07	.15	.31	.50	.69

Data collection

We conducted the experiment in a computer lab with standard PC configurations, 19-inch monitors (screen resolution 1280 × 1024), and connections to a local server for data collection. The experimental platform recorded all system events, including (a) initial security level, (b) progress in accumulating rows, (c) occurrence of alerts, (d) occurrence of attacks, (e) clearing rows, and (f) changes in security setting.

Results

Use of security features

We analyzed the frequency of changes in the security settings, using a three-way analysis of variance (ANOVA) with session, damage severity, and PA cost as independent variables. The frequency of changes in the settings of the security system significantly decreased over the course of the experiment, F(2, 72) = 7.56, p = .001, η_p² = .17. The comparisons showed that in the first session, participants changed the setting significantly more frequently (M = 1.6, SD = .81) than in the second (M = 1.3, SD = .52) and third (M = 1.1, SD = .30) sessions, t(78) = 1.975, p = .052, d = .44, and t(78) = 3.655, p < .001, d = .82, for the comparisons between Sessions 2 and 3, respectively. Thus, participants adopted a stable security level after they gained experience with the task.

We computed a weighted security level (WS) to analyze the use of security features. This measure is the sum of the products of the security levels (S_i) and the time a participant spent in the level during the session (D_i) over all seven security levels, as defined in Equation 1:

W S = (\sum_{i = 1}^{7} S_{i} * D_{i}) / (\sum_{i = 1}^{7} D_{i})

WS could have values between 1 and 7. If a participant spent the entire session (20 min) in Security Level 7, then for this participant, WS = 7. On the other hand, if a participant spent half of the time in the highest security level (i.e., 7) and the rest of the time in the lowest security level (i.e., 1), then for this participant, WS = ([7 × 10] + [1 × 10]) / (10 + 10) = 4.

We analyzed the security system settings with a three-way ANOVA with damage severity and PA cost as independent variables and session as a within-subject variable. Participants in the high-damage condition set significantly higher security levels (M = 4.43, SD = 1.08), compared to participants in the low-damage condition (M = 3.63, SD = 1.24), F(1, 36) = 13.76, p < .001, η_p² = .38. Also significant was the two-way interaction between the damage condition and PA cost, F(1, 36) = 5.11, p = .026, η_p² = .26. As Figure 4 illustrates, when PA cost was high, participants chose higher security levels when the damage from an attack was more severe, t(58) = 4.23, p < .001, d = 1.09. When PA cost was low, the damage had no significant effect on the security settings, t(58) = 1.067, p = ns.

Figure 4.

Weighted security levels set by participants as a function of protective actions costs and the damage from attacks.

Exposure to risk

We measure the exposure to risk participants chose in the Tetris game through the number of rows they accumulated on the screen before taking a protective action. The more rows they had on the screen, the larger was the potential loss from an attack (but also the larger the gain, if they saved the rows exactly before an attack occurred). It is often challenging to measure acceptable risk directly. Participants will clear rows when they reach their individual limit of exposure to risk. They may also clear rows when they receive an alert before they reach their individual limit and the number of rows exceeds the participant’s limit with an alert (which will be lower than the limit without an alert).

We used a number of measures to assess participants’ chosen exposure to risk, including the number of rows lost due to attacks, the number of rows saved, and the mean number of rows on the screen when a protective action was taken following an alert or without an alert.

Participants in the high-damage condition lost significantly more rows (M = 7.42, SD = 5.73) than participants in the low-damage condition (M = 4.5, SD = 3.77), F(1, 36) = 7.98, p < .001, η_p² = .18. The number of completed rows participants managed to save increased from the first session (M = 32.51, SD = 8.79) to the second (M = 38.250, SD = 9.63) and third (M = 39.89, SD = 9.18) sessions, F(2, 72) = 24.44, p < .001, η_p² = .40. The difference between the last two sessions was not significant. The two-way interaction Session × Damage was significant, as shown in Figure 5, F(2, 72) = 4.66, p = .015, η_p² = .11. In the first session, participants in the high-damage condition saved significantly fewer completed rows compared to participants in the low-damage condition. From the second session on, participants in both damage conditions accumulated similar gains. Thus, participants in the high-damage condition adopted a strategy that helped them compensate for the greater damage from attacks.

Figure 5.

Average gains in the three sessions for the high- and low-damage conditions.

The damage from an attack affected gains more than losses. To understand this difference, we conducted an ANOVA examining factors influencing the number of clear row actions. Participants in the high PA cost condition cleared rows significantly less often (M = 12.42, SD = 2.55) compared to participants in the low PA cost condition (M = 22.37, SD = 9.57), F(1, 36) = 22.98, p < .001, η_p² = .39. Figure 6 illustrates how the PA cost influenced the number of protective actions and gains across sessions. When the PA cost was high, the variability between participants diminished on both variables, leading to a much denser cluster compared to when the PA cost was low.

Figure 6.

Average gains in a session as a function of the number of clear rows actions for high and low protective actions costs.

We measured the acceptable exposure to risk with and without an alert through the number of rows a participant accumulated before performing a clear rows action. The number of rows participants saved with and without an alert served as estimates for the upper limits for the two types of risk exposure. These limits could have changed dynamically during the experiment, but we assume that they stabilized with experience. We analyzed the acceptable exposure to risk, as expressed by the number of completed rows participants accumulated before performing a protective action, using an ANOVA with PA cost, damage severity, session number, and existence of an alert as predictor variables.

The exposure to risk increased over time with means of 3.75 (SD = 1.88), 4.56 (SD = 2.28), and 5.39 (SD = 2.42) for the three sessions, respectively, F(2, 72) = 22.58, p < .001, η_p² = .28, with t(158) = 2.44, p = .016, d = 0.39, and t(158) = 2.23, p = .027, d = 0.35, for the comparisons between Sessions 1 and 2 and Sessions 2 and 3. Receiving an alert lowered the exposure to risk significantly (M = 4.00, SD = 2.32), compared to exposure to risk without an alert (M = 5.13, SD = 2.13), F(1, 36) = 21.46, p < .001, η_p² = .37. There was significantly more exposure to risk when the PA cost was high (M = 5.366, SD = 2.39) than when it was low (M = 3.770, SD = 1.89), F(1, 36) = 13.82, p < .001, η_p² = .28. The two-way interaction between the presence of an alert and the PA cost was also significant, F(1, 36) = 5.06, p = .037, η_p² = .12 (see Figure 7). Post hoc t tests showed that in the low PA cost condition, there was no significant difference in the exposure to risk with and without an alert, whereas in the high PA cost condition, exposure to risk was significantly greater without an alert, t(38) = 1.18, p = ns, and t(38) = 3.32, p = .008, d = 0.75, respectively. With an alert, there was no significant difference between the PA cost conditions, t(38) = 1.915, p = ns. Without an alert, the exposure to risk was significantly greater when the PA cost was high, t(38) = 4.76, p < .001, d = 1.16.

Figure 7.

Exposure of risk for participants in the high and low protective actions cost conditions with and without an alert.

Response to alerts

Overall, participants experienced 1,416 alerts and ignored 64% of them. Participants had a window of opportunity to save their unprotected gains following an alert and before possibly experiencing an attack. We estimated the odds ratio of carrying out a protective action (1) or ignoring the alert (0), using a mixed-effect logistic regression model with participant-specific random effects to capture the between-participants variability. Table 2 summarizes the odds ratios and the 95% confidence intervals, derived from the model.

Table 2:

Estimated Odds Ratio and Confidence Interval for the Likelihood of Carrying out a Protective Action Following an Alert

	Odds Ratio	Confidence Interval (95%)
Session: 2 vs. 1	0.779	0.556–1.092
Session: 3 vs. 1	0.522**	0.366–0.744
Damage: high vs. low	1.114	0.551–2.249
Protective actions cost: low vs. high	3.505**	1.735–7.079
Security level	0.779*	0.660–0.920
Number of exposed rows	1.880**	1.725–2.049

p < .01. **p < .001.

The PA cost had the strongest effect on the probability of responding to an alert. Low PA cost increased the odds of a protective action by a factor of 3.5. The number of exposed rows also contributed to the increased odds of a protective action following an alert. Each completed row increased the odds of a protective action by a factor of 1.9. In contrast, the transition between the first and the last session was associated with a decrease in the odds of a protective action by a factor of 0.52. Similarly, each increase in the security level decreased the odds of a protective action by a factor of 0.78.

Conclusions

The results show that the three behaviors in TriRB differ in their sensitivity to changes in PA cost and damage severity, suggesting that cyber risk taking indeed results from the combination of different behaviors. For example, PA cost had no significant effect on the setting of the security level, but it did affect the exposure to risk, and it affected the tendency to respond to alerts, regardless of the damage severity. Damage severity affected the mean security level chosen (with higher security levels when damage was more severe) and the exposure to risk only when PA cost was high.

The interaction with the system changed as participants gained experience. The changes in the interaction patterns are evidence for a complex learning process. Participants chose a level of security that depended on the damage caused by attacks. More severe consequences of an attack led to the use of higher security levels compared to situations when the severity of the attack was lower. With experience, the responses to alerts decreased when PA cost was high and the interaction became overall more risky. No similar trend was observed when PA cost was low.

Our results indicate that users’ willingness to take precautionary actions depends on the perceived benefits from these actions. Prospect theory (Kahneman & Tversky, 1979) can explain why the costs of security outweigh the possible cost of risky behavior. The PA costs are certain and given, whereas the costs from risky behavior (i.e., the damage from an attack) are probabilistic. Kahneman and Tversky (1979) and later research show a general tendency toward risk-seeking behavior in the loss domain, where people prefer an option in which they may incur a larger loss with some probability over a certain smaller loss.

Experiment 2: The Reliability of the Alerting System and PA Costs

The perceived reliability of alerts can affect users’ trust as expressed by responses to alerts. Users are more likely to trust reliable systems and to respond to the alerts they produce (e.g., Maltz & Meyer, 2001). Similarly, frequent false alerts may cause users to ignore the alerts and perhaps to decide not to use the system at all (Bliss et al., 1995; Cranor, 2008; Meyer, 2004; Sunshine et al., 2009).

The reliability of security systems depends on technological developments and the settings of the security system (e.g., the set of rules used by an Intrusion Detection System), and it may depend on the type of threat and the attacker’s method of operation (Egelman et al., 2008). Experiment 1 examined TriRB in a setting with a relatively reliable and sensitive security system (d′ = 2). Experiment 2 examined the effects of alerting reliability on TriRB with a specific interest in the combined effects of alerting reliability and PA costs.

Meyer (2004) identified two types of responses to alerts—compliance and reliance. Compliance is the tendency to perform a preventive action following an alert, and reliance is the tendency to refrain from performing a protective action without an alert. The two responses are two different behaviors, though not entirely independent of each other, and a user can develop reliance without compliance or show compliance without reliance (Dixon, Wickens, & McCarley, 2007; Meyer & Lee, 2013; Meyer, Wiczorek, & Günzler, 2014; Vashitz et al., 2009). To assess whether the exposure to risk with and without an alert shows indications of reliance (exposure to risk without an alert will be larger with the more reliable system), compliance (exposure to risk following an alert will be smaller for the more reliable system), or both, we used two levels of system reliability in the following experiment.

Based on the theoretical background in behavioral decision making and findings from the previous experiment, we predicted that changing the reliability of the security system will lead to differences in the security level users choose and in their responses to security indicators. With a reliable security system, users will use relatively high security levels and will tend to behave securely after receiving an alert. Furthermore, we predicted that similar to the previous experiment, raising the PA costs will lower users’ tendency to take protective actions and increase their exposure to risk. Such findings will substantiate the claim that the behaviors in TriRB respond differentially to a wide range of influencing factors.

Method

Participants

Participants were 40 students (20 female) between 19 and 29 years of age (M = 24.6, SD = 1.92). They also received 0.5 Israeli Shekels (about $0.12) for each completed row they saved.

Design and procedure

As in Experiment 1, participants completed a 3-min training session and then did three 20-min experimental sessions on three days. The experimental design was a 2 × 2 between-groups design with the PA cost (low = 7-s delay, high = 22-s delay) and the reliability of the alerting system (d′ = 3 for the high-reliability system and d′ = 1 for the low-reliability system) as independent variables. Table 3 shows the probabilities for TP and FP alerts in each security level for the two reliability levels. We randomly assigned participants to one of the four experimental conditions. The probability of an attack in a 20-s time interval was always 0.1, as in the first experiment.

Table 3:

Probability of True Positive (p[TP]) and Probability of False Positive (p[FP]) for the Seven Security Levels for High-Reliability (d′ = 3) and Low-Reliability (d′ = 1) Conditions

		Security Level
		1	2	3	4	5	6	7
d′ = 1	p(TP)	.006	.068	.312	.691	.933	.994	.999
d′ = 1	p(FP)	.001	.006	.068	.309	.691	.933	.994
d′ = 3	p(TP)	.691	.799	.879	.933	.967	.985	.994
	p(FP)	.006	.015	.034	.067	.122	.202	.309

Note. d′ = reliability.

Results

Use of security features

Participants changed the security level more often in the first session (M = 1.43, SD = .67) than in the second (M = 1.15, SD = .43) and third (M = 1.1, SD = .30) sessions, t(78) = 2.18, p = .033, d = 0.49, and t(78) = 2.78, p < .001, d = 0.62, for comparisons between the first session and the other two. There was no significant difference between the second and third sessions.

We analyzed the WS participants chose with a three-way ANOVA with the session as a within-subject variable and the system reliability and PA cost as between-subjects variables. The two-way interaction Session × Reliability was significant, F(2, 72) = 4.02, p = .022, η_p² = .03. As shown in Figure 8, participants in the high-reliability condition maintained a relatively stable level of security throughout the three sessions. In contrast, participants in the low-reliability condition chose lower security levels as the experiment progressed. A post hoc analysis indicated that in the third session, participants in the low-reliability condition used significantly lower levels of security compared to participants in the high-reliability condition, t(38) = 2.57, p = .014, d = 0.81.

Figure 8.

Average weighted security level in the three sessions as a function of the reliability of the security system.

Exposure to risk

The ANOVA of the number of completed rows participants lost due to attacks in each condition and session showed that, on average, participants in the high-reliability condition experienced significantly fewer losses (M = 3.77, SD = 2.88) compared to participants in the low-reliability condition (M = 6.18, SD = 4.36), F(1, 36) = 11.08, p = .002, η_p² = 0.24. The number of completed rows participants saved increased significantly over time, F(2, 72) = 40.09, p < .001, η_p² = .53, with means of 31.15 (SD = 10.08), 36.86 (SD = 10.66), and 41.75 (SD = 11.61) for the three sessions, respectively. Although the reliability of the security system influenced losses, gains were sensitive to the PA cost. Participants in the low PA cost condition saved significantly more completed rows (M = 42.125, SD = 11.74) compared to participants in the high PA cost condition (M = 31.050, SD = 8.33), F(1, 36) = 18.56, p < .001, η_p² = .34. Also, on average, participants in the low PA cost condition carried out more protective actions (M = 23, SD = 8.93) compared to participants in the high PA cost condition (M = 13.7, SD = 3.13), F(1, 36) = 27.76, p < .001, η_p² = .44. Figure 9 illustrates how the PA cost influenced both the number of protective actions and the gains across session. When the PA cost was high, the variance between participants diminished, expressed in a denser cluster, compared to the greater variability in the gains and in the number of protective actions when the PA cost was low.

Figure 9.

Average gains in a session as a function of the number of clear rows actions for high and low protective actions costs.

We analyzed the acceptable exposure to risk, as expressed by the number of completed rows participants accumulated before performing a protective action, using an ANOVA with PA cost, reliability of the security system, session, and existence of an alert as independent variables. Overall exposure to risk increased over time with means of 3.12 (SD = 1.40), 3.75 (SD = 2.12), and 4.60 (SD = 2.74) rows, for the three sessions, respectively, F(2, 72) = 16.75, p < .001, η_p² = .32. The exposure to risk was lower after an alert (M = 3.02, SD = 2.22) than without an alert (M = 4.63, SD = 1.94), F(1, 36) = 61.13, p < .001, η_p² = .63. The three-way interaction PA Cost × Session × Alert was significant, F(2, 72) = 4.47, p = .015, η_p² = .11. As illustrated in Figure 10, the high and low PA cost conditions generated two distinct patterns of exposure of risk that evolved during the three sessions. When the PA cost was high, participants gradually increased their exposure to risk without an alert while maintaining a relatively stable exposure to risk in the presence of an alert. Therefore, participants in the high PA cost condition increased the difference between the exposure to risk with and without an alert as the experiment progressed. In contrast, in the low PA cost condition, participants maintained a constant difference between the risk exposure with and without an alert, raising both similarly over time.

Figure 10.

Exposure to risk as a function of experimental session and the existence of an alert for participant in the high (a) and low (b) protective actions cost conditions.

Response to alerts

Overall, participants received 819 alerts and ignored 55% of them. We analyzed the responses to alerts with a logistic regression model that estimated the odds ratio of carrying out a protective action (1) or ignoring the alert (0). Table 4 summarizes odds ratios and confidence intervals derived from the model. High reliability of the security system, compared to low reliability, raised the odds of a protective action following an alert by a factor of 1.5. Low PA cost also raised the odds of a protective action, compared to high PA cost, by a factor of 1.4. Similarly, each increase in the number of exposed rows increased the odds of a protective action by a factor of 1.72. In contrast, the transition between the first and the last session was associated with a decrease in the odds of a protective action by a factor of 0.65, and each increase in the security level decreased the odds of a protective action by a factor of 0.79.

Table 4:

Estimated Odds Ratio and Confidence Interval for the Likelihood of Carrying out a Protective Action Following an Alert

	Odds Ratio	Confidence Interval
Session: 2 vs. 1	1.062	0.776–1.451
Session: 3 vs. 1	0.654*	0.460–0.924
Reliability: low vs. high	1.506**	1.139–1.991
Protective actions cost: high vs. low	1.408*	1.075–1.844
Security level	0.794**	0.680–0.924
Number of exposed rows	1.717**	1.576–1.878

p < .01. **p < .001.

Conclusions

This experiment shows how components of TriRB responded to variations in the reliability of the security system and the PA costs. The two reliability levels elicited significant differences in the use of security features, which became more evident as participants gained experience in the game. By the third session, the differences in the settings of the security system were significant. However, when examining the actual FP rates each security level yielded, we find that participants’ preferred setting in both reliability conditions led to very similar rates of .067 and .068 for the high- and low-reliability conditions, respectively. In contrast to the similar FP rates, the participants’ preferred settings yielded large differences for the TP rates, with rates of .933 and .312 for the high- and low-reliability conditions. Overall, it seems that with an unreliable system, some users abandoned it (i.e., used very low security levels). However, other users still considered alerts from an unreliable system valuable and used relatively high security settings. A reliable security system provided better protection compared to an unreliable system, as is evident from the significant differences in losses. On average, with a reliable security system, participants experienced smaller losses compared to participants who used an unreliable security system, regardless of the PA costs. However, the PA cost, rather than the reliability, had a significant effect on the gains participants accumulated during the interaction. This implies that the overall performance in a task was more sensitive to the costs of protective actions and that users developed interaction patterns that compensated for the need to interact with an unreliable security system. The notion is supported by the finding that exposure to risk was sensitive to PA cost, and combining high PA cost with an alert moderated the tendency to increase risk exposure as the experiment progressed. Furthermore, both high reliability and low PA cost raised the likelihood of complying with an alert. Reliability did not influence compliance and reliance. The selected level of security possibly mediated the influence of reliability, and participants’ risk exposure was more sensitive to the presence of an alert. As suggested by TriRB, it is possible that even within each experimental condition, there are multiple strategies for coping with security risks. For example, 1 participant actually used the lowest security level during all sessions and had an interaction pattern that depended only on the amount of unprotected gains without paying attention to alerts. This is one among a wide variety of possible strategies participants could use to maintain a desired level of risk taking.

Discussion

We demonstrated the existence of three risk-related behaviors, named the triad of risk-related behaviors. The three behaviors are not simply different manifestations of a general risk-taking tendency but rather different behaviors that are to some extent related to each other. When integrating the three dimensions, the model can provide a holistic view on cyber-risk-taking behavior. When examining each dimension separately, the model can inform about preferences and attitudes toward specific aspects of cyber risk, providing metrics for the quantitative evaluation of users’ risk taking. The user’s choice in one behavior (e.g., the setting of the security level) will affect the possible choices in the other behaviors (e.g., the tendency to respond to alerts). As was shown previously, a very cautious user who limits the exposure to risk will receive only a few correct alerts and will therefore have an alerting system that seems highly unreliable (Meyer & Bitan, 2002). Such users may not respond to alerts, because these are unlikely to indicate an actual problem. Thus, the evaluation of a user’s risk-related behavior needs to consider the complex interplay of the different activities and the properties of the environment in which a user acts. A user can behave more or less cautiously in numerous ways, and no single behavioral measure can reliably express a user’s risk taking in the cyber environment.

Although the three behaviors determine a user’s momentary level of risk, they actually occur at somewhat different points in time. Security features are usually set in advance, when first starting to use the system or occasionally during the use of the system. Such high-level, long-term interaction with security features may be seen as a strategic behavior. Indeed, in our study, participants did not extensively explore the different security levels, and they adjusted the security setting less as the experiment progressed.

The exposure to risk is a decision during an ongoing dynamic process. In our case, risk can increase over time, similar to the risk in the Balloon Analogue Risk Task (Lejuez et al., 2002). Users have to monitor the gradual increase in the riskiness of their situation and must decide whether it has become too risky to be comfortable so that a protective action is necessary. The ongoing monitoring of risk during a task resembles tactical risk management.

The responses to alerts are momentary reactions to discrete events that resemble the need to execute a maneuver in response to an emerging threat. Users decide whether to take a protective action after seeing an alert or whether to ignore it. This decision depends on the assessment of the likelihood that the alert indeed points to an impending attack and whether the current exposure to risk requires a protective action following an alert.

The results from the two experiments also inform us on the impact of normative and task-related factors on cyber-risk-taking behaviors. The cost of a protective action had a striking impact on risk-taking behaviors and was more influential than threat severity and alerting reliability. It is possible that when users balance usability and security, usability tends to be overweighed. Also, although reliability of an alerting system is often described by FP and false negative rates, in the context of cyber risk taking, it seems that users’ preferences are mainly influenced by the FP rate. Apparently there is less tolerance for the distractions false alerts create compared to their ability to indicate possible threats. Both exposure to risk and response to security indications were influenced by the amount of unprotected gains. Cyber assets (e.g., data, servers, networks) can be recovered if protected correctly, and users learn to moderate the amount of unprotected assets according to their preferences. These preferences are dynamic and evolve over time, showing a general trend of increase in risk taking. These findings, and the insights into cyber-risk-taking behaviors they provide, can guide the development of usable cybersecurity systems and can help in predicting users’ patterns of interaction.

In addition to the contributions our study makes to the understanding of risk, it demonstrates some of the methodological issues that arise when dealing with risk-related behaviors and the experimental environment required to study realistic issues that necessarily involve correlated variables. For instance, if users tend to limit their exposure to risk, they will usually have relatively few gains at stake when an alert is issued, which may lower the tendency to respond to alerts. Thus, we need hierarchic, fairly complex statistical models to analyze these situations. Isolating the different variables may create impoverished situations that fail to capture the complexities of users’ risk-related behaviors.

To conclude, we show that users’ risk-taking behavior in a cyber environment, as in other dynamic situations, expresses the combination of different but interrelated behaviors. We also show that although this topic is definitely complex, it can be subject to systematic, controlled research considering characteristics of the system, the environment, and the user. Such research should eventually lead to the generation of valid predictive models of user behavior that can serve to develop better systems and to decide on the optimal system settings.

Key Points

Risk taking in cybersecurity is complex and situation dependent.

We propose a triad of three risk-related behaviors (use of security features, exposure to risk, and response to indications) that affect users’ risk-taking behavior.

Using a micro-world environment, we demonstrate how properties of threats, the security system, and the situation affect risk-taking behavior.

Footnotes

Acknowledgements

This study is based on parts of the first author’s PhD dissertation at Ben-Gurion University of the Negev. The research was partly funded by Deutsche Telekom through T-Labs@BGU, was sponsored by the U.S. Army Research Laboratory, and was accomplished under Cooperative Agreement Number W911NF-16-2-0113. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Laboratory or the U.S. government. The U.S. government is authorized to reproduce and distribute reprints for government purposes notwithstanding any copyright notation herein.

Noam Ben-Asher is an Oak Ridge Associated Universities Senior Research Fellow at U.S. Army Research Laboratory, Adelphi, Maryland. Before this position, he was a postdoctoral fellow at the Dynamic Decision Making Laboratory at Carnegie Mellon University and at IBM Research. He received his PhD degree in human-factors engineering from Ben-Gurion University in 2011.

Joachim Meyer is professor in the Department of Industrial Engineering at Tel Aviv University, Israel; was on the faculty of Ben-Gurion University of the Negev, Israel; and held research positions at the Technion—Israel Institute of Technology and at the MIT AgeLab and the MIT MediaLab. He has an MA in psychology and a PhD in industrial engineering (1994) from Ben-Gurion University of the Negev in Beer Sheva, Israel.

References

Akhawe

Felt

A. P.

(2013). Alice in Warningland: A large-scale field study of browser security warning effectiveness. Paper presented at the USENIX Security Symposium 2013, Washington, DC. Retrieved from https://www.usenix.org/system/files/conference/usenixsecurity13/sec13-paper_akhawe.pdf

Albrechtsen

(2007). A qualitative study of users’ view on information security. Computers & Security, 26, 276–289.

Bahr

G. S.

Ford

R. A.

(2011). How and why pop-ups don’t work: Pop-up prompted eye movements, user affect and decision making. Computers in Human Behavior, 27(2), 776–783.

Ben-Asher

Gonzalez

(2015). Effects of cyber security knowledge on attack detection. Computers in Human Behavior, 48, 51–61.

Ben-Asher

Meyer

Parmet

Moeller

Engler

(2010). An experimental microworld for evaluating the tradeoffs between usability and security. Usable Security Experiment Reports (USER) Workshop in the Symposium on Usable Privacy and Security (SOUPS). New York, NY: ACM SIGCHI.

Bliss

Gilson

Deaton

(1995). Human probability matching behaviour in response to alarms of varying reliability. Ergonomics, 38, 2300–2312.

Botzer

Meyer

Bak

Parmet

(2010). User settings of cue thresholds for binary categorization decisions. Journal of Experimental Psychology: Applied, 16, 1–15.

Bravo-Lillo

Cranor

L. F.

Downs

Komanduri

(2011). Bridging the gap in computer security warnings: A mental model approach. IEEE Security & Privacy, 9(2), 18–26.

Bromiley

Curley

(1992). Individual differences in risk taking. In Yates

J. F.

(Ed.), Risk taking behavior (pp. 87–132). New York, NY: John Wiley.

10.

Buchanan

D′Amico

Kirkpatrick

(2016, October). Mixed method approach to identify analytic questions to be visualized for military cyber incident handlers. Paper presented at the 2016 IEEE Symposium on Visualization for Cyber Security (VizSec), Baltimore, MD.

11.

Canfield

C. I.

Fischoff

Davis

(2016). Quantifying phishing susceptibility for detection and behavior decisions. Human Factors, 58, 1158–1172.

12.

Cranor

(2008). A framework for reasoning about the human in the loop. In Proceedings of the 1st Conference on Usability, Psychology and Security (pp. 1–15). Berkeley, CA: USENIX Association.

13.

Dixon

S. R.

Wickens

C. D.

McCarley

J. S.

(2007). On the independence of compliance and reliance: Are automation false alarms worse than misses? Human Factors, 49, 564–572.

14.

Egelman

Cranor

Hong

(2008). You’ve been warned: An empirical study of the effectiveness of web browser phishing warnings. In Proceeding of the 26th Annual ACM SIGCHI Conference on Human Factors in Computing Systems—CHI 2008 (pp. 1065–1074). New York, NY: ACM.

15.

Green

Swets

(1966). Signal detection theory and psychophysics. New York, NY: Wiley.

16.

Herley

(2009). So long, and no thanks for the externalities: The rational rejection of security advice by users. In Proceedings of the 2009 ACM Workshop on New Security Paradigms (pp. 133–144). New York, NY: ACM SIGCHI.

17.

Herrero

Urueña

Torres

Hidalgo

(2016). My computer is infected: The role of users’ sensation seeking and domain-specific risk perceptions and risk attitudes on computer harm. Journal of Risk Research, 20, 1466–1479. doi: 10.1080/13669877.2016.1153504

18.

Dearman

Truong

(2010). Improving users’ security choices on home wireless networks. In Proceedings of the Sixth ACM Symposium on Usable Privacy and Security (pp. 1–12). New York, NY: ACM SIGCHI.

19.

Kahneman

Tversky

(1979). Prospect theory: An analysis of decision under risk. Econometrica: Journal of the Econometric Society, 47, 263–291.

20.

Kainda

Flechais

Roscoe

(2010, February). Security and usability: Analysis and evaluation. Paper presented at the 2010 International IEEE Conference on Availability, Reliability and Security, Krakow, Poland.

21.

Kott

Arnold

(2013). The promises and challenges of continuous monitoring and risk scoring. IEEE Security & Privacy, 11(1), 90–93.

22.

Lange

Kott

Ben-Asher

Mees

Baykal

Vidu

C. M.

. . .Madahar

(2017). Recommendations for model-driven paradigms for integrated approaches to cyber defense. arXiv preprint arXiv:1703.03306.

23.

Laughery

(2006). Safety communications: Warnings. Applied Ergonomics, 37, 467–478.

24.

Lejuez

C. W.

Read

J. P.

Kahler

C. W.

Richards

J. B.

Ramsey

S. E.

Stuart

G. L.

. . .Brown

R. A.

(2002). Evaluation of a behavioral measure of risk taking: The Balloon Analogue Risk Task (BART). Journal of Experimental Psychology: Applied, 8, 75–84.

25.

Loewenstein

Weber

Hsee

Welch

(2001). Risk as feelings. Psychological Bulletin, 127, 267–286.

26.

Macmillan

Creelman

(2005). Detection theory: A user’s guide. Mahwah, NJ: Lawrence Erlbaum.

27.

Maltz

Meyer

(2001). Use of warnings in an attentionally demanding detection task. Human Factors, 43, 217–226.

28.

Meyer

(2004). Conceptual issues in the study of dynamic hazard warnings. Human Factors, 46, 196–204.

29.

Meyer

Bitan

(2002). Why better operators receive worse warnings. Human Factors, 44, 343–354.

30.

Meyer

Lee

J. D.

(2013). Trust, reliance and compliance. In Lee

J. D.

Kirlik

(Eds.), The Oxford handbook of cognitive engineering (pp. 109–124). Oxford, UK: Oxford University Press.

31.

Meyer

Sheridan

T. B.

(2017). The intricacies of user adjustment of alerting thresholds. Human Factors, 59, 901–910.

32.

Meyer

Wiczorek

Günzler

(2014). Measures of reliance and compliance in aided visual scanning. Human Factors, 56, 840–849.

33.

Modic

Anderson

R. J.

(2014). Reading this may harm your computer: The psychology of malware warnings. Computers in Human Behavior, 41, 71–79.

34.

Möller

Ben-Asher

Engelbrecht

K.-P.

Englert

Meyer

(2011). Modeling the behavior of users who are confronted with security mechanisms. Computers & Security, 30, 242–256.

35.

Mosier

Skitka

(1996). Human decision makers and automated decision aids: Made for each other. In Parasuraman

Mouloua

(Eds.), Automation and human performance: Theory and applications (pp. 201–220). Mahwah, NJ: Lawrence Erlbaum.

36.

Kankanhalli

(2009). Studying users’ computer security behavior: A health belief perspective. Decision Support Systems, 46, 815–825.

37.

Olmstead

Smith

(2017). What the public knows about cybersecurity. Retrieved from http://www.pewinternet.org/2017/03/22/what-the-public-knows-about-cybersecurity/

38.

Parasuraman

(2000). Designing automation for human use: Empirical studies and quantitative models. Ergonomics, 43, 931–951.

39.

Parasuraman

Manzey

(2010). Complacency and bias in human use of automation: An attentional integration. Human Factors, 52, 381–410.

40.

Pfleeger

S. L.

Sasse

M. A.

Furnham

(2014). From weakest link to security hero: Transforming staff security behavior. Journal of Homeland Security and Emergency Management, 11(4), 489–510.

41.

Proctor

R. W.

Chen

(2015). The role of Human Factors/Ergonomics in the science of security: Decision making and action selection in cyberspace. Human Factors, 57, 721–727.

42.

Sasse

Brostoff

Weirich

(2001). Transforming the “weakest link”—A human/computer interaction approach to usable and effective security. BT Technology Journal, 19, 122–131.

43.

Schechter

Dhamija

Ozment

Fischer

(2007). The emperor’s new security indicators: An evaluation of website authentication and the effect of role playing on usability studies. In Proceedings of the 2007 IEEE Symposium on Security and Privacy (pp. 51–65). Piscataway, NJ: IEEE.

44.

Stobert

Biddle

(2014). The password life cycle: User behaviour in managing passwords. In Symposium on Usable Privacy and Security (SOUPS 2014). Retrieved from https://www.usenix.org/system/files/conference/soups2014/soups14-paper-stobert.pdf

45.

Sunshine

Egelman

Almuhimedi

Atri

Cranor

(2009). Crying wolf: An empirical study of SSL warning effectiveness. In Proceedings of the 18th Conference on USENIX Security Symposium (pp. 399–416). Berkeley, CA: USENIX Association.

46.

Vance

Anderson

B. B.

Kirwan

C. B.

Eargle

(2014). Using measures of risk perception to predict information security behavior: Insights from electroencephalography (EEG). Journal of the Association for Information Systems, 15, 679–722.

47.

Vashitz

Meyer

Parmet

Peleg

Goldfarb

Porath

Gilutz

(2009). Defining and measuring physicians’ responses to clinical reminders. Journal of Biomedical Informatics, 42, 317–326.

48.

Whitman

Mattord

(2011). Principles of information security. Boston, MA: Cengage Learning.

49.

Wilde

(1982). The theory of risk homeostasis: Implications for safety and health. Risk Analysis, 2, 209–225.

50.

Wogalter

M. S.

(2006). Handbook of warnings. Mahwah, NJ: Lawrence Erlbaum.

51.

Workman

Bommer

Straub

(2008). Security lapses and the omission of information security measures: A threat control model and empirical test. Computers in Human Behavior, 24, 2799–2816.

52.

Yue

W. T.

Çakanyıldırım

(2010). A cost-based analysis of intrusion detection system configuration under active or passive response. Decision Support Systems, 50, 21–31.