The Influence of Time Pressure and Persuasion Principles on Phishing Detection

Principles of Persuasion in Phishing Attacks

For a phishing attack to be successful, the recipient must comply with the phisher’s requests. Cialdini’s principles of persuasion are a collection of persuasive strategies that are used to bolster attempts at influencing others, often in marketing and sales contexts (Cialdini, 2009). They include the authority, reciprocity, commitment/consistency, scarcity, social proof, and liking principles. While often used with good intentions, persuasion strategies are also used for more nefarious purposes, like encouraging users to respond to phishing emails (Muscanell et al., 2014).

The authority principle proposes that people tend to obey authority figures, such as leaders and experts (Cialdini, 2009). This principle is seen in phishing emails that encourage users to respond by impersonating authority figures like government organizations, financial institutions, and organizational leadership (Muscanell et al., 2014, Wright et al., 2014).

The reciprocity principle suggests that people tend to feel a sense of obligation to repay others for what they have done for them (Cialdini, 2009). Thus, people are more likely to follow a request from those who have done something for them. In phishing, this often looks like prize giveaways, rewards for completing surveys, and Nigerian prince scams, which feature gifts in return for clicking a malicious link, providing personal information, or paying a fee (Muscanell et al., 2014; Lin et al., 2019).

The commitment/consistency principle states that once a person makes a commitment to something, they are more likely to act in ways that are consistent with that commitment (Cialdini, 2009). In phishing, this principle is demonstrated in emails that ask the recipient to follow-up on a previous interaction, whether it be with another individual (e.g., following up about a job application) or with a system (e.g., notification of account activity) (Muscanell et al., 2014; Taib et al., 2019).

The scarcity principle suggests that people are more likely to act when there is a sense of urgency to do so, such as when there is a rare opportunity or one that is only available for a limited time (Cialdini, 2009). In phishing, this principle displays as response deadlines and rare opportunities that encourage the user to respond before the opportunity is lost (Muscanell et al., 2014; Lawson et al., 2020).

The social proof principle is based on the tendency for people to look to others to decide the most appropriate behavior when faced with uncertainty (Cialdini, 2009). In phishing emails, this principle is expressed as emails addressed to entire organizations or groups of people (e.g., “Dear all”) or emails that tout the popularity of an action (e.g., “This offer is in high demand!”) (Muscanell et al., 2014, Wright et al., 2014).

Lastly, the liking principle proposes that people are more willing to comply with people who are likable, share similar interests and beliefs, or are attractive (Cialdini, 2009). In phishing, liking is typically expressed through extremely polite language and pretending to have similarities with the recipient (e.g., attending the same college, having mutual friends) (Muscanell et al., 2014; Lin et al., 2019).

Research on user abilities to detect phishing emails using these principles have found mixed results. For instance, in an email classification task, Lawson et al. (2020) found that users were generally liberal when responding to persuasive emails. Users correctly identified more phishing emails using the authority and scarcity principles, and fewer using liking and commitment/consistency. Similarly, Parsons et al. (2019) found that phishing emails using the commitment/consistency principle appeared safer to users, while those using the scarcity principle appeared the most dangerous. Research using naturalistic paradigms (e.g., simulated phishing emails sent to personal inboxes) also has mixed findings. Social proof (Lin et al., 2019), commitment/consistency (Lin et al., 2019), authority (Wright et al., 2014), and the use of no principles (Wright et al., 2014) have resulted in the lowest click-through rates; and authority (Lin et al., 2019), scarcity (Lin et al., 2019; Wright et al., 2014), liking (Lin et al., 2019; Wright et al., 2014), and social proof (Taib et al., 2019) have resulted in the highest click rates. Together, users appear to fall for attacks using liking and commitment/consistency the most, and those using the authority principle the least; meanwhile, results for the other principles are mixed.

One potential source of inconsistency is variation in methodologies. For instance, studies differ in how many emails participants respond to (Parsons et al., 2019; Lawson et al., 2020), whether the emails are found using online sources (e.g., databases, image searches) or created by researchers (Taib et al., 2019; Parsons et al., 2019), the specific traits that constitute each of the persuasion principles (Wright et al., 2014; Lin et al., 2019), and whether emails are being classified in a lab-controlled classification task or a naturalistic phishing simulation (Lawson et al., 2020; Taib et al., 2019). The present study aimed to clarify the effects of the persuasion principle by employing a task in which users classify emails that each contain only one of the six persuasion principles.

Time Pressure

Time pressure may also influence users’ abilities to detect phishing emails. While time pressure in the form of appeals to urgency are frequently researched (e.g., Lin et al., 2019; Wright et al., 2014), the impact of situational time pressure, such as a time limit for classifying all of the emails in an email task, is less studied. Research on situational time pressure and phishing detection has generally found that greater time pressure degrades detection abilities (Jones et al., 2019; Sarno & Neider, 2022; Butavicius et al., 2022). However, studies vary considerably in their time pressure manipulations. Specifically, the amount of emails classified or the amount of time allotted to classify an email (Sarno & Neider, 2022; Butavicius et al., 2022), the amount of time pressure participants are subjected to (Jones et al., 2019; Sarno & Neider, 2022; Butavicius et al., 2022), and at what level the time pressure is enforced (e.g., time limit for each trial vs the entire set) (Jones et al., 2019; Butavicius et al., 2022).

Whereas Jones et al. (2019) implemented time pressure through their instructions, Sarno & Neider (2022) manipulated the number of emails users classified under a common time limit, and Butavicius et al. (2022) limited the time each email was visible for; the present study aimed to examine how limits on the amount time to complete the entire classification task affects detection abilities. Participants may naturally pace themselves differently; allowing participants to control the length of time they spend on each email allows for more naturalistic observations of how users react to external time pressure. Additionally, no study has investigated interactions between time pressure and persuasive strategies in phishing emails. According to the Elaboration Likelihood Model (Cacioppo & Petty, 1984), persuasive information is processed differently depending on whether it is evaluated with the central route (e.g., evaluating pros and cons of complying with a request) or the peripheral route (e.g., quick, heuristic judgements about a request). Under time pressure, users may not have enough time to systematically process the persuasive appeals in emails, and as a result fall for phishing more easily.

The Present Study

The present study examined users’ abilities to classify emails as a function of time pressure (i.e., varying time limits to classify emails) and persuasive techniques used in emails. To expand on prior research on persuasive phishing emails, the present study investigated users’ abilities to distinguish phishing and legitimate emails, where each email embodied a single persuasion principle. Additionally, to further investigate the role of time pressure in phishing susceptibility, the present study employed a time pressure manipulation where users had varying deadlines to complete the classification task and a countdown timer to remind them of their remaining time throughout the task. Consistent with prior research, we expected that users would be worst at discriminating emails using the liking and commitment/consistency principles, and best at those using the authority principle. We also anticipated higher levels of time pressure to result in worse performance in the email classification task across all persuasion principles.

Participants

A total of 99 participants were recruited from an undergraduate psychology recruiting pool for this study and were compensated with course credit. The sample was 80.8% female and ranged in age from 18 to 25 (M = 18.64). To participate, participants had to be at least 18 years old and self-report normal/corrected vision and cognitive functioning.

Stimuli

The stimuli consisted of 60 static images of emails (50% legitimate, 50% phishing). The emails were collected from researchers’ inboxes, online datasets (e.g., university Phish Bowls), and image searches. The emails were selected based on the presence of persuasion principles (Muscanell et al., 2014). Five legitimate and five phishing emails were selected for each persuasion principle, and each email only included a single persuasion principle.

Procedure and Materials

After verbally providing informed consent, participants were escorted to a lab computer to complete an email classification task created in PsychoPy (Peirce et al., 2019) and a subsequent individual differences battery created in Qualtrics. In the email classification task, participants were instructed to classify a series of emails as “legitimate” or “not legitimate” via mouse click, and indicate how confident they were in each classification via a slider ranging from 1 (“not at all”) to 5 (“totally”). The order of the emails was randomized, and all participants saw the same randomized order. Participants were randomly assigned to one of three between-subjects conditions: low time pressure (22-minute time limit), high time pressure (8-minute time limit), or control (no time limit). The time limits were based on response times from a pilot study using the same stimuli. In the low and high time pressure conditions, a timer and progress indicator (a count of remaining emails) was displayed to participants in the bottom-right corner of the email task interface, and they were instructed to classify all the emails before the timer reached zero. After the timer reached zero, it continued counting down with negative numbers, allowing participants to classify any remaining emails.

After completing the email classification task, a Qualtrics survey automatically opened and presented participants with an individual differences battery. The battery consisted of demographic questions (e.g., gender, age, ethnicity, socio-economic status), a 7-item variant of the Cognitive Reflection Test (Frederick, 2005; Thomson & Oppenheimer, 2016), the Susceptibility to Persuasion Strategies Scale (Kaptein et al., 2012), the NASA Task Load Index (Hart & Staveland, 1988), and the time obedience subscale of the Time Styles Scale (Usunier & Valette-Florence, 2007). Data associated with individual differences, the Task Load Index, and confidence are not reported here. The experiment took about 45 minutes to complete.

Time Pressure, Persuasion, and Performance

To investigate the effects of the time pressure and persuasion principle manipulations, a series of 3x6 mixed ANOVAs were conducted at an alpha level of .05, with time pressure and persuasion principle as factors and measures of classification task performance (response time, sensitivity, response criterion) as dependent variables.

Response time results revealed a main effect for time pressure condition, F(2, 96) = 19.14, p < .001, η_p² = .29, a main effect for persuasion principle, F(3.94, 378.45) = 50.22, p < .001, η_p² = .07, and an interaction between the two, F(7.88, 378.45) = 3.71, p < .001, η_p² = .07 (see Figure 1). These results revealed that users responded as expected to the time pressure, where users in the high time pressure condition (M = 7.02 s) made quicker classifications than those under low time pressure (M = 11.26 s; p < .001), who were quicker than the users under no time pressure (M = 13.66 s; p < .05). In addition, these results also demonstrated that persuasion strategies influenced how quickly users made legitimacy judgements (see Table 1).

OPEN IN VIEWER

Figure 1.

Response times by persuasion principle and time pressure condition. Error bars reflect the standard error of the mean.

OPEN IN VIEWER

Table 1.

Means of performance outcomes by persuasion principle. Standard deviations are presented in parentheses.

Principle	RT (s)	d'	c	HR	FAR
Authority	12.44 (6.91)	-0.08 (1.22)	-0.09 (0.60)	0.51 (.25)	0.53 (.23)
Reciprocity	7.81 (4.59)	1.19 (1.21)	-1.12 (0.83) 0.20	0.91 (.14) 0.55	0.63 (.29)
Commitment	11.52 (5.48)	0.74 (1.20)	0.20 (0.71)	0.55 (.27)	0.34 (.22)
Scarcity	9.82 (4.59)	1.05 (1.19)	-0.54 (0.68)	0.77 (.20)	0.50 (.23)
Social Proof	11.06 (6.30)	1.21 (1.04)	0.23 (0.63)	0.63 (.20)	0.28 (.21)
Liking	11.22 (5.30)	1.31 (1.05)	0.35 (0.61)	0.59 (.20)	0.22 (.20)
Overall	10.65 (5.18)	0.67 (0.41)	-0.11 (0.30)	0.66 (.12)	0.42 (.13)

Sensitivity results revealed no main effect from time pressure, F(2, 96) = 1.37, p = .26, η_p² = .03, a main effect from persuasion principle, F(5, 480) = 21.70, p < .001, η_p² = .18, and no interaction between them, F(10, 480) = 1.44, p = .159, η_p² = .03, indicating that ability to distinguish between phishing and legitimate emails was affected by the persuasion principle used in the emails, but not by the presence of time pressure (see Figure 2). Overall, all participants struggled to classify emails, attaining an average sensitivity of .67.

OPEN IN VIEWER

Figure 2.

Sensitivity by persuasion principle and time pressure condition. Error bars reflect the standard error of the mean.

Similar to sensitivity, there was no main effect on response criterion from time pressure, F(2, 96) = .21, p = .81, η_p² < .01, a main effect from persuasion principle, F(4.01, 384.87) = 91.07, p < .001, η_p² = .49, and no interaction between them, F(8.02, 384.87) = 1.42, p = .188, η_p² = .03 (see Figure 3). These results indicated that users differed in their level of cautiousness depending on the persuasion strategy used, and not the level of time pressure.

OPEN IN VIEWER

Figure 3.

Response criterion by persuasion principle and time pressure condition. Errors bars reflect the standard error of the mean.

Together, while users made quicker judgements in response to time pressure, their performance was not otherwise affected by it. Rather, participants in all time pressure conditions struggled to classify the emails and were influenced more by persuasive strategies used in the emails.

Performance by Persuasion Principle

To summarize performance for emails with different persuasion principles, response time, sensitivity, response criterion, hit rate, and false alarm rate were collapsed across the three time pressure conditions (see Table 1).

Pairwise comparisons were conducted on each dependent variable to examine the relative influence of the persuasion principles. Response times indicated that users were quicker for every other principle than they were for the authority and commitment/consistency principles (p’s < .05), quicker with social proof and liking than authority (p’s < .05), quicker with scarcity than with either social proof or liking (p’s < .05), and quicker with reciprocity than any other principle (p’s < .001). Users were similarly sensitive to emails using the reciprocity, social proof, and liking principles (p’s > .05), less sensitive for emails using commitment/consistency principle (p’s < .05), and even less sensitive for emails using the authority principle (p’s < .001). Sensitivity for the scarcity principle was only significantly greater than that of the authority principle (p < .001). For response criterion, responses to authority were significantly more liberal than responses to social proof, liking, and commitment/consistency (p’s < .05), responses to scarcity were significantly more liberal than responses to authority (p < .001), and responses to reciprocity were significantly more liberal than to scarcity (p’s < .001).

Time Pressure

Echoing Sarno et al. (2022), response time data indicated participants were influenced by time pressure, with faster response times in the time pressure conditions. However, the present findings diverge from research that finds a speed-accuracy tradeoff, suggestingtime pressure significantly reduces users’ deception detection abilities (e.g., Jones et al., 2019; Butavicius et al., 2022). One reason for the absence of accuracy effects may be that the stimuli in the present study were too difficult to accurately classify under any level of time pressure, so time pressure did not elicit a meaningful difference in performance (i.e., a standard floor effect).

Persuasion Strategies

Overall, participants were brief and performed poorly but cautiously with their email classifications. These results are consistent with prior research, with studies on persuasive phishing reporting similarly low sensitivities and liberal response criteria (e.g., Lawson et al., 2020).

Prior studies have provided conflicting results about the persuasiveness of the authority principle (Lin et al., 2019), but typically find that it is the least persuasive persuasion principle (Lawson et al., 2020; Wright et al., 2014). However, users in the present study spent the longest amount of time evaluating, were less cautious with, and were by far least sensitive to emails containing the authority principle, suggesting that it may be a persuasive tactic under certain circumstances.

Similarly, users were relatively slow in their classifications of emails with the commitment/consistency principle, but also relatively risky and insensitive. These findings highlight how persuasive commitment/consistency is, echoing findings from Lawson et al. (2020) and Parsons et al. (2019), and diverging from Lin et al. (2019).

On the other hand, participants were the fastest, most cautious, and relatively sensitive to emails containing the reciprocity principle. These results demonstrate that users were able to accurately classify emails containing the reciprocity principle with relative ease. These findings are unique, as reciprocity typically does not stand out as one of the most or least persuasive principles in past research (e.g., Parsons et al., 2019; Wright et al., 2014; Lin et al., 2019).

Participants were faster classifying emails that contained the scarcity principle than any other principle, excluding reciprocity. However, participants had a moderately low sensitivity and were relatively cautious in their responses to these emails, indicating that users quickly conclude that such emails are phishing. These findings contrast with research that finds scarcity to be the one of the most persuasive strategies in phishing (Lin et al. 2019; Wright et al., 2014), instead supporting Lawson et al. (2020) and Taib et al. (2019).

Users were relatively slow and risky, but most sensitive to emails containing the liking and social proof principles. These findings suggest that while users may need time to identify whether liking and social proof emails are phishing, they are ultimately the best at classifying those emails. The findings about liking contrast with prior research, as most studies find the liking principle to be among the most persuasive persuasion principles (Lawson et al., 2020; Lin et al., 2019; Wright et al., 2014). Meanwhile, the findings for social proof echo Lin et al. (2019)’s findings but diverge from Taib et al. (2019)’s.

In summary, the present study found that users generally struggled to determine whether emails were phishing or not. They were worst at classifying emails using the authority and commitment/consistency principles, and best at identifying emails using the social proof, liking, and reciprocity principles. Discrepancies between the present and prior research may be due to methodological differences. Past studies vary in their definitions of the various principles (Wright et al., 2014; Lin et al., 2019), the detection task paradigm used (Lawson et al., 2020; Taib et al., 2019), whether the stimuli were created by researchers or came from actual email inboxes (Taib et al., 2019; Parsons et al., 2019), and how many classifications users make per persuasion principle (Parsons et al., 2019; Lawson et al., 2020). The present study used Muscanell et al. (2014)’s definitions of the principles, used a traditional email classification task with 60 trials, and used stimuli which contained one principle each and were sourced from actual email inboxes. Future research should aim to adopt standardized definitions with clear criteria for different persuasion principles, use representative stimuli, and adopt a standardized experimental paradigm so that the effect of persuasion principles can better be understood.

Limitations and Conclusions

The results of the present study suggest that users struggle to detect phishing emails, regardless of time pressure or persuasive phishing strategies. However, participants were best at identifying phishing in emails utilizing the social proof, liking, or reciprocity principles, and worst at discriminating emails containing the authority or commitment/consistency principles. While these findings help clarify the role of time pressure and persuasive tactics in phishing susceptibility, there are several limitations. First, the study used a relatively homogenous sample of undergraduate students. It is possible that different users may be more vulnerable to different principles. Future research should explore more diverse samples to better understand how susceptibility to persuasion strategies differs across populations (e.g., across the lifespan). the emails in the present study only included one persuasion principle each. Future research should use more representative sets of stimuli that better capture the diversity of phishing emails. Overall, the present study suggests that email content may play a larger role in users’ phishing detection abilities than situational variables like time pressure. Additionally phishing emails using persuasion tactics like appeals to authority or commitment/consistency may be more persuasive than those involving social proof, liking, or reciprocity.