How persuasive is a phishing email? A phishing game for phishing awareness

Abstract

Context:

In the current era of digital technology, social engineers are using various tactics to undermine human weaknesses. Social Engineers target human psychology to achieve their target(s) which are in the form of data, account details, or IT devices etc. According to our research, one of the first methods social engineers used to target victims is Phishing/Spear Phishing.

Objective:

The objective of this study is to utilize serious game to: i) educate players regarding phishing and spear-phishing attacks; ii) make aware and educate players regarding dangers associated with excessive online information disclosure.

Method:

In order to address the objectives we have: i) performed an in-depth literature review to extract insights related to social engineering, phishing, game design, learning functions, human interaction, and game-based learning etc; ii) proposed and aligned the game design with social engineering ontology concepts; iii) performed an empirical evaluation to evaluate the effectiveness of the designed board game.

Conclusion:

From this research study, we conclude that: i) PhishI game is useful in educating players regarding excessive online information disclosure and phishing awareness; ii) game-based learning is an effective method for inculcating and general cyber-related awareness in players.

Keywords

Security and privacy human and social aspects information assurance social engineering serious game collaborative learning empirical evaluation

1. Introduction

Information security plays a subtle yet significant role in our daily lives. It is of great importance for organizations around the globe to provide effective security protection. From a hacker’s perspective, in any organization, the employees are the critical junction through which information breach can occur. The information system security becomes more important in smart cities where all devices will (potentially) be connected using Internet of Things (IoT) [75], making societies more vulnerable to attacks [36]. Applications and Information included in IoT are more sensitive and need stringent security measures. Various studies show that human are the weakest link [27,34,50,61,63], who are vulnerable to attack, e.g. hospitals I.T systems, smart cars, and smart phones [36,75]. Imagine a situation where an attacker controls a house hold network system by guessing the weak password of the system. He can first lock the house and then increase its temperature or create other pressure-building situations so that the victim necessarily pays the ransom amount which is dreadful to its own right. Today’s technology makes a lot of good things happen such as, mobile payment, social networks, remotely controlling of house and vehicles, etc but, it also empowers malicious attackers which may lead to great harmful consequences without systematic protection. Human is often considered a weakest link in Information security chain and its necessary to aware and educate them.

Social engineering can be defined as “the art of influencing people to divulge sensitive information, and the process of doing so is known as a social engineering attack” [50]. “In information security terms, social engineering refers to an incident in which an information system is penetrated through the use of social methods” [68]. One of the social engineering methods is to use phishing attacks to get the desired information. Phishing [17] is one of the most dangerous threats to the world of information and technology. Phishing can be explained as a fraudulent activity in which attacker gets (secret) information such as user’s identity, account information (debit or credit card), pass-codes, etc. All of these activities are performed through spoofed emails, messages, and websites that seem original to the target victim. Phishing attacks follow deception theory which can be explained as “a message knowingly transmitted by a sender to foster a false belief or conclusion by the receiver” [11,13,44,47]. Generally, spoofed websites and emails are used to deceive and breach valuable information [11,44].

In cyberspace, spear phishing is one of the most severe threats to Internet security around the globe. Attackers aim to get users’ identities such as passwords and financial details by using spoofed emails and messages. The primary concern for companies today is to protect data from cyber-attacks. Recently1

¹
https://info.phishlabs.com/blog/2019-phishing-trends-intelligence-report-the-evolving-threat

^,2

https://www.businessinsider.com.au/scammers-have-dominos-australia-customer-data-and-are-sending-scarily-customised-spam-emails-2017-10

many events have taken place where the causes of breach were linked to human. Whether the breach is accidental or intentional, the victim bears the financial and reputational loss. Phishers mostly use fake emails or messages to persuade users to Phish. According to recent studies, the frequency and intensity of phishing attacks will increase in future, and hence, there is a strong need for public awareness and training [5,41,79]. Research studies showed that security awareness has a noticeable positive effect on users’ detection of deceptive attempts [35]. Social engineering attacks such as phishing attacks have severely impacted the world. As reported by Microsoft in 2014, annual world impact is US $5 billion. Spear phishing accounts for 91% of all attacks, being the most successful.

Many researchers across the world are working on mitigating the effects of cyber-attacks. Various studies have been performed where digital games are developed to train people regarding phishing emails [6,70]. Recent3

https://www.hackmageddon.com/category/security/cyber-attacks-timeline/

intense attacks show that there is a growing need to educate the public, as in digital era, nearly every individual own a smartphone connected to the Internet. In particular, there is a need to educate people regarding phishing [5,12,27,79]. Researchers conducted a training session for users using different education materials such as lectures, videos, and games. After conducting the training session, the results showed that participants who were playing the game could easily recognise a fake website [62]. Similarly, Junger et al. [40] concluded that due to lack of knowledge and defence strategies humans are vulnerable to phishing attacks. To mitigate this vulnerability, one have to receive training regarding phishing and spam emails.

Games provides scenario that are embedded in our daily lives and a learning experience in which players understand the rules, constraints, and things one should and shouldn’t do. Further, the motivational and game design elements help the participants to take the critical role and make bold decisions without worrying about the consequences [72]. Serious Games have been used as a possible way to engage participants/learners, and to transfer knowledge in a unique and easy way [22]. The effect of game-based learning can be seen in various fields, in various levels of education [72], e.g. maths [37], programming [31], collaborative airport management [28], and cyber security awareness [48,77], etc. Qian et al and Chang et al. from different fields performed experiments and analyzed the effects of game-based learning on students’ performances, they concluded that there are significant positive impact of game-based learning on players’ learning [14,56].

The motivation to design a game for cyber security is taken from the recent portal4

⁴

http://www.gamification.co/2016/03/02/teaching-kids-cybersecurity-game-based-training/

and from [10,57,77]. In a comparative study on game-based learning [52,53], the possible limitations were explored and how non-digital games can overcome those shortcomings were analysed. Moreover, research studies verify that non-digital game-based learning engenders a positive effect on student’s comprehension. Some of the limitations of the aforementioned digital games for education are that students and teachers need some prior experience of the game and need some investment to build software and a setup for education. Besides this, some of the benefits of non-digital game-based learning are that no technical prerequisite skills are needed for teachers and students; it is cost-effective, and is helpful in enhancing social interactions. The aforementioned discussion is the motivation for us to design a non-digital game for phishing awareness education.

Motivation and contribution: In order to protect individuals and organizations from phishing attacks, there are a few ways which can help us achieve that:

Building a phishing attack repository that accepts crowd reports on phishing emails and phone calls which can be found by searching for Fraudwatch and Millersmiles.

Understanding a phishing security test which would find out whether a given user is phishing prone or not. This would include phishing attempts to test how employees react on receiving suspicious emails and requests which would solicit confidential information. In our study, we have performed a pre-survey to check the initial knowledge level of the participants.

Security education by games that are carefully designed with in-depth knowledge about phishing, to explain the reasons for its effectiveness as an attack technique and to explain the methods used to stop phishing attacks.

The objective of the study is to design a game-based solution to thwart spear-phishing attacks, to create a know-how about the spear-phishing process, to educate people regarding the methods to identify the (signs of) phishing emails, to generate security requirements using phishing emails, and to further educate people regarding online information disclosure. In particular, the PhishI game is designed by using the detailed design findings from the research literature. Also, formulated a game design framework which can be used to extend the game design. Lastly, an empirical evaluation is performed to analyze the effectiveness of the learning activity.

Fig. 1.

Research protocol of the study.

The paper is divided into six sections. The first section introduces the topic and motivates the research problem. Section 2 further enlightens the game design rationale, the designed and proposed framework for the PhishI game, the various elements used in it, and its game process. Section 3 explains the empirical evaluation conducted to analyze the game effectiveness. Section 4 discusses the observation(s), the case study, and our methodology to evaluate the participants and their discussion. Section 5 explains the related work on phishing awareness and the gaps filled by our game. Lastly, Section 6 concludes our paper. The details can be seen in Fig. 1.

2. PhishI game assets and process

This section explains the game context and game elements as well as the game process for the PhishI game. Furthermore, this section shed light on the win/lose conditions, challenging part of the game from the participants perspective, and important design rationales.

2.1. Game elements

2.1.1. Scenario based learning – map/floor plan & assets

Scenario Based Learning is one of the ways where participants create scenarios in an hypothetical (game) scenario and further learn by brainstorming, interaction, and discussion. The same concept is used in building an hypothetical environment using storyline and game map.

Storyline of the game: The Yeovil district hospital recently announced a prize for anyone who can hack into their system and further tell them possible weaknesses in their security system. You, as a specialized team of hackers, are given the task to get that system down. So far, by initial analysis, it is seen that they have used one of the best security systems and software which makes it near impossible to hack by usual ways. Furthermore, the only possibility is through phishing or spear phishing attacks. You people are provided with important information extracted from social media (profiles) of the employees. Focus on one employee and try to gather as many information from the social media (Open Source intelligence). Then, draft a phishing email so that victim can install malware by this method and finally, by this way, you can undermine the system. Hospitals are and will be the target of ransomware attacks in future [1]. This is the main reason our first design of the game depicts hospital as an organization. Furthermore, the motivation of creating a controlled environment is taken from [57].

Map/Floor plan of the Hospital system is shown in Fig. 2. This map represents one of the site of the hospital where all the offices and important rooms are located. We can further see that in each department one of the human assets is located. A human asset can be a doctor, an IT person, Nurse, patient or intern. Each of the human asset is connected to the virtual world by his/her IT device(s). The blue rectangular box on the top right shows the virtual world which the hospital people use for connecting to the web.

Fig. 2.

Yeovil district hospital map adapted for game settings.

2.1.2. Role based learning – players play the role of attacker

An Attacker is a person who uses sociological and physiological principles on people to perform actions or to expose their confidential information or use deception via obfuscated image(s) or spoofed GUI component(s) of a computer program. In the game, the player plays the role of an attacker. Badge or Player role represents the character roles of the players in the game. Players role/badge is further motivated from the study [45,66]. As, in our game, we are only focusing on phishing attacks, we have only designed three identity card/badge for players. The motivation for designing this card is so that players can wear the hat of a social engineering attacker and can take actions in the game environment which they can use in real life. Figure 3 represents the role card/badge of the players.

Fig. 3.

Attacker role card.

Fig. 4.

Attack cards and social media cards.

2.1.3. Social engineering body of knowledge – attack type/techniques cards

Behind any game, there is a body of knowledge and skills that is being practised. We have used Mouton’s social engineering ontology [50] as the design know-how of the game. As the focus of the game is on Phishing awareness we have only designed the attacks related to phishing. Figure 4a represents the attack cards used in the game, and Table 1 lists the complete set of attacks and possible explanation of each attack.

Table 1
Attack type used in PhishI from literature

Attack type Explanation Reference

Phishing Phishing is a process in which attackers send a malicious email or SMS to the mass of victim, to get the financial or confidential information. Attackers pretend to be from a reputable organization. [10]

Spear phishing Spear phishing is a type of attack in which attackers target specific victim, i.e., person or organization(s). As this kind of attack is specific to the target, it is most successful in obtaining the information. [65]

Trojan horse attack Trojan horse is a type of malware which is used to trick the victim into installing it on his/her personal computers and further infect the system by using that malware or by providing a backdoor to the attacker. [29,32]

Need & greed attack The need of a person makes him/her vulnerable. In this type of attack, attackers target the victim using his/her vulnerability or need. This attack aims to use victims need and further make him/her greedy. [10]

Suggest your attack In this type of attack, players suggest their attack type. The motivation for providing this option is to help the players to think out of the box and can relate, and share their knowledge and experience. Nil

Attack type	Explanation	Reference
Phishing	Phishing is a process in which attackers send a malicious email or SMS to the mass of victim, to get the financial or confidential information. Attackers pretend to be from a reputable organization.	[10]
Spear phishing	Spear phishing is a type of attack in which attackers target specific victim, i.e., person or organization(s). As this kind of attack is specific to the target, it is most successful in obtaining the information.	[65]
Trojan horse attack	Trojan horse is a type of malware which is used to trick the victim into installing it on his/her personal computers and further infect the system by using that malware or by providing a backdoor to the attacker.	[29,32]
Need & greed attack	The need of a person makes him/her vulnerable. In this type of attack, attackers target the victim using his/her vulnerability or need. This attack aims to use victims need and further make him/her greedy.	[10]
Suggest your attack	In this type of attack, players suggest their attack type. The motivation for providing this option is to help the players to think out of the box and can relate, and share their knowledge and experience.	Nil

Table 2

Psychology/compliance principles/human behavioral patterns

Psychology to target	Explanation	Reference
Fear	By using different fear tricks, company’s employee gives access and share sensitive information. For example, new hiring employees may be the target.	[10,38]
To obey authority	Studies show that high level of obedience leads to social engineering attack. For example, attacker can target the obedient person to gain access the critical information.	[10,24,38,64]
Trust	Research shows that people may be easily target by excessive trust. Definitely, it is the first priority in business dealing but too much trust is harmful for companies data and leads to social engineering attack.	[10,38]
Need & greed	As studies shows that when an attacker knows the needs of a person, he/she wants to manipulate that person by his need and that make him/her greedy.	[10,24]
Guilt	If a person induces to someone may feel guilty. In this case, an attacker said him/her, if you want out from this guilt and should complete the task given by attacker.	[10]
Curiosity	The eagerness that wants to learn and get more and more about something is known as curiosity. For example attacker may use different promotional codes to win a cash or gift prize to get sensitive information.	[10]
Panic/time pressure	Research shows that attacker can create a panic situation to the victim. For example, an attacker may tell the victim that employee’s company in the loss and will fire soon, accept attacker’s offer, it will be beneficial for both the company and the person. The person may go in a panic situation and accept the offer of an attacker.	[10,24]
Emotional	An attacker may target his/her victim emotionally by getting critical information easily. These emotions may be positive or negative.	[10,38]
Social proof	People usually mimic what majority of the people do as in this way they are least likely to held responsible of unseen situation.	[24,64]
Deception	Usually, people abide to whom they feel attracted and similar, etc. So in this case deception principle become the strongest tool for attackers.	[24,38]
Suggest new way	This card gives an opportunity to players to suggest some new and innovative psychology to target victim.	Nil

2.1.4. Psychology needs of target victim – compliance principle /psychology cards

Social relations is the reasons why a target responds to the attackers’ requests. The reasons include: friendship, scarcity, authority and so on. Attackers used these psychology needs to attack the target victim. Figure 5 shows the compliance principles cards. We have used various compliance principles in our game which are shown in Table 2. One motivation to design this type of cards is that players may learn the psychological technique employed by social engineers.

Fig. 5.

Compliance principle or psychology to target adapted from [77].

2.1.5. Point of contact selection – social media cards

In the game PhishI, we have proposed social media cards which represent social media information (Open Source intelligence) present on various social media channels, e.g. Facebook, Twitter, etc. The motivation of using cards which are not extracted from the database is that the players can enjoy the game irrespective of their ability to use Internet, their access to a social media account, etc. The social media cards represent the information regarding targeted assets. Social media cards can be seen in Fig. 4b.

2.2. Game process

The PhishI game has four phases, each phase takes 15–20 minutes.

Phase 1: Victim selection: Game players select a target victim from the organization map shown in the Fig. 2. The first phase of attack preparation has four steps: selection of communication media, target devices/victim, attacking techniques, and attack material preparation. In PhishI, players have to choose the target, medium, technique, and compliance principle.

Phase 2: Gather information about target: In this phase, game players have to gather the information regarding the target victim from various social media sources (Open Source Intelligence) provided in the game. This concept is adapted from [4,23,42,54], which explains how the attackers use social media information of victim in planning the social engineering attack. In Phishi, we have designed social media cards which disclose sensitive important information of the victims. Using the information from the social media cards can help the player to learn the vulnerability associated with excessive online information disclosure. Additionally, players can learn how attackers can easily design and execute a spear-phishing attack.

Fig. 6.

Phishing message design cards used by players.

Phase 3: Draft phishing email: Here, game players have to draft a phishing email to the target victim using the information given above. The players can see the phishing message design cards (learning cards) in order to design a successful phishing email. Phishing message design cards shown in Fig. 6 are incorporated in the game PhishI as part of the learning strategy. The players will read the cards and try to incorporate the important steps needed to identify phishing emails. The steps are adapted from studies [15,25,62,78].

While constructing phishing emails, players refer to the phishing message design cards as discussed below:

URL: Try not to use URL with numbers, e.g. http:147.46.226.55/Paypal.html. As the victim can easily identify the phishing link.

Subdomain URL: Try to use famous company names and logos for communication, e.g. http://secure-sinin.ebay.com.ttps.us/. This link belongs to “.ttps.us” website instead of “ebay.com”.

Similar and deceptive domains: Try to use hyphen “-” with famous company name to make it more realistic, e.g. http://www.pay-pal.com/login/php.

Email content linked with need and greed: Make best use of the information gathered from social media or online media. Try to use victims needs to convince the victim.

Official Logo and email template: Try to use official logo and email template to make the email looks more realistic.

Time and date: Try to send email on data and time when there is minimum support available, e.g. during Christmas holidays, and at night, etc.

Customer service number/email: Use fake contact numbers so that when victim contacts by phone, he/she may use the number provided in the email.

Psychology to target: Make the best use of compliance principles. Exploit psychology weakness to target victim.

Send fake attachment: Send attachment containing malware or virus. Upon downloading and installing, the attacker may get access to victims’ computer.

Ask for personal details: Ask the victim to provide financial or personal information by clicking the fake link.

Phase 4: Spread the email: In the last phase, two player teams swaps the phishing email drafted for review. The phishing email reviewed by using the phishing message design cards (learning cards) given to the teams.

The step by step procedure of the game process is described below:

Select a target victim on the map whom you want to attack using phishing email.

Note the description of the victim in information gathering for future record.

On the virtual space area of the game, select five cards from the social media deck representing about the victim players.

Record all the information on the sheet given to the team.

Draft an email using the compliance principles/psychology cards.

Revise an email and review for possible improvements.

Review by the opponent teams.

Discussion Session.

2.3. Other game design considerations

2.3.1. Challenges

The players have to collect maximum information from various sources provided in the game.

The players have to draft the email which relates to the needs of the target victim and by using various compliance principles.

The players have to draft a phishing email with minimum deficiencies.

2.3.2. Win/lose conditions

The scoring system is divided into three areas, such as email structure (2 points), idea/scenario (2 points), and efficient usage of available information in drafting phishing email (2 points). The team which scores the most points wins the game. The decision will be done after the discussion session which is the last phase of the game.

2.3.3. What and where to teach in game:

In Table 3, we have explained which important learning aspects that we have embedded in different phases of the game.

Table 3
What and where to teach in PhishI

What to teach Where to teach

Too much sharing of personal information can be damaging (information disclosure). Phase 2: Information gathering

Identify fake/phishing email Phase 3: Drafting phishing email using hypothetical scenario making

Identify fake URL’s & subdomains Phase 3: Drafting phishing email using hypothetical scenario making

Phase 4: Using phishing message design cards

Identify similar and deceptive domains Phase 3: Drafting phishing email using hypothetical scenario making

Phase 4: Using phishing message design cards

Emails – need and greed (psychology) Phase 3: Drafting phishing email using hypothetical scenario making

Phase 4: Using phishing message design cards

Identify psychology targeting in phishing email Phase 3: Drafting phishing email using hypothetical scenario making

Phase 4: Using phishing message design cards

What to teach	Where to teach
Too much sharing of personal information can be damaging (information disclosure).	Phase 2: Information gathering
Identify fake/phishing email	Phase 3: Drafting phishing email using hypothetical scenario making
Identify fake URL’s & subdomains	Phase 3: Drafting phishing email using hypothetical scenario making
Phase 4: Using phishing message design cards
Identify similar and deceptive domains	Phase 3: Drafting phishing email using hypothetical scenario making
Phase 4: Using phishing message design cards
Emails – need and greed (psychology)	Phase 3: Drafting phishing email using hypothetical scenario making
Phase 4: Using phishing message design cards
Identify psychology targeting in phishing email	Phase 3: Drafting phishing email using hypothetical scenario making
Phase 4: Using phishing message design cards

2.3.4. Game design framework:

A game design is a function of several areas which together create an environment in which users enjoy and learn. For example, game patterns, user experience, psychological aspects, knowledge base, and learning functions are some factors influencing the learning experience. To design an effective serious game, we have adapted game design patterns and psychological needs [26,58] which are relevant to our game. We have focused on four main areas, which are social engineering domain knowledge, psychological needs of players, learning functions, and human interaction. Table 4 represents the mapping of domains in our game.

Table 4
Main concepts and embedding in PhishI game

Learning functions Mapped game design pattern’s (GDP) Incorporating in PhishI game Psychological needs satisfaction

Interpreting Pre-defined goals, gain information Social media cards, persona centered information, goal defined Need for autonomy, need for gaining information and understanding.

Analyzing Randomness, strategic knowledge Randomness of social media cards selection, attack type cards selection. Need for confidence, need for surprise/interest

Feedback Score, points Review session Need for companionship need for social relatedness, need for competence

Explaining Direct information Game rules, discussion between team members, discussion in review session Need for competence, need for achievement

Applying Game elements Phishing sheet, phishing message design cards, points, avatars of attackers, story of game Need for understanding

Learning functions	Mapped game design pattern’s (GDP)	Incorporating in PhishI game	Psychological needs satisfaction
Interpreting	Pre-defined goals, gain information	Social media cards, persona centered information, goal defined	Need for autonomy, need for gaining information and understanding.
Analyzing	Randomness, strategic knowledge	Randomness of social media cards selection, attack type cards selection.	Need for confidence, need for surprise/interest
Feedback	Score, points	Review session	Need for companionship need for social relatedness, need for competence
Explaining	Direct information	Game rules, discussion between team members, discussion in review session	Need for competence, need for achievement
Applying	Game elements	Phishing sheet, phishing message design cards, points, avatars of attackers, story of game	Need for understanding

3. Empirical evaluation

Phishing attacks generally result in billions of dollars annual losses. To address the issue of phishing and spear phishing, researchers around the globe have been working on various solutions. Two types of techniques are used to test the awareness of participants. One is the test based technique, and the other is the wild technique. For our study, we have used a combination of the techniques adapted from the cyber-phishing platform [33]. We have conducted a controlled test activity and used survey to assess the learning of the participants. In parallel, we have used the observation methodology to evaluate the learning and collect feedback.

Section 3 explains the recruitment process of the participants and describes how we have designed the evaluation and our goals for the study. It also explains the results of the study.

3.1. Recruitment process

The game sessions were conducted with multiple sessions on campus. Author of the paper, lab mates, colleagues were invited to take part in the pilot experiment to understand and perform the game. In order to come up with the comparison chart we have used Delphi approach for the conclusion, in case of disagreement. To perform our game activity, we advertised it by posting flyers and by dispatching messages to (different) departmental groups. A total of 63 participants (Masters and PhD students) participated in the activity. With in 63, 40 were male and 23 were females. 63 participants were further divided into thirty teams (three sessions). The participants belong to various department. Some are from school of software, others are from Department of Energy, Department of Computer Science, and Material Science. English language is used as an activity language.

3.2. Empirical evaluation design

According to recent studies published in Nature [59] and Springer [4], a same threat can be interpreted differently. In phishing attacks, an identical phishing email sent by an attacker made some of the targets click on the infected link while others ignore it. In this scenario, where different people have different perceptions regarding the same threat, a possible way is to provide an environment where players can learn in a friendly environment. Moreover, they can discuss and share their knowledge and views. Our PhishI game provides this environment to the participants where they design and develop the phishing email and discuss with peers to understand the essence of a phishing email.

Fig. 7.

Empirical evaluation flow and time division.

Table 5

Pre-game participants identification of URL

	Phishing URL	Not sure	Legitimate URL	Correct answer	Percentage
http://www.paypal.com/accept	7	11	45	Legitimate URL	71.4%
http://www.ebay.com	9	5	49	Legitimate URL	77.7%
http://147.91.75.1/ebay/	15	15	33	Phishing URL	23.8%
http://secure-signin.ebay.com.ttps.us/	19	27	17	Phishing URL	30.1%
http://www.msn-verify.com/	5	5	53	Phishing URL	7.9%

Figure 7 shows the detail of experiment timeline. In Table 5, we have shown the responses of the participants. The participants before the activity tried to identify the URL’s given on the paper sheet. From Table 5, we can see that participants correctly identified the famous URLs such as paypal, ebay but got confused while identifying URLs with numbers or combinations of famous websites. This pre-survey helped us identify the area(s) which need special attention in our game design.

Below are the goals for our study:

Goal 1: Game based learning by PhishI game and feedback collection

After the controlled activity, the players were asked to fill a survey questionnaire. Players were asked regarding the phishing attacks and their possible impacts on the individuals’ lives. For example, we asked: “It is extremely likely that my computer will be infected by a phishing attack in future.” The majority of the respondents’ agreed with the statement. Only one of the respondents disagreed. The diverse responses from the players depict that there is a room for improvement in the game which can be bridged by discussing the latest phishing attacks and their effects. This will, in turn, help the players know the importance of saving themselves from Phishing attacks. In the second question, the players were asked: “My chances of being targeted by phishing attacks are great.” The majority of the players agreed; however, some participants disagreed. This might be because the players trusted their anti-virus software to keep looking for such kind of emails. To account for this, players’ understanding of phishing can be further improved by adding limitations of Anti-viruses in the game. Finally, we asked the players: “I feel phishing attack will infect my computer in the future.” To this statement, players mainly selected the answers lied in the “agree” and “neutral” option.

Goal 2: Phishing knowledge and avoidance behavior

The URL survey page which was taken before the game playing session was then again used to check post-game learning. Table 6 represents the responses from the participants. We can see that the majority of the participants have correctly identified the answers which represent the overall positive learning of the players. If we compare Table 5 and 6, we can see how the identification of URL improved after playing the game session.

Table 6

Post-game participants identification of URL

	Phishing URL	Not sure	Legitimate URL	Correct answer	Percentage
http://www.paypal.com/accept	7	4	52	Legitimate URL	82.5%
http://www.ebay.com	4	3	56	Legitimate URL	88.8%
http://147.91.75.1/ebay/	51	2	10	Phishing URL	80.9%
http://secure-signin.ebay.com.ttps.us/	44	7	12	Phishing URL	69.8%
http://www.msn-verify.com/	52	1	10	Phishing URL	82.5%
http://www.amazoon.com/	47	5	11	Phishing URL	74.6%

Goal 3: Educating regarding online information disclosure

Social media provides an opportunity for people and companies to share ideas and experiences. However, on the other side, they divulge personal as well as public information as well as other’s information. So, information disclosure on social media allows an attacker to access sensitive information from the personal and organizational systems. Currently, companies are using social networking sites for educational, industrial, and marketing purposes. But there is a need to educate general public about online information disclosure as well [49,80]. In PhishI game, using the social media information available to design the phishing email will probably make players think of how their information can be used against them. Furthermore, this will also make them consider disclosure of information online as something that can be dangerous. This insight was also observed during discussion session at the end of the game session.

3.3. Research model and analysis

We have used quantitative approach (survey) to calculate the feedback regarding the game and the already existing phishing knowledge of the participants. The questions of the survey were adapted from the already published literature, as we believe that this will increase the credibility of the obtained results. The post survey was divided into two broad sections: the demographic questions and independent and dependent questions which helped to support the research model. Furthermore, we have adopted 5 points Likert scale to gather the behavioral responses of the participants. In our survey, value five represents (Strongly Agree) and value one expresses (Strongly Disagree). For our research model, we have adapted Technology Threat Avoidance Model (TTAT) from [7] and Technology Acceptance Model from [3]. The post survey questionnaire can be seen in the Table 18. In total, we got 63 responses which, after pre-processing,5

⁵
The responses which had missing data, or outliers (the participants selected same scale for all the survey questions), etc. were removed.

were reduced to 47. The research model shown in Fig. 8 was used to perform the analysis. The hypotheses of our study are mentioned below.

H1:

Fun to Play has a positive effect on Intention to play.

H2:

Ease to Play has a positive effect on Intention to play.

H12:

Ease to Play and Fun to Play have a positive effect on Intention to play.

H3:

Intention to Play has a positive effect on Game Based Learning.

H4:

Phishing Knowledge has a positive effect on Game Based Learning.

H34:

Intention to Play and Phishing Knowledge have a positive effect on Game Based Learning.

H5:

Game Based Learning has a positive effect on Avoidance Behavior.

Fig. 8.

Research model.

Accordingly, we have come up with four linear regression equations: $\begin{array}{l} (1) & Intention = β_{0} + β_{1} * Fun + β_{2} * Ease + μ_{1} \\ (2) & Game-Based Learning = α_{0} + α_{1} * Intention + α_{2} * Phishing Knowledge + μ_{2} \\ (3) & Avoidance Behavior = ρ_{0} + ρ_{1} * Game-Based Learning + μ_{3} \\ (4) & Avoidance Behavior = f (Fun, Ease) \end{array}$ where: μ are the disturbance terms with $E (μ) = 0$ and a finite variance. The first three equations are what we simply get from our model proposed above. The fourth equation can be derived simply by substituting second and first equation in the third one. Thus, in total, we have four equations to test. Before regressing variables, we can have a look at the graph matrix6

⁶

Graph matrix plots all the scatter plots between the specified variables. This enables us to get a “feel” for the data before any regression analysis.

(Fig. 9) which shows the general trends for our variables. Observing from the last row, we can see that the relation between our independent and dependent variables are generally positive – indicating towards acceptability of our model.

Fig. 9.

Graph matrix.

The regression results for our equations are shown in Table 7. From looking at the table, we can see that all7

⁷

The coefficient value for Fun to Play in the first equation comes out to be $- 0.047$ in our multivariate model. Later on, we will show that this negative value is due to multicollinearity and not a negative trend in the population.

the coefficients values are positive, i.e. a positive increase in independent variable is associated with a positive increase in y-variable. For some of the coefficients in the equations, the individual p-values come out to be statistically insignificant at a 5% confidence level. This insignificance is due to multicollinearity between variables.

We had earlier noted that (initially) it was generally hard for people to understand and apply phishing concepts. Although, from the perspective of our model, the level of Phishing knowledge of individuals is exogenously given,8

⁸

Lower as compared to other coefficient values in the table.

the difficulty to understand phishing concepts is partially explained by a relatively lower coefficient value for phishing knowledge, i.e. 0.17.

Table 7

Regression analysis

Equation	Independent variable	Dependent variable	p-value	Coefficients	R-squared	Overall F-stat *
1	Fun to play	Intention to play	0.816	−0.047	58.71%	0.0000
1	Ease to play	Intention to play	0.000	0.801 **	58.71%	0.0000
2	Intention to play	Game based learning	0.016	0.48	36.71%	0.0034
2	Phishing knowledge	Game based learning	0.167	0.17	36.71%	0.0034
3	Game based learning	Avoidance behavior	0.014	0.47	24.26%	0.0137
4	Fun to play	Avoidance behavior	0.243	0.27	46.74%	0.0000
4	Ease to play	Avoidance behavior	0.003	0.47	46.74%	0.0000

Robust standard errors are used wherever the residuals were heteroscedastic.

^**

For example, as an interpretation: a one unit increase in participants’ perception of Ease to Play is associated with a 0.8 unit increase in their Intention to Play, on average.

The correlation matrix for variables is shown in Table 8. For most of the variable combinations, there is a high correlation i.e. when one variable moves in one direction, the other one follows. So, for example, if a participant perceives a game as one that is fun to play, he/she is much likely to perceive it as an easy one too. Hence, when we add both of the variables together in one equation, one of the two turns insignificant once the other is controlled for. Although there is sufficient multicollinearity between our variables, the equations proposed above all are statistically significant at 5% confidence level (overall), which means that our model and the associated trends hold in the larger population.

Table 8

Correlation matrix

	Fun to play	Ease to play	Intention to play	Game based learning	Phishing knowledge	Avoidance behavior
Fun to play	1.00
Ease to play	0.86 *	1.00
Intention to play	0.65	0.77	1.00
Game based learning	0.34	0.48	0.59	1.00
Phishing knowledge	0.66	0.67	0.80	0.54	1.00
Avoidance behavior	0.64	0.67	0.90	0.49	0.73	1.00

All the variables are highly correlated i.e. participants’ perceptions move together across a range of variables. This will generate a tendency for our regression to suffer from multicollinearity.

4. Discussion

4.1. Observation

Observation is an important research method it is used when we need to study human behavior, human behavior, body language, facial expressions, etc [43,60,77]. During the game session, two of the researchers acted as the observers. One researcher sat within one of the teams and observed their arguments and discussion points. The other walked around the room for a general and more holistic appreciation of the game sessions. Researchers noted interesting discussion points, such as “Lets phish the nurse as she may not know about phishing,” “Normally doctors have little time in their schedules. Hence, if we try deceiving them via situations built around the persuasion principle of “urgency”, it might work,” etc. From the discussion points, we can conclude that the participants were thinking from the perspective of an attacker, and we can believe that this will help to learn various techniques and strategies used by the social engineers. Some of primary observations on which all the researchers agreed upon are given in Table 9.

Table 9
Primary observations and areas for improvement

Some of the important observations Interpretation Category

“What is social engineering? and phishing? Did you understand?” This may be due to the reason that these terminologies are not known by participants and they need time to understand. Once they got attack examples they can easily relate that with their real life examples. Other possible solution is to not use terminologies/jargon’s, so that understanding can be much easy. Game Pre-Session (PS)

“How to play the game?” As in the initial phase of the game, some of the participants faced issue in understanding the process. But once they understand the process/steps, they suggested reasonable phishing emails. Game process (GP)

“Lets target that lady?” On inquiring about this comment at the end of the game, the participants told that the reason for this is that “Lady may be an easy target and it seems easy to deceive them”. Game design (GD)

“We are winning ! Yo !” One of the emotional sentence made by the participant whose team scores more than the opponent. Game elements (GE)

Some of the important observations	Interpretation	Category
“What is social engineering? and phishing? Did you understand?”	This may be due to the reason that these terminologies are not known by participants and they need time to understand. Once they got attack examples they can easily relate that with their real life examples. Other possible solution is to not use terminologies/jargon’s, so that understanding can be much easy.	Game Pre-Session (PS)
“How to play the game?”	As in the initial phase of the game, some of the participants faced issue in understanding the process. But once they understand the process/steps, they suggested reasonable phishing emails.	Game process (GP)
“Lets target that lady?”	On inquiring about this comment at the end of the game, the participants told that the reason for this is that “Lady may be an easy target and it seems easy to deceive them”.	Game design (GD)
“We are winning ! Yo !”	One of the emotional sentence made by the participant whose team scores more than the opponent.	Game elements (GE)

Case study – Game instance: One particular instance of the game is shown in Table 10 for information. We believe that the case instance will help in better understanding on how the activity carries out.

Table 10

Players responses during game progression || an instance of game

Attributes/dimensions	Game instance for Team 1
1. Social media information (from social media cards)	Very much in need of new job
2. Vulnerability of human asset	Looking for new job
3. Psychology to target (from psychology cards)	Need and greed, curiosity
4. Phishing email drafted by Team 1 (Table 17 in Appendix)	With reference to your friend, we came to know that you are searching for a new job opportunity. We are contacting you from National Hospital. We are looking for experienced resources for our hospital. If you are interested kindly read the attached file related to our hospital and reply with your resume.
5. Score given by reviewers team	4 out of 6
6. Suggestion given by reviewer’s team	i) You must mention the name of a friend who is too social, so that the reader may trust this completely; ii) You can talk about some good salary package and position; iii) For further information you need to give him/her some cell number; iv) You can use female name and information as coordinator so that the chances of victim to respond and contact will be more.
7. Reply by Team No. 1 members	For suggestion i) The target victim may contact the friend mentioned and the attack may fail.
8. Suggestion given by an expert/teacher	i) Composed phishing email is reasonable. ii) You can also use social media channel (linked In, etc) to contact the victim in-order to first build TRUST and then launch phishing attack.
9. Reply by the Team No. 1 members	Yes, good suggestion.

4.2. How we have evaluated the participants knowledge?

4.2.1. Developing and reviewing PhishI emails

In this research study, knowledge of the participants were evaluated by various methods and techniques mentioned in research literature. Firstly, we observed the participants knowledge by analyzing the quality of phishing email generated by them. As discussed in game design section, the players have to use the given psychological principles, human weaknesses, attack techniques, and other valuable information to generate deceptive or phishing emails. The players, when generate a phishing email, possess working-knowledge on how any potential social engineer can use various techniques to target humans. Furthermore, discussion session (among different team members at the end of scenario generation session) helps to understand the context and other viable ways of attacks. The quality of phishing emails and discussion and feedback session reflects the learning outcomes of the game session. Some of the phished emails designed by the players during the activity session are shown in Appendix A. Scenario based learning is the technique used in industry for training and education for the past years [20,71]. This method of evaluation is adapted from [10,77].

4.2.2. Game feedback & survey [43,60]

To receive feedback related to the game activity, we asked players to fill the post-game survey. The results of the game, as shown in the results section, are overall positive. There is still much room for improvement. The feedback obtained by the researchers will be used to improve the game with the next version.

4.3. Limitations and validity threats

Some of the known limitations and validity threats are discussed below:

a) One of the limitation of this study is the (sample size) number of participants for the evaluation activity. As this is the preliminary evaluation, only 63 participants’ result is shown in the study. We realize this limitation and are planning to verify the results with more empirical evaluations in future. The motivation to report the results with 63 participants is: i) to report preliminary results to the community; ii) that many research studies have used less than 50 participants for initial or preliminary evaluation [34,67,77], supporting our rationale of using 63 participants to represent the research results; b) Another limitation is that we have taken participants solely from the university environment. We believe that, in future, the effectiveness of the game-based activity must be verified by inviting general public or industry participants; c) One of the limitations of experiment-based activity is that a layman can easily turn uninterested and bored. This effect “looking out of the window” is mentioned in [10] and first introduced by Ketil Stólen. To minimize the window effect during our study, we have implemented a two-step methodology. In the first step, the participants were filtered out using intrinsic motivation (if participants are interested by themselves, they would respond to the email or advertisement). Then, extrinsic motivators were embedded in the game using game elements as well to maintain the fun-element throughout the game.

a) Some threats, and limitations are associated directly with the type of research methodology selected. If we take qualitative data as the data collection method in the study, two of the most important possible shortcomings will be selective memory (partly remembering the event) and positive attribution (only reporting the positive results). In order to minimize these factors in our study, before the empirical evaluation session, all the researchers were assigned observational duties. During the activity session, the researchers noted important points and topics, which were further discussed with other fellow researchers at the end of the activity. Conducting the meeting soon after the evaluation activity helped minimize the effect of selective memory, and the participation of all the researchers helped to cross-report points which made the points reported authentic and true representative of what happened during the sessions; b) A comparative analysis of our study with respect to other phishing games can be seen in Table 11 and 12. The tables illustrate the places where our game has bridged the gaps existing in other games and the places where it is not comparable to other games. In this study, we have used limited number of phishing attacks, persuasion techniques, and human weaknesses for the proposed game. In future, our plan is to update these lists and further give participants more options to select, brainstorm, and learn from.

Threats to internal and construct validity [ 55 , 73 ]: Internal validity deals with researcher’s biasness and interpretation of the results, whereas the construct validity deals with the possible threats associated with the experiment or activity design. In order to minimize the internal validity of the study, we have controlled factors such as activity time distribution, learning context and material, teaching method, and the language of activity. Furthermore, to minimize the construct validity, survey questionnaire, URLs for identification and scenario-based analysis method are taken from published research literature (to minimize the biasness in designing or any effect on the results or result’s interpretation).

Threats to external validity [ 55 , 73 ] deals with the generalizability of the results. As this is a preliminary evaluation for the proposed activity, we cannot claim the results to be general (at this time), but still we believe that the results may not vary much (by analyzing the results from the empirical evaluation). Furthermore, external validity of the study is still to be verified by further empirical evaluations in future.

Others: a) One of the main limitations of our work, which was also earlier touched upon, is the level of effort needed by users to play this game. In our experiment, since the teams belonged to different departments and academic disciplines, the ‘effort-factor’ might have minimized. In future, we are planning to address this challenge by using Design Science principles which will help participants understand concepts easily; b) Other possible limitation can be biasness in filling the survey responses by the respondents. In our case, after the game activity, we categorically announced to remain fair in filling the survey as this will help in improving the game. Furthermore, the validity can be tested by performing more empirical evaluations in future; c) In this preliminary evaluation of the game, where sharing knowledge and healthy discussion is the main essence, we are not sure how this game will behave in case of single players (individuals playing the game). This can be a future task to first verify and further see how it is effective as compared to played by multiple players. d) In order to analyze how psychology factors affect victim, first we need to understand the pattern or deciphering of social engineering attacks. We have not evaluated this aspect in this particular version of the study. On the other hand, our research group is working on the above dimension. The study particularly focuses on emotional journey of victim during a phone scam (another kind of social engineering attack). The journey of the victim goes from E1 to E8. Starting from the Trust factor to becoming Curious and passing through Fear, Anger, Urgency and lastly Relaxed (temporarily).

5. Related work on phishing awareness techniques

After an in-depth literature review, it can be argued that a significant number of cyber-security studies have been published on the topic of phishing techniques and awareness. If we further categorize these studies, we can see that phishing literature can be categorized into technical and non-technical solutions to counter phishing. Furthermore, many studies showed that computer users are vulnerable to phishing due to many reasons. Some of them are: i) “look and feel” of the fake website is greatly comparable to the original website and hence, could effect user’s ability to identify the difference; [19] ii) end users are not adequately aware of the phishing attacks, techniques, and process; [21] iii) end users don’t observe and pay heed to indications of browser security [74].

Taxonomy and theoretical model for phishing: Ahmed et al. [5], developed a taxonomy by performing an in-depth literature review. The proposed taxonomy covered four essential dimensions which include i) communication media; ii) attacking techniques; iii) targeted devices for an attack; iv) countermeasures for phishing attacks. The researchers then, categorized the phishing studies into five areas i) machine learning; ii) text mining; iii) human users; iv) profile matching and v) others (honeypot countermeasures) to give a clear idea of the research studies. The researchers of the study believe that the mentioned taxonomy will be helpful in identifying phishing attacks and can be used as countermeasures to such attacks.

Phishing awareness evaluation: Due to an increase in the number of phishing attacks Jansson et al. [39] inquired how vulnerable their students are to such attacks. They designed an experiment which was divided into two phases. In the first phase, the target victims were categorized into different groups. Four different types of phishing emails were then generated, and each group received a unique type of phishing emails. To make phishing attacks more context-oriented, one phishing email explained that the university database has crashed and to retrieve their account information, they had to click on the provided link. In short, the results of the training with simulating phishing attacks has shown that this can be used to enhance user’s phishing resistance [39]. Kumaraguru et al. [44] designed an experiment and, as the first step, sent an email for volunteer participation in an activity. The participants were first given training using the game “Phish Guru” which helped them learn and understand the legitimate and phished URLs using an interactive fish-themed activity. After the training, the participants received emails at random hours (for seven days) concerning winning a game and/or other cash prizes via filling a simple form. The form asked for university credentials as a pre-requisite for participating in the lottery. The study concluded that the participants who took “Phish Guru” training were more effective in identifying phish emails and that they retained phishing knowledge for 28 days after the training. Furthermore, the author concluded that the participants who were between 18 to 25 years of age were more vulnerable to phishing attacks than the elder users.

Anti-phishing games: Sheng et al. [62], designed and evaluated a game “Anti-phishing Phil” for educating and making participants aware of phished URL’s. The digital game created a swimming pool or a water tank in which fish were placed at positions. Clicking over each fish revealed either a legitimate or a phished URL. The player, then, is a small fish which is new to this environment and by the help of its mentor identifies the legitimate and the phished URLs. For example, participant (the small fish) and its mentor swim to another fish at a position in the tank. Upon clicking on that fish, a URL pops up, which may be legitimate or phished. By using the knowledge from the tutorial at the start of this activity, the participant has to correctly identify the type of the email, which would earn him/her points. The study concluded that the participants who played the game were better able to identify phishing emails and that “Anti-phishing Phil” makes people more aware of phishing attacks viz-a-viz other educational methods based on reading, etc. M. Baslyman et al. [9], developed a board game named “Smells Phishy” that claimed to educate users about online phishing scams. The experiment was performed on 21 participants. In the start of the game, each player received “X” amount of money for the gameplay. The idea of the game revolves around purchasing items from a place on a map. The player, upon buying, may either successfully get the item or be phished. After it’s evaluation, it was established that users (after playing the game) could defend themselves better from phishing attacks. The results of the study confirmed that after playing the game, the user depicted better awareness of phishing scams and also learned the techniques to protect themselves.

5.1. Commercial tools and table comparison

PhishMe9

⁹
https://cofense.com

is a commercial company which aims to utilize human asset to minimize phishing attacks. It works on the concept of crowd sourcing. Studies have been published recently [28] where researchers have used crowd as source of information to counter phishing attacks. The goal of the PhishMe is to educate people via contextual knowledge. For example, if one of your close friends texts you in an unusually formal tone, you can take a safe bet that something has gone wrong. So once a client identifies phished stuff, he/she can report it to PhishMe, where an expert analyzes the threat and add the phishing email and its template to the database and also update all other users regarding the email. If we compare PhishMe with our proposed game, it turns apparent that our version covers many other dimensions of an attack scenario. For example, we have used game-based learning techniques (discussed in game design and rationale section). Moreover, we have not only focused on the context but also understood human needs as one of the means via which attackers manipulate people into installing malware or providing important information. Players by wearing the hat of an attacker can use various persuasion technique(s), attack vector and human needs to develop phishing emails. Phishing attacks by using emails can never have a zero success rate because of; i) new employees joining and transferring between companies; ii) new emotional triggers discovered; iii) new deception techniques, etc. In order to train our players, we gave them a list of URLs (phished vs legitimate) so that they can learn and identify the phished one by observation.

Wombat security10

¹⁰

https://www.wombatsecurity.com

is also a commercial organization that focuses on preventing people from falling for phished emails. Wombat developed a game-based solution known as Anti-Phishing phyllis. In the game, phyllis character teaches various important sections in emails that can be used to identify fraudulent emails. In the exercise, email is shown on a screen and a bubble appears at various places on the email. When the player hovers the pointer over that bubble, a box pops up and displays the options of “ignore” and “disharm.” Depending upon the option selected and the original correct answer, a tick or cross sign appears, signifying a correct or wrong attempt. Furthermore, it explains why the option selected is correct or wrong. In one single round, the player has to review three emails and, after the round, a summary is presented for player’s recap. If we compare Anti-Phishing phyllis with our proposed game, the main difference is that of the way in which the problem is dealt with. In wombat game, the player tackles the situation as a user − as a passive subject, but in our game, the player has to live the role of an attacker (active interaction) and use all the important lessons to draft a phished email. Definitely and additionally, our game includes a team-based learning component which is not present in these games.

Table 11

PhishI comparison with other similar games

Aspects	Characteristics	PhishI	Anti-phishing phil [62]	PhishGuru [44]	Smells phishy [9]
Role Playing	(Attack) Characters in game	✓	x	x	✓
Security context	Story line of the game	✓	✓	✓	✓
	Dynamic Nature of Map (Changeability)	some+	x	x	some+
	Map Used in the game for reference	✓	some+	x	✓
	Players play by moving on the Map	x	✓	x	x
Game element	Different type of Attack Cards	✓	x	x	✓
Game element	Dice for Randomness	x	x	x	✓
Security knowledge area	Game address social engineering issues	some+	some+	some+	some+
	Game educate network security related issues	x	x	x	x
	Game educate physical security related issues	x	x	x	x
Security mechanism	Making scenario	✓	x	✓	✓
	Attack mechanics	✓	x	x	✓
	Defence mechanics	x	x	x	some+
Game design	Multi-player	✓	x	x	x
	Digital game	x	✓	some+	x
	Game design base from research literature	✓	x	x	x
Targeted learning areas	Educating regarding spear phishing	✓	x	some+	x
	Educating regarding information disclosure	✓	x	x	x
	Eliciting phishing security requirements	✓	x	x	x
	Online social media information can be misused	✓	x	x	x
Team-based learning	Discussion session (knowledge/experience sharing)	✓	x	x	x

Table 12

PhishI comparison with other games adapted from [76,77]

Aspects	Characteristics	PhishI	CSRAG [77]	Ctrl-ALt-Hack [18]	Social engineering [10]	Dox3d
Role Playing	(Attack) Characters in game	✓	✓	✓	x	✓
Security context	Story line of the game	✓	✓	✓	x	✓
	Dynamic nature of map (changeability)	some+	some+	x	x	✓
	Map used in the game for reference	✓	✓	x	✓	✓
	Players play by moving on the map	x	✓	x	x	✓
Security mechanism	Attack mechanics	✓	✓	✓	✓	✓
	Defence mechanics	x	x	x	x	✓
	Making scenario	✓	✓	x	✓	x
Game element	Different type of attack cards	✓	✓	some+	✓	x
Game element	Dice for randomness	x	✓	✓	x	x
Security knowledge area	Social engineering issues	✓	✓	some+	✓	some+
	Network security related issues	x	✓	some+	x	✓
	Physical security related issues	x	✓	some+	x	some+
Security protection target	Mission for the team and player	✓	✓	✓	x	✓
Team-based learning	Discussion session	✓	✓	some+	✓	some+
Evaluation design	Different methods for evaluation	✓	✓	x	x	x
Targeted learning areas	Educating regarding spear phishing	✓	x	x	some+	x
	Educating regarding information disclosure	✓	x	x	x	x
	Eliciting phishing security requirements	✓	x	x	x	x
	Online social media information can be misused	✓	x	x	x	x

Comparison of awareness games: All of the tools which are developed to date have failed to reduce phishing attacks as the strategy, process, and techniques of phishing attacks are evolving with the advancement in technology. Moreover, it has been discovered that the best anti-phishing tools lose more than 20% of the phishing sites [16,19,62]. The deficiency is reinforced by the fact that most systems are dependent on individuals who make confidential decisions when they perform activities on the internet. The researchers of this study played the games mentioned in Table 11 & 12 and further summarized the characteristics of all the games based on various aspects such as security context, security mechanism, targeted learning areas, etc. Table 11 gives an in-depth comparison of the similar phishing awareness games, and on the other hand, a comprehensive comparison of cyber security (general) is shown in Table 12, which focus on generic cyber security awareness games. For all the non-digital games such as Dox3d, and Ctrl-Alt-Hack etc the games were ordered online. On the other hand, digital games mentioned in Table 11 were played online. Researchers used Delphi approach, it is used where there is a disagreement between the individual subjective analyses. If we look closely, we can further analyze that overall PhishI provides good coverage of the characteristics as compared to other mentioned games. Overall, we believe that PhishI is a game which not only trains players with spear phishing related concepts but also focuses on information disclosure awareness. In Table 11 & 12, ‘Some+’ identifies that this phase is present to a good extent or present in some scenarios of the game. ‘✓’ represents the complete presence of that phase in the game, and ‘X’ represents the absence of that phase from the game.

R. Zhao et al. [79], designed a toolkit that could automatically create fraudulent websites which seemed much like their original counterparts. Zhao et al. experimented on 194 users to check the usefulness of this toolkit. After the experiment, the results showed that more than 90% users became a target of illegitimate websites. Zhao et al believes that phishing attacks in future will increase and there is strong need to defend these attacks collectively. In order to minimize the phishing attacks our game offers many aspects as compared to other similar games discussed in Table 11 & 12: i) players adopt the role of attacker in the game environment and think from the view of an attacker, which helps them understand the perspective of the attacker first-hand; ii) team-based learning and discussion helps to better understand the situation and other concepts; iii) our game provides a know-how about persuasion techniques, attack vectors, and human weaknesses which are useful in deciphering phishing emails in real life; iv) the use of social media (cards) to collect relevant information helps players understand how excessive online information disclosure can be utilized by the attackers.

6. Conclusion

We are living in a world where physical conflicts have now moved to the cyber space, which is becoming an ever more insecure place [46]. In this situation, security awareness is of vital importance. Thus, this makes it more necessary than ever to devise a mechanism for better security awareness [51]. To draw the attention of all user groups, we need a method that is more engaging and easy to use [30]. We know that awareness training methods are only effective if used properly [51]. In the past decades, more and more effort is spent on making the operating system and network environment secure. Due to this, attackers have changed their targets from information systems to human elements to break into organizations. The number of attacks in the recent past include attackers who have used social engineering as their method of invasion in target organizations [2].

This paper introduced PhishI as a systematic approach to design serious games for security education. We define a game design framework that integrates the body of knowledge on social engineering, the psychological needs of organizational players, and the candidate game pattern that serve different needs. We use spear phishing as a key example to show how the proposed approach works, and then evaluated the learning effects of the generated game based on empirical data collected from student activity. In PhishI game participants are required to swap phishing emails and be able to comment on the effectiveness of the attack scenario, the effectiveness of this method can be justified as this method of evaluation is used in recent designed games such as [10,77]. Our results showed that students’ awareness of spear phishing risks is improved and that the resistance to potential first attack contact is enhanced. Furthermore, the game showed positive effect on participants’ understanding of excessive online information disclosure.

In future, we are planning to design and develop an automated tool supporting the game composition process, with knowledge repository of known phishing attack scenarios. Thus, some functionalities as shown by of the game PhishI will be shifted online.

Footnotes

Acknowledgments

Financial support from the Natural Science Foundation of China Project no. 61432020 is gratefully acknowledged. We thank Awaid Yasin for reviewing the paper.

Phishing email generated by players

The players were given a context of a company and according to the situation and various variables such as Human Asset (target asset) responsibility in organization, his/her interest (social media information), compliance principle, players need to generate an email. Below mentioned are some of the phished emails generated by the players during game play. These emails generated by the players are shown in Table 13, 14, 15, 16. 17 for understanding.

Post activity survey

The post survey questionnaire can be seen in the Table 18.

References

Hospitals become major target for ransomware, Network Security 2016(4) (2016), 1–2, https://www-sciencedirect-com.web.bisu.edu.cn/science/article/pii/S1353485816300319. doi:10.1016/S1353-4858(16)30031-9.

Abawajy, User preference of cyber security awareness delivery methods, Behaviour & Information Technology 33(3) (2014), 237–248. doi:10.1080/0144929X.2012.708787.

Abdullah,

Ward and

Ahmed, Investigating the influence of the most commonly used external variables of {TAM} on students’ perceived ease of use (PEOU) and perceived usefulness (PU) of e-portfolios, Computers in Human Behavior 63 (2016), 75–90, https://www-sciencedirect-com.web.bisu.edu.cn/science/article/pii/S0747563216303387 . doi:10.1016/j.chb.2016.05.014.

S.M.

Albladi and

G.R.S.

Weir, A semi-automated security advisory system to resist cyber-attack in social networks, in: Computational Collective Intelligence,

N.T.

Nguyen,

Pimenidis,

Khan and

Trawiński, eds, Springer International Publishing, Cham, 2018, pp. 146–156. ISBN 978-3-319-98443-8. doi:10.1007/978-3-319-98443-8_14.

Aleroud and

Zhou, Phishing environments, techniques, and countermeasures: A survey, Computers & Security 68 (2017), 160–196, https://www-sciencedirect-com.web.bisu.edu.cn/science/article/pii/S0167404817300810 . doi:10.1016/j.cose.2017.04.006.

N.A.G.

Arachchilage and

Cole, Design a mobile game for home computer users to prevent from phishing attacks, in: International Conference on Information Society (i-Society 2011), 2011, pp. 485–489.

N.A.G.

Arachchilage and

Love, A game design framework for avoiding phishing attacks, Computers in Human Behavior 29(3) (2013), 706–714, https://www-sciencedirect-com.web.bisu.edu.cn/science/article/pii/S0747563212003585 . doi:10.1016/j.chb.2012.12.018.

N.A.G.

Arachchilage and

Love, Security awareness of computer users: A phishing threat avoidance perspective, Computers in Human Behavior 38(Supplement C) (2014), 304–312, https://www-sciencedirect-com.web.bisu.edu.cn/science/article/pii/S0747563214003331 . doi:10.1016/j.chb.2014.05.046.

Baslyman and

Chiasson, “Smells phishy?”: An educational game about online phishing scams, in: 2016 APWG Symposium on Electronic Crime Research (eCrime), 2016, pp. 1–11. doi:10.1109/ECRIME.2016.7487946.

10.

Beckers and

Pape, A serious game for eliciting social engineering security requirements, in: 24th IEEE International Requirements Engineering Conference, RE 2016, Beijing, China, September 12–16, 2016, IEEE, 2016, pp. 16–25. doi:10.1109/RE.2016.39.

11.

Bergholz,

De Beer,

Glahn,

Moens,

Paaß and

Strobel, New filtering approaches for phishing email, Journal of Computer Security 18(1) (2010), 7–35. doi:10.3233/JCS-2010-0371.

12.

Bullee,

Montoya,

Junger and

P.H.

Hartel, Spear phishing in organisations explained, Inf. & Comput. Security 25(5) (2017), 593–613. doi:10.1108/ICS-03-2017-0009.

13.

D.B.

Buller and

J.K.

Burgoon, Interpersonal deception theory, Communication Theory 6(3) (1996), 203–242. doi:10.1111/j.1468-2885.1996.tb00127.x.

14.

C.-C.

Chang,

Liang,

P.-N.

Chou and

G.-Y.

Lin, Is game-based learning better in flow experience and various types of cognitive load than non-game-based learning? Perspective from multimedia and media richness, Computers in Human Behavior 71(Supplement C) (2017), 218–227, https://www-sciencedirect-com.web.bisu.edu.cn/science/article/pii/S0747563217300377 . doi:10.1016/j.chb.2017.01.031.

15.

K.L.

Chiew,

K.S.C.

Yong and

C.L.

Tan, A survey of phishing attacks: Their types, vectors and technical approaches, Expert Systems with Applications 106 (2018), 1–20, https://www-sciencedirect-com.web.bisu.edu.cn/science/article/pii/S0957417418302070 . doi:10.1016/j.eswa.2018.03.050.

16.

L.F.

Cranor,

Egelman,

J.I.

Hong and

Zhang, Phinding phish: An evaluation of anti-phishing toolbars, in: Proceedings of the Network and Distributed System Security Symposium, NDSS 2007, San Diego, California, USA, 28th February–2nd March 2007, The Internet Society, 2007, http://www.isoc.org/isoc/conferences/ndss/07/papers/phinding_phish.pdf .

17.

De Kimpe,

Walrave,

Hardyns,

Pauwels and

Ponnet, You’ve got mail! Explaining individual differences in becoming a phishing target, Telematics and Informatics 35(5) (2018), 1277–1287, https://www-sciencedirect-com.web.bisu.edu.cn/science/article/pii/S0736585317304677 . doi:10.1016/j.tele.2018.02.009.

18.

Denning,

Lerner,

Shostack and

Kohno, Control-Alt-Hack: The design and evaluation of a card game for computer security awareness and education, in: Proceedings of the 2013 ACM SIGSAC Conference on Computer & Communications Security, CCS ’13, ACM, New York, NY, USA, 2013, pp. 915–928. ISBN 978-1-4503-2477-9. doi:10.1145/2508859.2516753.

19.

Dhamija,

J.D.

Tygar and

M.A.

Hearst, Why phishing works, in: Proceedings of the 2006 Conference on Human Factors in Computing Systems, CHI 2006, Montréal, Québec, Canada, April 22–27, 2006,

R.E.

Grinter,

Rodden,

P.M.

Aoki,

Cutrell,

Jeffries and

G.M.

Olson, eds, ACM, 2006, pp. 581–590. doi:10.1145/1124772.1124861.

20.

Dix,

J.E.

Finlay,

G.D.

Abowd and

Beale, Human–Computer Interaction, 3rd edn, Pearson. ISBN 978-0-13-046109-4.

21.

J.S.

Downs,

M.B.

Holbrook and

L.F.

Cranor, Decision strategies and susceptibility to phishing, in: Proceedings of the Second Symposium on Usable Privacy and Security, SOUPS ’06, ACM, New York, NY, USA, 2006, pp. 79–90. ISBN 1-59593-448-0. doi:10.1145/1143120.1143131.

22.

Dunwell,

Petridis,

Arnab,

Protopsaltis,

Hendrix and

de Freitas, Blended game-based learning environments: Extending a serious game into a learning content management system, in: 2011 Third International Conference on Intelligent Networking and Collaborative Systems, IEEE, 2011, pp. 830–835. doi:10.1109/INCoS.2011.58.

23.

Edwards,

Larson,

Green,

Rashid and

Baron, Panning for gold: Automatically analysing online social engineering attack surfaces, Computers & Security 69 (2017), 18–34. doi:10.1016/j.cose.2016.12.013.

24.

Ferreira,

L.M.

Coventry and

Lenzini, Principles of persuasion in social engineering and their use in phishing, in: HCI (22), Lecture Notes in Computer Science, Vol. 9190, Springer, 2015, pp. 36–47.

25.

Fette,

Sadeh and

Tomasic, Learning to detect phishing emails, in: Proceedings of the 16th International Conference on World Wide Web, WWW ’07, ACM, New York, NY, USA, 2007, pp. 649–656. ISBN 978-1-59593-654-7. doi:10.1145/1242572.1242660.

26.

N.H.

Flores,

A.C.R.

Paiva and

Letra, Software engineering management education through game design patterns, Procedia – Social and Behavioral Sciences 228(Supplement C) (2016), 436–442, 2nd International Conference on Higher Education Advances, HEAd’16, 21–23 June 2016, València, Spain, https://www-sciencedirect-com.web.bisu.edu.cn/science/article/pii/S1877042816309934. doi:10.1016/j.sbspro.2016.07.067.

27.

W.R.

Flores and

Ekstedt, Shaping intention to resist social engineering through transformational leadership, information security culture and awareness, Computers & Security 59 (2016), 26–44. doi:10.1016/j.cose.2016.01.004.

28.

Freese, Game-based learning: An approach for improving collaborative airport management, in: European Conference on Games Based Learning, Academic Conferences International Limited, 2016, p. 835.

29.

Fuentes,

J.A.

Álvarez,

J.A.

Ortega,

L.G.

Abril and

Velasco, Trojan horses in mobile devices, Comput. Sci. Inf. Syst. 7(4) (2010), 813–822. doi:10.2298/CSIS090330027F.

30.

Giannakas,

Kambourakis and

Gritzalis, CyberAware: A mobile game-based app for cybersecurity education and awareness, in: Interactive Mobile Communication Technologies and Learning (IMCL), 2015 International Conference on, IEEE, 2015, pp. 54–58. doi:10.1109/IMCTL.2015.7359553.

31.

Güleç,

Yilmaz and

M.A.

Gozcu, Bireylerin Programlama Yeteneklerini ve Bilgi Seviyelerini Arttirmak Amaciyla Dusunulmus Ciddi Oyun Tabanli Ogrenme Catisi – CENGO(Serious Game-Based Learning Framework to Improve Programming Skills and Knowledge Levels of Individuals – CENGO), in: Proceedings of the 11th Turkish National Software Engineering Symposium, Alanya, Turkey, October 18–20, 2017,

Ç.

Turhan,

Coskunçay,

Yazici and

Oguztüzün, eds, CEUR Workshop Proceedings, Vol. 1980, CEUR-WS.org, 2017, pp. 171–183, http://ceur-ws.org/Vol-1980/UYMS17_paper_8.pdf .

32.

Gupta,

Singhal and

Kapoor, A literature survey on social engineering attacks: Phishing attack, in: 2016 International Conference on Computing, Communication and Automation (ICCCA), 2016, pp. 537–540. doi:10.1109/CCAA.2016.7813778.

33.

M.L.

Hale,

R.F.

Gamble and

Gamble, CyberPhishing: A game-based platform for phishing awareness testing, in: 2015 48th Hawaii International Conference on System Sciences, 2015, pp. 5260–5269, ISSN 1530-1605. doi:10.1109/HICSS.2015.670.

34.

Heartfield and

Loukas, Detecting semantic social engineering attacks with the weakest link: Implementation and empirical evaluation of a human-as-a-security-sensor framework, Computers & Security 76 (2018), 101–127, https://www-sciencedirect-com.web.bisu.edu.cn/science/article/pii/S0167404818301780 . doi:10.1016/j.cose.2018.02.020.

35.

Heartfield,

Loukas and

Gan, You are probably not the weakest link: Towards practical prediction of susceptibility to semantic social engineering attacks, IEEE Access 4 (2016), 6910–6928. doi:10.1109/ACCESS.2016.2616285.

36.

Hellaoui,

Koudil and

Bouabdallah, Energy-efficient mechanisms in security of the Internet of things: A survey, Computer Networks 127(Supplement C) (2017), 173–189, https://www-sciencedirect-com.web.bisu.edu.cn/science/article/pii/S1389128617303146 . doi:10.1016/j.comnet.2017.08.006.

37.

Host’oveckỳ and

Novák, Game-based learning: How to make math more attractive by using of serious game, in: Computer Science on-Line Conference, Springer, 2017, pp. 341–350.

38.

Jakobsson (ed.), Understanding Social Engineering Based Scams, Springer, 2016. ISBN 978-1-4939-6455-0. doi:10.1007/978-1-4939-6457-4.

39.

Jansson and

von Solms, Phishing for phishing awareness, Behaviour & Information Technology 32(6) (2013), 584–593. doi:10.1080/0144929X.2011.632650.

40.

Junger,

Montoya and

F.-J.

Overink, Priming and warnings are not effective to prevent social engineering attacks, Computers in Human Behavior 66 (2017), 75–87, https://www-sciencedirect-com.web.bisu.edu.cn/science/article/pii/S0747563216306392 . doi:10.1016/j.chb.2016.09.012.

41.

Ki-Aries and

Faily, Persona-centred information security awareness, Computers & Security 70 (2017), 663–674, https://www-sciencedirect-com.web.bisu.edu.cn/science/article/pii/S0167404817301566 . doi:10.1016/j.cose.2017.08.001.

42.

Krombholz,

Hobel,

Huber and

Weippl, Advanced social engineering attacks, Journal of Information Security and Applications 22 (2015), 113–122, Special Issue on Security of Information and Networks, https://www-sciencedirect-com.web.bisu.edu.cn/science/article/pii/S2214212614001343. doi:10.1016/j.jisa.2014.09.005.

43.

Kumar, Research Methodology: A Step-by-Step Guide for Beginners, 3rd edn, SAGE Publications Ltd, 2010, https://www.amazon.com/Research-Methodology-Step-Step-Beginners/dp/1446269973/ref=sr_1_2?ie=UTF8&qid=1530868080&sr=8-2&keywords="Research+methodology . ISBN 1849203008, 9781849203005.

44.

Kumaraguru,

Cranshaw,

Acquisti,

Cranor,

Hong,

M.A.

Blair and

Pham, School of phish: A real-world evaluation of anti-phishing training, in: Proceedings of the 5th Symposium on Usable Privacy and Security, SOUPS ’09, ACM, New York, NY, USA, 2009, pp. 3:1–3:12. ISBN 978-1-60558-736-3. doi:10.1145/1572532.1572536.

45.

Kyewski and

N.C.

Krämer, To gamify or not to gamify? An experimental field study of the influence of badges on motivation, activity, and performance in an online learning course, Computers & Education 118(Supplement C) (2018), 25–37, https://www-sciencedirect-com.web.bisu.edu.cn/science/article/pii/S0360131517302506 . doi:10.1016/j.compedu.2017.11.006.

46.

Le Compte,

Elizondo and

Watson, A renewed approach to serious games for cyber security, in: Cyber Conflict: Architectures in Cyberspace (CyCon), 2015 7th International Conference on, IEEE, 2015, pp. 203–216.

47.

L.K.

Marett and

J.F.

George, Deception in the case of one sender and multiple receivers, Group Decision and Negotiation 13(1) (2004), 29–44. doi:10.1023/B:GRUP.0000011943.73672.9b.

48.

Micallef and

N.A.G.

Arachchilage, Changing users’ security behaviour towards security questions: A game based learning approach, in: 2017 Military Communications and Information Systems Conference (MilCIS), IEEE, 2017, pp. 1–6.

49.

Morlok, Sharing is (not) caring – the role of external privacy in users’ information disclosure behaviors on social network sites, in: 20th Pacific Asia Conference on Information Systems, PACIS 2016, Chiayi, Taiwan, June 27–July 1, 2016,

Liang,

Hung,

P.Y.K.

Chau and

S.-I.

Chang, eds, 2016, p. 75, http://aisel.aisnet.org/pacis2016/75 .

50.

Mouton,

Leenen and

H.S.

Venter, Social engineering attack examples, templates and scenarios, Computers & Security 59 (2016), 186–209. doi:10.1016/j.cose.2016.03.004.

51.

Nagarajan,

J.M.

Allbeck,

Sood and

T.L.

Janssen, Exploring game design for cybersecurity training, in: Cyber Technology in Automation, Control, and Intelligent Systems (CYBER), 2012 IEEE International Conference on, IEEE, 2012, pp. 256–262. doi:10.1109/CYBER.2012.6392562.

52.

Naik, A comparative evaluation of game-based learning: Digital or non-digital games? in: European Conference on Games Based Learning, Vol. 2, Academic Conferences International Limited, 2014, p. 437.

53.

Naik, Non-digital game-based learning in the teaching of mathematics in higher education, in: European Conference on Games Based Learning, Vol. 2, Academic Conferences International Limited, 2014, p. 431.

54.

Paradise,

Shabtai and

Puzis, Detecting organization-targeted socialbots by monitoring social network profiles, Networks and Spatial Economics (2018), 1–31.

55.

Petersen and

Gencel, Worldviews, research methods, and their relationship to validity in empirical software engineering research, in: 2013 Joint Conference of the 23rd International Workshop on Software Measurement and the 8th International Conference on Software Process and Product Measurement, 2013, pp. 81–89. doi:10.1109/IWSM-Mensura.2013.22.

56.

Qian and

K.R.

Clark, Game-based learning and 21st century skills: A review of recent research, Computers in Human Behavior 63(Supplement C) (2016), 50–58, https://www-sciencedirect-com.web.bisu.edu.cn/science/article/pii/S0747563216303491 . doi:10.1016/j.chb.2016.05.023.

57.

Robles,

Norris,

Watson and

A.F.

Browne, Survey of non-malicious user actions that introduce network and system vulnerabilities and exploits, in: SoutheastCon 2018, 2018, pp. 1–5, ISSN 1558-058X. doi:10.1109/SECON.2018.8478938.

58.

Sailer,

J.U.

Hense,

S.K.

Mayr and

Mandl, How gamification motivates: An experimental study of the effects of specific game design elements on psychological need satisfaction, Computers in Human Behavior 69 (2017), 371–380. doi:10.1016/j.chb.2016.12.033.

59.

L.D.

Salay,

Ishiko and

A.D.

Huberman, A midline thalamic circuit determines reactions to visual threat, Nature 557(7704) (2018), 183–189, http://www.nature.com/articles/s41586-018-0078-2 . doi:10.1038/s41586-018-0078-2.

60.

M.N.K.

Saunders, Research Methods for Business Students, Pearson Education Limited, Harlow, Essex, England, 2016. ISBN 978-1292016627.

61.

Schaab,

Beckers and

Pape, Social engineering defence mechanisms and counteracting training strategies, Inf. & Comput. Security 25(2) (2017), 206–222. doi:10.1108/ICS-04-2017-0022.

62.

Sheng,

Magnien,

Kumaraguru,

Acquisti,

L.F.

Cranor,

J.I.

Hong and

Nunge, Anti-Phishing Phil: The design and evaluation of a game that teaches people not to fall for phish, in: Proceedings of the 3rd Symposium on Usable Privacy and Security, SOUPS 2007, Pittsburgh, Pennsylvania, USA, July 18–20, 2007,

L.F.

Cranor, ed., ACM International Conference Proceeding Series, Vol. 229, ACM, 2007, pp. 88–99. doi:10.1145/1280680.1280692.

63.

Siadati,

Nguyên,

Gupta,

Jakobsson and

N.D.

Memon, Mind your SMSes: Mitigating social engineering in second factor authentication, Computers & Security 65 (2017), 14–28. doi:10.1016/j.cose.2016.09.009.

64.

Silic and

Back, The dark side of social networking sites: Understanding phishing risks, Computers in Human Behavior 60(Supplement C) (2016), 35–43, http://eproxy2.lib.tsinghua.edu.cn:80/rwt/33/http/P75YPLUUMNVXK5UDMWTGT6UFMN4C6Z5QNF/science/article/pii/S0747563216301029 . doi:10.1016/j.chb.2016.02.050.

65.

Steer, Defending against spear-phishing, Computer Fraud & Security 2017(8) (2017), 18–20, https://www-sciencedirect-com.web.bisu.edu.cn/science/article/pii/S136137231730074X . doi:10.1016/S1361-3723(17)30074-X.

66.

R.B.

Svensson and

Regnell, Is role playing in requirements engineering education increasing learning outcome?, Requirements Engineering 22(4) (2017), 475–489. doi:10.1007/s00766-016-0248-4.

67.

Tang,

Bex,

Schriek and

J.M.E.M.

van der Werf, Improving software design reasoning – a reminder card approach, Journal of Systems and Software 144 (2018), 22–40, https://www-sciencedirect-com.web.bisu.edu.cn/science/article/pii/S0164121218301043 . doi:10.1016/j.jss.2018.05.019.

68.

Tetri and

Vuorinen, Dissecting social engineering, Behaviour & Information Technology 32(10) (2013), 1014–1023. doi:10.1080/0144929X.2013.763860.

69.

H.S.

Tsai,

Jiang,

Alhabash,

LaRose,

N.J.

Rifon and

S.R.

Cotten, Understanding online safety behaviors: A protection motivation theory perspective, Computers & Security 59 (2016), 138–150, https://www-sciencedirect-com.web.bisu.edu.cn/science/article/pii/S0167404816300190 . doi:10.1016/j.cose.2016.02.009.

70.

S.S.

Tseng,

K.Y.

Chen,

T.J.

Lee and

J.F.

Weng, Automatic content generation for anti-phishing education game, in: 2011 International Conference on Electrical and Control Engineering, 2011, pp. 6390–6394. doi:10.1109/ICECENG.2011.6056921.

71.

Van der Merwe, Scenario-based strategy in practice: A framework, Advances in Developing Human Resources 10(2) (2008), 216–239. doi:10.1177/1523422307313321.

72.

Vogeler, Game-based learning with OER in higher education: Development and evaluation of a serious game, in: European Conference on e-Learning, Academic Conferences International Limited, 2018, pp. 592–XX.

73.

Wohlin,

Runeson,

Höst,

M.C.

Ohlsson and

Regnell, Experimentation in Software Engineering, Springer, 2012. ISBN 978-3-642-29043-5. doi:10.1007/978-3-642-29044-2.

74.

Wu,

R.C.

Miller and

S.L.

Garfinkel, Do security toolbars actually prevent phishing attacks? in: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’06, ACM, New York, NY, USA, 2006, pp. 601–610. ISBN 1-59593-372-7. doi:10.1145/1124772.1124863.

75.

Yaqoob,

Ahmed,

M.H.

Rehman,

A.I.A.

Ahmed,

M.A.

Al-garadi,

Imran and

Guizani, The rise of ransomware and emerging security challenges in the Internet of things, Computer Networks (2017), http://eproxy2.lib.tsinghua.edu.cn:80/rwt/33/http/P75YPLUUMNVXK5UDMWTGT6UFMN4C6Z5QNF/science/article/pii/S1389128617303468. doi:10.1016/j.comnet.2017.09.003.

76.

Yasin,

Liu,

Li,

Fatima and

Jianmin, Improving software security awareness using a serious game, IET Software (2018), http://digital-library.theiet.org/content/journals/10.1049/iet-sen.2018.5095.

77.

Yasin,

Liu,

Li,

Wang and

Zowghi, Design and preliminary evaluation of a cyber security requirements education game (SREG), Information and Software Technology 95 (2018), 179–200, https://www-sciencedirect-com-443.web.bisu.edu.cn/science/article/pii/S0950584917301921 . doi:10.1016/j.infsof.2017.12.002.

78.

Zhang,

J.I.

Hong and

L.F.

Cranor, Cantina: A content-based approach to detecting phishing web sites, in: Proceedings of the 16th International Conference on World Wide Web, WWW ’07, ACM, New York, NY, USA, 2007, pp. 639–648. ISBN 978-1-59593-654-7. doi:10.1145/1242572.1242659.

79.

Zhao,

John,

Karas,

Bussell,

Roberts,

Six,

Gavett and

Yue, Design and evaluation of the highly insidious extreme phishing attacks, Computers & Security 70 (2017), 634–647, http://eproxy2.lib.tsinghua.edu.cn:80/rwt/33/http/P75YPLUUMNVXK5UDMWTGT6UFMN4C6Z5QNF/science/article/pii/S0167404817301633 . doi:10.1016/j.cose.2017.08.008.

80.

Zhitomirsky-Geffet and

Bratspiess, Professional information disclosure on social networks: The case of Facebook and LinkedIn in Israel, Journal of the Association for Information Science and Technology 67(3) (2016), 493–504. doi:10.1002/asi.23393.

How persuasive is a phishing email? A phishing game for phishing awareness

Abstract

Context:

Objective:

Method:

Conclusion:

Keywords

1. Introduction

1 https://info.phishlabs.com/blog/2019-phishing-trends-intelligence-report-the-evolving-threat

2.1. Game elements

2.1.1. Scenario based learning – map/floor plan & assets

2.2. Game process

2.3.1. Challenges

2.3.2. Win/lose conditions

2.3.3. What and where to teach in game:

3.1. Recruitment process

3.2. Empirical evaluation design

5 The responses which had missing data, or outliers (the participants selected same scale for all the survey questions), etc. were removed.

4.1. Observation

4.2.1. Developing and reviewing PhishI emails

4.2.2. Game feedback & survey [43,60]

4.3. Limitations and validity threats

5. Related work on phishing awareness techniques

5.1. Commercial tools and table comparison

9 https://cofense.com

Footnotes

Acknowledgments

Phishing email generated by players

Post activity survey

References

¹
https://info.phishlabs.com/blog/2019-phishing-trends-intelligence-report-the-evolving-threat

⁵
The responses which had missing data, or outliers (the participants selected same scale for all the survey questions), etc. were removed.

⁹
https://cofense.com