Dark Play of Serious Games: Effectiveness and Features (G4HE2018)

Abstract

Objective:

Choosing inappropriate or unethical actions in games is referred to as dark play. For a serious game on delirium for medical students, we aimed to investigate the potential differences between dark play and normal play on game effectiveness regarding abilities in advising care, learning motivation and engagement, and attitude toward delirious patients. Furthermore, we aimed to explore the use of different game features between the two types of play on empathy, self-efficacy, and consequences of care.

Methods:

We performed a two-arm randomized controlled trial including an exploratory qualitative approach with 157 medical students, who played the serious game “The Delirium Experience.” Participants were randomly allocated to either the dark play or normal play group. Participants had to give three recommendations for taking care of delirious patients, and complete both the Delirium Attitude Scale, and Learning Motivation and Engagement Questionnaire to study game effectiveness. To explore game features, open questions were asked.

Results:

We did not find difference between the two types of play in game effectiveness. “Patient's and nurse's perspective” seem to be an important game feature for being able to empathize with a patient in both groups. To support self-efficacy, “practice how to care” and “feedback in the game” were important in both study groups. “Being able to see the importance of good interaction with the patient” was reported important for self-efficacy in the dark play group, whereas this was “seeing the consequences of care” in the normal play group.

Conclusions:

There seems to be no change to game effectiveness when providing players the opportunity to use dark play in a serious game. A realistic view of another person's perspective could be an important game feature to increase empathy.

Introduction

Autonomy in gameplay and the corresponding freedom of choice often increases player satisfaction and provides a feeling of control.¹ However, this freedom of choice also gives players the opportunity to choose inappropriate or unethical actions in the game. Choosing inappropriate or unethical actions in a game is referred to as dark play. In a dark play situation, players use behavior in the game that in a normal situation would be unacceptable.² Because of the presence of autonomy in (video) games, dark play is an option to choose in almost all games. In serious game research, little attention has been paid to dark play as studies have mainly focused on serious games as a whole with a specific overall outcome (e.g., learning) and normal gameplay.^7–11 Many studies have investigated unethical or immoral actions in video games on potential adverse outcomes, and it seems that video games may increase antisocial behavior.^3,4 On the contrary, research has also demonstrated that exhibiting immoral behavior in video games leads to an improved awareness of moral norms,⁵ and players do not disengage from their morals during gameplay.⁶ It is unknown whether dark play affects game effectiveness. Furthermore, dark play provides an opportunity to explore various game features of serious games instead of studying serious games as a whole with a specific overall outcome.

One might argue that dark play in a serious game may increase the risk that students learn incorrect behavior, thereby risking a decrease in game effectiveness. However, learning involves observing behavior and the consequences of that particular behavior.¹² Serious games are intended to teach the player something, so the inappropriate actions in a serious game lead to negative consequences that are observed by the players and they receive feedback on their behavior. One could therefore hypothesize that the risk of potentially adverse effects of dark play in serious games might be low, but this needs to be studied.

Exploring different game features (e.g., competition/perspectives) is important to improve serious game design as game features define gameplay.^8,9,13,14 We define game features as a combination of game mechanics (components of the game) and dynamics (how the game continues depending on actions of the player).⁹ As the game mechanisms allow players to have autonomy, the game dynamics allow players to choose for dark play in the game. The type of play (normal or dark) of a serious game could emphasize different game features as the game progresses differently, allowing us to explore the use of those different features.

“The Delirium Experience” is a serious game that makes use of interactive video simulation with several gameplays, and is intended to train and educate medical students and staff on how to take better care of delirious patients.¹⁵ Delirium is an acute neuropsychiatric syndrome often experienced by older hospitalized patients, which has serious consequences, such as increased length of hospital stay, functional decline, institutionalization, and increased mortality.¹⁶ Adequate recognition and management of delirium is important to reduce its incidence and severity,^17,18 for which education and training of medical students and staff are needed.^17,19 The Delirium Experience focuses on delirium from both the patient's and caregiver's perspectives (trailer: www.youtube.com/watch?v=A-lLLP8Me0E).¹⁵ In The Delirium Experience, dark play results in adverse events and scenarios with an extremely frightened patient experiencing severe delirious episodes. Previous research with medical students has shown that The Delirium Experience increases the ability in advising care for delirious patients, self-reported knowledge on delirium, and learning motivation and engagement.²⁰ Because the potential influence of dark play was not studied in this previous research, we conducted a new trial.

The aim of this study was thus to examine the potential differences between dark play and normal play on game effectiveness regarding abilities in advising care for delirious patients, learning motivation and engagement, and attitude toward delirious patients. Furthermore, we aimed to explore the use of different game features between the two types of play on empathy for the patient, self-efficacy in the learner, and consequences of care.

Methods

Procedure, design, and study population

We performed a two-arm randomized controlled trial to study game effectiveness, including an explorative qualitative approach to explore the use of different game features in dark and normal play. Students were recruited among third year Bachelor Medical Sciences students who enrolled in an elective practical on delirium. To be included in this study, participants had to (a) be in their third year of preclinical education in December 2017 at the University Medical Center Groningen (UMCG), (b) enroll in the practical on delirium, and (c) give informed consent.

Students started with an optional conventional lecture on delirium. In addition, students could voluntarily sign up for the practical on delirium, in which the study conditions took place. Students were informed about the study in the information text about the practical. All students were given the opportunity to join the practical on delirium, even if they did not wish to participate in the study. This trial was not necessary to register according to the ICMJE (International Committee of Medical Journal Editors) guidelines, as our participants consisted of medical students who could voluntarily sign up for both the practical and the study. We used SurveyMonkey²¹ to ask students to give informed consent, provide anonymous digital questionnaires, and randomly allocate participants into one of the study arms (ratio 1:1). Before the intervention started, all participants completed questions on age, sex, experience with caring for older and delirious patients (slide-bar 0: no experience—100: a lot of experience), and whether they had attended the conventional lecture. Participants were asked to play The Delirium Experience twice: the first time to get familiar with the game, the second time randomly allocated in SurveyMonkey to either the dark or the normal play instruction. Players in the “dark play” condition were instructed to “take actions as a healthcare professional within the game that would make the symptoms of delirium as severe as possible.” Players in the “normal play” condition were instructed to try to “achieve high scores and provide the patient with the best possible care.”

The practical took place in classrooms of the UMCG. Participants were not allowed to talk during the practical to ensure they were not aware of the second study condition. Researchers were not aware of the assignment participants received. Because of the new and exploratory nature of this study we had no previous expectations of differences we could expect and therefore we did not perform a power calculation. From previous research we knew that ∼150 students would participate in this practical and expected this to be sufficient.

Intervention

In The Delirium Experience, the player experiences 4 days as a caregiver and the corresponding nights as the patient. During daytime, the player has to take care of a delirious patient and can choose different actions regarding medical care. Depending on the actions chosen, the delirious episodes of the patient differ in severity and the player gets different options to choose from the next day. Hence, if the player performs poorly as a caregiver the severity of the delirious episodes increases, which is referred to as dark play if this is done on purpose. Players receive feedback at the end of each day in the game on how they performed caring for a delirious patient and are given information how they could improve their care before they switch to the patient's perspective. As the game progresses differently depending on the choices of the player, it is possible to play either with dark or normal play.

Outcomes

Educational interventions on delirium should focus on translating obtained knowledge into practice and target attitudes toward delirious patients.^22,23 Furthermore, learning motivation and engagement are important as these can influence learning outcomes.²⁴ Therefore, we chose the following measures of game effectiveness: (a) abilities in advising care, (b) learning motivation and engagement, and (c) attitude toward delirious patients. To measure abilities in advising care for delirious patients, participants were asked to give three written recommendations in which participants describe how they would manage delirium in practice. We used a predefined rubric to assess the given recommendations: 0 (incorrect/not mentioned), 1 (topic mentioned), 2 (nonspecific recommendation), or 3 (specific recommendation) points per criterion (range 0–9; weighted kappa = 0.84).²⁰ Learning motivation and engagement was measured with the Motivation and Engagement Questionnaire to evaluate learning experiences (range 9–45 points; Cronbach's α = 0.92).²⁵ Attitude toward delirious patients was measured with the Delirium Attitude Scale (range 19–133 points; Cronbach's α = 0.79), adapted from the Dementia Attitude Scale as described in our previous study.²⁰

To explore different game features of dark play and normal play of The Delirium Experience, all participants were asked to answer three open questions, after playing the game. Questions were asked in the SurveyMonkey questionnaire and consisted of two lines of open text fields. The questions were formulated by three researchers (K.B.S., D.J., Sd.R.) and two medical students, and focused on game features considered important by the participants on the following: (a) empathy: “Could you experience what a delirious patient endures? If yes: what are the most important features of Delirium Experience accountable for this?,” (b) self-efficacy: “After playing Delirium Experience, did you feel better prepared to take care of a delirious patient? If yes: what are the most important features of Delirium Experience accountable for this?,” and (c) consequences of care: “By using Delirium Experience, did you gain insight in the consequences of your actions in taking care of a delirious patient? If yes: which insights did you gain?.” These concepts were chosen for exploration as educational aids on delirium should focus on empathizing with the patient and understanding the patients' needs, and the students' need to gain insight into how their care influences the patient.^22,23 Furthermore, self-efficacy has important implications for clinical performance.²⁶ Participants were also asked which game features they would like to see in The Delirium Experience, to provide insights into missing game features for improving empathy, self-efficacy, and consequences of care for future serious game development.

Data analysis

Data were checked for normality by judging histograms, skewness, and kurtosis. Continuous variables were analyzed with an independent sample t-test in cases of normal distribution, and a Mann–Whitney U test in cases of non-normal distribution. Values of P < 0.05 were considered statistically significant. Answers to the open questions were thematically analyzed with Atlas.ti 8 through constant comparison with open coding (categorizing data; K.B.S. and D.J.), axial coding (connecting themes; K.B.S.), and selective coding (selecting core themes; K.B.S. and Sd.R.). Percentages of how often a theme was used in each group were calculated. The three most reported themes were collected for both groups. During data coding, the researcher was blinded to the study condition.

Results

In total, 159 students participated in the practical delirium in December 2017; 157 students gave consent to participate in this study. The dark play group consisted of 85 participants, 72 participants were included in the normal play group. The median (Mdn) age (interquartile range, IQR) of participants was 20 (1) and 83% were women. Baseline variables are given in Table 1. There were no important differences between the groups on the baseline variables. Data provided from the game showed participants played the version to which they were assigned.

Table 1.

Baseline Variables

	Total	Dark play	Normal play
Number of participants	157	85	72
Age, in years^a	20 (1)	20 (1)	21 (1)
Female^b	130 (83)	70 (82)	60 (83)
Previous experience with older patients, (0–100)^c	35.2 (24.7)	37.2 (26.8)	32.8 (21.8)
Previous experience with delirious patients, (0–100)^a	2 (10)	3 (12.5)	2 (6)
Lecture attended^b	141 (90)	73 (86)	68 (94)

Median (interquartile range).

Number (%).

Mean (standard deviation).

Game effectiveness

None of the measures on given recommendations, learning motivation and engagement, and attitude showed a significant difference between the two arms. Scores on given recommendations for the care for delirious patients in the dark play group, Mdn = 6 (IQR = 2), did not differ significantly from the normal play group, Mdn = 7 (IQR = 1), U = 2.72, P = 0.224. On average, participants in the dark play group scored M = 34.3 (SD = 4.8) points on learning motivation and engagement, whereas the normal play group scored M = 34.4 (SD = 4.5). This difference, 0.18 (CI: −1.29 to 1.67) t(0.25), P = 0.803, was not significant. Attitude in the dark play group Mdn = 93.5 (IQR = 14), did not differ significantly from the normal play group Mdn = 94.5 (IQR = 13.5), U = 2.72, P = 0.416.

Game features

Participants mentioned the same game features influencing their ability to empathize with a delirious patient: patient's and nurse's perspective, patient's experience and response, and videos of hallucinations (Fig. 1). The main enabling feature in the game to empathize with the patient was the perspective in the game. Participants specifically mentioned the patient's perspective, some participants mentioned switching between patient's and nurse's perspective. The patient's experience and response allowed participants to gain better insight into how the patient experiences delirium and how his response to certain actions could be clarified. The videos of the hallucinations allowed participants to gain a better understanding of hallucinations and how realistic they are to patients.

FIG. 1.

Important game features according to participants in dark play and normal play.

The two game features most commonly reported by participants as important to support their self-efficacy were the same in both groups: practicing what to do and feedback. However, the third theme (e.g., game feature) was different: seeing the interaction with the patient was a commonly reported theme in the dark play group, whereas in the normal play group this was experiencing the consequences of care. Participants in the dark play group felt supported in their self-efficacy owing to experiencing the importance of good interaction with a patient and how this affects the patient experiencing delirium, for example, “do not deny the hallucinations” and “try to comfort the patient.” Participants in the normal play group mentioned being able to see the consequences of their actions on the patient and the progression of delirium as important.

Game features that were important to get insight in the consequences of choices in caring for a delirious patient were the same in both groups: practicing what to do, interaction with the patient, and being able to see the consequences of care. The game allowed participants to practice what to do in caring for a delirious patient, which actions they should perform, and in which order. This was reported as important for gaining insight into the consequences of care on delirium. Furthermore, the different examples of how to interact with a delirious patient, for example, how to comfort and communicate with the patient, were also reported as important. Finally, participants reported gaining insight as to how much their actions could influence the patient's wellbeing and experiences of delirium owing to actually seeing these consequences in the game.

Regarding suggestions for improvements, two similar themes were reported: more feedback and more patient's perspective and response in both groups and one other theme: options as a patient in the dark play group and more information and explanation in the normal play group. Participants would like to get more feedback in the game, mainly on the correct order of actions, and explanations on why certain actions are correct or incorrect. In addition, both groups reported more patient's perspective and direct response of the patient to actions of the nurse as improvements. The dark play group missed options to choose from in the patient's perspective to influence the progression of the game as they were able to in the nurse's perspective. The normal play group was in need of more background information on both the patient and delirium in general.

Discussion

This study did not find differences between the two types of play in game effectiveness, defined as abilities in advising care for delirious patients, learning motivation and engagement, and attitude toward delirious patients. We also explored possible use of different game features between the two types of play on empathy, self-efficacy, and consequences of care. We found that for both groups the most reported game features were the same for empathy and consequences of care, but not for self-efficacy and suggestions for game improvement.

As serious games are intended to be safe environments to practice skills that can be transferable to a real-life setting,²⁷ exploration of options and strategies is an important game feature, as this is often not possible nor ethical in real life.²⁸ As we did not find differences between dark play and normal play on game effectiveness regarding ability in advising care for delirious patients, attitude toward delirious patients, or on learning motivation and engagement, there seems to be no risk of decreasing game effectiveness when providing players with the opportunity to play dark play in a serious game. However, as autonomy and freedom of choice increases player satisfaction and feeling of control,¹ it is important to maintain freedom of choice in serious games. The values of the game effectiveness were also in line with a previous study on the effects of The Delirium Experience,²⁰ which suggest stable learning outcomes of the game.

Although previous research showed that simple (e.g., cartoon-like) designs are as effective as more realistic designs,¹⁴ in this study, participants reported the patient's and the nurse's perspective, and the videos of a patient with real-time hallucinations were important for being able to empathize with a patient. In The Delirium Experience, the perspectives are realistic and performed by professional actors. The results of this study therefore suggest a realistic design might be important for players' ability to empathize. Participants indicated that perspective change from patient to nurse were important game features for being able to empathize with someone else, which has also been reported in previous research regarding training moral sensitivity and depression.^28,29

Although The Delirium Experience does not contain performance graphs, it does contain feedback and scores after each day in the game, which provides guidance to the players on how to continue and improve. This seemed to be important for supporting self-efficacy.²⁹ Previous research has also shown the importance of performance graphs (e.g., feedback) on game satisfaction.³⁰ Gaining insights into the direct and delayed consequences of actions are important for training moral sensitivity.²⁹ It also seems this is important in empathizing with someone else and supporting self-efficacy. In The Delirium Experience patient's experiences and response, seeing the importance of interaction and consequences of care are all related to direct or delayed consequences visualized in the game. Finally, both groups found practicing to care for a delirious patient in a serious game was important, which is also what serious games are intended for.²⁷ This may explain why we did not find differences between the two groups for abilities in advising care; participants in both groups were able to practice how to, or how not to, care for a delirious patient.

This study has some limitations. First, the randomization in SurveyMonkey has led to 8% more participants in the dark play group than in the normal play group. However, the groups did not differ in baseline variables. Second, there were technical issues with the plug-in, causing 14 students (9%) to play the game in dyads rather than individually. As this concerned only a few students, who were equally divided over both the dark play and normal play groups, and because their results were in line with results of the other students, we do not think that this has influenced the results. Third, although these results give insight into serious game features we did not study whether or how these game features influence student's clinical competencies in practice, which are important for patient care in the future.

Results of this study give rise to further research into the game features. For example, in this game, it was not possible to make decisions as a patient, only to watch; it would be interesting to study the effects of being able to do so. In the normal play group, players needed more explanation and background information on the case and on the topic. This was not reported in the dark play group, which makes it interesting for further test directly whether it is the presence of (more) feedback on incorrect actions in dark play that decreases the need for further explanation. Furthermore, in this study, participants were allocated to either dark or normal play, whereas in a natural situation, players would choose for dark or normal play. It would be therefore interesting to study game effectiveness in a more natural play situation, as players who choose dark play might already have different characteristics. Further research should also focus on why players choose dark play, and whether players who choose dark play are different from players who choose normal play.

Conclusion

Dark play seems not to have adverse effects on acquired knowledge, learning motivation and engagement, or attitude toward delirious patients in a serious game. A realistic view of another person's perspective, and changing perspectives, seem to be important game features to generate empathy. Being able to practice in a safe environment and seeing the results of one's actions seem to be important game features to support self-efficacy.

Footnotes

Author Disclosure Statement

No competing financial interests exist.

References

Ryan

, Rigby

, Przybylski

. The motivational pull of video games: A self-determination theory approach. Motiv Emot, 2006; 30:344–360.

Mortensen

, Linderoth

, Brown

. The Dark Side of Game Play: Controversial Issues in Playful Environments. Routledge Taylor & Francis Group, New York; 2015.

Calvert

, Appelbaum

, Dodge

, et al. The American Psychological Association Task Force assessment of violent video games: Science in the service of public interest. Am Psychol, 2017; 72:126–143.

Greitemeyer

, Mügge

. Video games do affect social outcomes. Personal Soc Psychol Bull, 2014; 40:578–589.

Grizzard

, Tamborini

, Lewis

, et al. Being bad in a video game can make us morally sensitive. Cyberpsychol Behav Soc Netw, 2014; 17:499–504.

Tamborini

, Bowman

, Prabhu

, et al. The effect of moral intuitions on decisions in video game play: The impact of chronic and temporary intuition accessibility. N Media Soc, 2018; 20:564–580.

Hamari

, Koivisto

, Pakkanen

. Do persuasive technologies persuade? - A review of empirical studies. In Persuasive Technology: 9th International Conference. PERSUASIVE 2014; Padua, Italy; May 21–23, 2014; 8462:118–136.

Boyle

, Hainey

, Connolly

, et al. An update to the systematic literature review of empirical evidence of the impacts and outcomes of computer games and serious games. Comput Educ, 2016; 94:178–192.

Hunicke

, LeBlanc

, Zubek

. MDA: A Formal Approach to Game Design and Game Research. Work Challenges Game AI. 1–4; 2004. https://www.cs.northwestern.edu/∼hunicke/MDA.pdf

10.

Kharrazi

, Lu

, Gharghabi

, Coleman

. A scoping review of health game research: Past, present, and future. Games Health J, 2012; 1:153–164.

11.

Maheu-Cadotte

M-A

, Cossette

, Dubé

, et al. Effectiveness of serious games and impact of design elements on engagement and educational outcomes in healthcare professionals and students: A systematic review and meta-analysis protocol. BMJ Open, 2018; 8:e019871.

12.

Bandura

. in Handbook of Socialization Theory and Research. College Publishing Company, Chicago; 1969, pp. 213–262.

13.

Arnab

, Lim

, Carvalho

, et al. Mapping learning and game mechanics for serious games analysis. Br J Educ Technol, 2015; 46:391–411.

14.

Wouters

, van Nimwegen

, van Oostendorp

, van der Spek

. A meta-analysis of the cognitive and motivational effects of serious games. J Educ Psychol, 2013; 105:249–265.

15.

Effectieve Ouderen Zorg. Delirium Experience. (2017). www.deliriumexperience.nl (accessed May 26, 2017).

16.

American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (DSM). American Psychiatric Association, Washington; 2013.

17.

Kalish

, Gillham

, Unwin

. Delirium in older persons: Evaluation and management. Am Fam Physician, 2014; 90:150–158.

18.

Miller

MO.

Evaluation and management of delirium in hospitalized older patients. Am Fam Physician, 2008; 78:1265–1270.

19.

Steis

, Fick

. Are nurses recognizing delirium? A systematic review. J Gerontol Nurs, 2008; 34:40–48.

20.

Buijs-Spanjers

, Hegge

, Jansen

, et al. A web-based serious game on delirium as an educational intervention for medical students: A randomized controlled trial. JMIR Serious Games, 2018; 6:e17.

21.

SurveyMonkey. (2018). https://nl.surveymonkey.com (accessed July 4, 2018).

22.

Fisher

, Gordon

, MacLullich

AMJ

, et al. Towards an understanding of why undergraduate teaching about delirium does not guarantee gold-standard practice—results from a UK national survey. Age Ageing, 2015; 44:166–170.

23.

Teodorczuk

, Mukaetova-Ladinska

, Corbett

, Welfare

. Reconceptualizing models of delirium education: Findings of a Grounded Theory study. Int Psychogeriatr, 2013; 25:645–655.

24.

Carini

, Kuh

, Klein

. Student Engagement and Student Learning: Testing the Linkages*. Res High Educ, 2006; 47:1–32.

25.

Dankbaar

MEW

, Richters

, Kalkman

, et al. Comparative effectiveness of a serious game and an e-module to support patient safety knowledge and awareness. BMC Med Educ, 2017; 17:30.

26.

Leigh

. High-Fidelity Patient Simulation and Nursing Students' Self-Efficacy: A Review of the Literature. Int J Nurs Educ Scholarsh, 2008; 5:1–17.

27.

Wattanasoontorn

, Boada

, García

, Sbert

. Serious games for health. Entertain Comput, 2013; 4:231–247.

28.

Cheek

, Fleming

, Lucassen

, et al. Integrating Health Behavior Theory and Design Elements in Serious Games. JMIR Ment Heal, 2015; 2:e11.

29.

Katsarov

, Christen

, Mauerhofer

, et al. Training moral sensitivity through video games: A review of suitable game mechanisms. Games Cult, 2017. https://journals-sagepub-com-s.web.bisu.edu.cn/doi/abs/10.1177/1555412017719344

30.

Sailer

, Hense

, Mayr

, Mandl

. How gamification motivates: An experimental study of the effects of specific game design elements on psychological need satisfaction. Comput Hum Behav, 2017; 69:371–380.