Exploring the Potential of Using a Text-Based Game to Inform Simulation Models of Risky Migration Decisions

Abstract

Background

In this paper, we explore the potential of games to collect empirical data for informing agent-based simulation models of migration. To examine the usefulness of game-based approaches, we conducted a simple, yet carefully designed psychological experiment.

Methods

In a preregistered study, we used a novel, immersive experimental setting to investigate the risky migration decisions made by migrants and non-migrants. Participants (284 migrants and 284 non-migrants) played a choice-based interactive fiction game—a fully text-based game where players progress by selecting from a list of possible actions—that involved making three risky migration decisions. In one condition, participants were shown a non-linear progress bar and explicit acknowledgements of the choices they made to promote perceived agency: the feeling that one’s actions have a non-trivial impact on the game. In the other condition, the progress bar was linear, and the explicit acknowledgements were omitted.

Results

Our experimental manipulation was successful; participants in the former condition self-reported higher perceived agency than participants in the latter condition, as did migrants compared to non-migrants. Nevertheless, condition and migrant status did not meaningfully affect the risky migration decisions participants made in the game.

Conclusion

These findings indicate that the results of generic studies on risky migration decisions conducted on non-migrants can potentially inform simulation models of migration. However, these findings were obtained from a single experiment, and thus warrant replication and further research before definitive conclusions can be drawn. Furthermore, a simple text-based game may be too superficial to allow deep insights into the idiosyncrasies of migration decision-making. This suggests a possible trade-off between clear interpretability of the results and the usefulness for informing simulation models of complex social processes, such as migration.

Keywords

migration decision-making risk-taking text-based game simulation models perceived agency

Background

Highly complex and dynamic social processes, such as migration, require analytical tools that can make meaningful statements about their underlying mechanisms. Simulation models, such as agent-based models, offer an appealing possibility for designing and analyzing such processes in silico, provided the models reflect the broader social reality and the way that the modelled individuals—agents—make decisions and interact with one another. However, the information needed to inform such models is typically so specific to the individual model and research questions that generic survey or experimental data are inappropriate, and therefore more bespoke approaches and data collection may be required.

In this paper, we explore the potential of using simple text-based games to inform the parameters and specification of an agent-based simulation model of migrant journeys (Bijak et al. 2021). The main motivation behind using games in this context is to obtain results that are as psychologically realistic as possible, while enabling their rigorous analysis through proper experimental control. To gain insights into the usefulness of game-based approaches for such purposes, we conducted a simple, yet carefully designed psychological experiment, focused on risk preferences in the context of risky migration decisions. In this paper, we start by presenting the background that motivated our intervention and methodological choices, which are discussed in more detail immediately after. Next, we report the results before finishing with a critical discussion, focused on the trade-offs between the simplicity of the proposed approach and the insights that can be gained for modelling purposes.

Migration Research

Experimental research has been largely neglected in migration studies, with a few notable exceptions (e.g., Baláž & Williams, 2015). This is likely due to the common criticism that experimental conditions cannot accurately replicate and thus measure real-world social phenomena, especially those as complex as human migration. Indeed, due to its countless factors, dynamics, and their intricate interactions, human migration has been considered impossible to fully measure or understand (International Organization for Migration, 2018). However, de Haas (2021) criticized what he labelled the “‘migration is too complex’ fallacy” (p. 3), pointing out that the notion of complexity highlights that experiences such as human migration are multi-faceted, but does not imply they are devoid of patterns or structure. In line with this argument, experimental research aims to abstract and reproduce facets of real-world phenomena to identify their patterns and consequently establish causal relationships (Kihlstrom, 2021; Orne, 1962). Therefore, provided experimental findings are understood within context, using experimental methods to clarify and predict certain aspects of our reality is not only justified, but often necessary to make sense of the world around us.

Nevertheless, perfect reproduction of social contexts in artificial settings is currently impossible. Accordingly, despite causal relationships being possible to establish through experimental control, questions remain over the extent to which these relationships can be generalized to real-world settings (i.e., ecological validity; Andrade, 2018). To that end, other methods, such as ethnographic research, are largely considered more ecologically valid than experimental methods. Ethnography refers to the study of social interactions, behaviors, and group perceptions through qualitative research methods, such as interviews and participant observation (Falzon, 2015; Shah, 2017). In a recent example related to migration decisions, Belabbas et al. (2022) conducted qualitative interviews with Syrian and Afghan refugees to determine what factors were relevant in shaping their journeys to Europe. Although such ethnographic research provides rich, contextually detailed data, it does not provide the experimental control or quantitative data that are often needed to explain and predict specific aspects of real-world phenomena.

In summary, there is a clear trade-off between ecological validity and experimental control in social science research. In fact, Kothgassner and Felnhofer (2020) referred to this problematic trade-off as a “Gordian Knot” (p. 216), highlighting its ongoing relevance. It has long been noted that immersive games can decrease the extent of this trade-off (Kozlov & Johansen, 2010; Lin-Greenberg et al., 2021), as they allow researchers to create more realistic environments and at the same time control and manipulate variables to determine their individual impacts and thus establish cause-and-effect relationships. With the exponential advancement of technology, it is possible that the compromise between ecological validity and experimental control will one day disappear, but this is not yet the case. In this paper, we explore the possibility of using games to cut, or at least loosen, this Gordian Knot in the context of risky migration decisions.

Immersion and Player Agency

Murray (1997) defined virtual reality as a stirring narrative in any medium, such as a game. McMahan (2003) adopted this definition and proposed three requirements for making virtual realities and games immersive, offering players the feeling of being inside a virtual world: (a) the user’s expectations of the game must match the game’s mechanics closely; (b) the game’s mechanics must be consistent even if they do not match those of real life; and (c) the user’s actions must have a non-trivial impact on the game. The game mechanics here refer to the rules that simulate the virtual world inside a game (Mizutani et al., 2021).

Since all three requirements are needed to attain immersion, failing to achieve one should theoretically lead to a complete or at least partial loss of immersion. When using games in experimental research, the first two requirements will likely be achieved by default. This is due to the standard procedure for conducting experiments and the typical design of games used in this setting. Firstly, clear instructions are usually given to participants before they take part in an experiment. Therefore, if an experiment involves a game, participants will likely be given information on how to play it beforehand, so their expectations should match the game’s mechanics. Secondly, most of the time, games created for experimental research are simple by design since their purpose is to investigate a limited number of outcome variables, so their mechanics are inherently consistent. Consequently, to manipulate immersion in games used for experimental research, we must typically focus on the final requirement, which refers to a concept commonly known as player agency—the ability for a player’s actions to have a non-trivial impact on the game.

We are not the first to use player agency as an indicator of immersion. Indeed, immersion is considered an umbrella construct that consists of different sub-dimensions (Wagner & Liu, 2021), and previous studies have regarded player agency as a sub-dimension of immersion (Denisova & Cairns, 2015; Guo & Lo, 2023; Qin et al., 2009). To ensure player agency, multiple story arcs and decision trees are often designed and created. However, it is also possible to make users feel as if their actions have a non-trivial impact on the game even when they do not. Fendt et al. (2012) created two versions of the same text-based interactive story to compare the effects of simulated player agency and real player agency. Since Murray (1997) defined player agency as the power to take meaningful action and see the results of our decisions, Fendt et al. simulated agency by presenting players with explicit acknowledgements of their decisions. In contrast, real agency was created by generating different outcomes depending on the player's decisions. Results showed that participants reported similar feelings of agency regardless of whether it was real or simulated.

The Present Study

The present study’s research question was the following: Does immersion narrow the gap between migrants and non-migrants in the context of risky migration decisions? Comparability of migration decisions among migrants and non-migrants is desirable considering the limited number of migrants on online data collection platforms, as well as the potential ethical issues associated with asking migrants to relive their potentially traumatic migration decisions. To address our research question, we created a choice-based interactive fiction game, namely, a fully text-based game where players progress by selecting from a list of possible actions (Hausknecht et al., 2020). Our game shows similarities to interactive digital narratives and interactive storytelling. However, definitions for both of these terms emphasize the need for different paths to be available in the story, demanding player agency (Bostan & Marsh, 2012; Koenitz et al., 2018). Considering that our game does not ensure player agency, but rather only simulates it, we decided not to define it with these terms.

In our game, participants, who included both migrants (broadly defined in our study as individuals who migrated to a new country before and were old enough to remember their migration journey) and non-migrants, made three risky migration decisions embedded within the context of a migrant journey to advance through a story. Consistent with commonly used measures of risk-taking (e.g., the Columbia Card Task and the Balloon Analogue Risk Task; Figner et al., 2009; Lejuez et al., 2002), these decisions involved a binary trade-off between a greater reward that involved risk and a smaller reward that involved no risk. A prototype game was created for similar purposes in Bijak et al. (2023), but its complex design introduced several confounds that made the results difficult to interpret. Therefore, we simplified the design in our study. Specifically, participants took part in one of two conditions: agency or no-agency. In the agency condition, our game promoted perceived agency by showing participants explicit acknowledgements of the choices they made and having a non-linear progress bar. In the no-agency condition, our game did not promote perceived agency by omitting these acknowledgements and having a linear progress bar. We hypothesized that the risky migration decisions of migrants and non-migrants would be more similar in the agency condition than in the no-agency condition.

Methods

Transparency and Openness

All data, analytic code, and materials needed to replicate this study are available on OSF (https://osf.io/7a4vr/). This study was preregistered (https://aspredicted.org/sn4v5.pdf). We obtained ethical approval to conduct this study from the University of Southampton Faculty of Environmental and Life Sciences Ethics Committee (approval no. 68015.A1).

Participants

Pilot Study

For the pilot, we recruited 285 participants from Prolific (https://www.prolific.com/) for the pre-screener, which assessed participants’ suitability for the study. A total of 60 participants reported having migrated to a new country before, but four of these reported not remembering their migration journey. These four participants were excluded, and the remaining 56 that reported remembering their migration journey were included as migrants (∼20% of the sample). We then selected a matched sample (based on age, gender, highest level of education, marital status, and annual household income) of 56 non-migrants from the remaining 225 participants. We matched the migrant and non-migrant samples based on these demographic variables to control for their potential effects on the dependent variables (see the Experimental Design section). These 112 participants (56 migrants and 56 non-migrants) were invited to take part in our experiment, but only 92 complied (∼18% attrition rate). Specifically, 25 migrants and 23 non-migrants participated in the agency condition, and 17 migrants and 27 non-migrants participated in the no-agency condition. For further sampling and demographic details, see Method Supplement S1.

The sample size for the experiment in the pilot study was intended to be ∼20% of the sample size for the experiment in the main study. The sample size for the pre-screener was not specified in advance. Instead, we simply stopped data collection once we had recruited 56 migrants. After the pilot, two minor formatting changes were made, neither of which impacted the main body of the experiment. For further details on these minor formatting changes, see Method Supplement S1.

Main Study

Subsequently, for the main study, we recruited 1,850 participants from Prolific for the pre-screener. A total of 385 participants reported having migrated to a new country before, but 21 of these reported not remembering their migration journey. These 21 participants were excluded, and the remaining 364 that reported remembering their migration journey were included as migrants. We then selected a matched sample (based on age, gender, highest level of education, marital status, and annual household income) of 364 non-migrants from the remaining 1,465 participants. These 728 participants (364 migrants and 364 non-migrants) were invited to take part in our experiment, but we stopped data collection once we reached our preregistered sample size (284 migrants and 284 non-migrants). For further sampling and demographic details, see Method Supplement S1.

The sample size for the main study was based on an a priori power analysis in G*Power 3.1 that indicated 566 participants were required to detect a small-to-medium effect with a mixed factorial ANOVA (n ≈ 566, f = .15, 1‒β = .90, α = .05). We increased the sample size to n = 568 so that an equal number of migrants and non-migrants could take part in each of the two conditions. The sample size for the main study’s pre-screener was based on: (a) the planned sample size for the experiment; (b) the attrition rate between the pre-screener and the experiment, which our pilot study found to be ∼18%; and (c) the percentage of Prolific participants that reported having migrated to a new country before and remembering their migration journey, which our pilot study found to be ∼20%. Despite the pilot study being almost identical to the main study, we did not combine their data. This is because the migrants and non-migrants were demographically matched within, but not across, the pilot study and the main study. Therefore, combining their data results in a demographically unmatched sample of migrants and non-migrants, which could account for any differences between the two groups.

Experimental Design

A 2×2×3 mixed factorial experimental design was used. The between-subjects independent variables were the condition that participants took part in (agency or no-agency) and participants’ migrant status (migrant or non-migrant). The within-subjects independent variable was the risky migration decisions in the game (first, second, or third). The dependent variables were participants’ responses to the risky migration decisions and their self-reported perceived agency. Age and gender were measured as demographic variables.

Choice-Based Interactive Fiction Game

We created the choice-based interactive fiction game on Qualtrics (https://www.qualtrics.com/). Despite not being game design software, Qualtrics allows for the implementation of HTML, CSS, and JavaScript, which are commonly used to create text-based games. Our game included a prologue, three risky migration decisions, and an epilogue. The prologue involved a trivial decision that did not provide a measure of the dependent variable. Each of the following three risky migration decisions involved a binary trade-off between a greater reward that involved risk and a smaller reward that involved no risk, and they were all embedded within the context of a migrant journey. For example, the first risky migration decision involved taking a shorter but more treacherous route or taking a longer but safer route (all the risky migration decisions are available on OSF: https://osf.io/7a4vr/). To make the decisions as realistic as possible, we drew inspiration from both first-hand and second-hand accounts of migrant journeys. These were obtained from the website Telling the Real Story (https://www.tellingtherealstory.org/en/) and from YouTube videos published by various news channels (e.g., BBC News and Sky news). The epilogue did not involve any decisions.

The development of our game followed an iterative process (Bannon, 1995; Raghothama & Meijer, 2018). Each version of the game was shared among the authors and sent to personal contacts that included migrants and non-migrants. The game was then updated accordingly and once again sent out for feedback. This was done until no further improvements were suggested.

Two versions of the game were made, one for the agency condition and one for the no-agency condition. There were two differences between the two versions of the game. First, at the beginning of each decision, participants in the agency condition, but not in the no-agency condition, were shown an explicit acknowledgment of the choice they made in the preceding decision. This acknowledgement was one or two sentences long and did not provide any additional useful information. The trivial decision in the prologue was used to show (or not show) an explicit acknowledgement of this choice in the following risky migration decision, and thus vary the perceived agency between conditions from the first decision. Second, the progress bar in the agency condition displayed two different branches at each decision, one of which was subsequently filled in, indicating the choice the participant made. This created the illusion that different choices led to different outcomes. In contrast, the progress bar in the no-agency condition was linear.

Procedure

Device restrictions were applied on Prolific, which suggested that participants access the experiment through a computer. Since this was a remote online study, participants could use any web browser and computer of their choosing. To navigate through the experiment, a mouse (or touchpad) and keyboard were necessary.

Participants were first recruited for the pre-screener. Before starting, they were shown a combined information sheet and consent form. After reading the form and providing informed consent by clicking a button at the bottom of the web page, participants were asked to provide their Prolific IDs. They were then asked for their age, gender, highest level of education, marital status, annual household income, whether they had ever migrated to a new country before, and, if they replied yes to the preceding question, whether they were old enough to remember their migration journey. For the response options of each demographic question, see Method Supplement S1. Participants were then debriefed and redirected back to the Prolific website. The pre-screener took approximately 2 min to complete.

Participants that passed the pre-screener were then invited to take part in the experiment. Before starting, the procedure was identical to that of the pre-screener. That is, the same combined information sheet and consent form was used, and participants indicated they read the form, gave informed consent, and provided their Prolific IDs in the same way. They then took part in the agency condition or the no-agency condition. In both conditions, participants navigated through a choice-based interactive fiction game. However, the game promoted perceived agency in the agency condition, but not in the no-agency condition.

After completing the game, participants were asked to indicate the extent to which they felt as though their actions were meaningful within the context of the story, and to what extent they felt as though they were able to see the consequences of their decisions, both on a scale of 0–100 (adapted from Fendt et al., 2012). Finally, they were asked for their age, gender, whether they had ever migrated to a new country before, and, if they replied yes to the preceding question, whether they were old enough to remember their migration journey. For the response options of each demographic question, see Method Supplement S1. Participants were then debriefed and redirected back to the Prolific website. Both conditions took approximately 10 min to complete, but since progression was entirely self-paced, completion times varied between participants.

Results

Risky Migration Decisions

We coded participants’ responses into binary variables, with 0 indicating the choice of a non-risky option and 1 indicating the choice of a risky option. A 2 (condition) ×2 (migrant status) ×3 (decision) mixed factorial ANOVA was conducted on participants’ responses. The same ANOVA was repeated twice, once with age added as a factor and once with gender added separately. The results from these three ANOVAs are shown in Table 1. Post-hoc Tukey tests based on estimated marginal means were then conducted for the main effect of decision across all three ANOVAs, since it was significant with a BF₁₀ > 3. BF₁₀ = Bayes factor that quantifies the empirical evidence in favour of the alternative hypothesis. When a Bayes factor is between one-third and three, it is considered evidentially weak or anecdotal, and thus researchers typically conclude that a significant effect is absent (Giolla & Ly, 2019). Conversely, when a Bayes factor is greater than three, researchers typically conclude that a significant effect is present. The results from these three post-hoc Tukey tests are shown in Table 2. In the ANOVA with gender added as a factor, the main effects of condition and gender as well as the interaction between migrant status, gender, and decision were significant, but with a BF₁₀ < 3. Therefore, post-hoc independent samples t-tests based on estimated marginal means were conducted for the two main effects (see Table S1), and their interaction is visualized in Figure S1. Overall, these results suggest that migrants did not significantly differ from non-migrants in their risky migration decisions, and that the condition participants took part in (agency or no-agency) did not meaningfully impact their risky migration decisions.

Table 1.

Mixed Factorial ANOVAs.

ANOVA	df	F	p	η²_G	BF ₁₀
ANOVA 1
condition	1, 564	3.71	.055	.002	0.30
migrant	1, 564	0.09	.771	< .001	0.06
condition:migrant	1, 564	0.09	.771	< .001	0.08
decision	1.88, 1062.53	62.09	< .001	.067	6.69 $\times$ 10 ²⁴
condition:decision	1.88, 1062.53	0.01	.987	< .001	0.01
migrant:decision	1.88, 1062.53	1.22	.293	.001	0.04
condition:migrant:decision	1.88, 1062.53	0.28	.741	< .001	0.03
ANOVA 2
condition	1, 544	2.20	.139	.001	0.37
migrant	1, 544	1.63	.203	< .001	0.09
age	5, 544	1.53	.178	.005	0.01
condition:migrant	1, 544	1.76	.185	.001	0.14
condition:age	5, 544	0.54	.744	.002	0.01
migrant:age	5, 544	0.80	.549	.002	0.00
condition:migrant:age	5, 544	1.40	.221	.004	0.06
decision	1.88, 1021.92	19.19	< .001	.022	1.90 $\times$ 10 ¹²
condition:decision	1.88, 1021.92	0.05	.948	< .001	0.01
migrant:decision	1.88, 1021.92	0.85	.421	< .001	0.04
age:decision	9.39, 1021.92	0.59	.812	.003	0.00
condition:migrant:decision	1.88, 1021.92	0.05	.938	< .001	0.03
condition:age:decision	9.39, 1021.92	1.33	.213	.008	0.06
migrant:age:decision	9.39, 1021.92	0.43	.924	.003	0.00
condition:migrant:age:decision	9.39, 1021.92	1.05	.395	.006	0.07
ANOVA 3
condition	1, 555	4.27	.039	.003	0.44
migrant	1, 555	0.07	.787	< .001	0.07
gender	1, 555	3.96	.047	.002	0.46
condition:migrant	1, 555	0.10	.754	< .001	0.11
condition:gender	1, 555	1.00	.317	< .001	0.16
migrant:gender	1, 555	0.74	.390	< .001	0.15
condition:migrant:gender	1, 555	0.08	.781	< .001	0.16
decision	1.88, 1042.70	61.21	< .001	.067	4.15 $\times$ 10 ²⁴
condition:decision	1.88, 1042.70	0.00	> .999	< .001	0.02
migrant:decision	1.88, 1042.70	1.36	.256	.002	0.03
gender:decision	1.88, 1042.70	0.27	.746	< .001	0.02
condition:migrant:decision	1.88, 1042.70	0.32	.715	< .001	0.04
condition:gender:decision	1.88, 1042.70	0.93	.391	.001	0.09
migrant:gender:decision	1.88, 1042.70	3.71	.027	.004	1.09
condition:migrant:gender:decision	1.88, 1042.70	0.90	.403	.001	0.13

Note. Rows containing a significant effect are presented in bold. Rows containing a significant effect with a BF₁₀ > 3 are presented in bold and underlined. ANOVA 1: dependent variable = participants’ responses (0 [non-risky] or 1 [risky]); factors = condition (agency or no-agency), migrant status (migrants or non-migrants), and decision (first, second, or third). ANOVA 2: same as ANOVA 1 but with age (18–24, 25–34, 35–44, 45–54, 55–64, or 65 or over) added as a factor. ANOVA 3: same as ANOVA 1 but with gender (male or female) added as a factor.

Table 2.

Post Hoc Tukey Tests.

Post hoc Tukey test	M ₁	M ₂	df	t	p	d	BF ₁₀
Post hoc Tukey tests ANOVA 1
decision 1 - decision 2	.17	.39	564	-8.46	< .001	-0.63	> 1,000
decision 1 - decision 3	.17	.14	564	1.09	.519	0.07	0.01
decision 2 - decision 3	.39	.14	564	9.73	< .001	0.69	> 1,000
Post hoc Tukey tests ANOVA 2
decision 1 - decision 2	.16	.36	544	-4.71	< .001	-0.56	204.23
decision 1 - decision 3	.16	.14	544	0.57	.838	0.05	0.01
decision 2 - decision 3	.36	.14	544	5.40	< .001	0.62	56.78
Post hoc Tukey tests ANOVA 3
decision 1 - decision 2	.17	.39	555	-8.44	< .001	-0.62	0.20
decision 1 - decision 3	.17	.15	555	1.02	.567	0.06	0.01
decision 2 - decision 3	.39	.15	555	9.60	< .001	0.68	0.65

Note. Rows containing a significant effect are presented in bold. Rows containing a significant effect with a BF₁₀ > 3 are presented in bold and underlined. All post-hoc Tukey tests were based on estimated marginal means. ANOVA 1: dependent variable = participants’ responses (0 [non-risky] or 1 [risky]); factors = condition (agency or no-agency), migrant status (migrants or non-migrants), and decision (first, second, or third). ANOVA 2: same as ANOVA 1 but with age (18–24, 25–34, 35–44, 45–54, 55–64, or 65 or over) added as a factor. ANOVA 3: same as ANOVA 1 but with gender (male or female) added as a factor. The results from post hoc Tukey test 3 may be misleading due to the significant interaction between migrant status, gender, and decision in ANOVA 3. Post-hoc Tukey tests 1, 2, and 3 tested the pairwise comparisons between each decision (first, second, and third) from ANOVAs 1, 2, and 3, respectively.

Perceived Agency

We summed participants’ ratings for the two perceived agency questions to create a composite measure of perceived agency ranging from 0–200. A Welch’s independent samples t-test showed that perceived agency was significantly higher in the agency condition (M = 156.41, SD = 36.14) than in the no-agency condition (M = 146.39, SD = 39.92), t(560.48) = 3.14, p = .002, 95% CI [3.74, 16.29], d = 0.26, BF₁₀ = 10.89. A Welch’s independent samples t-test showed that perceived agency was significantly higher for migrants (M = 155.82, SD = 37.97) than non-migrants (M = 146.99, SD = 38.32), t(565.95) = 2.76, p = .006, 95% CI [2.54, 15.12], d = 0.23, BF₁₀ = 3.73. Overall, these results suggest that participants’ perceived agency was significantly higher in the agency condition compared to the no-agency condition, and that migrants’ perceived agency was significantly higher than non-migrants’ perceived agency.

Discussion

In this preregistered study, we investigated the risky migration decisions (embedded within the context of a migrant journey) of migrants (broadly defined in our study as individuals who migrated to a new country before and were old enough to remember their migration journey) and non-migrants within a novel and immersive experimental setting. We found that migrants did not significantly differ from non-migrants in their risky migration decisions. This refutes a fundamental assumption in our research question, which asks whether immersion narrows the gap between migrants and non-migrants in the context of risky migration decisions, and thus assumes that a gap exists in the first place. We made this assumption because of the received wisdom that migrants are more risk-taking than non-migrants (Baláž & Williams, 2011; Goldbach & Schlüter, 2018; Jaeger et al., 2010; Lübke et al., 2021; but see Ceriani & Verme, 2018; Mironova et al., 2019 for opposite findings for conflict-driven migration). However, most of the previous research investigated risk attitudes through Likert scales that ranged from a complete unwillingness to take risks to a complete willingness to take risks. Although this type of self-report measure has been shown to be a reliable predictor of certain risk behaviors (Dohmen et al., 2011), it is vastly different from the context-rich and migration-specific risky decisions used in our study, which may explain the differing results. Indeed, consistent with the aim of our study, Czaika et al. (2021) discussed the multi-faceted nature of migration decisions and recommended conducting scenario-based studies that experimentally manipulate the decision environment to capture this complexity and thus learn more about migration-specific risk-taking.

We also found that the condition participants took part in (agency or no-agency) did not meaningfully impact their risky migration decisions. This result is noteworthy considering that our experimental manipulation worked; participants’ perceived agency was significantly higher in the agency condition compared to the no-agency condition. Additionally, migrants’ perceived agency was significantly higher than non-migrants’ perceived agency, despite their comparable risky migration decisions. Therefore, we can conclude that perceived agency, despite being said to promote immersion and consequently decrease the degree of trade-off between ecological validity and experimental control (Kozlov & Johansen, 2010; Lin-Greenberg et al., 2021; McMahan, 2003), did not affect participants’ risky migration decisions in our study. This finding raises an interesting possibility: it might not be necessary to invest the time and effort to promote perceived agency when investigating risky migration decisions in an experimental setting if it does not lead to different results. However, this study is the first to examine the effects of perceived agency and migrant status on migration-specific risk-taking in an immersive, experimental setting, which also raises the possibility that our design is too simplistic (Bijak et al., 2023). Although participants generally reported moderate to high levels of perceived agency in both conditions, perhaps it is necessary to create more realistic games with immersive 3D environments for the effects of perceived agency to emerge.

It is worth noting that participants’ risk-taking varied significantly between different migration decisions. However, this result was expected, and does not carry any meaningful practical or theoretical insights. Although all decisions involved a binary trade-off between a greater reward that involved risk and a smaller reward that involved no risk, these decisions were placed within rich, idiosyncratic contexts. It is therefore unsurprising that they elicited different levels of risk-taking. Accordingly, the focus of our study was not on differences between participants’ decisions on their own, but rather whether these decisions interacted with condition and/or migrant status, which they did not.

Notably, age and gender did not meaningfully impact participants’ risky migration decisions. This is surprising considering the large body of work on the demography of risk that suggests risk aversion increases with age and that females are more risk averse than males (Donkers et al., 2001; Halek & Eisenhauer, 2001; Hartog et al., 2002). Furthermore, the comparatively small body of work comparing the demography of risk between migrants and non-migrants has also found significant age and gender differences. For example, Baláž and Williams (2011) found significantly greater risk aversion amongst non-migrant women compared to migrant women, whereas Jaeger et al. (2010) found greater risk aversion in migrants compared to non-migrants regardless of both age and gender. Nevertheless, this research assessed willingness to take risks in general rather than willingness to take migration-specific risks, which, as acknowledged by Baláž and Williams (2011), raises questions about causality; although there is likely an association between generic and specific measures of risk-taking, the strength of this association is up for debate. The disparity between our results and those of previous studies highlights this issue as well as the importance of using migration-specific risk measures to gain more nuanced insights into migrant and non-migrant risk-taking.

The general concept of validity in the context of games can be described as the degree of correspondence between the game and whatever aspect of reality it attempts to model (Peters et al., 1998). When a game is used as a research tool, it should ideally be designed to make participants behave similarly to how they would in reality. To do this, Peters et al. (1998) suggested that the game should appear realistic to participants, resemble reality, and have high predictive validity (i.e., the extent to which the game can predict what happens in reality). To meet these criteria and therefore increase the validity of a game, Peters et al. recommended: (a) working systematically and participatively to adjust and improve the game; (b) presenting the game to both researchers and future game players (in our study, these were migrants and non-migrants) and asking for their opinion on the degree of correspondence between the game and reality; and (c) testing the game extensively.

In our view, the iterative process that we used for the development of our game (see the Choice-Based Interactive Fiction Game section) followed all of Peters et al.’s (1998) criteria. That being said, we could have taken further steps to ensure a high degree of correspondence between our game and reality, such as incorporating a validity questionnaire at the end of our study (van Lankveld et al., 2017). Nevertheless, some perspectives emphasize the notion that the only valid representation of reality is reality itself (Raghothama & Meijer, 2018). This idea is linked to the aforementioned trade-off between ecological validity and experimental control (see the Migration Research section); highly controlled experiments that allow for causality to be established often lack generalizability, and loosely controlled experiments that are generalizable often cannot establish causality (Klabbers, 2009, 2018). Therefore, although games can hold significant explanatory power, they cannot offer comprehensive predictions or explanations of the aspect(s) of reality they attempt to model. This view aligns with the core objective of this paper: to inform the design of future migration experiments with the aim of achieving more accurate (but certainly not perfect) predictions, explanations, and consequently simulation models of migration.

Limitations and Directions for Future Research

Our study has several potential limitations. Firstly, our risky migration decisions were not psychometrically validated. To do this, researchers typically examine the convergent validity between different measures that assess the same construct by examining the extent to which they produce similar results. When doing so, it is recommended to distinguish between general and specific risk-taking measures (Bran & Vaidis, 2020). However, to the best of our knowledge, no other migration-specific risk measures were available at the time of our study.

Secondly, as mentioned above, each migration decision in our game was unique in terms of the specific risk, reward, and context it involved. Consequently, one could argue that each migration decision measured participants’ risk-taking in that specific setting, and that collapsing risk categories across all questions was inappropriate. Indeed, prior to conducting our study, we considered matching each migration decisions’ risk, reward, and setting. However, we decided against it because it would undermine the experiment’s ecological validity; a migrant’s journey in real-life involves various risky decisions that do not all involve the same information or context (Czaika et al., 2021). Therefore, to holistically measure risk-taking in the context of migration, this variation must be accounted for.

Lastly, we determined whether participants were migrants or non-migrants by asking them whether they had ever migrated to a new country before and, if they said yes, whether they were old enough to remember their migration journey. These questions do not allow for distinctions to be made between different types of migrants (e.g., those that migrated within or between countries) or different reasons for migrating (e.g., security, poverty, family, study, or new opportunities). Despite this, our game had participants assume the role of an asylum seeker that made a long, hazardous journey across various countries. Migrants with similar experiences might have been more immersed in our game and thus may have responded differently. Furthermore, if the migrants in our study could not relate to the experiences described in our game, it might have prompted them to adopt a new set of precepts about reality, personal goals, and identity while playing (Bowman & Lieberoth, 2018). Although this can be seen as a consequence of immersion, we consider it undesirable in the context of our study; our aim was for participants to respond as closely as possible to how they would in reality, not to adopt alternate identities that could impact their decision-making. Therefore, information about the type of migrant that was recruited for our study may have been useful since this could have affected their migration-specific risk-taking. However, as noted in The Present Study section, there are potential ethical issues associated with asking migrants to relive their potentially traumatic migration journeys and decisions, which would likely be further exacerbated for asylum seekers.

Some additional considerations for future research include how individual differences between migrants and non-migrants (e.g., attitudes toward risk and uncertainty; Czaika et al., 2021) interact with their risky migration decisions in immersive contexts. Furthermore, our game was only in English. Although the participants in our study were all fluent in English, playing the game in their native language might have increased immersion and consequently their propensity for making meaningful decisions.

To the best of our knowledge, this study is the first to examine the effects of perceived agency and migrant status on migration-specific risk-taking in an immersive, experimental setting. Our experimental manipulation was successful; participants in the agency condition reported higher feelings of perceived agency than participants in the no-agency condition. Furthermore, migrants reported higher feelings of perceived agency than non-migrants. Despite this, neither condition nor migrant status meaningfully affected risky migration decisions. Nonetheless, additional work is still needed to examine how risk-taking patterns may differ between standard self-report surveys and immersive, context-rich decision-making settings.

Conclusion

In our experiment, participants’ perceived agency in a text-based game and migrant status did not affect their risky migration decisions. This suggests that generic studies on risky migration decisions conducted on non-migrants may be enough to inform simulation models of migration, since neither immersive contexts nor specific participant demographics impacted the data required to make agent-based models more psychologically realistic. However, although differences in perceived agency did not translate into differences in our dependent variable of interest—risky migration decisions—they may impact other outcome measures. This is worth exploring considering we found that perceived agency can be easily manipulated through the adaptation of a linear progress bar into a non-linear progress bar, and the addition of explicit acknowledgements of choices made by participants.

Our findings also indicate that a text-based game may be too simple and artificial to allow for deep insights into the idiosyncrasies of migration decision-making, and this information might need to be sought elsewhere, for example, in thorough ethnographic studies (see Belabbas et al. 2022). This interpretation is consistent with an earlier suggestion (Bijak et al. 2023) of a trade-off between clear, interpretable experimental results and extensive, data-rich inputs needed to inform simulation models of complex social processes, such as migration. In our case, on the one hand, the straightforward experimental design of our study allowed us to isolate the impacts (or lack thereof) of various factors on human decisions, as reported in this paper. On the other hand, such a manageable and interpretable design was still likely to be too simple to reflect the complex reality being modelled.

So, can games help inform the construction and design of agent-based simulation models? In light of our results, the answer depends on the purpose of modelling. If the aim is to include realistic decision parameters under various circumstances in the models, then simple games may fail to capture the complex reality, and more complex and realistic games may be difficult to interpret. If, however, the objective of using games is to help understand the mechanisms involved in human decision-making, and create a plausible design of an agent-based simulation model, then this avenue may be more promising. This research path is worth pursuing in future studies at the intersection of gaming and simulation modelling.

Supplemental Material

Supplemental Material - Exploring the Potential of Using a Text-Based Game to Inform Simulation Models of Risky Migration Decisions

Supplemental Material for Exploring the Potential of Using a Text-Based Game to Inform Simulation Models of Risky Migration Decisions by Ariana Modirrousta-Galian, Toby Prike, Philip A. Higham, Martin Hinsch, Sarah Nurse, Souhila Belabbas, and Jakub Bijak in Simulation & Gaming

Footnotes

Acknowledgments

We thank Jason Hilton and Peter W. Smith for discussions and help with testing the study.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was funded by the European Union’s Horizon 2020 research and innovation programme; European Research Council grant 725232 BAPS: Bayesian Agent-Based Population Studies.

Disclaimer

This article reflects the authors’ views, and the Research Executive Agency of the European Commission is not responsible for any use that may be made of the information it contains.

ORCID iD

Ariana Modirrousta-Galian

Data Availability Statement

All data, analytic code, and materials needed to replicate this study are available on OSF (https://osf.io/7a4vr/). This study was preregistered ().

Supplemental Material

Supplemental material for this article is available online.

Author Biographies

Ariana Modirrousta-Galian is a PhD student in Experimental Psychology at the University of Southampton.

Dr Toby Prike is a Lecturer of Psychology at the University of Adelaide.

Professor Philip A. Higham is Professor of Experimental Psychology at the University of Southampton.

Dr Martin Hinsch is a Research Associate at the University of Glasgow.

Dr Sarah Nurse is a Research Fellow in Social Statistics and Demography at the University of Southampton.

Dr Souhila Belabbas is a Lecturer at the University of Oran-2.

Professor Jakub Bijak is Professor of Statistical Demography at the University of Southampton.

References

Andrade

(2018). Internal, external, and ecological validity in research design, conduct, and evaluation. Indian Journal of Psychological Medicine, 40(5), 498–499. https://doi.org/10.4103/ijpsym.ijpsym_334_18

Baláž

Williams

A. M.

(2011). Risk attitudes and migration experience. Journal of Risk Research, 14(5), 583–596. https://doi.org/10.1080/13669877.2010.547256

Baláž

Williams

A. M.

(2015). Experimental research methods in migration: From natural to true experiments. Population, Space and Place, 23(1), Article e1974. https://doi.org/10.1002/psp.1974

Bannon

L. J.

(1995). From human factors to human actors: The role of psychology and human-computer interaction studies in system design. In Baecker

R. M.

Grudin

Buxton

W. A. S.

Greenberg

(Eds.), Readings in human-computer interaction: Toward the year 2000 (2nd ed., pp. 205–214). Elsevier. https://doi.org/10.1016/B978-0-08-051574-8.50024-8

Belabbas

Bijak

Modirrousta-Galian

Nurse

(2022). From conflict zones to Europe: Syrian and Afghan refugees’ journeys, stories, and strategies. Social Inclusion, 10(4), 211–221. https://doi.org/10.17645/si.v10i4.5731

Bijak

Higham

P.A.

Hilton

Hinsch

Nurse

Prike

Reinhardt

Smith

P.W.

Uhrmacher

A.M.

Warnke

(2021). Towards Bayesian Model-Based Demography: Agency, Complexity and Uncertainty in Migration Studies. In: Methodos series, vol 17. Cham: Springer. https://doi.org/10.1007/978-3-030-83039-7

Bijak

Modirrousta-Galian

Higham

P. A.

Prike

Hinsch

Nurse

(2023). Investigating immersion and migration decisions for agent-based modelling: A cautionary tale [version 2; peer review: 2 approved with reservations]. Open Research Europe, 3, 34. https://doi.org/10.12688/openreseurope.15581.2

Bostan

Marsh

(2012). Fundamentals of interactive storytelling. AJIT-e: Online Academic Journal of Information Technology, 3(8), 19–42. https://doi.org/10.5824/1309-1581.2012.3.002.x

Bowman

S. L.

Lieberoth

(2018). Psychology and role-playing games. In Zagal

J. P.

Deterding

(Eds.), Role-playing game studies: A transmedia approach (pp. 245–264). Routledge. https://doi.org/10.4324/9781315637532-13

10.

Bran

Vaidis

D. C.

(2020). Assessing risk-taking: What to measure and how to measure it. Journal of Risk Research, 23(4), 490–503. https://doi.org/10.1080/13669877.2019.1591489

11.

Ceriani

Verme

(2018). Risk preferences and the decision to flee conflict. In Policy Research Working Papers. https://doi.org/10.1596/1813-9450-8376

12.

Czaika

Bijak

Prike

(2021). Migration decision-making and its key dimensions. The ANNALS of the American Academy of Political and Social Science, 697(1), 15–31. https://doi.org/10.1177/00027162211052233

13.

de Haas

(2021). A theory of migration: The aspirations-capabilities framework. Comparative Migration Studies, 9, Article 8. https://doi.org/10.1186/s40878-020-00210-4

14.

Denisova

Cairns

(2015). First person vs. third person perspective in digital games: Do player preferences affect immersion? In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, Republic of Korea, 2015, 145–148. https://doi.org/10.1145/2702123.2702256

15.

Dohmen

Falk

Huffman

Sunde

Schupp

Wagner

G. G.

(2011). Individual risk attitudes: Measurement, determinants, and behavioral consequences. Journal of the European Economic Association, 9(3), 522–550. https://doi.org/10.1111/j.1542-4774.2011.01015.x

16.

Donkers

Melenberg

Van Soest

(2001). Estimating risk attitudes using lotteries: A large sample approach. Journal of Risk and Uncertainty, 22(2), 165–195. https://doi.org/10.1023/A:1011109625844

17.

Falzon

M.-A.

(2015). Multisited field studies. In Wright

J. D.

(Ed.), International encyclopedia of the social & behavioral sciences (2nd ed., pp. 103–108). https://doi.org/10.1016/B978-0-08-097086-8.12211-1

18.

Fendt

M. W.

Harrison

Ware

S. G.

Cardona-Rivera

R. E.

Roberts

D. L.

(2012). Lecture notes in computer science: Vol. 7648. Achieving the illusion of agency (pp. 114–125). Springer. https://doi.org/10.1007/978-3-642-34851-8_11

19.

Figner

Mackinlay

R. J.

Wilkening

Weber

E. U.

(2009). Affective and deliberative processes in risky choice: Age differences in risk taking in the Columbia Card Task. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35(3), 709–730. https://doi.org/10.1037/a0014983

20.

Giolla

E. M.

(2019). What to do with all these Bayes factors: How to make Bayesian reports in deception research more informative. Legal and Criminological Psychology, 25(2), 65–71. https://doi.org/10.1111/lcrp.12162

21.

Goldbach

Schlüter

(2018). Risk aversion, time preferences, and out-migration. Experimental evidence from Ghana and Indonesia. Journal of Economic Behavior & Organization, 150, 132–148. https://doi.org/10.1016/j.jebo.2018.04.013

22.

Guo

C.-H.

(2023). An empirical framework for understanding a player’s sense of agency in games. International Journal of Human-Computer Interaction. Advance online publication. https://doi.org/10.1080/10447318.2023.2241286

23.

Halek

Eisenhauer

J. G.

(2001). Demography of risk aversion. The Journal of Risk and Insurance, 68(1), 1–24. https://doi.org/10.2307/2678130

24.

Hartog

Ferrer‐i‐Carbonell

Jonker

(2002). Linking measured risk aversion to individual characteristics. Kyklos, 55(1), 3–26. https://doi.org/10.1111/1467-6435.00175

25.

Hausknecht

Ammanabrolu

Côté

M.-A.

Yuan

(2020). Interactive fiction games: A colossal adventure. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05), 7903–7910. https://doi.org/10.1609/aaai.v34i05.6297

26.

International Organization for Migration. (2018). World migration report 2018. https://publications.iom.int/system/files/pdf/wmr_2018_en.pdf

27.

Jaeger

D. A.

Dohmen

Falk

Huffman

Sunde

Bonin

(2010). Direct evidence on risk attitudes and migration. The Review of Economics and Statistics, 92(3), 684–689. https://doi.org/10.1162/REST_a_00020

28.

Kihlstrom

J. F.

(2021). Ecological validity and “ecological validity”. Perspectives on Psychological Science, 16(2), 466–471. https://doi.org/10.1177/1745691620966791

29.

Klabbers

J. H. G.

(2009). The magic circle: Principles of gaming & simulation (3rd and rev. ed.). Sense Publishers. https://doi.org/10.1163/9789087903107

30.

Klabbers

J. H. G.

(2018). On the architecture of game science. Simulation & Gaming, 49(3), 207–245. https://doi.org/10.1177/1046878118762534

31.

Koenitz

Di Pastena

Jansen

de Lint

Moss

(2018). The myth of ‘universal’ narrative models. In Rouse

Koentiz

Haahr

(Eds.), Lecture notes in computer science: Vol. 11318. Interactive storytelling (pp. 107–120). Springer. https://doi.org/10.1007/978-3-030-04028-4_8

32.

Kothgassner

O. D.

Felnhofer

(2020). Does virtual reality help to cut the Gordian knot between ecological validity and experimental control? Annals of the International Communication Association, 44(3), 210–218. https://doi.org/10.1080/23808985.2020.1792790

33.

Kozlov

M. D.

Johansen

M. K.

(2010). Real behavior in virtual environments: Psychology experiments in a simple virtual-reality paradigm using video games. Cyberpsychology, Behavior, and Social Networking, 13(6), 711–714. https://doi.org/10.1089/cyber.2009.0310

34.

Lejuez

C. W.

Read

J. P.

Kahler

C. W.

Richards

J. B.

Ramsey

S. E.

Stuart

G. L.

Strong

D. R.

Brown

R. A.

(2002). Evaluation of a behavioral measure of risk taking: The Balloon Analogue Risk Task (BART). Journal of Experimental Psychology: Applied, 8(2), 75–84. https://doi.org/10.1037/1076-898x.8.2.75

35.

Lin-Greenberg

Pauly

R. B. C.

Schneider

J. G.

(2021). Wargaming for international relations research. European Journal of International Relations, 28(1), 83–109. https://doi.org/10.1177/13540661211064090

36.

Lübke

Décieux

J. P.

Erlinghagen

Wagner

G. G.

(2021). Comparing the risk attitudes of internationally mobile and non-mobile Germans. In Erlinghagen

Ette

Schneider

N. F.

Witte

(Eds.), The global lives of German migrants (pp. 85–100). Springer. https://doi.org/10.1007/978-3-030-67498-4_5

37.

McMahan

(2003). Immersion, engagement, and presence: A method for analyzing 3-D video games. In Wolf

M. J. P.

Perron

(Eds.), The video game theory reader (pp. 67–86). Routledge. https://www.researchgate.net/publication/284055280_Immersion_engagement_and_presence_A_method_for_analyzing_3-D_video_games

38.

Mironova

Mrie

Whitt

(2019). Risk tolerance during conflict: Evidence from Aleppo, Syria. Journal of Peace Research, 56(6), 767–782. https://doi.org/10.1177/0022343318824632

39.

Mizutani

W. K.

Daros

V. K.

Kon

(2021). Software architecture for digital game mechanics: A systematic literature review. Entertainment Computing, 38, Article 100421. https://doi.org/10.1016/j.entcom.2021.100421

40.

Murray

J. H.

(1997). Hamlet on the holodeck: The future of narrative in cyberspace. The MIT Press.

41.

Orne

M. T.

(1962). On the social psychology of the psychological experiment: With particular reference to demand characteristics and their implications. American Psychologist, 17(11), 776–783. https://doi.org/10.1037/h0043424

42.

Peters

Vissers

Heijne

(1998). The validity of games. Simulation & Gaming, 29(1), 20–30. https://doi.org/10.1177/1046878198291003

43.

Qin

Patrick Rau

P.-L.

Salvendy

(2009). Measuring player immersion in the computer game narrative. International Journal of Human-Computer Interaction, 25(2), 107–133. https://doi.org/10.1080/10447310802546732

44.

Raghothama

Meijer

(2018). Rigor in gaming for design: Conditions for transfer between game and reality. Simulation & Gaming, 49(3), 246–262. https://doi.org/10.1177/1046878118770220

45.

Shah

(2017). Ethnography? Participant observation, a potentially revolutionary praxis. HAU: Journal of Ethnographic Theory, 7(1), 45–59. https://doi.org/10.14318/hau7.1.008

46.

van Lankveld

Sehic

J. C.

Meijer

S. A.

(2016). Assessing gaming simulation validity for training traffic controllers. Simulation & Gaming, 48(2), 219–235. https://doi.org/10.1177/1046878116683578

47.

Wagner

Liu

(2021). Creating immersive learning experiences: A pedagogical design perspective. In Hui

Wagner

(Eds.), Creative and collaborative learning through immersion: Interdisciplinary and international perspectives (pp. 71–87). Springer. https://doi.org/10.1007/978-3-030-72216-6_5

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.14 MB