Taking a Glimpse Into the Future by Playing?

Abstract

Background. This article analyses whether an experimental policy-simulation might help political scientists in making predictions about prospective situations. Within the presented simulation-game, two groups of randomly assigned participants discussed the same issues but had different demographic structures. The ways in which their discussions transpired and results of the negotiation might provide a glimpse at how decisions might be made by societies in the future.

Purpose. This article presents this special simulation-game. Furthermore, it seeks to discuss whether policy-simulations can be designed as experiments, whether there is a benefit, and for whom.

Method. This study is based on both qualitative and quantitative data. The partici-pants in the simulation-game have been questioned in a pretest and in two post-tests. Furthermore, six pre- and post-interviews were conducted with involved experts.

Results. The results show that the simulation-game can provide some ideas about the future, if perhaps not necessarily a wholly accurate image of the future. Observing experts and the participants gained some knowledge about how future generations might work together and where potential conflicts might be.

Conclusion. Policy-simulations can be designed as experiments and might help to provide a glimpse into future, but designing such simulation-games is even more com-plex than designing simulationgames for a solely educational use.

Keywords

alternative purpose games designing simulations political science simulation-based learning survey-design

Introduction

The didactic and pedagogical uses of simulation-games in teaching and learning are largely accepted at this point, even though some requirements – like comprehensive assessments of simulation-based learning – may not have been entirely met yet (Asal & Kratoville, 2013; Baranowski & Weir, 2015; Duke, 2000; Patranek, Corey, & Black, 1992; Rünz, 2015; Thatcher, 1990; Usherwood, 2015; Van Dyke, Declair, & Loedel, 2000; Wolfe & Crookall, 1998). Nevertheless, Guasti, Muno, and Niemann (2015) state: “so far simulations have rarely been considered as a research tool. And although simulations do not qualify as experiments, because of the lack of random assignment of participants and control groups, they can be seen as quasi-experiments (Guasti et al., 2015, p. 211).” In their article the authors promoted using simulation-games as a “multi-dimensional resource” (ebd.) for combining educational and research purposes.

This article intends to present and analyse a policy-simulation that aims at the abovementioned experimental design to function as both an educational and a research tool. By doing so, this article will address whether simulation-games have a benefit by using them as experiments and for whom there might be a benefit. Secondly, this article will show how the setting of the used simulation-game tries to fulfil these needs, and thirdly, it will present one way in which such a simulation-game can be evaluated.

Design and Background of the Simulation-Game

The Parliament of Generations¹ was created in 2013 for the German Federal Ministry of Education and Research within the “Science Year 2013 – The Demographic Opportunity” that is a yearlong series of different activities. Additionally, the PdG was played in 2016 in cooperation between the Bavarian State Parliament and the Akademie für Politische Bildung (Academy for Civic Education), Tutzing. The survey presented in this article is a result of this second iteration of the PdG. The author of this article was part of the team that created as well as conducted the PdG.

The purpose of creating the PdG was to teach and learn about the effects of demographic changes within German society. Furthermore, the PdG should help to understand whether and in which ways demographic changes might have an impact on the process of policy-making within parliaments. However, the design of the simulation-game fulfils most of the abovementioned requirements for also being a truly experimental simulation-game: a randomised assignment of participants as well as a control group. Furthermore, the selection of the participants is based on the demographic characteristics of age, gender and migration background related to the demographic structure of Germany society in 2016 and as predicted for 2050.

With these choices, we tried to connect learning and researching within simulation-games as they have customarily been combined in other sectors: “Simulations have long been part of scientific research methods. Meteorologists use computer simulations to help predict the path of weather fronts; economists use them to make economic forecasts for an economy; military strategists use simulations to conjecture about the course of events during military campaigns; and the list could go on (Boyer & Smith, 2015, p. 315).”

Asal and Blake suggest a couple of decisions that game designers have to make before creating a simulation-game (Asal & Blake, 2006). Their framework suggests structuring around design choices that can be briefly describes as “real or fictional case”, “level of complexity”, “participants” as well as the “decision-making process (Asal & Blake, 2006, p. 7).”

The PdG is based on a fictional case-design. It involves a parliament – or more accurately, two parliaments – which are comprised of groups defined by the different generations that they represent, not by political parties: The 1^st Generation contains members who are 15-30 years old; the 2^nd Generation those who are 31-50 years of age; the 3^rd Generation includes those 51-66 years old and the 4^th Generation features persons older than 66 years. These generations are instructed to work together as if they are separate parliamentary groups. The participants were selected through an application process. The Members of the Bavarian State Parliament were asked to promote the project among their constituencies in Bavaria and interested people could apply for participation. The selection of the participants was based on the named demographic characteristics: age, gender and migration background. The result was that there were two parliaments: one randomised by the characteristics for 2016 and the other via the same criteria for 2050 – and both discussed the same issues.

The simulation-game’s level of complexity is very high as two different issues are discussed: a proposal for developing regions by strengthening facilities to secure livelihoods in rural areas and a campaign for higher-quality education. Both issues are very complex and designed for creating conflicts between the different generations. Discussing the education topic, the participants have to decide whether they want to prioritise increasing efforts for strengthening lifelong learning or primary education, just to mention one point of the discussions. Furthermore, the process of decision making enhances the complexity of the simulation-game. The participants have to work in group meetings within their generation, have to participate in one of two thematic committees and have to vote within the plenary about both issues.

The objectives of the simulation-game are to give the participants insights into the way demographic changes in societies might affect the process of decision making and provide insights into the overall shift of interests. It might influence the decisions of parliaments if a greater number of older people with different cultural backgrounds are living in a society. Secondly, the participants should gain insights into parliamentary work in general. Furthermore, selected experts observed the simulation-game and tried to get insights into the way decision-making processes might change as the societies’ sociodemographic backgrounds shift, which might be relevant for experts as well as politicians and other interested parties.

Design of the Survey

The data for this project are collected by a panel survey with three times of measurements as well as semi-structured interviews with experts. Both the survey as well as the interviews complement each other but are not built upon each other, like mixed methods are (Flick, 2008; Teddlie & Tashakkori, 2010).

Within the participants-based longitudinal survey, the participants were confronted with the same online questionnaire two weeks prior to their participation in the simulation-game (time of measurement 1), four weeks after the simulation-game (time of measurement 2) and again one year after the simulation-game (time of measurement 3). The range for taking part in each single time of measurement was limited to two weeks. The relatively longer period between the treatment and the second time of measurement involved weighting the benefits against the disadvantages: it may be that the internal validity could be harmed by a longer period; for example, if the participants should learn about demographic change in another way between treatment and measurement. However, it is also possible that what we have told them about the experiment in the simulation could influence the participants, or that they could remember what they answered at the first time of measurement, which could create more problems (de Vaus, 2010, p. 72 f.). Furthermore, the more measurements are conducted, the less likely is that the responses over time will be constant (de Vaus, 2010).

The questionnaire itself includes two major parts: one part aims at the personal background of the participants in order to gain knowledge about their sociodemographic backgrounds. Within the second part, the participants were asked about their views on demographic changes in societies. Additionally, a question for constructing an individual identification code was included in the questionnaire. This enabled me to match the answers afterwards without violating the participants’ anonymity.

The questions within the questionnaire were mostly closed questions with four possible answers as well as one neutral variable (“don’t know”). There were no filter questions: all participants had to answer every single question – skipping questions was not possible (Bryman, 2016; ESOMAR ESOMAR World Research, 2011, p. 230; 2011, p. 230; Toepoel, 2017).

This study also includes the experiences of six experts who participated in the PdG along with the participants. The experts’ area of expertise covered all relevant parts of the simulation-game: game-designing as well as the issues discussed in the simulation-game. They themselves participated in the simulation-game as experts with their knowledge about the content of the simulation-game to inform and help the participants on the one hand, and had the objective to overserve the participants’ behaviour on the other hand (Angrosino & Rosenberg, 2011; Creswell, 2007). They were asked to participate in two interviews to gain knowledge about their view on the simulation-game. All six experts participated in the semi-structured interviews, which were conducted four weeks prior to the simulation-game as well as four weeks afterwards (Kvale, 2007, p. 36). The interviews had a duration of 35 minutes on average. Through this design, the organizer of the simulation-game hoped to deepen the insights into the way decision-making processes will change among different demographic configurations. Getting insights into this specific knowledge of the experts gained by the observation was the subject of the interviews.

Even if “flexibility is a key requirement of qualitative interviewing” (King & Horrocks, 2010, p 35) and the interviewer should be able to react to new issues within the interview, an interviewer’s guide outlined major topics for structuring the interviews in the case at hand (King & Horrocks, 2010, p 35f.). The selected questions used for the interviewers’ guide were based on the systematization developed by Michael Patton (1990). For the first round of interviews, after a short introduction with basic information about the purpose and proceedings of the interviews, the attendees were asked to describe what they understood the main goals of the PdG to be. This was followed by questions about the structure of the simulation-game as well as the design of the simulation-game itself. Lastly, questions about the content discussed within the simulation-game were prescribed to end the interview. These questions were then adjusted for the second round of interviews. Within this second round, the experts were asked whether their experiences of the simulation-games had met their expectations formulated in the previous interview.

Description of the Participants

In order to introduce the analyses of the participants’ views on demographic change and the simulation-game, within the following section the participants will be briefly described.

Demographics of the Participants

In total, 133 people participated in the simulation-game. 64 belonged to the group who represented 2016, 69 to the group who represented 2050 [see Table 1].

Table 1.

Participants in the PdG.

	2016	2050	All
1st Generation (15-30 years)	12 (19 %)	13 (19 %)	25
2nd Generation (31-50 years)	20 (31 %)	17 (25 %)	37
3rd Generation (51-66 years)	16 (25 %)	15 (22 %)	31
4th Generation (67+ years)	16 (25 %)	24 (35 %)	40
Total	64	69	133

A more detailed allocation within the different demographic requirements is shown in Table 2, whereby Table 2 shows the results of the projection, not of the de facto allocation of participants. The underlying projection of the population was created specifically for this project by Nora Sánchez Gassen.²

Table 2.

Calculation of Participants (Age, Gender and Migrant Background) in Percent.

Projected Participants “2016”
	No Migrant Background		Migrant Background
Age	Male	Female	Male	Female	All
1^st Generation (15-30 years)	7.6 %	7.3 %	3 %	2.7 %	20 %
2^nd Generation (31-50 years)	11.8 %	11.5 %	3.8 %	3.9 %	31 %
3^rd Generation (51-66 years)	11.1 %	11.4 %	2 %	2.1 %	27 %
4^th Generation (67+ years)	8.7 %	11.4 %	1 %	1.1 %	22 %
Total	39 %	41 %	11 %	9 %	100 %
Projected Participants “2050”
	No Migrant Background		Migrant Background
Age	Male	Female	Male	Female	All
1^st Generation (15-30 years)	5.1 %	5.1 %	3.8 %	3.6 %	17 %
2^nd Generation (31-50 years)	6.8 %	6.5 %	5.7 %	5.1 %	24 %
3^rd Generation (51-66 years)	7.7 %	7.5 %	4 %	3.6 %	23 %
4^th Generation (67+ years)	12.7 %	14.5 %	3.9 %	4.4 %	36 %
Total	32 %	33.3 %	17.3 %	17.3 %	100 %

Summarizing Tables 1 and 2, some participants who are older and have a migrant background were not adequately represented. In their place, some groups were slightly overrepresented, foremost younger people in the 2050 group. Nevertheless, the requirements for getting a representative sample were almost reached. Migrant background is defined by the organizers as that the individual in question, or one of his or her parents, has been born abroad or as a migrant in Germany. Most important for the project were the changes within the distribution of the different generations. There are projected to be more people in the older demographics in 2050 than in 2016. Therefore, the 4th Generation in 2050 has the majority instead of the 2nd Generation, as in 2016.

Response Rates of the Participants on the Questionnaire

The response rate of participants to the questionnaire has been very high (Berrens, Bohara, Jenkins-Smith, Silva, & Weimer, 2003; Heerwegh, 2005; Kaplowitz, Hadlock, & Levine, 2004). At the time of the pre-test 74 per cent of all participants completed the questionnaire. At the second time of measurement 57 per cent participated in the survey, or in total 76 persons. Even at the second follow-up, 32 per cent (43 persons) participated in the survey.

Having a closer look at the response rates, Figure 1 illustrates that older people participated at a higher rate in the last follow-up questionnaire in comparison to the other generations, especially in the last measurement. These rates of responses are astonishing because of the method used, the online questionnaire, in which older people normally do not participate as much as younger people do (Goyder, 1987; McFadden & Winter, 2001). Even if this may causes a small bias within the sample, a statistical correction for the analysis is not helpful because of the smaller N after a correction. Furthermore, this is just a slight deviation.

Figure 1.

Response rates (in per cent) at the different times of measurements split by age.

Participants’ Level of Interest in Politics

Finally, a closer look into the participants’ political background shows one problem of the survey: a selection-bias. Most of the participants stated that they have a strong or very strong interest in political matters [Figure 2].

Figure 2.

Interest in politics (in per cent) at the different times of measurements.

This self-estimated interest might be caused by the method of recruitment of the participants. Some of those involved in the recruitment process involved Members of the Bavarian State Parliament, who spread the call for applications in the regions that they come from through announcements on their websites or in their local newspapers. However, other Members directly got in contact with potential participants. Both methods might have produced the effect that largely people with a high level of personal interest in politics, as well as knowledge of political processes applied for participation.

Summarizing the Demographics of the Participants

Within the described simulation-game, two groups discussed the same issues. Some factors have to be kept in mind when working with the resulting data: there is a slight overrepresentation of older people in the responses to the questionnaire as well as a selection-bias among all participants because of the fact that more people with a high level of interests in politics participated in the simulation-game.

Participant Feedback

The questionnaire aims to gather general data about the participants but foremost to test the participants’ view on demographic change in societies on the one hand and the self-estimated personal value of their own participation in the simulation-game. Because of these dual goals, questions pursuing both are embedded and especially the questions regarding demographic change have been addressed within the simulation-game in detail. The participants discussed the two named topics during the simulation-game and were confronted multiple times with the changes in society caused by demographics and their impact on politics.

Participants’ Knowledge About Demographic Change

In answer to one of the questions, participants must evaluate their own knowledge about demographic changes. Overall, the average knowledge of the participants about demographic changes is good. At the first time of measurement before participating in the simulation-game, the average was at 3.6 on a scale from 1 = very low, up to 5 = very good (M = 3.60, SD = .66). One year after the simulation-game, the indicated knowledge was the highest with 4.0 on average (M = 4.00, SD = .66) [Figure 3].

Figure 3.

Means of the self-estimated knowledge about demographic change at the different times of measurements.

A repeated measurement analysis of variances (rmANOVA) is significant (F(1.99, 83.77) = 3,57, p > .05), but there was a change in the means after the second measurement, too.

There are some possible explanations. It could be that the participants gained more knowledge on demographic change through other intervening variables. Furthermore, it might be possible that more participants with a higher self-estimated knowledge participated in the last measurement or that the participants changed their estimation of their knowledge about demographic changes. All three reasons are equally possible.

Effect of Participation

Next, the participants were asked about their perception concerning the effect on their own view on demographic change of participating in the simulation-game. 45 per cent stated that participating in the PdG had changed the nature of their own views [Figure 4], time of measurement 2, very strong and strong combined) Nevertheless, a large number did not see any effect on their own views, or did not see a strong effect. A paired-sample t-test did not show significant changes between the second and third times of measurement (t(42) = -1.96, p = .056).

Figure 4.

Effect of participation on participants’ view on demographic change (in per cent) at the second and third times of measurement.

After one year, at the third time of measurement, more than half of the participants who answered the questionnaire stated that participating in the simulation-game had changed their view on demographic changes. This is rendered even more interesting by the fact that the same participants estimated themselves as having a very high level of knowledge about demographic changes.

Personal Views on Demographic Change

Additionally, the participants were asked if they thought that a demographic change might be a challenge or an opportunity for societies [Table 3]. In all three times of measurement most participants stated that it was a challenge. Furthermore, most of the participants answered at all three times of measurements that - in their opinion - a demographic change in societies has a strong influence on (the process) of political decision making and the decisions themselves [Figure 5]. There was no significant change in this belief among the participants (rmANOVA is not significant (F(1.77, 74.28) = .41, p = .64)).

Table 3.

Demographic Change Is a….

	Time ofmeasurement 1	Time ofmeasurement 2	Time ofmeasurement 3
… challenge	72 %	62 %	79 %
… opportunity	23 %	38 %	19 %
Do not know	5 %	0 %	2 %

Figure 5.

Impact of demographic change on political decisions.

Expert Feedback

In order to conduct the analysis with the experts, I followed Mayring’s instructions for structuring summarizing qualitative content analysis (Mayring, 2014). Within this qualitative analysis he proposes seven steps to reduce and compare the material. By doing so, the main messages of the experts could be compared and related to one another (Mayring, 2014, p. 65f.). The focus of this section is on the experts’ views on (1) the simulation-game itself, (2) the conclusions they have drawn about the impact of the demographic change on political decisions as well as (3) whether benefits might be taken from this special simulation-game, and if so, by whom.

Goals of the Simulation-Game

At the beginning of the interviews the experts were asked which were – in their point of view – the most important goals of the PdG. All of them distinguished between three goals: the participants should learn about demographic changes and the implications these changes might cause; they should learn about the ways in which decisions are made in modern parliaments and that it is very difficult to reach consensus between different interests; and that it should be analysed whether and in which ways demographic changes might affect decision-making processes. When asked about their own prioritisation of these goals, most of the experts said that the participants’ gaining knowledge about demographic changes was most important. Two of them said that analysing effects of demographic changes on decision-making processes was most important. One expert described these goals as interdependent. Following this opinion, the PdG might cause some kind of reciprocal learning: an experiment wherein all, the participants and the operators (and experts), might learn a lot. The participants might learn about demographic changes, the others about the way these changes might influence decisions.

Design of the Simulation-Game

A second set of questions referred to the design of the simulation-game. All experts underlined that the methods used in the PdG were unique and that they did not know any other simulation-game trying to combine education and scientific research in that way. Some of the experts were worried about the complexity of the simulation-game. They expressed their concerns that this complexity might overstress the participants, especially the older ones. Nevertheless, almost all experts made another very important point: they highlighted that the simulation-game was a truly fictional case, not related to the topics discussed but to the division of the participants into different groups by age (the generations), not by political interest or party affiliation. This is a major difference to the way modern parliaments normally are organised and, like one expert said, nobody assumes that in 2050 interests and the organisation of parliaments will be by age and not by political affiliation.

Results of the Simulation-Game

In the post-interviews all experts were satisfied with the simulation-game itself and with what they could observe during the simulation-game. Summarizing the processes within the simulation-game, there were conflicting interests between the middle-generations, 2nd and 3rd Generations (31 – 50 years and 51 – 66 years old) on the one side and the youngest and oldest generation, 1st and 4th Generations (15-30 and 65+ years old) on the other side. Especially the youngest and oldest generation worked together as a kind of coalition.

In view of these circumstances, the experts drew some interesting conclusions. One expert told me in the first interview that in his or her field of interest there are contradictory assumptions about the way the different generations will work together in the future. One assumption states that foremost the oldest and the youngest generations will work together. Others hypothesize that these two generations will mostly be interested in their own needs. The same expert told me in the post-interview that his/her observations during the PdG indicate that the first assumption might be more likely.

Another expert described an assumption in his/her field of interest stating that future societies must take care not to leave the middle-generations behind. This middle-aged group (especially the 2nd Generation at the PdG), also called the ‘Sandwich-Generation’, has to deal with caring for both the youngest and oldest people. But their number will be smaller than they are today. Combined with the assumption that the youngest and oldest generations might cooperate more closely, this might give this middle-aged group the feeling that the needs of those who give most are heard least (e.g. Abramson, 2015; Boyczuk & Fletcher, 2016). The expert formulated concerns that this feeling could cause the generation to generate some kind of protesting behaviour. This process was observed within the PdG, too. The 2nd generation could not get their interests represented within the process of formulating proposals to vote on and strongly tried to prevent the proposals from reaching a majority at the final voting. The expert told me that this observation confirmed that the described assumption might be accurate.

Discussion

Within this discussion of the simulation-game, I want to address three major points: Firstly, this special simulation-game might have a benefit for experts. Secondly, the major goals of this simulation-game remain in teaching participants. Lastly, the simulation-game at hand seems to have a selection-bias, which had an impact on the way the participants could learn.

Interviewing the experts involved in the simulation-game, most of them talked about a benefit for their own work. They were able to gain new information about effects of demographic changes on societies in general and decision-making processes in particular. Furthermore, they were able to examine existing assumptions and get indications of which of these assumptions might be correct or not. However, the benefits of this simulation-game can just provide some hints about the future. It cannot provide a perfect prediction of the future. For getting such a thing, other major variables we probably do not yet know would have to be included. Furthermore, creating as well as conducting this kind of simulation-game is even more complicated than others are. This complexity has an effect on the participants, too. The materials for preparation have to describe the way the simulation-game works in detail; during the simulation-game, a number of supporting staff is needed.

The main objective for the participants in this simulation-game still is to learn about the subject of the simulation-game. Within the PdG, the participants said that they were convinced of the importance of learning about the effects of demographic changes on societies in general and decision-making processes in particular. They also reported that they had learned about politics in general. However, the way the experts could use the experimental design of the simulation-game is different from how the participants could use it.

Furthermore, the manner of recruiting the participants for the PdG produced a selection-bias. The participants were very well informed about politics before they participated in the simulation-game. Following the abovementioned analysis of the responses on the questionnaires, the participants have not learned much and have not changed their minds about impacts of democratic changes. Even if the participants are randomly assigned by the criteria age, sex and migration background, other criteria might be important, too. The issues discussed in the simulation-game, education as well as livelihood in rural areas, suggest that educational background and the place of residence might be proper criteria, too, next to other criteria not mentioned so far. The migration background was not of importance during the simulation-game. Sometimes participants argued based on their experiences regarding their migration background; however, this was not dominant during the discussions.

Conclusion

Experimental simulation-games are possible and they might provide specific added value. The simulation-game described here, the Parliament of Generations, is one example off how this can be done. Everyone, experts as well as participants, can learn by participating in a simulation-game like this. They can be the named ‘multi-dimensional resource’ for combining teaching and research. Or, like Rünz argues “researchers of political simulations should broaden their perspective by addressing research questions that go beyond pedagogical goal assessment (Rünz, 2015, p. 276).” Nevertheless, designing and conducting this simulation-game seems to be even more difficult than doing so with traditional simulation-games. Furthermore, the results presented here cover this unique simulation-game, the PdG. To produce stronger results about the real value of experimental simulation-games, more games, and different ones, should be conducted and focussed upon.

Nevertheless, we were able to demonstrate that experimental simulation-games with randomized groups of participants can be realized. In order to do so, a fictional scenario was created within which the participants were divided into two major groups. One group was structured according to the demographics of 2016, the other with the projected demographics for 2050. By doing so, differences in the distribution of the majorities have been shown. The group of 2050 was older and there were more participants with migration backgrounds. Demographic changes of societies became visible. The focus of the PdG was to show the influence of these changes on decision-making processes.

Experts from different fields of interests attended the simulation-game as observers. Both participants and experts reported that they had gained new information about demographic changes. Furthermore, some of the experts could connect this information with existing assumptions on effects of demographic changes in their respective fields of interest, for example the behaviour of the middle-aged generations in the simulation-game. Even if this new knowledge does not offer a perfect image of the future, this experimental simulation-game might give a glimpse at the future.

Footnotes

Acknowledgements

A warm thank you to the participants of the simulation. Without their dedication, the simulation and the survey would have failed. A special thank you goes to the Akademie für Politische Bildung, Tutzing, the anonymous experts, as well as to the team of the Bavarian State Parliament.

Declaration of Conflicting Interests

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author received no financial support for the research, authorship, and/or publication of this article.

Notes

Author Biography

Robert Lohmann is a research and teaching assistant at the Jean-Monnet-Chair for European Politics at the University of Passau, Germany. He has a Master’s degree in Science of Democracy from the University of Regensburg, Germany, where he worked for three years at the Chair for International Politics and Transatlantic Relations before he moved to Passau. Furthermore, he has been permanent visiting lecturer at the Akademie für Politische Bildung (Academia for Civic Education), Tutzing, Germany, from 2012 to 2019 with the responsibility for simulation-games. His research focus is on European politics and the system of the European Union as well as on the effects of simulation-based learning.

Contact: robert.lohmann@uni-passau.de

References

Abramson

T. A.

(2015). Older adults: The “panini sandwich” generation. Clinical Gerontologist, 38(4), 251–267. doi:10.1080/07317115.2015.1032466

Angrosino

Rosenberg

(2011). Observations on observation: Continuities and challenges. In Denzin

N. K.

Lincoln

Y. S.

(Eds.), The SAGE handbook of qualitative research (pp. 467-496). Los Angeles, CA: SAGE.

Asal

Blake

E. L.

(2006). Creating simulations for political science education. Journal of Political Science Education, 2(1), 1–18. doi:10.1080/15512160500484119

Asal

Kratoville

(2013). Constructing international relations simulations: Examining the pedagogy of IR simulations through a constructivist learning theory lens. Journal of Political Science Education, 9(2), 132–143. doi:10.1080/15512169.2013.770982

Baranowski

M. K.

Weir

K. A.

(2015). Political simulations: What we know, what we think we know, and what we still need to know. Journal of Political Science Education, 11(4), 391–403. doi:10.1080/15512169.2015.1065748

Berrens

R. P.

Bohara

A. K.

Jenkins-Smith

Silva

Weimer

D. L.

(2003). The advent of internet surveys for political research: A comparison of telephone and internet samples. Political Analysis, 11(1), 1–22. doi:10.1093/pan/11.1.1

Boyczuk

A. M.

Fletcher

P. C.

(2016). The ebbs and flows: Stresses of sandwich generation caregivers. Journal of Adult Development, 23(1), 51–61. doi:10.1007/s10804-015-9221-6

Boyer

Smith

E. T.

(2015). Developing you own in-class simulations: Design advice and a ‘commons’ simulation example. In John

Miller

W. J.

Simon

(Eds.), Handbook on teaching and learning in political science and international relations (pp. 315-326). Cheltenham, UK: Edward Elgar.

Bryman

(2016). Social research methods (5th ed.). Oxford, UK: Oxford University Press.

10.

Caswell

Sánchez Gassen

(2015). The sensitivity analysis of population projections. Demographic Research, 33, 801–840. doi:10.4054/DemRes.2015.33.28

11.

Creswell

J. W.

(2007). Qualitative inquiry and research design: Choosing among five approaches. Thousand Oaks, CA: SAGE.

12.

de Vaus David

(2010). Research design in social research (Reprinted). Los Angeles, CA: SAGE.

13.

Duke

R. D.

(2000). A personal perspective on the evolution of gaming. Simulation & Gaming, 31(1), 79–85. doi:10.1177/104687810003100107

14.

ESOMAR World Research. (2011). World research codes and guidelines: ESOMAR guideline for online research. Online-resource (29.11.2017). Retrieved from https://www.esomar.org/uploads/public/knowledge-and-standards/codes-and-guidelines/ESOMAR-GRBN-Online-Research-Guideline-October-2015.pdf

15.

Flick

(2008). The SAGE Qualitative Research Kit. Managing quality in qualitative research. Los Angeles, CA: SAGE.

16.

Goyder

(1987). The silent minority: Nonrespondents on sample surveys. Cambridge, UK: Polity Press.

17.

Guasti

Muno

Niemann

(2015). Introduction – EU simulations as a multi-dimensional resource: From teaching and learning tool to research instrument. European Political Science, 14(3), 205–217. doi:10.1057/eps.2015.18

18.

Heerwegh

(2005). Effects of personal salutations in e-mail invitations to participate in a web survey. Public Opinion Quarterly, 69(4), 588–598. doi:10.1093//nfi053

19.

Kaplowitz

M. D.

Hadlock

T. D.

Levine

(2004). A comparison of web and mail survey response rates. Public Opinion Quarterly 68(1), 94–101. doi:10.1093/poq/nfh006

20.

King

Horrocks

(2010). Interviews in qualitative research. Los Angeles, CA: SAGE.

21.

Kvale

(2007). The SAGE Qualitative Research Kit. Doing interviews. London, England: SAGE.

22.

Mayring

(2014). Qualitative content analysis: Theoretical foundation, basic procedures and software solution. Klagenfurt. Retrieved from http://nbn-resolving.de/urn:nbn:de:0168-ssoar-395173

23.

McFadden

Winter

(2001). Experimental analysis of survey response bias over the internet: Some results from the Retirement Perspectives Survey. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.194.4676&rep=rep1&type=pdf

24.

Patranek

C. F.

Corey

Black

(1992). Three levels of learning in simulations: Participating, debriefing, and journal writing. Simulation & Gaming, 23(2), 174–185. doi:10.1177/1046878192232005

25.

Patton

M. Q.

(1990). Qualitative evaluation and research methods. Newbury Park, CA: SAGE.

26.

Rünz

(2015). Beyond teaching: Measuring the effect of EU simulations on European identity and support of the EU. European Political Science, 14(3), 266–278. doi:10.1057/eps.2015.23

27.

Sánchez Gassen

N. E.

(2015). Germany’s future electors: Developments of the German electorate in times of demographic change. Wiesbaden, Germany: Springer Research.

28.

Teddlie

Tashakkori

(2010). Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioural sciences. Los Angeles, CA: SAGE.

29.

Thatcher

D. C.

(1990). Promoting learning through games and simulations. Simulation & Gaming, 21(3), 262–273. doi:10.1177/1046878190213005

30.

Toepoel

(2017). Online survey design. In Lee

R. M.

Blank

Fielding

N. G.

(Eds.), The SAGE handbook of online research methods (2nd ed., pp. 184-202). Los Angeles, CA: SAGE.

31.

Usherwood

(2015). Building resources for simulations: Challenges and opportunities. European Political Science, 14(3), 218–227. doi:10.1057/eps.2015.19

32.

Van Dyke

G. J.

Declair

E. G.

Loedel

P. H.

(2000). Stimulating simulations: Making the European Union a classroom reality. International Studies Perspectives, 1(2), 145–159. doi:10.1111/1528-3577.00014

33.

Wolfe

Crookall

(1998). Developing a scientific knowledge of simulation/gaming. Simulation & Gaming, 29(1), 7–19. doi:10.1177/1046878198291002