Details and Dynamics: Mental Models of Complex Systems in Game-Based Learning

Abstract

Background. Although the effectiveness of game-based learning (GBL) is well-supported, much less is known about the process underlying it. Nevertheless, developing a mental model that matches the game system, which in turn models a real-world system, is a promising proposed process.

Aim. This article explores the first steps in model matching: identifying the entities and (complex) relations in a game system.

Method. Participants (N = 30) played the analog game DOMINION and completed a multi-step mental model mapping exercise. Categories of entities in mental model maps were inductively identified with grounded theory coding, while complex relations in mental model maps were identified via content analysis.

Results. Participants described formal game entities, player actions, sociality, learning processes, and subjective experience in their mental model maps. Participants identified very few complex relations—and no feedback loops—in their mental model maps.

Conclusions. Games—and analog games specifically—provide a breadth of resources for model matching and GBL. Through gameplay, learners come to affix conceptual meanings to material objects, a process dubbed lamination.

Keywords

analog games complex systems game-based learning GBL mental models model matching

The ability to understand systems, or systems thinking, is a crucial competency within science, technology, engineering, and math (National Research Council, 2012), organizational studies (Checkland, 1999), social sciences (Garson, 2009), and social psychology (Levine & Doyle, 2002). The systems at issue in these disciplines share a degree of complexity that makes understanding them inherently difficult (Sterman, 1994). Modeling complex systems with simplified games or gameful simulations—themselves complex systems in miniature—is thought to facilitate an understanding of those systems that can then be applied in non-game contexts (Landriscina, 2013). This process of developing a cognitive understanding of game systems in relation to non-game systems is not yet well understood—including the first steps of developing understandings of the components and features of complex systems encountered during gameplay. This study explored that potential, finding that while participants identified and externalized a broad range of entities in an analog game system, identification of complex relations was limited.

Complex Systems in Games and Life

Many conceptualizations of games treat them as systems of interrelated entities (e.g., Giddings, 2009; Taylor, 2009). Because the components of systems vary so dramatically—from material to immaterial, human to nonhuman, static to processual (Juul, 2011)—they are here referred to generically as entities. Broadly, systems are composed of various entities and their interrelations (Groesser & Schaffernicht, 2012). While ludic, or gameful, systems function to make games challenging and fun, systems with similar properties exist naturally in non-game contexts. For instance, a relatively simple ecosystem might include one population of predators and one of prey. As predators eat prey, predator populations grow and prey populations shrink. Eventually, finding prey becomes difficult, and predator populations dwindle. As predation declines, prey populations recover and grow. The notion underlying game-based learning is that understanding systems can be facilitated by playing games or gameful simulations modeling their dynamics (Martinez-Garza & Clark, 2017).

Recent meta-analyses provide support for the effectiveness of game-based learning across subjects and age groups (Clark, Tanner-Smith, & Killingsworth, 2016; Wouters & van Oostendorp, 2017). Specific examples of learning systems from games and gameful simulations span elementary students learning genetic and cellular systems (Corredor, Gaydos, & Squire, 2013) and Newtonian physics (White, 1984); middle school students learning economic systems (Foster, 2011); and undergraduates learning environmental systems (Stave, Beck, & Galvan, 2015), governmental systems (Nishikawa & Jaeger, 2011), and economic systems (Doyle, Radzicki, & Trees, 2008).

Systems thinking competencies exist at every level of Bloom’s revised learning taxonomy (Anderson & Krathwohl, 2001; see Stave & Hopper, 2007). The lowest levels of systems thinking involve identifying a system’s components or entities. At higher levels, systems thinking involves understanding how entities relate to each other. At yet higher levels, it involves using those understandings to make predictions, guide interventions, or build systems. As a first step in understanding the process of learning game systems, this study focuses on the foundational systems thinking competencies of identifying the entities within a system and how they influence each other. These competencies are conceptualized as understanding detail system complexity and dynamic system complexity, respectively.

Detail Complexity

System complexity due to the number and kinds of parts within a system is detail complexity (Senge, 2006). The more entities in a system, the greater its detail complexity. Detail complexity makes understanding complex systems difficult, likely because it strains cognitive working memory capacity by requiring individuals to attend to more bits of information simultaneously (cf. Sweller, van Merrienboer, & Paas, 1998). The breadth of types of entities—e.g., conceptual and material, human and nonhuman—in a system may also be a component of detail complexity, requiring individuals to integrate disparate entity types into their understanding of a single system (cf. Doyle et al., 2008). Identifying a system’s parts is foundational to higher-level learning about complex systems (Stave & Hopper, 2007).

Dynamic Complexity

System complexity emerging from particular kinds of relations among entities in a system is dynamic complexity (Senge, 2006). Common heuristics for these relations typically treat them as direct, linear, deterministic, and immediate (Sterman, 1994; Sweeney & Sterman, 2007). Five types of relations, however, do not conform to these heuristics and are therefore considered complex: indirect relations, nonlinear relations, stochastic relations, time-delayed relations, and feedback loops. Because they violate heuristic understandings of causal relations, the prevalence of these individual complex relations contributes to a system’s overall dynamic complexity. An indirect relation is one in which an entity influences another via at least one other (Schaffernicht & Groesser, 2013). Whereas simple, linear relations involve a constant proportion of input to output, nonlinear relations change depending on the input (Sweeney & Sterman, 2007)—the magnitude of one entity’s influence on another depends on the quantity or strength of the influencing entity (e.g., curvilinearity of exponential and logarithmic relations). Because they involve randomness, stochastic relations are non-deterministic (Resnick & Wilensky, 1998)—the influence of one entity on another is probabilistic. Time-delayed relations involve a span of time between an event or an input and its consequences (Sweeney & Sterman, 2007); the influence of one entity on another does not occur immediately, but after some temporal lag. In combination, relations among entities of a system can form feedback loops (Sweeney & Sterman, 2007), such that an entity influences itself, often through indirect relations.

Learning Complex Systems as Mental Model Development

Although the cognitive mechanisms underlying game-based learning are not yet well understood, it has been suggested that learning about a complex system through gameplay is a process that involves understanding and forming a cognitive representation, or mental model, of that system (Martinez-Garza & Clark, 2017). Mental models of complex systems are cognitive representations of systems that include both a) the entities an individual perceives to be part of that system and b) how those entities relate to each other (Doyle & Ford, 1999). Based on perceptions of, interactions with, and communication about a system—along with pre-existing stocks of knowledge and experience—individuals are thought to develop mental models specific to that system (Landriscina, 2013). Through repeated gameplay, mental models should become more closely aligned with the game system’s structure, a process known as model matching (Boyan & Sherry, 2011). When game systems represent real-world systems, mental models of game systems should ostensibly be transferrable or applicable to those real-world systems (Martinez-Garza & Clark, 2017). Like systems, mental models can be characterized in terms of detail complexity (quantity and breadth of entities) and dynamic complexity (complex types of relations).

Complexity of Game Mental Models

Given the scant state of research on model matching (Boyan & Sherry, 2011) and learning systems from gameplay (Davidsen & Spector, 2015), this study explores the first steps of the model matching process described above: identifying entities (detail complexity) and discerning complex entity relations (dynamic complexity). We ask:

Research Question 1: What discrete entities do individuals include in their mental models of a complex game system (detail complexity)?

Research Question 2: To what extent do individuals identify complex relations among entities in their mental models of a complex game system (dynamic complexity)?

Method

Participants

Participants (N = 30) were recruited from a large, public university and the surrounding community. Sampling was performed to saturation: the point at which no new information is captured, leading to diminishing empirical returns (Bowen, 2008; Özesmi & Özesmi, 2004). To identify the saturation point, data was coded as it was collected, constantly comparing the new to the old (Glaser, 1965) until no more than one new entity had been identified in the most recent three cases. Saturation was reached at N = 30. Of these, 27 were enrolled in college, 17 identified as female and 13 as male, 27 as white, and the average age was 27 (SD = 12.5, range = 18-69). Two participants indicated that they had minimal experience with the stimulus game, while all others reported no experience.

Stimulus Materials

Participants played the modern analog game, DOMINION (Vaccarino, 2008; see Figure 1). In DOMINION, players play as feudal lords expanding their domains. Each turn, players play action cards that provide special abilities, acquire new cards by playing treasure cards representing their kingdoms’ resources, and eventually acquire victory cards representing their lands worth points at the end of the game. Participants played with the action cards recommended by the rulebook for first-time players: Cellar, Moat, Village, Woodcutter, Workshop, Militia, Remodel, Smithy, Market, and Mine, as well as the standard treasure cards (Copper, Silver, and Gold) and victory cards (Estate, Duchy, and Province). These action cards allowed players to draw more cards (Cellar, Moat, Village, Smithy, Market), play more action cards (Cellar, Village, Market), acquire more and costlier cards (Woodcutter, Workshop, Market), upgrade their cards (Remodel, Mine), attack each other (Militia), and defend against attacks (Moat).

Figure 1.

Layout of DOMINION during study sessions.

DOMINION was selected because it contains the five complex relations described above: indirect, nonlinear, stochastic, time-delayed relations, and feedback loops. In DOMINION, an example of an indirect relation is that between treasure cards and victory cards. Although the goal of DOMINION is to earn the most points by acquiring victory cards, it is not possible to acquire the most valuable victory cards without first using treasure cards to acquire actions cards (e.g., Village, Workshop, Mine) and more valuable treasure cards (i.e., Silver, Gold). As such, action cards and other treasure cards are intermediaries in the relation between treasure cards and victory cards. Furthermore, the relation between treasure cards and victory cards is nonlinear: more valuable victory cards (e.g., Province) cost more treasure, but yield an increasingly greater proportion of points to treasure. Specifically, 1-point Estate victory cards cost 2 treasure (1:2), 3-point Duchy victory cards cost 5 treasure (3:5), and 6-point Province victory cards cost 8 treasure (3:4). The ratio of points acquired to treasure spent increases from .5 to .6 to .75. Many relations in DOMINION are also stochastic, because each player’s deck of cards is repeatedly randomized by shuffling. The particular combination of cards available to each player each turn, therefore, involves randomness, as the cards drawn from players’ decks are randomized. Furthermore, the effect of newly acquired cards are time-delayed, since they do not come into play until they are shuffled into a deck and then drawn into a player’s hand on a future turn. DOMINION also contains many feedback loops, such as one involving victory cards. Because victory cards in players’ decks do not allow players to do anything besides score points at the end of the game, whenever these victory cards are drawn into players’ hands, they make it more difficult to play other useful cards (e.g., action and treasure cards), and therefore more difficult to acquire more victory cards. In this feedback loop, acquiring victory cards suppresses future acquisition of victory point cards.

An analog game was selected for this study to lower potential barriers to participants’ model matching, as evidence suggests that tangible objects facilitate information recall (Patten & Ishii, 2000) and cue abstract, decontextualized cognitive orientations toward media (cf. Kaufman & Flanagan, 2016a); that explicit rulesets should facilitate understanding of system relations (cf. Zagal, Rick, & Hsi, 2006); and that these conditions promote understanding of games as systems rather than as discrete components (Kaufman & Flanagan, 2016b). In addition to playing with the same set of introductory action cards, to further standardize participants’ exposure to the game, a) participants were provided identical individual player mats to organize their cards and b) game rules were modified so that all participants played exactly 10 rounds, confirmed by the designer as sufficient to gain an understanding of the game (Vaccarino, 2016).

Procedure

Demographics and DOMINION experience metrics were collected through an online questionnaire, following which participants signed up for an in-person study session. Demographics were used only for participant description and to gender-balance in-person study sessions, while DOMINION experience was used to group participants with DOMINION experience together (lest they unduly influence others’ zero-time learning). During in-person sessions, participants individually signed informed consent forms before playing DOMINION in groups of two to four (depending on participant follow-through). Because the goal of this exploratory study was to elicit “naïve” mental models, to avoid influencing the particular entities and relations within DOMINION that participants found salient during gameplay, they were instructed to learn and play the game by relying on a simplified game rulebook without researcher intervention. After the gameplay session, they individually completed a multi-step procedure designed to elicit a mental model map depicting their understanding of DOMINION as a system of interrelated entities. Sessions lasted approximately 90 minutes.

Because mental models are cognitive representations (Doyle & Ford, 1999), they are not directly accessible. As such, some procedure for externalizing them is necessary. To construct their mental model maps, participants individually followed written instructions designed to externalize their mental models of the game they had just played, an approach modified from Brandstädter, Harms, and Großschedl (2012). See Figure 2 for an example mental model map from this study.

Figure 2.

An example mental model map transcribed by the researcher.

As an exploratory investigation into mental model form and content, the mental model mapping exercise was designed to be as open-ended as possible so as not to influence participants’ descriptions with a more structured procedure or more specific prompts. The first step in this mapping exercise asked participants to think of the things in the game they thought were important and to write each on a separate sticky note. After participants completed the first prompt, they were given a second prompt and a large blank piece of paper on which to arrange their sticky notes. The second prompt was divided into small pages as an instruction booklet of several steps. Participants were asked to a) arrange their sticky notes on the large, blank piece of paper before b) indicating relations among them with arrows. They were then asked to c) write a description of each relation next to every arrow. Participants were encouraged to revise as they saw fit.

Results

Entities in Game Mental Models (Detail Complexity)

RQ1 asked about the entities individuals included in their mental models of a game system. After transcribing mental model maps to a spreadsheet for analysis, iterative grounded theory coding (Glaser & Strauss, 1967) was used to inductively categorize the entities that participants wrote on sticky notes. Participants wrote an average of 7.69 (SD = 3.16) entity descriptions in their mental model maps, for 227 total. Although mental model mapping instructions asked participants to list entities on separate sticky notes, participants tended to write richer, more detailed descriptions than anticipated. To account for the possibility that participants described multiple entities on one sticky note, data units for coding were as small as single words or phrases. Entities were assigned 348 unique codes, which were collapsed into 10 subcategories within five overarching categories (see Table 1).

Table 1.

Summary and Examples of Categories of Entities in Participants’ Mental Model Maps.

Category	Category description	Sub-category	Sub-category description	Participants (N = 30)	Entity descriptions (N = 227)	Example (coded words bolded)
Formal game entities	Referred to things described within the rulebook, displayed on game components, or related to the components used to play the game	Material objects	Tangible objects that were part of the game	53.3% (16)	16.3% (37)	How do you know when to buy the green cards
		Ludic concepts	Concepts that were meaningful to individuals because of their role within the game, including how they were described in rulebooks, the game-specific abilities that they allowed, and the game-specific goals that they were used to achieve	76.7% (23)	28.6% (65)	Getting action cards with the “draw” feature
		Narrative concepts	Concepts that involved imagining the meaning of game components, actions, and other players in terms of the fictional story unfolding through gameplay	13.3% (4)	2.6% (6)	Competition with fellow monarchs
Player actions	Behaviors participants performed during gameplay	–	–	53.3% (16)	17.6% (40)	When do you start using all your actions instead of just buying more of those
Sociality	Other participants in the study	Other players	General references to other players	10.0% (3)	2.2% (5)	The game is fun and easy to learn. I could see it being a lot more fun with a large group of people.
		Orientations to other players	Either a competitive or a cooperative orientation toward the other participants	20.0% (6)	4.8% (11)	Cooperation/helped each other (more fun than the object)
		Surveillance of other players	Observing and tracking behaviors performed by other participants	13.3% (4)	1.8% (4)	Need to keep track of other players actions at all Times
Learning processes	The process of making sense of ludic concepts, material objects, and actions	–	–	40.0% (12)	13.2% (30)	It was important to talk to the other players to make sure we all understood
Subjective experience	Participants’ overall experience of playing the game	Confusion and uncertainty	The affective and cognitive experience of confusion and uncertainty about gameplay	56.7% (17)	18.5% (42)	I think we all might not have had a full grasp of the rules but it was still an enjoyable experience
		Other reactions and evaluations	Other affective and cognitive reactions, evaluations, and reflections on gameplay	80.0% (24)	23.8% (54)	Game looked cheap at first but was actually really fun to play

Note. Percentages do not sum to 100% because codes were not exclusive. The participants column reports the number and percent of participants who included at least one entity of the respective category in their mental model maps. The entity descriptions column reports the number and percentage of entity descriptions that included at least one entity of the respective category/sub-category.

Because of the unanticipated richness of participants’ descriptions, the co-occurrence of entity categories within individual descriptions was explored with Bonferroni-adjusted chi-square tests between each pair of entity subcategories (see Table 2). Material objects, ludic concepts, and player actions all co-occurred within entity descriptions more frequently than expected by chance.

Table 2.

Co-Occurrence of Entity Categories Within Entity Descriptions.

Category	Sub-category	1	2	3	4	5	6	7	8	9
Formal game entities	1. Material objects
	2. Ludic concepts	.33*(10.1%)
	3. Narrative concepts	−.07^a(0.0%)	−.04a(0.4%)
4. Player actions		.42*(8.8%)	.22*(8.8%)	−.00^a(0.4%)
Sociality	5. Other players	−.07^a(0.0%)	−.10^a(0.0%)	−.03^a(0.0%)	−.07^a(0.0%)
	6. Orientations to other players	−.10^a(0.0%)	−.05^a(0.9%)	.22*^a(0.9%)	−.10^a(0.0%)	.11^a(0.4%)
	7. Surveillance of other players	.03^a(0.4%)	−.01^a(0.4%)	−.02^a(0.0%)	.11^a(0.9%)	−.02^a(0.0%)	−.03^a(0.0%)
8. Learning processes		−.14^a(0.4%)	−.16(1.3%)	−.06^a(0.0%)	−.15(0.4%)	.12^a(0.9%)	.09^a(1.3%)	−.05^a(0.0%)
Subjective experience	9. Confusion and uncertainty	−.03(2.6%)	−.03(4.8%)	−.08^a(0.0%)	−.07(2.2%)	.01^a(0.4%)	−.06^a(0.04%)	.02^a(0.4%)	−.05(1.8%)
	10. Other reactions and evaluations	−.22*(0.4%)	−.26*(1.8%)	−.09^a(0.0%)	−.23*(0.4%)	.13^a(1.3%)	.02^a(1.3%)	−.08^a(0.0%)	−.04(2.6%)	−.16(1.8%)

Note. Bonferroni-adjusted significance threshold.

One or more cells in the chi-square were expected to have five or fewer occurrences. Table contains phi for bivariate chi-square tests. Values in parentheses are percent of descriptions (N = 227) in which entity categories co-occurred.

p < .001.

Entity Relations in Game Mental Models (Dynamic Complexity)

RQ2 asked about the extent to which individuals identified complex relations among the entities in their mental models of a game system. Descriptions of relations in participants’ mental model maps were content analyzed for the five types of complex relations described in the preceding literature review using a codebook grounded in extant literature characterizing the qualities of these complex relations (available from the author upon request). Participants drew an average of 6.90 (SD = 3.78) arrows between pairs of entity descriptions in their mental model maps, for 207 arrows total. They described on average 4.93 (SD = 3.33) of these arrows, for a total of 148 descriptions. Some participants (n = 18) drew arrows, but did not explain all of their meanings—undescribed arrows were excluded from analysis due to ambiguity. The first author and a second independent coder (iteratively trained using random subsamples of the data) non-exclusively coded relations in 100% of the sample, because an initial review of data indicated infrequency of complex relations in the data. Intercoder reliability ranged from low for nonlinear (κ = −.02, 93.2% agreement), feedback (κ =.00, 98.6% agreement), delayed (κ = .32, 96.1% agreement), and indirect (κ = .34, 90.8% agreement) relations to high for stochastic relations (κ = 1.00, 100% agreement). Because intercoder reliability metrics that account for chance agreement are highly sensitive to infrequency (see Quarfoot & Levine, 2016) and complex relations were rare in the dataset (< 10%), low reliabilities were deemed acceptable, and disagreements were resolved via discussion with the aid of the original codebook.

Overall, participants described primarily simple relations, with a smaller but substantial number of indirect relations (see Table 3). Participants’ mental model maps included few time-delayed, stochastic, or nonlinear relations. See Figure 3 for exemplars of these relations from this dataset, with the exception of feedback loops.

Table 3.

Summary of Relation Types in Participants’ Mental Model Maps.

Code	Participants (N = 30)	Entity relations (N = 207, described n = 148)
Indirect	23.3% (7)	13.5% (20)
Time-delayed	6.7% (2)	2.0% (3)
Stochastic	3.3% (1)	1.4% (2)
Nonlinear	3.3% (1)	1.4% (2)
Feedback loop	0.0% (0)	0.0% (0)
Simple	86.7% (26)	81.8% (121)
Undescribed	60.0% (18)	28.5% (59)

Note. Percentages for Undescribed entity relations were calculated from the total (N = 207); percentages for all other relations were calculated from the subset of described relations (n = 148). Values in parentheses are ns. Totals do not sum to 100% because only Undescribed and Simple were exclusive codes.

Figure 3.

Example a) simple, b) indirect, c) time-delayed, d) stochastic, e) and nonlinear relations transcribed by the researcher.

Post Hoc Analysis

Multiplayer analog games involve substantial shared experiences among players, including not only the events of the game, but also interactions and communication among players. Therefore, it would be expected that individuals’ mental models of game systems would be influenced in similar ways by these shared experiences. As such, to explore the possibility that shared experiences led to overlap in mental model maps among participants within the same group, a series of one-way random effects ANOVAs was run to calculate intra-class correlations (ICC). For a given variable, ICC is the proportion of that variable’s between-group variance to its total variance. Greater ICC indicates a greater degree of similarity within each group (McCoach & Adelson, 2010), and therefore indicates greater interdependence among observations. For groups of four participants, even an ICC as small as .10 would deflate standard errors in statistical tests by a factor of .88 (McCoach & Adelson, 2010). An inspection of ICCs indicated that mental model maps of participants in the same group exhibited a degree of similarity in terms of the types of entities described (Table 4), co-occurrence of entity categories within entity descriptions (Table 5), and the inclusion of indirect relations (Table 6).

Table 4.

Variance and Intraclass Correlation Coefficients for Entity Categories in Participants’ Mental Model Maps.

Category	Sub-category	σ²	τ₀₀	ICC
Formal game entities	1. Material objects	1.91	0.27	0.125
	2. Ludic concepts	4.60	0.20	0.042
	3. Narrative concepts	0.29	0.00	0.000
4. Player actions		2.79	0.10	0.036
Sociality	5. Other players	0.23	0.04	0.149
	6. Orientations to other players	0.77	0.00	0.001
	7. Surveillance of other players	0.10	0.02	0.161
8. Learning processes		0.85	0.36	0.296
Subjective experience	9. Confusion and uncertainty	3.17	0.27	0.080
	10. Other reactions and evaluations	2.43	0.00	0.001

Note. σ² = within-group variance; τ₀₀ = between-group variance; ICC = τ₀₀/ (τ₀₀ + σ²).

Table 5.

Variance and Intraclass Correlation Coefficients for Co-Occurring Entity Categories in Participants’ Mental Model Maps.

Co-occurring categories	σ²	τ₀₀	ICC
Material objects + Ludic concepts	0.38	0.09	0.189
Player actions + Material objects	1.24	0.12	0.086
Player actions + Ludic concepts	1.35	0.01	0.005

Note. σ² = within-group variance; τ₀₀ = between-group variance; ICC = τ₀₀/ (τ₀₀ + σ²).

Table 6.

Variance and Intraclass Correlation Coefficients for Complex Relations in Participants’ Mental Model Maps.

Code	σ²	τ₀₀	ICC
Indirect	1.50	0.27	0.151
Delayed	0.16	<0.01	<0.001
Stochastic	0.13	<0.01	<0.001
Nonlinear	0.13	<0.01	0.001

Note. σ² = within-group variance; τ₀₀ = between-group variance; ICC = τ₀₀/ (τ₀₀ + σ²).

Discussion

After playing an analog complex-system game, participants attempted to externalize their mental models of the game system via mental model maps. In their maps, participants identified a broad range of entities across five main categories—formal game entities, player actions, sociality, learning processes, and subjective experience. The surprising degree of co-occurrence of some entity subcategories suggests an intimate relationship among them. Despite this apparent detail complexity, mental model maps suggested lower dynamic complexity, as participants had limited success at identifying and externalizing complex relations, infrequently documenting indirect, time-delayed, stochastic, and nonlinear relations. Additionally, mental model maps exhibited within-group consistency indicative of interdependence among mental models of participants in the same group. While other factors, such as additional game instruction and more guidance in mental model mapping might have facilitated understanding of DOMINION, these findings may also suggest a limited ability to learn complex systems from a single gameplay session of an analog game. These and other factors are discussed below.

Phenomenological Resources for Game-Based Learning

The diversity of entity categories in participants’ mental model maps aligns with conceptualizations of games as systems of interrelated and diverse human and non-human agents (e.g., Giddings, 2009; Taylor, 2009). This diversity can inform the use of games for learning, as the range of entities can be considered as potential resources to recruit for learning. These resources include not only game mechanics, challenges (Boyan & Sherry, 2011), physical game structures (Siegler & Ramani, 2009), and player actions (Laski & Siegler, 2014), but also narrative elements, interactions among players, and the subjective experience of gameplay. The consequences of aligning these game elements with learning objectives requires further research (Boyan & Sherry, 2011).

Lamination of Formal Game Entities

One unexpected finding in the present study—the frequent co-occurrence of formal game entity sub-categories—may suggest a fruitful research direction. Material objects (e.g., cards) frequently co-occurred with ludic concepts (e.g., victory points) in participants’ mental model maps, suggesting that they are perceived and experienced as not merely interrelated (cf. Juul, 2011), but at times inseparable. As such, these material objects and conceptual entities are interpreted here as, in a sense, laminated together. The metaphor of lamination refers to the description of game entities as simultaneously and inextricably both material and ludic. Material and conceptual layers of entities, though they could be teased apart analytically, were fused—laminated—in participants’ descriptions. Characterizing these material-semiotic entities (cf. Haraway, 1991) as laminated echoes conceptualizations of the ontological hybridity of gameplay (Leino, 2012)—that gameplay inextricably intertwines entities across ontological domains, such as the material and conceptual.

The lamination of conceptual information to material objects may be central to the process of learning from analog games, and a number of gameplay characteristics likely contribute to this process. While rules that define the role of material objects (see Evans, 2013) likely contribute to lamination, the frequent co-occurrence of player actions with both material objects and ludic concepts in participants’ mental model maps additionally suggests that the embodied enactment of gameplay laminates ludic concepts onto material objects. By employing material objects in the enactment of in-game actions, players come to conceptually understand the roles and relations of material objects within the game. This link between action and understanding echoes the insight of situated cognition scholars that knowledge is a product of contextual action, and therefore cannot be divorced from it (Brown, Collins, & Duguid, 1989). In fact, the centrality of player actions to the meaning of gameplay may be key to the unique potential of game-based learning. If it were not, students could learn as much from simply reading a static text like a rulebook (cf. Sitzmann, 2011).

Overcoming Barriers to Game-Based Learning

The infrequency of complex relations in participants’ mental model maps suggests that participants may have had limited abilities to develop and externalize their mental models corresponding to a complex game system after playing it a single time, confirming the repeated finding that understanding complex systems is a great challenge (Sterman, 1994). Furthermore, these findings echo others that identifying discrete entities is easier than identifying the particularities of relations among them (Boughzala, Chourabi, Lang, & Feki, 2017). This study’s findings suggest three potential barriers to learning complex systems. First, many participants overtly described feelings of confusion and uncertainty in their mental model maps, which may have been a response to feeling overwhelmed by the game (coded as subjective experience). Second, variability in the particular categories of entities that participants included in their mental model maps may indicate that games can be too expansive. In other words, some games may include too many potential entities and relations on which to focus, potentially distracting from the entities that are key to potential learning objectives and leading to heterogeneous learning outcomes. Third, although rule-defined goals are often part of conceptualizations of games (e.g., Juul, 2011), despite competitive goals defined by the rules, several participants described cooperative orientations (coded as orientations to other players), suggesting that instructors cannot take for granted learners’ adherence to these same goals. Instead, individuals may engage in counterplay (Apperley & Dieter, 2010)—rejecting a game’s nominal goals, norms, and rules in favor of their own.

These barriers are copacetic with three recently suggested interventions for enhancing learners’ abilities to identify complex relations in game systems. First, to address learner confusion, game expansiveness, and counterplay, additional instruction related to the game system—i.e., context integration (Wouters & van Oostendorp, 2017)—should support mental model matching by focusing learners on key entities and clarifying the nature of entity relations. While in this study more guided game instruction likely would likely have reduced confusion, increased consistency of mental model maps, and increased the identification of complex relations, allowing participants to learn the game independently was done to avoid influencing the particular entities and relations that participants found salient. Second, to address learner confusion, playing a game multiple times (Clark et al., 2016) should facilitate mental model matching by providing learners more opportunities to iteratively test their understandings of the relations in the game system (Landriscina, 2013). Third, to address learner uncertainty and game expansiveness, simpler games (Teach & Murff, 2014) should enable learners to more readily identify complex relations by reducing cognitive load associated with learning complex systems from games (Sitzmann, 2011). A game containing several instances of only one type of complex relation, for example, could facilitate identification of that complex relation by limiting the game’s overall dynamic complexity.

Interdependence of Mental Models

Three particularly notable findings emerged from the post-hoc analysis of within-group interdependence of participants’ mental model maps. First, the three categories of entities with the greatest within-group consistency all centrally involved other players: learning processes, surveillance of other players, and other players generally. This finding may suggest, unsurprisingly, that social interaction and inter-player communication influences participants’ mental models. Therefore, when this interaction is part of intended instructional outcomes, instructors should attempt to facilitate the type of desired interaction among learners. Second, both the inclusion of material objects and the co-occurrence of material objects with ludic concepts were substantially consistent within groups, indicating that group dynamics may influence the conceptual meanings players assign to the material objects in game systems. Third, of the complex relations participants included in their mental model maps, only the inclusion of indirect relations was substantially consistent within groups. However, because indirect relations were the only type of complex relation included by more than two participants, this finding may indicate that group processes influence dynamic complexity of mental models more broadly.

Limitations & Future Research

As with most exploratory, qualitative research, this study’s grounded and content analyses were subject to the researchers’ interpretive lenses and to the generalizability limitations of a small sample size. Further work is needed to determine whether and how these patterns may replicate in larger and more diverse samples, across games with different characteristics, and in various contexts. Further, all potential procedures to externalize mental models are necessarily incomplete and subject to potential demand effects (Doyle et al., 2008). The procedure in this study was designed a) to be as non-leading as possible, b) to facilitate participants’ representations of potentially non-sequential mental models, and c) to reduce cognitive demand during recall. Nevertheless, participants may have implicitly learned more about the game system than they were able to explicitly externalize via mental model mapping.

Participants departed from mental model mapping instructions by describing multiple entities within individual sticky notes and not labeling all arrows, which were addressed by non-exclusive coding and excluding undescribed relations from analysis. The unexpected richness of entity descriptions, which may have obscured complex relations by aggregating them into individual descriptions, facilitated an analysis of co-occurrence among entity categories. Nevertheless, mental model measurement methods require further refinement. For example, the relatively unguided and unfamiliar mental model mapping procedure used in this study may not have captured participants’ complete mental models of DOMINION. While more guidance during this procedure would likely have facilitated the identification of more complex relations, including feedback loops, researcher intervention was avoided for this exploratory study so as not to influence the kinds of relations participants included in their mental model maps. Future research should address these methodological limitations by measuring mental models with multiple, convergent methods, e.g., providing item banks for mapping tasks (Markóczy & Goldberg, 1995) to prevent over-aggregation, using standardized formalizations (Marne & Labat, 2014; Tang & Hanneghan, 2011) to facilitate mental model map comparisons, extracting causal relations from narrative descriptions (Doyle et al., 2008) to reduce demand effects, using closed-ended measures (Kaufman & Flanagan, 2016b), and measuring mental model-related task performance (Korteling, Helsdingen, & Sluimer, 2016).

A number of areas for future research would extend this line of research and address limitations of this exploratory study, including systematically addressing the role of game characteristics, in-game communication, player interactions, contextual factors, and individual learner differences in the process of mental modeling. Given the interdependence of mental models of players within the same group, this research should attend to the role of group micro-dynamics—such as player communication, interactions, and shared experiences—in mental model development (see Duncan & Berland, 2012). As a potential metric for learning and as a validation of mental model complexity measures, the relationship between mental model complexity and game performance should be systematically investigated (see Pirnay-Dummer & Kopainsky, 2011). To more completely understand the game-based learning process, future research should address the entire learning process, from the first steps of mental model development—recognizing entities and interrelations of entities in a system—to transferring mental models of game systems to non-game systems (Martinez-Garza & Clark, 2017). Future research should also examine how instructional techniques may improve individuals’ understandings of complex systems when coupled with games or gameful simulations in ways that leverage the phenomenological learning resources identified above, including the lamination of formal game entities.

Conclusion

This study indicated that games provide a breadth of potential resources for learning complex systems, although discrete entities (detail complexity) may be easier to identify and externalize than complex relations (dynamic complexity) in initial game system engagement. In analog games, manually manipulating material objects to perform game actions laminates conceptual information onto those objects, enabling learners to experience first-hand the relations and processes embedded in game systems. To fully tap the potential of game-based learning, more work needs to be done to understand the process through which learners understand games and transfer that understanding to non-game contexts—as well as game characteristics and instructional techniques that facilitate this process.

Footnotes

Acknowledgements

The authors gratefully acknowledge the assistance of Cathlin V. Clark-Gordon during content analysis, as well as Mathew M. Martin and Elizabeth Cohen for their thoughtful and critical feedback during the development of this article.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Author Biographies

Joe A. Wasserman (MA, West Virginia University) is a PhD student in the Department of Communication Studies at West Virginia University and Graduate Student Lab Manager at WVU’s Interaction Lab (#ixlab). His research focuses on learning complex systems and systems thinking through games and simulation, with an emphasis on analog games.

Contact: jowasserman@mix.wvu.edu

Jaime Banks (PhD, Colorado State University) is an assistant professor in the Department of Communication Studies at West Virginia University and a research associate at WVU’s Interaction Lab (#ixlab). Her research focuses on the intersections of identity, agency, and human-technology relations, with an emphasis on videogame avatars and social robots.

Contact: jabanks@mail.wvu.edu

References

Anderson

L. W.

Krathwohl

D. R.

(2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives, Abridged edition. New York, NY: Addison Wesley Longman.

Apperley

Dieter

(2010). Counterplay. The Fibreculture Journal, 16. Retrieved from http://sixteen.fibreculturejournal.org/

Boughzala

Chourabi

Lang

Feki

(2017). Feedback on the integration of a serious game in the data modeling learning. In Proceedings of the 50th Hawaii International Conference on System Sciences (pp. 735-742). Retrieved from http://scholarspace.manoa.hawaii.edu/handle/10125/41238

Bowen

G. A.

(2008). Naturalistic inquiry and the saturation concept: A research note. Qualitative Research, 8, 137-152. doi:10.1177/1468794107085301

Boyan

Sherry

J. L.

(2011). The challenge in creating games for education: Aligning mental models with game models. Child Development Perspectives, 5, 82-87. doi:10.1111/j.1750-8606.2011.00160.x

Brandstädter

Harms

Großschedl

(2012). Assessing system thinking through different concept-mapping practices. International Journal of Science Education, 34, 2147-2170. doi:10.1080/09500693.2012.716549

Brown

J. S.

Collins

Duguid

(1989). Situated cognition and the culture of learning. Educational Researcher, 18, 32-42. doi:10.3102/0013189X018001032

Checkland

(1999). Systems thinking, systems practice: Includes a 30-year retrospective. New York, NY: Wiley.

Clark

D. B.

Tanner-Smith

E. E.

Killingsworth

S. S.

(2016). Digital games, design, and learning: A systematic review and meta-analysis. Review of Educational Research, 86, 79-122. doi:10.3102/0034654315582065

10.

Corredor

Gaydos

Squire

(2013). Seeing change in time: Video games to teach about temporal change in scientific phenomena. Journal of Science Education and Technology, 23, 324-343. doi:10.1007/s10956-013-9466-4

11.

Davidsen

P. I.

Spector

J. M.

(2015). Critical reflections on system dynamics and simulation/gaming. Simulation & Gaming, 46, 430-444. doi:10.1177/1046878115596526

12.

Doyle

J. K.

Ford

D. N.

(1999). Mental models concepts revisited: Some clarifications and a reply to Lane. System Dynamics Review, 15, 411-415. doi:10.1002/(SICI)1099-1727(199924)15:4<411::AID-SDR181>3.0.CO;2-R

13.

Doyle

J. K.

Radzicki

M. J.

Trees

W. S.

(2008). Measuring change in mental models of complex dynamic systems. In Qudrat-Ullah

Spector

J.M.

Davidsen

P.I.

(Eds.), Complex decision making: Theory and practice (pp. 269-294). Berlin, Germany: Springer Berlin Heidelberg. doi:10.1007/978-3-540-73665-3_14

14.

Duncan

S. C.

Berland

(2012). Triangulating learning in board games: Computational thinking at multiple scales of analysis. In Martin

Ochsner

Squire

(Eds.), Proceedings of Games+Learning+Society 8.0 (pp. 90-95). Madison, WI: ETC Press.

15.

Evans

(2013). Translating board games: Multimodality and play. The Journal of Specialised Translation, 20, 15-32.

16.

Foster

A. N.

(2011). The process of learning in a simulation strategy game: Disciplinary knowledge construction. Journal of Educational Computing Research, 45, 1-27. doi:10.2190/EC.45.1.a

17.

Garson

G. D.

(2009). Computerized simulation in the social sciences: A survey and evaluation. Simulation & Gaming, 40, 267-279. doi:10.1177/1046878108322225

18.

Giddings

(2009). Events and collusions: A glossary for the microethnography of video game play. Games and Culture, 4, 144-157. doi:10.1177/1555412008325485

19.

Glaser

B. G.

(1965). The constant comparative method of qualitative analysis. Social Problems, 12, 436-445. doi:10.1525/sp.1965.12.4.03a00070

20.

Glaser

B. G.

Strauss

A. L.

(1967). The discovery of grounded theory: Strategies for qualitative research. Chicago, IL: Aldine Publishing.

21.

Groesser

S. N.

Schaffernicht

M. F. G.

(2012). Mental models of dynamic systems: Taking stock and looking ahead. System Dynamics Review, 28, 46-68. doi:10.1002/sdr.476

22.

Haraway

(1991). Simians, cyborgs, and women: The reinvention of nature. New York, NY: Routledge.

23.

Juul

(2011). Half-real: Video games between real rules and fictional worlds. Cambridge, MA: MIT Press.

24.

Kaufman

Flanagan

(2016a). High-low split: Divergent cognitive construal levels triggered by digital and non-digital platforms. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (pp. 2773-2777). New York, NY: Association for Computing Machinery. doi:10.1145/2858036.2858550

25.

Kaufman

Flanagan

(2016b). Playing the system: Comparing the efficacy and impact of digital and non-digital versions of a collaborative strategy game. In Proceedings of the First International Joint Conference of DiGRA and FDG. Retrieved from http://www.digra.org/digital-library/forums/13-digra-fdg2106/

26.

Korteling

H. J. E.

Helsdingen

A. S.

Sluimer

R. R.

(2017). An empirical evaluation of transfer-of-training of two flight simulation games. Simulation & Gaming, 48, 8-35. doi:10.1177/1046878116671057

27.

Landriscina

(2013). Simulation and learning: A model-centered approach. New York, NY: Springer.

28.

Laski

E. V.

Siegler

R. S.

(2014). Learning from number board games: You learn what you encode. Developmental Psychology, 50, 853-864. doi:10.1037/a0034321

29.

Leino

O. T.

(2012). Untangling gameplay: An account of experience, activity and materiality within computer game play. In Sageng

J. R.

Fossheim

Larsen

T. Mandt

(Eds.), The philosophy of computer games (Vol. 7, pp. 57-75). Dordrecht, The Netherlands: Springer Netherlands.

30.

Levine

R. L.

Doyle

J. K.

(2002). Modeling generic structures and patterns in social psychology. In Davidsen

P. I.

Mollona

Diker

V. G.

Langer

R. S.

Rowe

J. I.

(Eds.), Proceedings of the 20th International Conference of the System Dynamics Society. Retrieved from http://www.systemdynamics.org/conferences/2002/index.html

31.

Markóczy

Goldberg

(1995). A method for eliciting and comparing causal maps. Journal of Management, 21, 305-333. doi:10.1016/0149-2063(95)90060-8

32.

Marne

Labat

J.-M.

(2014). Model and authoring tool to help teachers adapt serious games to their educational contexts. International Journal of Learning Technology, 9, 161-180. doi:10.1504/IJLT.2014.064491

33.

Martinez-Garza

M. M.

Clark

D. B.

(2017). Two systems, two stances: A novel theoretical framework for model-based learning in digital games. In Wouters

van Oostendorp

(Eds.), Instructional techniques to facilitate learning and motivation of serious games (pp. 37-58). Cham, Switzerland: Springer International Publishing. doi:10.1007/978-3-319-39298-1_3

34.

McCoach

D. B.

Adelson

J. L.

(2010). Dealing with dependence (part I): Understanding the effects of clustered data. Gifted Child Quarterly, 54, 152-155. doi:10.1177/0016986210363076

35.

National Research Council. (2012). A framework for k-12 science education: Practices, crosscutting concepts, and core ideas. Washington, DC: National Academies Press.

36.

Nishikawa

K. A.

Jaeger

(2011). A computer simulation comparing the incentive structures of dictatorships and democracies. Journal of Political Science Education, 7, 135-142. doi:10.1080/15512169.2011.564915

37.

Özesmi

S. L.

(2004). Ecological models based on people’s knowledge: A multi-step fuzzy cognitive mapping approach. Ecological Modelling, 176, 43-64. doi:10.1016/j.ecolmodel.2003.10.027

38.

Patten

Ishii

(2000). A comparison of spatial organization strategies in graphical and tangible user interfaces. In Proceedings of DARE 2000 on Designing Augmented Reality Environments (pp. 41-50). New York, NY: Association for Computing Machinery. doi:10.1145/354666.354671

39.

Pirnay-Dummer

Kopainsky

(2011, April). Tracking performance, understanding and learning in complex game-like environments. Paper presented at the Annual Meeting of the American Educational Research Association, New Orleans, LA.

40.

Quarfoot

Levine

R. A.

(2016). How robust are multirater interrater reliability indices to changes in frequency distribution? The American Statistician, 70, 373-384. doi:10.1080/00031305.2016.1141708

41.

Resnick

Wilensky

(1998). Diving into complexity: Developing probabilistic decentralized thinking through role-playing activities. The Journal of the Learning Sciences, 7, 153-172. doi:10.1207/s15327809jls0702_1

42.

Schaffernicht

M. F. G.

Groesser

S. N.

(2013). Absent or distant? On the use of indirect causal links and chains of causal links to compare mental models of dynamic systems. In Eberlein

Martínez-Moyano

I. J.

(Eds.), Proceedings of the 31st International Conference of the System Dynamics Society. Retrieved from http://www.systemdynamics.org/conferences/2013/index.html

43.

Senge

P. M.

(2006). The fifth discipline: The art and practice of the learning organization. New York, NY: Doubleday/Currency.

44.

Siegler

R. S.

Ramani

G. B.

(2009). Playing linear number board games—But not circular ones—Improves low-income preschoolers’ numerical understanding. Journal of Educational Psychology, 101, 545-560. doi:10.1037/a0014239

45.

Sitzmann

(2011). A meta-analytic examination of the instructional effectiveness of computer-based simulation games. Personnel Psychology, 64, 489-528. doi:10.1111/j.1744-6570.2011.01190.x

46.

Stave

K. A.

Beck

Galvan

(2015). Improving learners’ understanding of environmental accumulations through simulation. Simulation & Gaming, 46, 293-321. doi:10.1177/1046878114531764

47.

Stave

Hopper

(2007). What constitutes systems thinking? A proposed taxonomy. In Sterman

Oliva

Langer

R. S.

Rowe

J. I.

Yanni

J. M.

(Eds.), Proceedings of the 25th International Conference of the System Dynamics Society. Retrieved from http://www.systemdynamics.org/conferences/2007/index.htm

48.

Sterman

J. D.

(1994). Learning in and about complex systems. System Dynamics Review, 10, 291-330. doi:10.1002/sdr.4260100214

49.

Sweeney

L. B.

Sterman

J. D.

(2007). Thinking about systems: Student and teacher conceptions of natural and social systems. System Dynamics Review, 23, 285-311. doi:10.1002/sdr.366

50.

Sweller

van Merrienboer

J. J. G.

Paas

F. G. W. C.

(1998). Cognitive architecture and instructional design. Educational Psychology Review, 10, 251-296. doi:10.1023/A:1022193728205

51.

Tang

Hanneghan

(2011). State-of-the-art model driven game development: A survey of technological solutions for game-based learning. Journal of Interactive Learning Research, 22(4), 551-605.

52.

Taylor

T. L.

(2009). The assemblage of play. Games and Culture, 4, 331-339. doi:10.1177/1555412009343576

53.

Teach

R. D.

Murff

(2014). Are the business simulations we play too complex? Developments in Business Simulation and Experiential Learning, 35, 205-211.

54.

Vaccarino

D. X.

(2008). Dominion. Rio Rancho, NM: Rio Grande Games.

55.

Vaccarino

D. X.

(2016). I’m Donald X., designer of Dominion, Kingdom Builder and others; AMA. Retrieved from https://www.reddit.com/r/boardgames/comments/4ivtt3/im_donald_x_designer_of_dominion_kingdom_builder/

56.

White

B. Y.

(1984). Designing computer games to help physics students understand Newton’s laws of motion. Cognition and Instruction, 1, 69-108. doi:10.1207/s1532690xci0101_4

57.

Wouters

van Oostendorp

(2017). Overview of instructional techniques to facilitate learning and motivation of serious games. In Wouters

van Oostendorp

(Eds.), Instructional techniques to facilitate learning and motivation of serious games (pp. 1-16). Cham, Switzerland: Springer International Publishing. doi:10.1007/978-3-319-39298-1_1

58.

Zagal

J. P.

Rick

Hsi

(2006). Collaborative games: Lessons learned from board games. Simulation & Gaming, 37, 24-40. doi:10.1177/1046878105282279