Simulation & Gaming

Purpose. This article summarizes the nine contributions to the symposium on system dynamics. The goal of the symposium is to investigate how research in system dynamics is contributing to simulation-gaming, and how the more general field of simulation-gaming is influencing work in system dynamics.

Summary of articles. We summarize the nine contributions. They include five articles on basic research in learning and teaching principles for system dynamics, three articles on interactive learning environments, and a final critical reflection article which summarizes the other eight and suggests future research needs.

Conclusion. The main problem still to be addressed is improvement in and measurement of learning and performance.

Background. Most research in the area of dynamic decision making in general and stock-flow failures in particular is conducted with the help of computerized simulations as task environments or paper-based tasks of simple dynamic systems.

Purpose. The purpose of this article is to investigate if stock-flow failures persist in naturalistic decision making environments.

Methods. For this purpose, a tangible stock-flow experiment is used. It asks participants to meet a target level of water in a glass by pouring water through a funnel as fast as possible. We then compare their performance with a purely cognitive task (a manufacturing task).

Results. Participants on average did not significantly perform better in a tangible stock-flow task than in a paper-based test of a comparable task. In addition, individual performance in the tangible task cannot be related to performance in a similar paper-pencil stock-flow task.

Conclusions. An implication of this study is that tangible stock-flow tasks are as difficult for humans to control as are purely cognitive tasks. Further research should address individual differences between the two task modes (tangible vs. cognitive). A limitation of this study is the usage of only one tangible stock-flow task.

Background. Stock-flow (SF) problems are ubiquitous in nature, ranging from the accumulation of water in a tub to the accumulation of CO₂ in the atmosphere. However, research on SF failure repeatedly demonstrates that people have severe difficulties understanding even the most basic SF problems.

Purpose. This study tested the hypothesis that people’s understanding of SF problems depends on the presentation format used. Specifically, we expect SF failure to decrease when avoiding previously used scientific formats comprising coordinate systems and graphs, and SF problems are presented in verbal formats.

Method. Participants (N = 107) solved a range of different SF problems with experimentally varied presentation formats (verbal vs. graphic). We assessed fundamental understanding of graphs and graphical versus verbal production of stocks and in- and outflows.

Results. Solution rates show that (a) SF failure is at least partially caused by specifics of the presentation format used previously; (b) fundamental misunderstandings in the construction of graphs can explain previous findings; and (c) the majority of participants arrived at the correct solution when SF problems were presented verbally.

Conclusion. The present study indicates that people are able to solve SF problems when they are presented in accessible formats. This result bears implications for simulation-based learning and assessment, and for the communication of SF problems.

Background. Poor public understanding of accumulation principles obstructs public engagement in environmental decision-making. Simulation-based learning environments (SBLEs) show promise for improving understanding.

Purpose. This study tested the hypothesis that use of stand-alone, self-contained, online SBLEs can improve user understanding of and ability to apply basic principles of accumulation.

Method. We used pretest/posttest measures to examine (a) whether user understanding of accumulations principles increased after exposure to a set of two SBLEs in an undergraduate Environmental Science course (n =126), and (b) how the extent of simulation use affected depth of user understanding.

Results. Understanding improved significantly after simulation use. Further, extent of simulation use affected performance although it is not a simple linear relationship. Those who ran the simulation a moderate amount of times (total of 14-24 runs) scored better than the low-range group (0-13 runs, p < .05) and better than the high-range users (25+ runs, p < .10). These results hold when controlled for prior systems understanding, graphing ability, motivation, and science background.

Conclusion. The results support the value of simulations for building operational understanding of accumulations and suggest design considerations that may further increase the effectiveness of such SBLEs.

In complex simulation-based learning environments, participants’ learning and performance may suffer due to demands on their cognitive processing, their struggle to develop adequate mental models, failure to transfer what is learned to subsequent learning or activities, and fear of failure. This study investigates an instructional strategy addressing those four problems, which we call prior exploration strategy. It was implemented in a simulation requiring participants to optimize a developing nation’s per capita income. The prior exploration strategy allows participants to manipulate and see the results of a simulation model in practice mode before they manage a similar simulation in a more final mode. The strategy was assessed in an experiment comparing participants using the prior exploration strategy with participants studying equivalent content in a non-exploratory fashion. The dependent variables were performance within the simulation and improvement of participants’ understanding. The prior exploration strategy significantly improved participants’ performance, as measured by per capita income. It also significantly improved some aspects of the participants’ understanding (e.g., their understanding of the nation’s debt accumulation), but not others (e.g., their understanding of the need to balance the nation’s health, education, and infrastructure investments; those that appear to have complex interrelations).

Background. Inquiry learning environments increasingly incorporate simulation and modeling facilities. Students acquire knowledge through systematic experimentation with the simulations and express that knowledge in runnable computer models.

Aim. As inquiry and modeling activities are new and demanding for students, support for learning is needed. This article reports three experimental studies that examine how students’ inquiry and modeling activities can be supported.

Need for support. Study 1 was an empirical assessment of students’ support needs. It compared a group of domain novices to two more knowledgeable reference groups in order to determine the novices’ support needs.

Model progression and worked examples. In Studies 2 and 3, the need for support was addressed by model progression (gradually increasing task complexity) and worked-out examples, examining the effect of those interventions on students’ performance and learning. Results suggest positive effects due to both increasing model complexity and providing worked examples that show what the activities in each model progression phase entail and how they should be performed.

Implications. The pattern of results across the three studies are discussed with regard to students’ use of available resources, influence of prior knowledge, and the relationship between performance and learning.

Global negotiations to reduce greenhouse gas (GHG) emissions have so far failed to produce an agreement. Even if negotiations succeeded, however, a binding treaty could not be ratified or implemented in many nations due to inadequate public support for emissions reductions. The scientific consensus on the reality and risks of anthropogenic climate change has never been stronger, yet public support for action in many nations remains weak. Policymakers, educators, the media, civic and business leaders, and citizens need tools to understand the dynamics and geopolitical implications of climate change. The WORLD CLIMATE simulation provides an interactive role-play experience through which participants explore these issues using a scientifically sound climate policy simulation model. Participants playing the roles of negotiators from major nations and stakeholders negotiate proposals to reduce GHG emissions. Participants then receive immediate feedback on the implications of their proposals for atmospheric GHG concentrations, global mean surface temperature, sea level rise, and other impacts through the C-ROADS (Climate Rapid Overview and Decision Support) policy simulation model used by negotiators and policymakers. The role-play enables participants to explore the dynamics of the climate and impacts of proposed policies using a model consistent with the best available peer-reviewed science. WORLD CLIMATE has been used successfully with students, teachers, business executives, and political leaders around the world. Here, we describe protocols for the role-play and the resources available to run it, including C-ROADS and all needed materials, all freely available at climateinteractive.org. We also present evaluations of the impact of WORLD CLIMATE with diverse groups.

Background. Research shows that learning and task performance improve when participants in management exercises understand the structure of the system they control. However, the majority of business simulators are black-boxes.

Aim. This article introduces structural debriefing, which is a debriefing activity aimed at helping students learn about causal relationships, feedbacks, accumulations, and delays within a black-box simulation.

Method. A structural debriefing can be prepared and facilitated by following the Structural Debriefing Protocol.

Results. A pilot study was conducted in which undergraduate students participated in a structural debriefing of The LITTLEFIELD TECHNOLOGIES, a popular simulation for teaching principles of operations management. The students were able to complete all eight steps of a structural debriefing, but required considerable time (three academic terms) to do so. Not every instructional simulation will require all the steps or such a large time commitment.

Conclusion. The successful completion of the pilot study demonstrates that structural debriefing is a useful debriefing technique. However, to be effective, the scope and format of a structural debriefing activity must suit practical and pedagogical considerations.

Background: Sustainable management of the municipal water supply is a priority issue in the policy agenda of Maui County, Hawaii. The Maui Economic Development Board launched the WaterStory initiative to educate residents about the determinants and history of Maui’s water resource development. A System Dynamics model was developed and coupled with a user interface to create WATERSTORY, an interactive learning environment (ILE).

Purpose: The purpose of this study was to analyze the benefits of WATERSTORY for the identification and assessment of sustainable water management policies in Maui.

Method: WATERSTORY is a System Dynamics-based ILE that shows the cross-sectoral impacts of users’ decisions. Both, the simulation model and the ILE resulted from group modeling and ILE testing sessions.

Results: The development of the WATERSTORY ILE facilitated the involvement of local communities in the discussion of water management problems and identification of possible solutions.

Conclusions: The study suggests that developing a simple ILE is essential in raising the interest of a wide audience involved in sustainable development policy making. After stakeholders gain confidence in the benefit of the ILE, it becomes easier to integrate simulation exercises into planning processes at national, regional and local levels.

Purpose. This article critically discusses the contributions in this symposium and the progress in the fields of system dynamics and interactive learning environments.

Two Perspectives. For this purpose, the article uses two different perspectives. One perspective is system dynamics or model-based policy analysis and design. The other perspective is the learning sciences, that is, the theories and principles of learning.

Commentaries. With these two perspectives in mind, this article reflects on each of the eight contributions to this symposium and gives critiques and suggestions.

Recommendation. The results and insights generated by these contributions suggest that the greatest need for future research on system dynamics and its contribution to simulation-gaming is demonstration of improvements in learning and performance.