Abstract
Research suggests that well-designed games can be good for learning under the right conditions. How such games are designed remains poorly understood, as studies have focused more on whether games can produce learning than on how such games work or how they can be reliably developed. That is, though the design of a game is considered essential to its effectiveness, educational games lack a theory-informed definition and have predominantly shared design in terms of “principles” or “heuristics.” The aim of this paper is to discuss how we define and share educational game design and why design is important for improving educational game research and development.
Keywords
The past decade’s interest in using games to help students learn has shown that games can be effective educational tools (R. E. Clark, 2007; Honey & Hilton, 2011; Young et al., 2012). The supporting evidence spans multiple disciplines, ranging from sociocultural (Gee, 2003; Steinkuehler, 2004) to neurological (Bhatt & Camerer, 2005; Green & Bavelier, 2008), and researchers have examined games that are specifically designed for learning (D. B. Clark, Sengupta, Brady, Martinez-Garza, & Killingsworth, 2015) as well as those that are designed for commercial entertainment (Golub, 2010; Nardi, 2010; Pirius & Creel, 2010; Turkay & Adinolf, 2010). However, some are not quite satisfied with knowing that games may be effective and have voiced the need to further develop research (including games research) that can make larger-scale, longer-term, systems-level impact (Honey & Hilton, 2011; Penuel, Fishman, Cheng, & Sabelli, 2011). They argue that investigations into whether games can be effective should give way to investigations into how or under what conditions they are effective (D. B. Clark, Tanner-Smith, & Killingsworth, 2015; Tobias & Fletcher, 2011). While game-based learning has shown potential, what is needed are ways to reliably convert that potential into action. Educational games research has the task of connecting learning theory to the reliable production of learning outcomes, especially for use in authentic settings. In order to do so, I argue, research needs to overcome a major obstacle: It must improve how design is conceptualized and discussed. In particular, educational games research must clarify definitions and develop robust ways to share products and processes associated with design so that the community may reliably produce, use, and test educational games and their associated theories.
What Is Design?
Before proceeding, it is important to clarify how design is being used here, which is more or less SchÖn’s (1984) definition: Design is an activity characterized by reflection-in-action, in which designers draw connections between the immediate design problem and their own prior experiences, relying on repertoires of design they have built up through professional practice or experiences. During design, the problem at hand is framed so that it is familiar and understandable. During this activity, the conditions of the world “talk back” to the designer, triggering a reframing of the problem based on the evolving constraints that the design must meet. When a problem is well defined, generating design solutions means meeting the initial constraints of the project. When a problem is not well defined, the designer must actively identify the similarities and differences between the current design (product and process) and prior work in order to evaluate potential solutions’ relevance or applicability. Rather than consider design as a logical process of theory instantiation, SchÖn’s definition considers design to be a practice. As a practice, it is something that can be improved.
Such a view of design is not the only reasonable definition, though it is favored here because of its striking resemblance to Pickering’s (1995) mangle of practice. Scientists in practice, Pickering explains, work with their nonhuman tools to strike a balance of activity. They do so by “tuning” their instruments, their material counterparts. However, because their material counterparts are not fully predictable in the outcomes they produce (they “resist”), scientists must react (“accommodate”) and adjust their own practice. Scientific activity, then, is characterized by a dialectic of resistance and accommodation. Like science, design may be viewed as a practice that produces a mangle through a dialectic. As often as educational game designers are researchers, design should resemble science.
SchÖn’s and Pickering’s perspectives on design are not the only useful ones, and the field may apply other definitions as needed. Proposing and testing alternative definitions of design theories are important first steps forward, as educational games research currently tends to consider design as essential but not important enough to theorize. Consider the development of Quest Atlantis (now Atlantis Remixed), a virtual world within which student players engage in prosocial and scientific narratives. Quest Atlantis and Atlantis Remixed have been played by tens of thousands of students across United States, China, Singapore, Denmark, and Australia, making it a good example of what a successful educational game project might look like (Barab, Dodge, Thomas, Jackson, & Tuzun, 2007; atlantisremixed.org). The design of the original virtual environment and its accompanying sociotechnical structures was clearly important to the project and to the authors as they explain that to create the game, they practiced empowerment design, or critical design intended to “transform individuals and those contexts in which they function” (Barab et al., 2002; Barab, Dodge, Thomas, et al., 2007, p. 278). Further, they critically discussed the inevitability of embedding values into games, highlighting the importance of addressing the unspoken or underlying agendas that cannot be dissociated from educational curricula (Barab, Dodge, Thomas, et al., 2007). The design for Quest Atlantis was clearly important insomuch as it was useful for achieving the designer’s intention or as a means for implementing underlying theories: “Central to our work in the Quest Atlantis Project (QA),” the authors write, “has been designing a context for learning that sits at the intersection of education, entertainment, and social action” (Barab, Dodge, Tuzun, et al., 2007, p. 155). They convey their intentions as well as the importance of being critical in those regards but do not go on to convey the ways that their intentions were insufficient for producing their intended learning outcomes via their designs. Barab, Dodge, Tuzun, et al. (2007), like many others in educational game research, treat design as essential but as a means to an end (Diehl et al., 2013; Filsecker & Kerres, 2014; Habgood & Ainsworth, 2011; Kafai, 1996).
The problem to be addressed is thus: How do we, researchers and designers, frame the space between the theory of interest and the products and outcomes? Schön’s (1984) definition is a useful start, as it suggests questions to ask of Barab, Dodge, Tuzun, et al. (2007), including the following: (a) How did they interpret the design problem at hand? (b) What sorts of development activity did they find useful in addressing these problems? and (c) How did the problem and their solution change over time? In Pickering’s terms, Barab and others might explain the resistances that were encountered when they tried their designs and then discuss how such resistances were accommodated through the course of development. If neither SchÖn nor Pickering is seen as adequate, alternative definitions of design could be considered: design as a process of proactive failure analysis (Petroski, 2008), as a process of exploring constraints (Gross, 1985), or in terms of how and why things work, especially in relation to people (Norman, 2013). There are many definitions of design to choose from (or invent). Deciding on which one to use matters less than deciding to approach design head-on. This is a necessary shift for advancing how we understand what constitutes good educational game design.
How Is It Shared?
At the same time that we address how we define design, we must also give serious consideration to how we share it. Designing good games (educational or otherwise) is difficult. In the commercial game development industry, this difficulty is addressed in various ways, including books, blogs, and postproduction reviews (“postmortems”; see Wawro, 2015) as well through formal education structures, like game design classes and degrees. These outlets regularly address how to design games and how to make designs better, calling it an art (Schell, 2008) or a craft (Achterman, 2011) and treating it as something that can be theorized, practiced, and discussed in order to be improved. By way of sharing their activities, practices, processes, and lessons learned, the commercial industry provides various means for entertainment game designers to advance their work.
In educational game research and development, this sort of sharing and attention to design is less frequent. Educational game research tends to share design via journal articles that list design principles or outline the essential steps that one must take in order to create or recreate an educational game. For example, the Games-To-Teach research team identifies seven principles that guided its development work across 14 games, such as the principle to “design educational action games by turning simulations into simulation games” (Squire et al., 2003, p. 19). Amory (2007), explaining educational game properties broadly, presents the game object model (GOM), a theoretical framework that relates pedagogy to game elements. The components of GOM are akin to objects in object-oriented programming. They maintain particular characteristics that can be inherited based on class relationships and can be interrelated. Van Staalduinen and de Frietas (2011) present a four-dimensional framework (learner, context, pedagogy, and representation) and 25 components (e.g., theme, sensory stimuli) that can be combined to meet the particular learning or user needs that are identified early in a design process. Similarly, Landers (2014) seeks a parsimonious definition of serious games before connecting game attributes to learning outcomes. Finally, drawing principles out of their successful design, the authors of the Quest Atlantis project develop a list of the learning affordances of their virtual environment and its sociotechnical structures for learning: learning and achievement, narrative engagement, identity development, collaborative participation, communication, and reflexivity.
There is nothing inherently wrong with sharing design principles. However, they are insufficient for conveying a game’s design by themselves. The problem is that these forms of design knowledge tend to consider design as a logical translation of intent to artefact, placing less attention on the choices (including missteps) that produce outcomes, on the context of the design, and on its use. For example, Van den Akker (1999) argues that the primary outcomes of design research are principles that characterize the artefact or the steps necessary to create the artefact. Such principles can be conveyed in the following form:
If you want to design intervention X [for the purpose/function Y in context Z], then you are best advised to give that intervention the characteristics A, B, and C [substantive emphasis], and to do that via procedures K, L, and M [procedural emphasis], because of arguments P, Q, and R. (Van den Akker, 1999, p. 9)
Principles and heuristics can clearly capture the general character of the design and the logic of the designers. They can be useful for making general recommendations on how others can begin to recreate desired outcomes. The importance of context is also acknowledged, as it provides ecological validity or explains the generalizability of the findings (Van den Akker, 1999).
This approach to design is especially popular in instructional design. For example, Gauthier, Corrin, and Jenkinson (2015) compare a digital study aid for vascular anatomy that used game elements (e.g., leader boards and power-ups) to a study aid without game elements. Sun, Ye, and Wang (2015) examine two commercial games, Cut the Rope and Angry Bird Space, for their incorporation of mechanics that induce trial and error, progressive challenges, and feedback: three elements that are useful for consolidating previously learned concepts. Villalta et al. (2011) recommend attending to six key principles that should be considered when designing educational classroom multiplayer games: (a) on-screen information, (b) game mechanics, (c) game progression, (d) methodology, (e) collaboration, and (f) holism. Villalta et al. go on to describe a game that they developed using these principles and how each principle was instantiated. These studies approach educational games in different ways but share a common theme. They take the idea that games can be broken down into elemental components and apply it to design. Deconstructing games is useful, especially for talking about or analyzing games. Its application to design is problematic, however.
To understand why this sort of approach to design is problematic, reconsider both SchÖn’s (1984) definition of design as “reflection-in-action” and Pickering’s (1995) mangle of practice. As exhibited above, educational games research, like science, has tended to report on design (though not learning) via post facto accounts of the representations produced and to summarize findings in terms of principles and heuristics. Sharing decontextualized representations of the products of design in this way is akin to the same problematic commitments Pickering identifies in science. Such accounts do not accurately reflect what happens during scientific practice, nor do they accurately represent design. That is, for Pickering, design principles fail to address the dialectic inherent to design activity. From a reflection-in-action approach, heuristics and principles miss how the world “talks back” to designers, forcing them to reflect on the problem and on their solution. For both Pickering and SchÖn, what is needed are ways to clearly convey the intentions, the materials, the context, and the interactions between these components as they evolve over the course of development. Their approach contrasts current studies that attempt to decompose games into collections of essential, educational components.
How Do We Learn From Others’ Design?
In order to better specify what to share, formats can be borrowed from other fields. Journals, like the International Journal of Designs for Learning and Educational Designer, have begun to explicitly encourage the sharing of educational design knowledge. These journals not only provide publication outlets but also have begun to establish formats for how such knowledge can be shared, including design cases, teaching cases, and design narratives. Design cases are a formal way of sharing design experiences, or precedent, with others. They assume design to be a primarily abductive activity of drawing connections between prior experiences or cases and the current, situated design problem in order to create a solution (Boling, 2010). Because this perspective considers precedent to be essential, educating designers means helping them to develop repertoires of design experiences directly or vicariously and encouraging design students to learn to see the world in terms of design.
Similar to design cases are teaching cases, or case studies that are presented in a way that is useful in an instructional setting. In particular, teaching cases present students with settings in order to provide opportunities for students to apply their general design principles to an imagined, safe scenario. By reading about a design problem and its context, proposing and justifying their own design solutions, and discussing and reflecting on their solutions and others’, students can use teaching cases to practice thinking like a designer (Ertmer & Quinn, 2007; Ertmer & Russell, 1995).
Finally, literature from the field of computer-supported collaborative learning specifically recommends writing a design narrative, particularly when conducting design-based research (Hoadley, 2002). Design narratives are characterized by their presentation of a plot that describes and relates the important development events as they unfold over time in a particular setting. Such narratives are an important means by which design information may be reported to the research community, providing the opportunity for others to understand and adapt the original research design in different contexts (Hoadley, 2002). In particular, they are intended to promote replication, providing readers with an understanding of the context of a design in order to replicate its outcomes elsewhere.
Implications for Design
Adopting design cases, teaching cases, and design narratives is generally intended to improve how we share design and shift the field toward better accounting for design activity. One goal of this shift is specifically for designers, as improving sharing will presumably lead to the design of better games. By conveying how new educational games come about, we allow others to identify what might have done better or what we ought to do next, improving our practice of educational game design. Relatedly, formal attempts to teach students how to design serious games (e.g., Michigan State University’s Serious Games program) raise the question: What does course material look like for students of educational game design? Teaching cases, like design cases, are useful because they provide the opportunity to review educational game designs, curated for lessons or principles that contributed to prior games’ successes and failures.
Design narratives and other approaches that convey the context of development as it unfolds (e.g., critical design ethnographies) can also improve educational game design. In particular, if coupled with theories that provide structure to the dimensions of change that are possible for educational game design, design narratives may help to identify new considerations for design. Consider, for example, Bielaczyc’s (2006) four dimensions of classroom structures that can be designed: the cultural beliefs dimension, the practices dimension, the socio-techno-spatial relations dimension, and the dimension of interaction with the “outside world.” The beliefs dimension includes the way that teachers and students think about learning and knowing, their identities, and the purpose of the tools or technology being used. The practices dimension includes students’ and teachers’ activities. This includes both what they do with the tool and without it as well as the social structures of the participants. The socio-techno-spatial relations dimension refers to relationship between the physical and cyberspace as well as the configurations of the students, teachers, and tools within. Finally, the interaction dimension considers the way that knowledge is brought into the activity, produced by the activity, and consumed by others and the way that students interact with others. Together, these dimensions constitute Bielaczyc’s social infrastructure framework, characterizing the aspects of classroom learning that are amenable to design and identifying the variables that are important for effectively integrating learning technologies. Applying the social infrastructure framework to educational game design can help to identify dimensions that can be changed or designed, and in particular, dimensions that are valuable for learning. In terms of improving design, applying the framework with a design narrative may help to push designers to consider, for example, the educationally valuable discourses or social activities that emerge from the game community.
Implications for Research
Improving how we define and share design will not only benefit how we carry it out but will also have implications for education research more generally. For example, by definition, design highlights the relationship between the artefact and its context, thus providing a theoretical means for addressing issues related to the sustained systemic impact of our research. Further, because educational games research is nascent, there is a strong need to replicate and test the research that has already been conducted and to be critical of the findings in order to test the limits of our theories. Improving how we share designs should enable this replication and testing, as the games that we design should be shared and played across contexts.
If research findings cannot be replicated, design may provide a means for understanding what did not work and why. For example, Petroski (2008) defines design as a process of “proactive failure analysis,” in which identifying how a designed artefact fails (or is perceived to fail) is the first step for improving it. Shortcomings of our design (both processes and products) provide another lens to account for the outcomes of a study, as the theories that are instantiated in an educational game are not the only driving factors for its success (or failure).
Additionally, addressing design means articulating what we define as failed design. Articulating failures in turn helps to shed light on our values, allowing the interrogation of our own or others’ agendas with regard to the development and instantiation of game-based educational curricula. This practice of sharing failures is important not only because designs may be improved upon through failure analysis but also because researchers and designers may be overly narrow in determining failure. Petroski (2008) writes, “Maker and user, let alone middle man, can have different expectations of what constitutes acceptable performance” (p. 106). By addressing design failures in our work, we may open new avenues for the participating parties to identify areas of improvement. This democratization of development may be important if we expect the technologies that we create to be used and valued by students and teachers alike.
Conclusion
The rationale presented here is intended to encourage a critical examination of how we talk about and share design. This push to attend to design is twofold. First, educational games’ multi-disciplinarity has made it difficult to interpret the various findings across fields, and definitions of important terms are sometimes at odds with one another across research paradigms (i.e., “learning”). Attending to design may help bring together the various perspectives that have already been applied to games. Explicitly defining design theories and improving how we share our design knowledge should enable the development of common artefacts and processes, a necessary first step for replicating findings, iterating on solutions, and moving research across disciplines.
Second, the call to attend to design is intended to try to stymie the tendency to create education research–based games that produce significant findings but see limited application in the world, especially once grant funding ends. Attending to design should be useful for developing games that have the sorts of educational impact that research suggests. Moving forward as a field inherently means advancing theories of learning while advancing design, as the two must support one another in order for our theories be practically useful and thus effective.