Being an Educator and Game Developer: The Role of Pedagogical Content Knowledge in Non-Commercial Serious Games Production

The Teacher Deficit Model

The educational possibilities of games have been discussed under concepts such as serious games, games for change and game-based learning (GBL). What is significant for a large part of the literature on these issues is that it often has a conceptual and hypothetical character (Arnseth, 2006; Berg Marklund, 2013), rarely addresses the hands-on work of teachers (Linderoth, 2014) and sometimes rests on corporate ideologies rather than academic values (Peterson, 2011; Selwyn, 2014). As Berg Marklund (2015) notes, there is a discrepancy between the promises of GBL found in the conceptual literature and the somewhat modest effects of using games in education shown in empirical studies (for an overview of the empirical evidence regarding effects of GBL see Boyle et al., 2016; Connolly, Boyle, Macarthur, Hainey, & Boyle, 2012; Hainey, Connolly, Boyle, Wilson, & Razak, 2016). This discrepancy is partly explained with the argument that teachers fail to understand the educational potential that games have, in what could be called a teacher deficit model of GBL. The argument is that teachers have problematic attitudes towards GBL and hence do not implement it in productive ways. Bourgonjon et al. (2013), for instance, stress that teacher education should discuss games in terms of quality of education so that “teachers will be more inclined to accept game-based learning as a merit for their practice.” (p. 32). Others have made similar claims about the need to educate teachers on how to implement GBL (Becker, 2007; Ketelhut & Schifter, 2011; Rosas et al., 2003). However, as we argue below, there are other explanatory formats besides the teacher deficit model that point more towards the games themselves and how well they are designed to fit educational practices.

Game Production and Serious Games

Game production studies have shown how creative and economical interests can come into conflict during game development (Sandqvist, 2010). According to Tschang (2007), developers have strategies for maintaining creative control over their games in relation to game publishers (including developing their own licenses and hiring famous ‘star’ developers to their teams). Still, Tschang stresses, the economical risk with big budget games makes product decisions conservative.

In relation to production studies on serious games Marfisi-Schottman, Sghaier, George, Tarpin-Bernard, and Prévôt (2009) argue the importance of balancing educational value with fun and attractive game-features in order to make serious games products economically viable. The lack of balance between education and entertainment value has also been argued to have to do with the composition of competencies within game development teams. Marne, Wisdom, Huynh-Kim-Bang, and Labat (2012) argue that one of the main problems with serious games is that if they are mainly created by game developers they tend to be fun but might lack the qualities that facilitate knowledge acquisition. On the other hand, teachers and instructors might design efficient instructional material but fail to utilize game characteristics and design patterns that are motivating and engaging. For Marne et al. (2012) the solution to this is to include teachers in the development of serious games and to develop methodologies and taxonomies that facilitate collaboration between the two groups. A related argument that also avoids the teacher deficit model is made by Bedwell, Pavlas, Heyne, Lazzara, and Salas (2012) who have developed a taxonomy in order to better measure the effectiveness of serious games. Their idea was to design a tool that would make it easier to study the relation between specific game designs and learning outcomes. Fullerton, Swain, and Hoffman (2008) go one step further in describing potential tensions between game design and education. Unlike Marne et al. (2012) and Marfisi-Schottman et al. (2009) who consider it a problem if educational games are not fun enough, Fullerton and her colleges argue that for educational games (as well as political games, news games and art games) enjoyment and fun might not even be useful concepts. Instead, the authors suggest that some games have to be designed with a different mindset. From such a perspective, educational game development is not simply about merging the ‘fun elements’ of games, with the ‘useful content’ of an educational field. Developing serious games in specific fields will instead have to be based upon an understanding of how the pedagogical content can be presented and taught through games as a cultural form.

In his seminal paper about how to understand the skills teachers have, Shulman (1986) coined the concept pedagogical content knowledge (PCK). His core idea is that teachers not only know their subject matter, but also know it as an object for teaching. This means knowing what topics in a given field that are generally taught (selection) as well as knowing the most suitable forms of representation for an idea in order to make it comprehensible to a learner. Shulman stresses that since there are no optimal forms of knowledge representation, teachers must have different sets of tools at their disposal, some that derive from research and others that come from experience. Shulman’s concept has been further developed in the field of educational technology (Koehler & Mishra, 2005), where the importance of PCK when implementing digital tools in educational settings is stressed.

Here, we use Shulman’s concept in order to reframe the call for balance between fun and educational value put forth in some game production studies (Marfisi-Schottman et al., 2009; Marne et al., 2012). Our interpretation of these papers is that they are pointing towards the importance of bringing PCK into serious games production, and seeing teachers as experts during the development.

The Study – PCK and Game Development

In this paper we analyze two cases of game development where games were developed by people who had various degrees of PCK and experience in game development. The focus of the analysis is on how PCK, or lack thereof, manifests itself in serious games production (and self reported accounts of serious games production). The games that were produced in the game division as well as some of the games made by the teachers in the university course were developed for specific educational settings by the people who were also involved in teaching in these settings. This meant that theses games were not made with commercial interests in mind. Instead the games solely gained their value as teaching resources. The case studies employed short-term fieldwork and observation as well as semi-structured qualitative interviews. In this sense, multiple methods were used for triangulation, “the combination of multiple methodological practices, empirical materials, perspectives, and observers in a single study” in order to increase the depth and complexity of our understanding of the studied phenomena (Denzin, 2012, p. 82). Results from each case were then juxtaposed in a cross-case analysis. For the sake of protecting the identity of the informants the two studied environments have been anonymized. Informed consent has been given verbally in all cases and is documented in interview recordings. The aim of the case studies and the cross-case analysis was to approach the following questions:

Case 1 – The National Defence College

The first case was a qualitative study conducted at a game division sorting under a national defense college. This game division conducts research, development and education, testing how war games can be used within military education, especially for the application of tactical theories. The main assignment of the game division is to give advanced support concerning the use of games in military activities, mainly training and education.

Among other tasks, the game division is continuously developing their own digital game platform for training officers. Scenarios built on this platform are implemented in the defense college’s education programs for training officer students (cadets) in tactics. The case study builds on interviews and participatory observation in this educational setting, in what can be termed a short-term focused field study (Higginbottom, Pillay, & Boadu, 2013; Knoblauch, 2005; Wall, 2015). One of the authors observed game-based exercises and interviewed both current and former developers, game managers and instructors. In the study, emphasis was put on producing knowledge with the employees at the game division and the study was part of their research activities. The participatory observation was comprised of observation of a two-day game-based exercise. This exercise used the game division’s own game platform, which they themselves had developed. Additionally, some of the day-to-day work of the division was observed during two days. The researcher engaged in informal group conversations with the group working on development and use of GBL at the division. Nine interviews were also conducted. Six of these were done with personnel or former personnel at the game division. These interviews focused on the design and implementation of the game platform. The interviews ranged from 64 – 104 minutes in length. Three of the interviews were with students at the defence college. These focused mainly on the students’ perspective on the implementation of the game platform, in order to provide background information regarding its use in training. The student interviews were shorter, ranging from 24 – 37 minutes. All interviews were recorded (one supplemented with additional notes due to low audio quality), transcribed and translated into English. For further details on the interviews see appendix A.

Case 2 – University Courses in Educational Game Design

The second case studied seven iterations of a university course in educational game design over the course of four years. In this course, one of the authors was a teacher and an active participant observer. The courses were all 15 ECTS (European Credit Transfer System) and involved a small game design project. In total, 104 participants passed the seven courses. Through a continual cycle of reflection and action, the participant researcher refined his own teaching practices and methods, in what is called a first person action research (see Kemmis, 2009; Reason & McArdle, 2004). This approach sought first and foremost to improve the teaching of GBL to students, but through continuous observation and reflection on the students’ work in the course an understanding of their particular design problems was developed, which in turn is used here in the analysis of PCK in game development.

The course attracted three main categories of participants, with varying background knowledge. The first category was comprised of teachers, who had already finished their degree and had teaching experience. They generally took the course as further training for professional development. Most of the participants in this category displayed a high degree of PCK. The second category consisted of gamers (some with actual experience of game development) who mainly took the course in order to improve their skills in game design. Generally, they had no previous PCK and, in many cases, had low motivation with regards to the pedagogical parts of the course. The final category consisted of teacher students, who took the course as an optional additional course, or students in a master program on educational technology. Participants in this category displayed some degree of PCK and were mostly acquainted with games.

As illustrated by Table 1 the expertise among the informants varied across the cases and across different categories of participants. Additionally, the two cases differ substantially in some other aspects. First and foremost, at the game division, game development was part of their everyday profession, whilst for most the course participants it was an assignment done as a part of their course. However, in the course some of the games were made by teachers, who just as the employees at the game section had PCK and developed games to be used in their own teaching (two of the projects in the course were also made in collaboration with companies and used in employee training). However, other projects in the course had a more tentative character and were developed in relation to the students’ perceptions of potential future practices. The analytically critical aspect here is if and how the participants display PCK during the development of their games. This means that the analysis does not focus on differences between the two cases but rather on how PCK and the non-commercial aspect of the games being developed structured the production. As displayed by Table 1 the different sets of skills and experiences provide a naturally organized frame for studying divergent approaches to game design, enabling contrastive analysis of decision making practices and approaches to tensions between goals of design and education.

OPEN IN VIEWER

Table 1.

Differences Regarding PCK, Gaming and Game Development Experience Among the Informants.

	Informant category	Pedagogical content knowledge	Gaming experience	Game development experience
Case 1	Employees at the game division	High	Some	High
Case 2	Teachers	High	Low	None
	Gamers	None	High	Some
	Teacher students	Some	Some	None

Educational Goals as Added Complexity

In the game design courses one reoccurring problem that was addressed during supervision was that the course participants who we here have labeled ‘gamers’ treated the educational aspect of the game as superfluous. This was mainly because the students had started the ideation of their game design with other components than the educational aspects of the game in mind. For example, one game project started with the idea of seeing how hidden information and deduction mechanics from various mystery/detective board games (that is, a specific type of gameplay), could be combined with the theme of Schrödinger’s Cat. In this, and other projects that started with strong ideas about specific mechanics or themes, educational aspects were treated as a mere additive to an already finished design. In order to fulfill the project criteria (and thereby the criteria for passing the course) of creating an educational game, students sometimes used rather elaborate arguments about far transfer ¹ between gameplay and other domains. These were arguments in a format similar to ‘chess players get better at maths’, a kind of argument that does not withstand empirical scrutiny (see Sala & Gobet, 2017). However, this sort of discourse is present in commercial GBL (Peterson, 2011) and is echoed in the projects designed by ‘gamer’ students. Their games received a thin varnish of educational theorizing but the lack of hands-on PCK and any form of teaching experience was apparent in the lack of any deep coupling between the game design and specific content.

In contrast to this approach, the course participants who were teachers grounded their design process in PCK. Even though the projects made by students in this category tended to have the clearest educational goals, the course participants still had to choose the specific curricular subjects they wanted the game to focus on. This sometimes meant redefining their initial, often overambitious, ideas about what the game could accomplish and settle for smaller, more focused learning goals with clearer relationships between the domain’s field of knowledge and the design of the game. Some course participants who had experience in commercial game design pointed out that taking the learning objectives into account when building their games was rather different from what they were used to. Two course participants who had worked as designers of board- and role-playing games stressed that they found it more challenging to ideate their game with educational aspects in mind. The fact that these games should teach something constrained their design.

As stated, adhering to a specific form of gameplay could sometimes mean that the students lost control over the educational goal. Either the goal became so general or generic that it might be questioned if the game actually could claim to support any form of skill or knowledge acquisition (as in the designs based on far transfer), or the course participants had to settle with a pre-defined relationship between gameplay and field of knowledge. Since the course participants’ main goal with their games was to fulfill the demands in the course, they could allow themselves to be vague regarding the field of knowledge. In a professional educational practice, this vagueness is not a sustainable option and would likely be one incentive for teachers to disregard the product. As teachers work first and foremost with the field of knowledge of their subjects in mind, and of the educational goals of their teaching, a game with only a diminutive connection to a field of knowledge and to learning outcomes is of questionable value as a tool for teaching. In contrast, a closer coupling between goals of game design and education is displayed in the interviews from the game division at the defense college, following below.

Instructional Goals as a Starting Point

Unlike some of the game development in the course, the defense college’s gaming division developed games that had the clear goal of being used in courses at the officer training program. Yet, the employees stressed that, to some degree, they also could have problems with unclear and sometimes changing learning objectives. The division has a long history of trying out different game forms for military training. Both the director of the division as well as a former lead researcher mentioned cases where they had seen game-based exercises failing due to a too narrow focus on a specific game technology or game genre (similar to the game design students discussed previously). In these cases, the problem had been that the projects had started with the goal of utilizing or developing a specific technology. This technology focus was described as a problem since the issue of what students learned became secondary, putting the cart before the horse.

The current director of the game division also pointed out that different instructors who participated in the game-based exercises could have their own specific interests that in some cases lead to vagueness in terms of educational goals. The director who also was the lead game developer as well as the main game facilitator stressed that it is a constant responsibility to stay on track when running the games:

It is also a responsibility to make sure that you do not drift away too far and make things too broad. You have seen our exercise and there are a lot of things going on at the same time. Some instructor that is interested in a specific thing may want to boost what they think is important. However, that happens at the cost of other important things. So you must have an idea about what this shall be and make sure that you stay on track. (Director of the game divison)

In his focus on subject content as the natural starting point in the process of developing their educational games the director displays PCK where overall educational goals are prioritized over individual instructors’ interests. This knowledge is understood as the basis that should guide the whole development process. The director stresses that if you have this focus, many parts of the game will simply unfold:

When we have designed our game platforms we have tried to focus on the fact that the students are to be exposed to different problems and choice situations. And the game /. . ./ is supposed to create an environment where these problems arise. So the game becomes what it has to become if these problems are to arise. (Director of the game division)

The game division has a history of working with digital games as well as board games and scenarios with role-playing elements. This means that they do not start by locking the development to a specific technology – as stated earlier, they have found that this leads to problems in relation to the educational goals. In this sense, the game section is developing tools to meet their educational goals, and let those goals guide to suitable technology and materials. In relation to commercial game development, it is likely quite rare that designers find themselves in a situation where it is not predetermined if they are to develop a digital or an analogue game. However, if the task is to find the best practice given PCK in regard to a certain field, this makes sense:

So what I am saying is, why relate a board game to our digital platform? Imagine you are to give a course module and have to choose between A and B, then it is all about the kinds of problems you want to emerge, that have to emerge. And a digital game platform has some benefits because we can in an easy way make sure that you can’t see my units and I can’t see yours, the classical hidden movement thing, so that speaks for a digital solution. If that is not important, maybe we can do the board game. The same thing, does it work to take turns, you see what I do, then you can consider what you are to do and move your troops. Or does it have to be continuous time where you do not have a clear separation between moves. These are the things you need to consider and it affects how the game must be. /. . ./ It’s not about whether I prefer this or that. This is how we have tried to work and then the game grows from that. (Director of the game division)

As stated in the quote above, both game genre and game mechanics, in addition to the underlying technology and materials of the game, are subordinate to the educational goals and the students’ opportunity to learn. The problems that they want to expose the students to are the starting point for the development process, rather than a layer of educational content added on top of an already established technological and game-mechanical framework.

Breaking Design Heuristics

One result of our study is that when PCK was displayed it sometimes meant breaking design heuristics suggested in the literature on game development. One such case has to do with transparency and usability regarding the interface. Adams and Rollings (2007), for instance, stress that game interfaces should provide good feedback, limit the number of steps to take actions, permit easy reversal of actions, minimize physical stress and avoid straining the player’s short-term memory. Schell (2015) has also pointed out the importance of transparent game design that helps divert cognitive load off the player. While these suggestions make sense in relation to games where the main purpose is entertainment, the issue is not so clear in educational game design. For instance, Linderoth (2012) has argued that if a game relies on interface designs where the relevant/best interaction possibilities are visually highlighted there is a risk that the educational outcome is lost. With such a design, players do not have to learn to discern relevant affordances in the situation. Instead, perceptual cues provided by the interface will make sure that they can progress in the game. One of the programmers and technical developers at the game division related such a design heuristic to his PCK:

If you look at the interface from the gamer’s perspective, then I think a bigger deal is that compared to other software, the developer has a natural instinct that usability and UX shall help the user. The user shall get as much help and support as possible, you put as much complexity as possible within the system so that the user is cognitively offloaded and gets an as simple task as possible. That is put to a test here, cause part of what you want to achieve in the game situation is that they should be exposed to a certain type of complexity. So that is kind of strange and I guess somewhat different to develop a game for an entertainment purpose. (Technical developer at the game divison)

In other words, when designing with an educational outcome in mind, the goal might not be to make the interface transparent to the player, allowing him or her more easy access to the game world. Rather, it may very well be that the learning goal is precisely to be able to make the relevant perceptual distinctions in that interface. A lessened complexity of the user interface might constitute a case of throwing the baby out with the bathwater – it will allow the player a smoother experience as well as easier and faster progression and completion of the game, while forgetting that the intended educational outcomes lies precisely in being able to handle that perceptually complex environment. Schell (2015), for instance, discusses how strange it is that children’s games do not map the same action to both mouse actions:

Smart designers break this rule of thumb, and make the left and right mouse buttons both map to the same action, so that either button can be pressed successfully. Really, why wouldn’t you do this for every game that only needs one mouse button? (p. 244)

From a learning perspective this design would not provide an opportunity for the player to actually learn how to separate between the functions of different mouse buttons. While making perfect sense from the perspective of entertainment purposes, it is a clear example of a how principles from game development can come in conflict with educational potential.

In the two studied cases the informants do not only break interface design heuristics, but PCK also challenges some principles of game design. For instance, the fundamental idea that games should strive to provide the player with meaningful choices (Schell, 2015, p. 181) was disregarded for educational purposes. In the course a few participants created a game called “The board game”. The goal of this game was to create a more diverse board of directors at a company. Inspired by Gramsci’s concept of hegemony the group made a game where the players tried to take action to challenge the domination of just one group being represented on the board. However, the game was based on an algorithm that would recreate the hegemonic structure that the players started with. In that sense the choices that the players made in the game are meaningless, however that was also the point of the game. A similar design principle has been used in well-known games where meaningless choices are used to difficulty or futility of action, such as in DEPRESSION QUEST (Quinn, 2013) and September 12^th (Frasca, 2007).

These examples echo the suggestion put forth by Fullerton et al. (2008), that educational games should not necessarily be designed to be fun. Some games can have other aesthetics than being fun, such as persuasive games (Bogost, 2007). While enjoyment is a well-established aesthetic for games in general, it is severely limiting to ask that every educational game should use fun or enjoyment as a design goal. In many cases, making the games fun, and therefore giving players incentives to engage with it (and, by extension, learn from it) is seen as a primary motivation for using serious games (Barab, Thomas, Dodge, Carteaux, & Tuzun, 2005). However, when the idea of a game is to illustrate a feature of a real system or to make the player reflect on serious issues, the game can still have educational value despite not being obviously fun or enjoyable. One crucial difference between mainstream game development and the cases studied here has to do with the differences in economic conditions. As Schell (2015) stresses, one aspect of game development is to make games that earn a profit. In both the studied cases, the game development rests on educational rather than commercial values, and the appeal of the game on a commercial market is not considered in the design. However, this is not simply a reflection of an existing difference between entertainment games and serious games, as many serious games are made as commercial products as well.

Limitations and Suggestions for Further Research

While this has been a study based on two limited cases of game development in very different environments, it nevertheless points to important lessons for further research in the field of educational game design. By showing that there are common elements to game design in both of these very different settings, we hope to provide some impetus for further empirical studies of design work of GBL, as well as for an approach to GBL that takes PCK into account.