Can Serious Games Contribute to Developing and Sustaining 21st Century Skills?

Abstract

Serious games (SG) are innovative tools that are widely recognized as having considerable potential to foster and support active learning. This article addresses the question of whether and how SG can contribute to the development of the so-called “21st century skills” in education. This article starts by characterizing the current need for 21st century skills and the identification of these core skills. Thereafter, it reports on a literature review of studies analyzing SG impact on the development of one or more 21st century skills; and finally, it analyzes which, among the most relevant game characteristics, are those that could facilitate 21st century skills development. This study offers a multifold perspective on the use of SG to support 21st century skills development that may be helpful for both teachers and SG designers.

Keywords

education serious games 21st century skills game-based learning competence-based education

Introduction

There is a need to develop new skills to face future challenges and prepare students for new jobs and technologies, including those job roles that do not yet exist (Fisch & McLeod, 2009). In society in general and education in particular, transition from the 20th century to the 21st century has brought important changes. First, according to Prensky (2004), technology has become “an entire strategy for how to live, survive and thrive in the 21st Century” (p. 2). Prensky observes that youth today communicate, buy and sell, search information, and socialize differently. The 21st century context requires a new set of competences beyond the obvious information and communication technology (ICT) literacy: communication, collaboration, social and cultural skills, creativity, critical thinking, problem-solving, productivity in a globalized world, learning to learn skills, self-direction, planning, flexibility, risk taking, conflict management, and a sense of initiative and entrepreneurship (Voogt & Pareja Roblin, 2012). Such new skills and competences are commonly defined as “21st century skills” in opposition to the 20th century skills (Wisniewski, 2010), based on the 3 R’s of Reading, wRiting, and aRithmetic and focusing on the “lower levels of Bloom’s taxonomy—knowledge, comprehension and application” (p. 57). According to Trilling and Fadel (2009), the 21st century skills term defines the new set of skills required to succeed in learning, working, and living. These 21st century skills should be developed by everyone, from primary school pupils to lifelong students and those involved in informal learning settings. To sustain the development of 21st century skills, the effectiveness of adopting active learning methodologies that encourage self-regulated learning is widely acknowledged (Bell & Kozlowski, 2008). Active learning methodologies involve activities that emphasize the development of learner skills (Bonwell & Eison, 1991). Active learning covers forms of learning in which the learner has some control over the input of information (Cohn, Atlas, & Ladner, 1994). It enhances the learner engagement in learning tasks and offers significant added value to provide learners with the abilities to cope with new challenges, solve problems (Edens, 2000), and adapt to changes in technology, knowledge, and jobs.

Games are a form of active learning that enable the learner to have some control of the game activity and engage in interaction. Game-based learning (GBL) is grounded in active learning methodologies and encourages learning activities by building on engagement and challenges to achieve the intended learning objectives. Games can therefore be considered among the tools to develop new skills for the 21st century (Prensky, 2006; Redecker et al., 2011) as well as being “central nodes in the organization of contemporary leisure culture, computer-mediated interaction, visual culture, and information societies” (Simon, 2006, p. 64). Playing games could develop a “gamer disposition” that could foster learners to thrive on change, discover new insight, and learn how to overcome obstacles (Brown & Thomas, 2008). Although games, in general, could be analyzed for their contribution to 21st century skills, in this article, we delimit our focus on education-oriented games or serious games (SG) as “games in which education (in its various forms) is the primary goal, rather than entertainment” (Michael & Chen, 2005, p. 17).

In this article, we tackle the issue of whether and how SG can support the development of 21st century skills in the various modalities and levels of formal and informal education. The article aims to answer the following research questions:

Research Question 1: What exactly do we mean by 21st century skills?

Research Question 2: Is there any evidence that SG are being used to develop 21st century skills? (whether)

Research Question 3: What are the game characteristics supporting 21st century skills development? (how)

To answer the first question, we introduce the nature and type of the so-called 21st century skills and the core skills in the following sections. To answer the second question, we conduct a literature review of SG and provide evidence of the use of SG for the development of at least one of the core 21st century skills. The analysis of the SG identified in the literature to support 21st century skills enables us to answer the third research question in this article by analyzing the characteristics of SG that support each of the 21st century skills.

Finally, we discuss the current opportunities and limits in the use of SG to develop 21st century skills, as well as the steps in game design and the teaching use of games that could help achieve this objective.

Identification of 21st Century Skills

Education has been traditionally focused on knowledge transmission in a context of formal educational settings. In the final decades of the 20th century, a learner-centered approach to education enabled a competency-based orientation in formal and informal settings to be introduced (Grant & Associates, 1979; Spady, 1977). In competency-based education, the emphasis is not on the curriculum and domain-specific knowledge but is oriented to a learner-centered and outcome-based approach. Despite the wide controversy on the distinction between competences and skills, in this article we follow Sue, Arredondo, and McDavis (1992) who characterize competence as a set of knowledge (K), beliefs and attitudes (A), and skills (S). In the context of digital games, Che Pee (2011, p. 22) consider competence as a “combination of knowledge, skills, and behavior leading to an individual being able to perform a certain task to a given level.” Accordingly, skill is the part of the competence related to the ability to behave effectively and engage certain attitudes and knowledge in action-oriented situations (Argyris & Schon, 1974). Carmeli and Tishler (2006) define skills as the “ability to do something in an effective manner” (p. 13). The characterization of the core or key skills in each educational system and society is one of the main challenges for educational policy makers. The knowledge society requires a new set of skills to cope with the knowledge-based economy and uncertain worldwide changes (Fisch & McLeod, 2009). This has led to the definition of the so-called 21st century skills and their respective frameworks of reference (Binkley et al., 2011; Dede, 2010; Redecker et al., 2011).

Since the first references to 21st century skills in the final decades of the last century (Jones, 1996; Stuart & Dahm, 1999), much has been published in the literature about these skills in an attempt to identify them precisely. Various definitions and classifications were proposed over the following years, and researchers have considered the characterization of 21st century skills from many perspectives, for example:

Binkley and colleagues (2011) defined 21st century skills as ways of thinking, working, and living in connected, media-rich worlds;

Redecker and colleagues (2011) in the Joint Research Center report for the EU Commission looked at the issue from the perspective of three general categories: (1) personal skills (initiative, resilience, responsibility, risk taking, and creativity); (2) social skills (team, networking, empathy, compassion, and co-constructing); and (3) learning skills (managing, organizing, metacognitive skills, and failing forward).

From an ICT-focused perspective, Dede (2010) included the following in a list of newly required abilities: “information problem solving” or “the ability to rapidly filter huge amounts of incoming data; extracting information valuable for decision making;” and “the ability to separate signal from noise in a potentially overwhelming flood of incoming data” (p. 5).

Furthermore, educational organizations have developed structured frameworks to define 21st century skills based on educational needs analysis, including a categorization of skills and a characterization of each of the skills in relation to the framework. Figure 1 shows two exemplar 21st century skills frameworks by the Partnership for 21st century skills (P21, 2007) and EnGauge.

Figure 1.

Partnership for 21st century skills (left) and EnGauge (right).

In the cited literature on 21st century skills, there is no consensual shared definition of the concept—nor is there a list of skills. As underlined by Kickmeier-Rust and Dietrich (2012, p. 681), the major challenge hindering the possibility of providing a shared definition and formal model for 21st century skills arises from “the unclear, probably vague and highly informal nature of these 21st century skills.” However, different authors and educational organizations have developed different conceptualizations of such skills by building on each other’s ideas so as to avoid a “Tower of Babel” situation (Dede, 2010). The resulting frameworks are largely consistent and generally focus on skills such as innovative thinking, creative problem solving, metacognitive abilities, communication, and collaboration. According to Kickmeier-Rust and Dietrich (2012, p. 680), 21st century skills involve “meta-skills such as problem solving, non-linear thinking, creativity, or communication skills.”

With the final aim of studying the impact of SG on the skills required to live in and be proactive actors in the knowledge society, we refer to the skills reported by Voogt and Pareja Roblin (2012). These authors proposed an extensive list of the skills found in six relevant frameworks on 21st century skills, namely, P21, 2007, EnGauge 21st century skills (Metiri Group & North Central Regional Educational Laboratory, 2003), assessment and teaching of 21st century skills (ATCS), National Educational Technology Standards and International Society for Technology in Education (NETS/ISTE) framework, competences for new millennium learners by Organization for Economic Cooperation and Development (OECD, 2005), and Center for Social and Economic Research (Gordon et al., 2009). Voogt and Pareja Roblin developed a meta-analysis of these six frameworks to identify the skills included in these different frameworks and then categorized the skills according to their shared presence. The categorization of 21st century skills made by these authors is shown in Table 1, which also shows the level of frequency of each skill in the frameworks.

Table 1.

Summary of 21st Century Skills Mentioned in Six Related Frameworks.

Mentioned in all frameworks	Mentioned in most frameworks	Mentioned in a few frameworks	Mentioned only in one framework
Communication Collaboration ICT literacy Social and/or cultural skills	Creativity Critical thinking Problem solving Develop quality products/productivity	Learning to learn Self-direction Planning Flexibility, and adaptability	Risk taking Manage and solve conflicts Sense of initiative and entrepreneurship

The skills identified by Voogt and Pareja Roblin (2012), namely, communication, collaboration, ICT literacy, and social and/or cultural skills, appear in all six revised frameworks, as well as matching two of the 4Cs proposed by Frydenberg and Andone (2011), namely, critical thinking and problem-solving, communication, collaboration, creativity, and innovation skills. ICT literacy is widely recognized as a key skill in the knowledge society (Bawden, 2008; Ferrari, 2012). Following these results, we can assume that, in addition to the ICT skills clearly emerging during the last decades, communication and collaboration are two major skills that we must take into account for success in modern society. These skills go far beyond the traditional 3Rs (Reading, wRiting, and aRithmetic) of the 20th-century skills panorama and are a promise to help students and professionals to meet present demands of lifelong learning.

Investigating Whether SG Can Contribute to Developing 21st Century Skills

SG researchers agree on the value of games as tools to develop the skills needed in the knowledge society (Kickmeier-Rust & Dietrich, 2012; Pivec, 2007; Redecker et al., 2011, Westera, Nadolski, Hummel, & Wopereis, 2008). According to Pivec, games enable an active learning approach that encourages “critical thinking, group communication, debate and decision making” (p. 391). Additionally, Westera, Nadolski, Hummel, and Wopereis (2008) highlight the contribution of SG to developing skills, especially those “needed in the information age: self-regulation, information skills, networked co-operation, problem solving strategies and critical thinking” (p. 420).

To understand whether SG can profitably be used to this end, we have searched the literature for specific case studies demonstrating the effectiveness of SG in developing 21st century skills.

Method

To answer the second research question, “Is there any evidence that SG are used to develop 21st century skills?” we have reviewed studies that analyze the use of SG for the development of these skills. The search strategy has been developed in the ISI Web of Knowledge, ProQuest, Scopus, and Google Scholar, using the following keywords in the search engines: “serious game”, “skill”, “competency”, and “competence” and then combining the keyword “serious games” and the name for each of the skills identified by Voogt and Pareja Roblin (2012; see Table 1).

In Table 2, we report on studies analyzing the potential of SG to develop each of the competences identified by Voogt and Pareja Roblin (2012). The goal of this selection of examples of SG is to identify exemplary SG that will facilitate a subsequent analysis of the game characteristics that support 21st century skills. The search strategy was developed in the EBSCO database, combining the terms associated with each of the 21st century skills and the synonyms appearing in the literature. The terms used for the search strategy corresponds to each of the terms in the first column of Table 2, combined with the terms “serious games”, “digital game based learning”, and “SG”. When the result of the 21st century skills and serious game provided more than 10 results, the articles were selected according to their impact in other studies referring to them. However, the 21st century skills of “ICT literacy”, “computer literacy”, “digital literacy”, or “eCompetence” did not provide any results, and none of the articles explicitly declared the learning objective of ICT literacy, computer literacy, digital literacy, or eCompetence. In this article, we discuss this specific research difficulty in the Discussion section. In Table 2, the frequency column (freq.) includes the four skills considered in all the frameworks revised by Voogt and Pareja Roblin (2012) under the term “all”; the term “most” includes the skills considered in most frameworks (i.e., P21, EnGauge, ATCS, and NETS/ISTE); the term “few” includes the skills included in just a few frameworks; and finally, the term “one” includes skills considered in only one framework.

Table 2.

Games Whose Effectiveness for Sustaining 21st Century Skills Has Been Reported in the Literature.

Freq.	21st century skills	Serious game examples related to each of the 21st century skills
All	Collaboration/teamwork, the ability of team members to coordinate themselves to accomplish tasks (Stevens & Campion, 1999) contributing to decision making to accomplish a common goal, assuming responsibility, and a certain leadership for collaborative tasks	Most SG encourage individual skills development. However, despite this result from the literature review, we selected four representative studies as examples of the use of multiplayer games that develop teamwork skills. Two of these studies address teamwork in terms of group development and leadership. The Leadership Game is a multiplayer online SG intended to develop leadership skills in teamwork (Ferretti, Brami, Quero, & Fiorese, 2010). NoviCraft team building and leadership solution is an immersive multiplayer 3D application for assessing, building, and developing leaders, teams, and teamwork (Häkkinen, Bluemink, Juntunen, & Laakkonen, 2009). Two other games focus on the decision-making process of teamwork. Linehan, Lawson, and Doughty (2009) developed a serious game to encourage collaborative decision-making behavior. The SG MetaVals aims to develop collective decision making in dyads (Romero, Usart, & Almirall, 2011; Usart, Romero, & Almirall, 2011)
All	Communication—the ability to “articulate thoughts and ideas clearly and effectively through speaking and writing” (Buck Institute for Education, 2012)	The literature review enabled the identification of two SG aspects contributing to communication skills development through the use of SG. First, by allowing the players to interact among themselves in a virtual environment: this is the case of VECTOR, a cultural SG that places learners in a virtual foreign town for language learning (Johnson, 2007). Second, communication skills are developed by interacting with a virtual player. In this line, the SG deLearyous aims to develop an environment in which users can improve their communication skills by interacting with a virtual character (Vaassen & Daelemans, 2011)
All	ICT literacy/computer or digital literacy/eCompetence as the ability to use computers and digital devices for personal and professional purposes	We should consider the development of ICT literacy not only as an indirect result of the use of SG as digital environments but also as a learning objective that some SG could be designed to support. First, we should consider SG as a digital environment requiring a certain level of ICT literacy. In this sense, one of the challenges in the use of SG in formal education is the lack of ICT literacy among teachers and some of the adult learners that could be considered as digital migrants (Prensky, 2001). However, SG helps to develop digital literacy through computer-based learning and human computer interaction (HCI) developed through gameplay. Prensky (2003) claims that SG contributes to developing ICT literacy for teachers and learners. More generally, Subrahmanyam, Greenfield, Kraut, and Gross (2001) consider that for skills to be enhanced by gameplaying, players must already possess such skills to some degree. Therefore, learners engaged in SG require a basic level of ICT literacy, but could benefit from the use of SG to develop this skill. For this reason, SG designers should consider HCI studies to develop ease of use among learners and increase acceptance among those with low levels of ICT literacy. Second, some SG develop ICT literacy skills—such as JobScout (http://myjobscout.org). This game aims to increase digital literacy to support employment searches on the Internet
All	Social skills/cultural skills/cultural heritage awareness/citizenship as the ability to understand the social and cultural context, to act as citizens and the ability to interact with others in a given social context	Anderson and colleagues (2010) developed a review of games in order to develop cultural heritage awareness. One of the interests Anderson and colleagues highlight in the use of SG for cultural heritage awareness is a virtual and augmented reality. More recently, Mortara and colleagues (2013) analyzed the use of SG for cultural heritage awareness in the context of Japanese culture and etiquette. They analyzed the impact of the game through Bloom’s taxonomy, but also took into account the gaming and learning experience of the students, which stressed the interest of games for long-term retention and learners’ engagement. Icura (Mortara et al., 2013). Civilization (Apperley, 2006) Revolution (Francis, 2006) Virtual Egyptian Temple (Jacobson, & Holden, 2005). Enhanced-learning environments with creative technologies for bilateral negotiations (ELECT BiLAT) is an SG immersive learning environment that teaches the preparation, execution, and understanding of bilateral meetings in a cultural context (Hill et al., 2006), and so contributes to the development of cross-cultural awareness (Lane et al., 2008). Similar to this SG, the Tactical Language and Culture Training System (TLCTS) is a story-centric SG mostly used to train soldiers in cultural, verbal, and nonverbal communication (Johnson, & Wu, 2008)
Most	Creativity as the ability to generate original, valuable, or useful ideas (Sternberg & Lubart, 1995) in individual and collaborative contexts (Romero, Hyvönen, & Barberà, 2012)	There is a growing number of SG aiming to develop the skill of creativity and we identify four examples. Pannese, Hallmeier, Hetzner, and Confalonieri (2009) analyze the uses of storytelling and serious games to foster learner creativity and encourage awareness. In the study of Song (2008), individual games stimulated learner creativity which was then transferred to the group blogs. Ott and Pozzi (2011) analyze the use of mind games to elicit creative attitudes in the context of primary education. Finally, creating games through game building environments such as Kodu supports learner creativity and ICT literacy by helping the players to learn programming (MacLaurin, 2009)
Most	Critical thinking as the ability to develop critique and “disciplined self-directed thinking” (Paul, 1990)	Gameplaying might increase learner critical thinking and problem-solving skills (Katz, 2000; Prensky, 2000) if the game avoids “quick answers and fast action for reflection and critical thinking” (Michael & Chen, 2005, p. 13). For Graesser, Chipman, Leeming, and Biedenbach (2009, p. 93) “games are deep in the sense that the content and skills tap deep reasoning, critical thinking, complex systems, causal chains and networks.” Jenkins and colleagues (2009) observe that SG “effectively engage students (notably, female students responded very well) in critical thinking about authentic scientifically based scenarios and enhance their interest in these scenarios” (p. 459) Despite the fact that SG are discussed as a learning environment that could foster critical thinking if they have a certain level of complexity, we only identified one SG specifically addressing this 21st century skill, the Critical Thinking Training Game. This SG has been designed to specifically support critical thinking as one of the key skills required by the U.S. Navy
Most	Problem solving as the ability to develop a sequence of actions to achieve a goal (Cohen & Feigenbaum, 1982)	Certain approaches to SG gameplay involve problem-solving skills. Adventure games are especially good at developing problem solving skills. In The Adventures of Jasper Woodbury developed by the Cognition and Technology Group at Vanderbilt University (1997), learners are engaged in completing the game and solving the problems they encounter
Most	Develop quality products/productivity (all except ATCS) as the ability to “set and meet high standards and goals for delivering quality work on time; demonstrating diligence and a positive work ethic” (BIE, 2012)	Despite McGonigal’s (2011) arguments regarding the benefits of games for improving professional and personal words, games are often perceived as an unproductive waste of time, especially among nongamer women (Royse, Lee, Undrahbuyan, Hopson, & Consalvo, 2007). The literature review allows us to identify only a few examples of SG aimed at explicitly developing quality orientation or productivity. The Good Productivity Game (Lutzker & White-Blackburn, 1979) develops productivity skills by providing just-in-time feedback. In Taskville (Nikkila, Linn, Sundaram, & Kelliher, 2011), productivity increases while the game is played
Few	Learning to learn (ATCS, EU) is “a set of meta-skills that enable (lifelong) learners to successfully construct and shape their own learning process” (European Commission, 2002, p. 31)	In the context of face-to-face games, Raser (1969, p. 115) notes that “gaming is, in essence, a process of developing skills that increase one’s ability to learn new facts and abilities.” In digital games, Pivec (2012) affirms that games could develop the “learning to learn” skill by experimenting within a safe environment. Raybourn (2009) describes the development of the multiplayer computer game—America’s Army Adaptive Thinking & Leadership—that was created to train intercultural competence and learning-to-learn skills related to metacognitive agility (self-awareness and self-regulated learning)
Few	Self-direction/self-regulation (P21, EnGauge, and OECD). In relation to learning-to-learn skills, self-direction is defined as the ability to set goals and plan for the achievement of those goals. Vohs and Baumeister (2004, p. 2) define “self-regulation” as “the exercise of control over oneself, especially with regard to bringing the self into line with preferred (thus, regular) standards”	SG involve different degrees of self-direction and self-regulation, depending on the game mechanics and the possibility for learners to choose and plan the goals, and adapt their behavior to achieve goals. Virtual games are one of the categories of SG that enable learners to have a higher degree of self-direction and self-regulation. According to Lane (2007), self-regulation is demanded for playing in the TLCTS language and cultural virtual game, as part of the intercultural development within the gameplay. Wagner (2008) analyzes the use of massively multiplayer online role-playing games (MMORPG) as constructivist learning environments enabling 21st century skills development. He argues that MMORPG could help learners develop self-regulation skills because they must be dedicated and disciplined to succeed in the game. He discusses MMORPG MIT Revolution and Indiana University’s Arden: The World of William Shakespeare (both of which are modifications of the commercial game Neverwinter Nights) In the field of health care and well-being, SG could foster self-regulation through the use of physiological feedback to support self-regulation skill development. An example is the Auction Game analyzed by Jercic et al. (2012) to develop player emotion regulation in the context of financial decision making
Few	Planning (EnGauge and OECD) as the ability to analyze a situation and create a mental simulation of future actions	SG allow different degrees of action planning and strategy during the game. Some of the games in the category of strategy involve the development of planning, as well as flexibility and adaptability skills. In the SG RollerCoaster Tycoon (www.rollercoastertycoon.com), players create companies or build theme parks and so develop strategic thinking and planning skills (British Broadcasting Corporation, 2002)
Few	Flexibility and adaptability (P21 and EnGauge) as the ability to reconfigure current and future actions (planning) in response to contextual demands	Similar to planning potential, SG enable different degrees of flexibility and adaptability according to the changing contextual demands or programmed critical incidents. Simulation Roulette is a health care game where the learners deal with random complications and prove their flexibility and adaptability skills (Frederick, Corvetto, Hobbs, & Taekman, 2011). In a military context, the ATL training simulation is a multiplayer simulation created for the Adaptive Thinking & Leadership (ATL) Program to train special force team leaders; in particular, it aims to develop adaptability, effective communication, analyze ambiguous situations, teach self-awareness and innovative thinking, and critically use effective problem-solving skills (Raybourn, Deagle, Mendini, & Heneghan, 2005)
One	Risk taking (EnGauge) as the ability to make decisions under uncertainty	SG are risk-free environments that facilitate discovery learning, curiosity, and perseverance (Kirriemuir, 2002). This game characteristic makes SG ideal for developing risk taking and management skills and transferring them to real life. An example is Hot Shot Business (Jason, 2003), where the player takes the risk of opening a self-financed business or obtaining a loan from a bank. Another SG that aims to support risk taking in other fields, such as food safety, is Serious Gordon (Mac Namee et al., 2006)
One	Manage and solve conflicts (OECD) as the ability to identify and manage conflicts in different situations	The SIREN project aims to develop social games for conflict resolution based on natural interaction. Within this project, Cheong and colleagues (2011) study the value of serious games for conflict resolution skills; they have developed an SG prototype to support the skill of conflict resolution (Berger, Liapis, & Yannakakis, 2012). In another approach, Smith (2004) denounces the risks of conflict being transferred in multiplayer games based on cheating and antisocial behavior
One	Sense of initiative and entrepreneurship (EU) as “a process by which individuals—either on their own or inside organizations—pursue opportunities without regard to the resources they currently control” (Stevenson & Jarillo, 1990, p. 23)	There is an increasing number of entrepreneurship SG in management education aiming at developing this skill within students and adult learners. We selected two examples representing two approaches. The first approach is a business simulation that helps to develop entrepreneurship skills. The SG SimVenture offers a holistic view of the common processes of running a small business. SimVenture has been analyzed as an engaging environment for encouraging entrepreneurship skills (King & Newman, 2009; Williams, 2011) The second approach is the design of an SG that develops the abilities, conceptions, and theories related to entrepreneurship. In this line, Babson Business School has developed “a game to support learning about how entrepreneurs think under conditions of risk, uncertainty, and unknowability.” The game is based on the theory of effectuation (Sarasvathy, 2008) and is designed to replace a case study for in-class discussion on entrepreneurial thinking. Overall, the use of SG is part of the method approach; it gives students a different environment to practice entrepreneurship

Note. SG = serious games; 3D = three-dimensional; VECTOR = Virtual environment cultural training for operational readiness; ICT = information and communication technology; ATCS = assessment and teaching of 21st century skills; P21 = Partnership for 21st century skills; OECD = Organization for Economic Cooperation and Development; MIT = Massachusetts Institute of Technology; SIREN = Social games for conflict resolution.

Results show a diversity of SG used to train each of the 21st century skills. Most of the studies focus on supporting one of the skills. However, we can consider that all games studied encourage ICT literacy skills because they focus on the use of a computer-based environment with a certain level of human–computer interaction; therefore, users need basic ICT skills in order to significantly play the game.

The analysis of the examples of SG aiming to support social and cultural skills leads us to observe that these games are based on communication; and therefore, communication skills are in some cases already supported as a metaskill during the development of social and cultural skills. The development of interpersonal abilities such as social, cultural, and communication skills is supported by games that employ virtual worlds where the players can interact with other players or virtual characters. These games are also used to develop communication skills, combining the potential of virtual worlds to develop cultural, social, and communication skills.

Among the results of this literature review, only one result showed an SG that is intended to develop all the 21st century skills, namely, ATCS, an SG that “addresses the issue of assessment and teaching of 21st century skills” (ATCS; Griffin, McGaw, & Care, 2012, p. 4). To articulate the 21st century skills, the ATCS has grouped these skills into two main categories: collaborative problem-solving and ICT literacy.

How SG Can Support the Development of 21st Century Skills

After identifying studies focused on the use of SG to develop 21st century skills, we observe that certain skills are better developed within certain categories of games. Virtual worlds facilitate complex collaboration games to develop communication, cultural, and decision making (Lane et al., 2008); while strategy games contribute to the development of planning, flexibility, and adaptability skills (Frederick, Corvetto, Hobbs, & Taekman, 2011). In this section, we aim to identify which are the most relevant characteristics embedded into SG that support the development of one or more 21st century skills. To identify the SG characteristics that could support skills development, we first produced a game characteristics classification by reviewing the pioneer studies on analogical GBL, and subsequently, we focused on computer-based games and SG. In the following section, a classification analyzes each of the SG characteristics in relation to their potential for developing and sustaining 21st century skills.

Garris, Ahlers, and Driskell (2002) reviewed the literature on game characteristics. Their study focuses on the evolution of game characteristics since the 1950s, when Wittgenstein (1953, 1958) highlighted the difficulty of defining the essential characteristics of games as he could not see any properties common to all games. Later, Crookall, Oxford, and Saunders (1987) distinguished between games and simulations. The authors noted that games do not necessarily intend to represent the real world as simulations do. In this line, Thomas and Macredie (1994) claimed that the core characteristic of games is that actions have no real-world consequences; this enables learners to practice safely without fear of failure. Based on their literature review, Crookall, Oxford, and Saunders identified various game characteristics, rules, and strategies—and the costs of losing a game. De Felix and Johnston (1993) studied the structural components of a game, such as dynamic visuals, interaction, rules, and goals. Malone (1981) introduced the characteristics of challenge, fantasy, complexity, and control to identify the level of engagement in educational games.

According to Alessi and Trollip (2001), game characteristics can include competition and rules. Other authors focus on the characteristics of interactivity (Thornton & Cleveland, 1990), challenging activities (Malone, 1981), choices (Hannafin & Peck, 1988), and fantasy (Charsky, 2010). These characteristics could challenge structured and organized activity (Crawford, 2003) and help learners acquire and practice skills (Charsky, 2010).

Thanks to these game characteristics, SG motivate learners to complete learning activities and provide them with authentic learning experiences where entertainment and learning are seamlessly integrated (Gee, 2005; Prensky, 2001). SG designers use these characteristics consciously and apply them differently to differing game genres (racing games, fighting games, and role-playing games [RPGs]) to facilitate learning and skill development objectives.

Game Characteristics Classification

After reviewing the game characteristics identified by different SG researchers, this section defines a classification of SG characteristics in relation to their potential for developing and sustaining 21st century skills. Table 3 presents a classification of game characteristics supporting the development of 21st century skills. It is based on the works of Garris et al. (2002) and Charsky (2010) and shows five main characteristics (left column/bold items). The game characteristics classification of Garris et al. (2002), fantasy, sensory stimuli, challenge, mystery, and sense of control that games provide foster a state of flow (Csikszentmihalyi, 1990). Flows’ theory underlies the highly accepted game characteristics framework of Garris et al. (2002) and the subsequent development of Charsky (2010). In the light of Charsky’s work, a second level of classification (right column) is presented that helps make the classification more complete. We describe the characteristics of the classification in relation to their potential for developing and sustaining 21st century skills subsequently:

Table 3.

Charsky (2010) Game Characteristics Classification.

Competition and goals	Competition
	Collaboration
	Complex collaboration
Choice	Expressive
	Strategy
	Tactical
Rules
Fantasy	Fidelity
Fantasy	Context
Challenge

Competition and goals

Clear rules, goals, and feedback on progress toward goals are needed in SG, specifically rules and goals (Garris, Ahlers, & Driskell, 2002). Charsky divides this category into three different characteristics.

Competition

Needed to make learning enjoyable and motivate learners to complete the game. The achievement of goals reflects the effort invested. When completed, we assume that learning took place.

Collaboration

Helps to teach leadership principles: delegation, empowerment, crises handling, application of knowledge, and information analysis.

Complex collaboration

A greater degree of collaboration, as found in some multiplayer online role-playing games, can help develop a flexible knowledge structure among gamers.

Rules

This game characteristic is defined by Charsky (2010) as the constraints that limit actions in games and such constraints may help learners achieve lower order skills.

Choice

The number of options and decisions a gamer has before or during gameplay are as follows:

Expressive

This involves choosing an avatar and designing an identity for a game. It can be used to develop empathy, a sense of pride, identity, progress in the game, and contributes to immersive learning.

Strategy

This provides a suitable level of difficulty and a powerful learning experience. Debriefing could play a key role in assisting learners to understand their errors, so that they can correct or overcome them.

Tactical

The gamer chooses a path or help. This involves decision making and adaptation to “pushed” learning by providing key information, assistance, coaching, or guidance.

Fantasy

Fidelity

The use of graphics, audio, video, three-dimensional virtual worlds, and artificial intelligence to represent reality. This represents the game designer’s attempt to provide motivating and exciting gameplay. The learner needs to engage with exact replicas of devices or places when learning how to develop in a real environment (as in simulations).

Context

The setting, narrative, story, scenario, characters, back story, problem, and so on, for the gameplay (Crawford, 2003). A well-written story with resounding characters and plot can enhance the authenticity. The gamer is not simply moving along with the story but is crafting the narrative with decisions and performance that can create a more authentic experience (Gee, 2005; Rollings & Adams, 2003). Learning through authentic scenarios could involve learning complex management and administrative practices.

Challenges

The challenge is the instructional content—answering the math problem—and many edutainment games consist of drill and practice activities that are sugar coated with game characteristics. The gamer typically acquires skills that lead to more complex challenges and thus more opportunities to acquire more skills and knowledge (Gee, 2003). Guiding the gamer to organize and reflect on their understanding is as important in a serious game as facilitating knowledge acquisition and transfer (Moreno & Mayer, 2005). Understanding could be deepened by offering an optimal level of difficulty and uncertain goal attainment (Garris et al., 2002).

It is important to remark that although we have listed each characteristic separately, they are inherently interdependent, and all have a similar overall purpose in motivating the learner.

Matching Game Characteristics and 21st Century Skills

To identify how to better tailor SG to fulfill the objective of learning 21st century skills, we have paired each of the game characteristics with the 21st century skills identified previously, based on the qualitative analysis of the literature reviewed in Table 2. Table 4 shows the degree of relevance of each of the game characteristics (competition, strategy, etc.) for each of the 21st century skills. Four levels of correspondence have been identified: (1, High, XXX) an unquestionable relationship between the presence of the game characteristic and the potential for developing the skill; (2, Medium, XX) where the presence of the game characteristic could potentially develop the skill; (3, Low, X) where a relationship between the game characteristic and the skill is possible in certain conditions; and (4, Not defined) where a priori, a clear relationship between the game characteristic and the development of the skill cannot be established.

Table 4.

Pairing Serious Game Characteristics and 21st Century Skills.

Game characteristics skills	(a) Competition and goals			(b) Choice			(c) Rules	(d) Fantasy		(e) Challenge
Game characteristics skills	Competition (a.1)	Collaboration (a.2)	Complex collaboration (a.3)	Expressive (b.1)	Strategy (b.2)	Tactical (b.3)		Fidelity (d.1)	Context (d.2)
Collaboration/teamwork	X	XXX	XX	X	X	X	X			X
Communication	X	XXX	XX	X			X			X
ICT literacy	X	XX	XX	XX			X			X
Social and/or cultural skills; citizenship		XX	XX	XX			XX	XX	XX
Creativity		X	X	X				XX	XX
Critical thinking	X	X	XX		XXX	XXX	X	X	X
Problem solving	X	X	XX		XX	XX	X	X	X
Develop quality/productivity	X	X	X		X	X		X	X
Learning to learn		X	X		X	X	X			X
Self-direction	X	X	XX	X	XXX	XXX
Planning	X	X	XX		XXX	XXX	X			X
Flexibility and adaptability	X	X	XX		XXX	XXX	X		X
Risk taking	XX	X	X		X	X	XX			X
Manage and solve conflicts	XX	XX	XXX		XX	XX	X			X
Sense of initiative and entrepreneurship	X	X	XX	X	XXX	XXX	X	X	X	X	XX

It can be seen that there is a direct connection between the competition and goal characteristics for the development of collaborative and social skills. For the three different choice characteristics, it is important to note that the “expressive” characteristic is used for working on cultural and expression skills, as well as for ICT development skills. However, strategy and tactical choices in SG could sustain the same skills: critical thinking, problem solving, learning to learn, self-direction, planning, flexibility and adaptability, risk taking, and a sense of initiative and entrepreneurship. Game rules could help develop citizenship, adaptability, and productivity skills. Fantasy characteristics are related to creativity—but few other skills; however, fidelity and context could help learning to learn, flexibility, adaptability, and social and cultural skills. The great degree of flexibility and range of plausible paths to success could help learners develop a flexible knowledge base that can be applied to a variety of real-world situations (Koster, 2013; Spiro, Feltovich, Jacobson, & Coulson, 1991). Such a knowledge base can be related to creativity, critical thinking, self-direction, planning, flexibility, and adaptability, as well as sense of initiative and entrepreneurship. Finally, the challenge characteristic in games is core for many authors, but this characteristic does not develop many skills in itself, although it might be essential for training concepts of entrepreneurship.

In summary, we can say that the key characteristics of SG enable the developing and sustaining of 21st century skills. Figure 2 shows the degree to which each game characteristic contributes to the development of 21st century skills.

Figure 2.

Game characteristic contribution to 21st century skills.

Discussion

Complex collaboration is the game characteristic that contributes the most to the development of 21st century skills. Complex collaboration and collaboration results from Table 4 show the importance of these key characteristics when designing and implementing SG to develop 21st century skills. Choice, strategy, and tactical characteristics are three important features to develop these skills. Both of these game characteristics require learners to play in a computer-based environment with a certain complexity, where the learner is allowed to develop strategies and tactics within the game mechanics. We would like to highlight that, despite the underlying presence of the “ICT literacy” skill in most of the SG analyzed, the search methodology developed to identify the SG having been specifically designed or used for developing these skills has not been identified. We assume the use of digital SG implicitly includes this key skill of the 21st century skills framework as a part of the digital artifice the students are proposed to use. Additional studies in these aspects through qualitative methodologies should be developed in order to understand how SG designers and instructors aim to develop (or not) their learners’ ICT skills.

According to the results found in Table 2, providing authentic and enriched learning scenarios is the second most important game characteristic that contributes to the acquisition of 21st century skills. This characteristic can also be related to game complexity. Competition with other learners, or with the SG, is another relevant characteristic in the development of these skills. We can analyze these results in the light of the current economic situation, where competition is practiced worldwide in most human activities.

Game rules with endogenous and exogenous reinforcement also contribute to a lesser degree to the development of 21st century skills. All of these characteristics propose behaviorist approaches to games, where learner actions are guided through rules and internal or external reinforcements—instead of acting in a complex environment of collaboration, strategy, tactics, and authenticity. Graphic fidelity and fantasy are the two game characteristics that least contribute to skills development.

Fidelity has been argued as a game characteristic that could boost knowledge and skills transfer, especially in the context of cultural heritage awareness (Petridis, Dunwell, de Freitas, & Panzoli, 2010). However, a high level of fantasy, as opposed to fidelity, is also argued to encourage the development of 21st century skills (Thomas & Brown, 2007). Our results show that these two game characteristics provide the smallest contributions to 21st century skills support in SG.

Conclusion

Our investigation, where we have mainly examined SG from the viewpoint of characteristics, indicates that SG could be productively adopted to support the development of those 21st century skills required to live and perform successfully in modern society. The analysis of the literature review on the use of SG for 21st century skills development has shown that most SG have not been primarily created with the aim of developing 21st century skills—despite the fact that they are used and analyzed with this purpose. To the knowledge of the authors, only ATCS (2012) has been explicitly conceived to develop 21st century skills (focusing on collaborative problem-solving and ICT literacy). Most SG have been conceived with domain-specific learning objectives, although the development of one or more other 21st century skills has been considered as a complementary objective. SG are still mostly designed from a curriculum perspective and have not been adapted to the competence-based approach of current educational policies. These results suggest the importance of convincing SG designers that the development of 21st century skills should be one of the key educational purposes and also suggest that the analysis of skills development transfer from SG should be further pursued.

Despite wide agreement on the usefulness of SG in supporting the development of 21st century skill frameworks (Kickmeier-Rust & Dietrich, 2012; Pivec, 2007; Redecker et al., 2011; Westera et al., 2008), the transfer of these skills from SG to the professional and personal worlds is a subject of further research. Some researchers consider that the virtual worlds of SG could “facilitate by modes of transfer that are no longer about fidelity between worlds, but are about the power of imagination to explore the differences and similarities between them” (Thomas & Brown, 2007, p. 169). However, other authors consider that there is a lack of scientific evidence about skills transfer from SG to professional and personal worlds. According to Squire (2002, p. 18), “research on transfer gives very little reason to believe that players are developing skills that are useful in anything but very similar contexts.” Further empirical studies should be conducted to analyze the level of transferability of each of the core 21st century skills in different types of game (Sawyer & Smith, 2008) and according to game characteristics (Charsky, 2010; Garris et al., 2002).

In terms of game characteristics, the review of SG has highlighted that some skills are better developed with certain game characteristics that involve collaboration, competition, strategy, and tactical support—all of these being characteristics of games that involve a gameplay open to complexity and action scenarios, cooperation, and decision making. These characteristics are commonly implemented in strategy games and virtual words, especially those open to learner communication and cooperation. These complex characteristics are hallmarks of entertainment games and are less frequently found in educational games (as these games are usually developed with smaller budgets). This observation points to the need to transfer the complexity of entertainment-orientated games in terms of strategy, communication, and cooperation toward skill-oriented games.

The three main intertwined questions in the introduction remain open and hopefully invite further and more in-depth research. The first question regards 21st century skills assessment in the context of SG; how should SG include assessment of skills and which skills require human assessment in the context of collaboration, strategy, and tactics in open scenarios; and how is this assessment supported within the computer-based environment of SG (Romero, Usart, Popescu, & Boyle, 2012). The second question regards how SG should be deployed to maximize effectiveness when used in the development of 21st century skills; which are the best and most effective learning/teaching techniques and educational approaches; what should the role of the teacher be; how can collaboration and competition be orchestrated within SG; and finally, how is it possible to build on informally acquired skills (Redecker et al., 2011). The final question relates to the impact of noneducational games on the development of 21st century skills. Although this study focuses on educational SG in the context of formal learning, we should also consider the impact that entertainment games may have in the development of skills such as ICT literacy, strategy, competition, and the other skills required to win. This future analysis could include an analysis of entertainment games as educational tools that help learners prepare for modern life by engaging in games that require and contribute to the development of 21st century skills.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received the following financial support for the research, authorship, and/or publication of this article: Some of the research reported in the study has been partially funded by the European Commission (Research. Executive Agency, grant no 270404).

References

Alessi

S. M.

Trollip

S. R.

(2001). Multimedia for learning: Methods and development (3rd ed.). Boston, MA: Allyn and Bacon.

Anderson

E. F.

McLoughlin

Liarokapis

Peters

Petridis

de Freitas

(2010). Developing serious games for cultural heritage: A state-of-the-art review. Virtual Reality, 14, 255–275.

Apperley

T. H.

(2006). Virtual Unaustralia: Videogames and Australia’s colonial history. Proceedings of the Cultural Studies Association of Australasia’s Annual Conference, Canberra, Australia.

Argyris

Schon

(1974). Theory in practice: Increasing professional effectiveness. San Francisco, CA: Jossey Bass.

Bawden

(2008). Origins and concepts of digital literacy. In Lankshear

Knobel

(Eds.), Digital literacies: Concepts, policies & practices (pp. 17–32). New York, NY: Peter Lang.

Bell

B. S.

Kozlowski

S. W. J.

(2008). Active learning: Effects of core training design elements on self-regulatory processes, learning, and adaptability. Journal of Applied Psychology, 93, 296–316.

Berger

Liapis

Yannakakis

(2012). Prototyping an adaptive educational game for conflict resolution. In ITS 2012 Workshop: Emotion in Games for learning. Crete, Greece.

Binkley

Erstad

Herman

Raizen

Ripley

Rumble

(2011). Defining 21st century skills, draft white paper. Melbourne, Australia: University of Melbourne.

Bonwell

C. C.

Eison

J. A.

(1991). Active learning: Creating excitement in the classroom. ASHE-ERIC Higher Education Report 1. Washington, DC: George Washington University.

10.

British Broadcasting Corporation. (2002). Video games stimulate learning. BBC NEWS. Retrieved June 10, 2014, from http://news.bbc.co.uk/2/hi/uk_news/education/1879019.stm

11.

Brown

J. S.

Thomas

(2008, February 14). The gamer disposition. Harvard Business Review. Retrieved from http://blogs.hbr.org/cs/2008/02/the_gamer_disposition.html

12.

Buck Institute for Education (BIE). (2012). Review of the 21st century skills frameworks. Retrieved July 26, 2013, from http://www.bie.org/research/21st_century_skills

13.

Carmeli

Tishler

(2006). An empirical analysis of the relative importance of management team. International Journal of Manpower, 27, 9–36.

14.

Charsky

(2010). From edutainment to serious games: A change in the use of game characteristics. Games and Culture, 5, 177–198.

15.

Cheong

Y.-G.

Khaled

Grappiolo

Campos

Martinho

Ingram

G. P.

Yannakakis

(2011). A computational approach towards conflict resolution for serious games. In Proceedings of the 6th International Conference on Foundations of Digital Games (pp. 15–22). Bordeaux, France: ACM.

16.

Che Pee

(2011). Computer games use in an educational system. Unpublished PhD thesis, University of Nottingham, Nottingham, England.

17.

Cognition and Technology Group at Vanderbilt University. (1997). The Jasper Project: Lessons in curriculum, instruction, assessment, and professional development. Mahwah, NJ: Erlbaum.

18.

Cohen

P. R.

Feigenbaum

E. A.

(Eds.). (1982). The handbook of artificial intelligence (Vol. 3). Los Altos, CA: William Kaufmann.

19.

Cohn

Atlas

Ladner

(1994). Improving generalization with active learning. Machine Learning, 15, 201–221.

20.

Crawford

(2003). Chris Crawford on game design. Boston, MA: New Riders.

21.

Crookall

Oxford

R. L.

Saunders

(1987). Towards a reconceptualization of simulation: From representation to reality. Simulation/Games for Learning, 17, 147–171.

22.

Csikszentmihalyi

(1990), Flow: The psychology of optimal experience. New York, NY: Harper Collins.

23.

Dede

(2010). Comparing frameworks for 21st century skills. In Bellanca

Brandt

. (Eds.), 21st century skills: Rethinking how students learn (Vol. 20, pp. 51–76). Bloomington, IN: Solution Tree Press.

24.

de Felix

Johnston

R. T.

(1993). Learning from video games. Computers in the Schools, 9, 199–233.

25.

Edens

K. M.

(2000). Preparing problem solvers for the 21st century through problem-based learning. College Teaching, 48, 55–60.

26.

European Commission. (2002). European report on quality indicators of lifelong learning: Fifteen quality indicators. Brussels, Belgium: EC Directorate-General for Education and Training.

27.

Ferrari

(2012). Digital competence in practice: An analysis of frameworks (Report EUR 25351 EN). Luxembourg: Publications Office of the European Union. Retrieved June 21, 2014 from http://ftp.jrc.es/EURdoc/JRC68116.pdf

28.

Ferretti

Brami

Quero

Fiorese

(2010). Analysis of current practices and user needs (WP2 Need analysis). (No. 2.1; pp. 1–138). Padova, Italy: CNR ISTC.

29.

Fisch

McLeod

(2009). Did you know? Retrieved March 19, 2013, from http://www.youtube.com/watch?v=PHmwZ96_Gos

30.

Francis

(2006) Revolution: Learning about history through situated role play in a virtual environment. In Proceedings of the American educational research association conference (Vol. 1). San Francisco, CA.

31.

Frederick

H. J.

Corvetto

M. A.

Hobbs

G. W.

Taekman

(2011). The “simulation roulette” game. Simulation in Healthcare, 6, 244–249.

32.

Frydenberg

M. E.

Andone

(2011). Learning for 21st century skills. IEEE's International Conference on Information Society, London, England.

33.

Gordon

Halász

Krawczyk

Leney

Michel

Pepper

Wiśniewski

(2009). Key competences in Europe: Opening doors for lifelong learners across the school curriculum and teacher education. CASE Network Reports, 87, 328.

34.

Garris

Ahlers

Driskell

J. E.

(2002). Games, motivation, and learning: A research and practice model. Simulation & Gaming, 33, 441–467.

35.

Gee

J. P.

(2003). What video games have to teach us about learning and literacy. New York, NY: Palgrave/Macmillan.

36.

Gee

J. P.

(2005). What would a state of the art instructional video game look like. Innovate: Journal of Online Education, 1, 159.

37.

Graesser

A. C.

Chipman

Leeming

Biedenbach

(2009). Deep learning and emotion in serious games. In Ritterfeld

Cody

Vorderer

(Eds.), Serious games: Mechanisms and effects (pp. 81–100). New York, NY and London: Routledge, Taylor & Francis.

38.

Grant

associates (1979). On competence: A critical analysis of competence-based reforms in higher education. San Francisco, CA: Jossey-Bass.

39.

Griffin

McGaw

Care

(2012). Assessment and teaching of 21st century skills. Dordrecht, The Netherlands: Springer.

40.

Häkkinen

Bluemink

Juntunen

Laakkonen

(2012). Multiplayer 3D Game in Supporting Team-Building Activities in a Work Organization. In ICALT (pp. 430–432). Rome, Italy.

41.

Hannafin

M. J.

Peck

K. L.

(1988). The design, development, and evaluation of instructional software. New York, NY: Macmillan Publishing.

42.

Hill

R. W.

Belanich

Lane

H. C.

Core

M. G.

Dixon

Forbell

Hart

(2006). Pedagogically structured game-based training: Development of the ELECT BiLAT simulation. Proceedings of the 25th Army Science Conference(ASC 2006), Orlando, FL .

43.

Jacobson

Holden

(2005, June). The virtual egyptian temple. World Conference on Educational Multimedia, Hypermedia and Telecommunications. pp. 4531–4536.

44.

Jason

(2003). Building a business simulation for kids: The making of disney’s hot shot business. ACM Computer Entertainment, 1, 18.

45.

Jenkins

Camper

Chisholm

Grigsby

Klopfer

Osterweil

Tan

(2009). From serious games to serious gaming. In Ritterfeld

Cody

Vorderer

(Eds.), Serious games: Mechanisms and effects (p. 459). New York, NY: Routledge.

46.

Jercic

Astor

P. J.

Adam

M. T. P.

Hilborn

Schaaff

Lindley

Eriksson

(2012, June 10–13). A serious game using physiological interfaces for emotion regulation training in the context of financial decision-making. Proceedings of the 20th European Conference on Information Systems (ECIS 2012), Barcelona, Spain .

47.

Johnson

W. L.

(2007 ). Serious use of a serious game for language learning. The 13th International Conference on Artificial Intelligence in Education. Amsterdam, The Netherlands: IOS Press. pp. 67–74.

48.

Johnson

W. L.

(2008). Assessing aptitude for learning with a serious game for foreign language and culture. In Woolf

. (Eds.), Intelligent tutoring systems (pp. 520–529). Berlin, Germany: Springer-Verlag.

49.

Jones

L. K.

(1996). Job skills for the 21st century: A guide for students. Phoenix, AZ: Oryx Press.

50.

Katz

(2000). Up, up, down, down. Slashdot.org. Retrieved June 20, 2014, from http://slashdot.org/features/00/11/27/1648231.shtml

51.

Kickmeier-Rust

M. D.

Dietrich

(2012). A domain model for smart 21st century skills training in game-based virtual worlds. ICALT 2012 Proceedings, 680–681. Rome, Italy .

52.

King

Newman

(2009). Evaluating business simulation software: Approach, tools and pedagogy. On the Horizon Journal, 17, 368–377.

53.

Kirriemuir

(2002). Video games and gaming consoles. The relevance of video games and gaming consoles to the higher and further education learning experience. Retrieved June 19, 2013, from http://www.jisc.ac.uk/uploaded_documents/tsw_02-01.rtf

54.

Koster

(2013). A theory of fun for game design (2nd ed.). Sebastopol, CA: O’Reilly Media.

55.

Lane

H. C.

(2007). Metacognition and the development of intercultural competence. In Woolf

B. P.

Aïmeur

Nkambo

Lajoie

(Eds.), Workshop on Metacognition and SRL. Artificial Intelligence in Education (AIED) (pp. 23–32). Marina del Rey, CA.

56.

Lane

H. C.

Hays

M. J.

Core

Gomboc

Forbell

Rosenberg

(2008). Coaching intercultural communication in a serious game. Proceedings of the 16th International Conference on Computers in Education, Taipei, Taiwan, 35–46.

57.

Linehan

Lawson

Doughty

(2009). Tabletop prototyping of serious games for ‘soft’ skills training. IEEE First International Conference Games and Virtual Worlds for Serious Applications. Los Alamitos, CA: IEEE. pp. 182–185.

58.

Lutzker

J. R.

White-Blackburn

(1979). The good productivity game: Increasing work performance in a rehabilitation setting. Journal of Applied Behavior Analysis, 12, 488.

59.

Mac Namee

Rooney

Lindstrom

Ritchie

Boylan

Burke

(2006, November 22–24). Serious gordon using serious games to teach food safety in the kitchen. 9th International Conference on Computer Games: AI, Animation, Mobile, Educational & Serious Games. Dublin: Dublin Institute of Technology.

60.

MacLaurin

(2009). Kodu: End-user programming and design for games. Proceedings of the 4th International Conference on Foundations of Digital Games, New York, NY: ACM, 18–19.

61.

Malone

T. W.

(1981). What makes computer games fun? Byte, 6, 258–277.

62.

McGonigal

(2011). Reality is broken why games make us better and how they can change the world. New York, NY: The Penguin Press.

63.

Metiri Group & North Central Regional Educational Laboratory. (2003). EnGauge 21st century skills: Literacy in the digital age. Chicago, IL: NCREL.

64.

Michael

Chen

(2005). Serious games: Games that educate, train, and inform. Boston, MA: Thompson Publishing.

65.

Moreno

Mayer

R. E.

(2005). Role of guidance, reflection, and interactivity in an agent based multimedia game. Journal of Educational Psychology, 97, 117–128.

66.

Mortara

Catalano

C. E.

Bellotti

Fiucci

Houry-Panchetti

Petridis

(2013). Learning cultural heritage by serious games. Journal of Cultural Heritage, 15, 318–325.

67.

Nikkila

Linn

Sundaram

Kelliher

(2011, May 7–12). Playing in taskville: Designing a social game for the workplace. Proceedings of the Gamification: Using Game Design Elements in Non-Gaming Contexts Workshop, in conjunction with SIGCHI 2011. Vancouver, BC .

68.

Organization for Economic Cooperation and Development. (2005). The definition and selection of key competencies: Executive summary. Paris, France: OECD. Retrieved June 20, 2014, from http://www.oecd.org/pisa/35070367.pdf

69.

Ott

Pozzi

(2011). Digital games as creativity enablers for children. Behaviour & Information Technology, 31, 1011–1019.

70.

Paul

(1990). Critical thinking: How to prepare students for a rapidly changing world. Santa Rosa, CA: Foundation for critical thinking.

71.

Partnership for 21st Century Skills. (2007). Framework for 21st century learning. Retrieved June 20, 2014, from http://www.p21.org/about-us/p21-framework

72.

Petridis

Dunwell

de Freitas

Panzoli

(2010, March 25–26). An engine selection framework for high fidelity serious games. Proceedings of the 2nd International Conference on Games and Virtual Worlds for-Serious-Applications, VSgames’10, Braga, Portugal .

73.

Pivec

(2007). Editorial: Play and learn: Potentials of game-based learning. British Journal of Educational Technology, 38, 387–393.

74.

Pivec

(2012, 7–8 June). Game-based research in education and action training. International Conference on the Future of Education, Florence.

75.

Prensky

(2000). Digital game-based learning. New York, NY: McGraw Hill.

76.

Prensky

(2001). Digital natives, digital immigrants. On the Horizon, 9, 1–6.

77.

Prensky

(2003). Digital game-based learning. ACM Computers in Entertainment, 1, 1–4.

78.

Prensky

(2004). The emerging online life of the digital native. New York, NY: Games2train. Retrieved June 07, 2014 from http://www.bu.edu/ssw/files/pdf/Prensky-The_Emerging_Online_Life_of_the_Digital_Native-033.pdf

79.

Prensky

(2006). “Don’t bother me mom—I’m learning:” How computer and video games are preparing your kids for 21st century success—And how you can help! Saint Paul, MN: Paragon House.

80.

Raser

J. R.

(1969). Simulation and society: An exploration of scientific gaming. Boston, MA: Allyn and Bacon.

81.

Raybourn

E. M.

(2009). Intercultural competence game that fosters metacognitive agility and reflection. Third International Conference Online Communities and Social Computing. Berlin/Heidelberg, Germany: Springer-Verlag. pp. 603–612.

82.

Raybourn

E. M.

Deagle

Mendini

Heneghan

(2005). Adaptive thinking & leadership simulation game training for special forces officers. The Interservice/Industry Training, Simulation & Education Conference (I/ITSEC) ( Vol. 2005). Orlando, FL: NTSA.

83.

Redecker

Leis

Leendertse

Punie

Gijsbers

Kirschner

(2011). The future of learning: Preparing for change: JRC-IPTS. Retrieved July 25, 2013, from http://ipts.jrc.ec.europa.eu/publications/pub.cfm?id=4719

84.

Rollings

Adams

(2003). Andrew rollings and ernest adams on game design. Berkeley, CA: New Riders Publishing.

85.

Romero

Hyvönen

Barbera

(2012). Creativity in collaborative learning across the life span, Creative Education, 3, 422–429.

86.

Romero

Usart

Almirall

(2011). Serious games in a finance course promoting the knowledge group awareness. EDULEARN11 Proceedings, Barcelona, Spain: IATED, 3490–3492.

87.

Romero

Usart

Popescu

Boyle

(2012). Interdisciplinary and international adaption and personalization of the metavals serious games. In Ma

Oliveira

Hauge

Duin

Thoben

K.-D.

(Eds.), Serious Games Development and Applications (Vol. 7528, pp. 59–73). Berlin/Heidelberg, Germany: Springer.

88.

Royse

Lee

Undrahbuyan

Hopson

Consalvo

(2007). Women and games: Technologies of the gendered self. New Media and Society, 9, 555–576.

89.

Sarasvathy

S. D.

(2008). Effectuation: Elements of entrepreneurial expertise. Cheltenham, England: Edward Elgar.

90.

Sawyer

Smith

(2008, February). Serious games taxonomy. Presented at the Serious Games Summit during the Game Developers Conference 2008, California.

91.

Simon

(2006). Beyond cyberspatial flaneurie: On the analytic potential of living with digital games. Games and Culture, 1, 62–67.

92.

Smith

J. H.

(2004, September 19–22). Playing dirty—Understanding conflicts in multiplayer games. 5th Annual conference of The Association of Internet Researchers. Brighton, United Kingdom: The University of Sussex.

93.

Song

C. R.

(2008). Educational games with blogs: Collaborating to motivate second language undergraduate critical thinking. Online Information Review, 32, 557–573.

94.

Spady

W. G.

(1977). Competency-based education: A bandwagon in search of a definition. Educational Researcher, 6, 9–14.

95.

Spiro

R. J.

Feltovich

P. J.

Jacobsen

M. J.

Coulson

R. L.

(1991). Cognitive flexibility, constructivism, and hypertext: Random access instruction for advanced knowledge acquisition in ill-structured domains. Educause Review, 5, 24–33.

96.

Squire

(2002). Cultural framing of computer/video games. Game studies, 2, 90.

97.

Sternberg

R. J.

Lubart

T. I.

(1995). Defying the crowd: Cultivating creativity in a culture of conformity. New York, NY: Free Press.

98.

Stevenson

H. H.

Jarillo

J. C.

(1990). A paradigm of entrepreneurship: Entrepreneurial management. Strategic Management Journal, 11, 17–27.

99.

Stevens

M. J.

Campion

M. A.

(1999). Staffing work teams: Development and validation of a selection test for teamwork settings. Journal of Management, 25, 207–229.

100.

Stuart

Dahm

(1999). 21st century skills for 21st century jobs: A report of the U.S. Department of Commerce. Washington, DC: U.S. Department of Education, U.S. Department of Labor, National Institute for Literacy and Small Business Administration.

101.

Subrahmanyam

Greenfield

Kraut

Gross

(2001). The impact of computer use on children’s and adolescents’ development. Journal of Applied Developmental Psychology, 22, 7–30.

102.

Sue

D. W.

Arredondo

McDavis

R. J.

(1992). Multicultural counseling competencies: A call to the profession. Journal of Counseling & Development, 70, 477–486.

103.

Thomas

Brown

J. S.

(2007). The play of imagination: Extending the literary mind. Games and Culture, 2, 149–172.

104.

Thomas

Macredie

(1994). Games and the design of human-computer interfaces. Educational and Training Technology International, 31, 134–142.

105.

Thornton

G. C.

Cleveland

J. N.

(1990). Developing managerial talent through simulation. American Psychologist, 45, 190–199.

106.

Trilling

Fadel

(2009). 21st century skills: Learning for life in our times. San Francisco, CA: Jossey-Bass.

107.

Usart

Romero

Almirall

(2011). Impact of the feeling of knowledge explicitation in the learners’ participation and performance in a collaborative game based learning activity. In Ma

Fradinho Oliveira

Madeiras Pereira

(Eds.), Serious Games Development and Applications (Vol. 6944, pp. 23–35). Berlin/Heidelberg, Germany: Springer.

108.

Vaassen

Daelemans

(2011). Automatic emotion classification for interpersonal communication. Proceedings of the 2nd Workshop on Computational Approaches to Subjectivity and Sentiment Analysis, Stroudsburg, PA: Association for Computational Linguistics, 104–110.

109.

Voogt

Roblin

N. P.

(2012). A comparative analysis of international frameworks for 21st century competences: Implications for national curriculum policies. Journal of Curriculum Studies, 44, 299–321.

110.

Vohs

K. D.

Baumeister

R. F.

(2004). Understanding self-regulation: An introduction. Handbook of Self-Regulation: Research, Theory, and Applications, 1–9.

111.

Wagner

M. D.

(2008). Massively multiplayer online role-playing games as constructivist learning environments in K-12 education: A delphi study (Unpublished doctoral dissertation proposal). Minnesota: Walden University.

112.

Westera

Nadolski

R. J.

Hummel

H. G. K.

Wopereis

I. G. J. H.

(2008). Serious games for higher education: A framework for reducing design complexity. Journal of Computer Assisted Learning, 24, 420–432.

113.

Williams

(2011). Impact of business simulation games in enterprise education. 2010 University of Huddersfield Annual Learning and Teaching Conference. pp. 11–20.

114.

Wisniewski

M. A.

(2010). Leadership and the millennials: Transforming today’s technological teens into tomorrow’s leaders. Journal of Leadership Education, 9, 53–68.

115.

Wittgenstein

(1953). Philosophical investigations. New York, NY: Macmillan.

116.

Wittgenstein

(1958). The blue and brown books. New York, NY: Harper & Row.