Pacing in Serious Games: Exploring the Effects of Presentation Speed on Cognitive Load,Engagement and Learning Gains

Abstract

Background. To determine the optimal speed of computer-paced serious games, the interplay of cognitive load variables and game engagement variables can serve as combined criteria.

Intervention. In order to test the effects of higher or lower game speeds on different types of cognitive load and engagement, the FRESH FOOD RUNNER game was developed, which teaches the harvest seasons of different fruits and vegetables.

Method. In this study, N=58 6^th grade primary school students are randomly assigned to play the serious game FRESH FOOD RUNNER at different speeds. Pre-and post-tests are used to assess learning gains, in combination with rating scales for different types of cognitive load and game engagement.

Results. Results show that highest learning gains and lowest levels of extraneous cognitive load are prevalent at medium speed settings, which are neither too fast nor too slow. A similar pattern is apparent for student ratings of game engagement.

Discussion and Conclusion. In consequence, learning gains, cognitive load and engagement variables can be regarded as suitable criteria to determine the optimal speed of serious games.

Keywords

cognitive load engagement game-based learning pacing serious games

Background

Research on digital games that are specifically designed for educational purposes (subsequently referred to as serious games) proposes that there is a sweet spot of complexity and challenge where a serious game provides an optimal level of both motivational involvement and cognitive effort for learning – ultimately leading to the state of flow, where players are completely immersed and focused (Boyle et al., 2012; Granic et al., 2014; Jabbar & Felicia, 2015). In order to design serious games that are capable of inducing these levels of engagement, numerous aspects need to be aligned, such as educational content, story, goals, tokens, variables, rules, controls, algorithms, visuals, sounds, possibilities of social interactions and indicators of success (Landers, 2014). In order to reduce complexity, research needs to address the impact of different configurations on engagement and learning while varying one aspect at a time and studying the impact on relevant outcome variables (I. Mayer et al., 2014). One simple yet fundamental aspect in many games is the pacing. While presentation speed has been a topic of empirical investigation with regard to other types of instructional media such as instructional videos and animations – mostly leading to the conclusion to give learners control over the presentation speed (Hasler et al., 2007; Karich et al., 2014; Tabbers & de Koeijer, 2010; Wouters et al., 2008), it has seldom been the focus of research on serious games. The reason for this might be due to the tendency that serious games have rarely employed fast-paced gaming formats such as jump-and-run-games, racing-simulations or ego-shooters, where slowing down is not an option (Boyle et al., 2016; Connolly et al., 2012). Instead, most serious games seem to be built around the design patterns of strategy, simulation and story-based games, or puzzles and quizzes, where players have more time to think strategically. However, when designing serious games which revolve around an idea of speed, it becomes apparent that it is very difficult to give general recommendations with regard to the pacing. Adequate speed is dependent on the complexity of variables, rules, decisions and controls involved in the game. Players of different skills are likely to consider different game speeds as appropriate. Furthermore, speed in serious games is not necessarily a constant but a variable that can be adjusted during gameplay, e.g. by adaptively increasing or decreasing speed based on player skill, by slowly increasing speed as players reach higher levels, or simply by letting players control the speed in order to play the game as quickly as possible. As there are many different types of gameplay-patterns, it is almost impossible to give general advice on how fast initial speed settings should be. As speed is an inherent characteristic of gameplay and related to a pre-defined challenge, some games allow for pausing the game for practical reasons, but do not include the functionality for players to adjust the pacing to their preferences (Bergeron, 2006). Game designers typically determine the base-speed settings of their games by pragmatic playtesting, in order to make the game at the same time playable, challenging and fun. In addition, game design can include more complex relations between play time and event time, where game speed can be understood as the relation of these two aspects, which can be slowed down, rewinded, warped or cut (Juul, 2004). For games with relatively fixed game speeds however, psychological models and measures might prove to be helpful to make more informed decisions when designing serious games. To determine the optimal presentation speed of a serious game, the interplay of a variety of game engagement and cognitive load measures can serve as combined criteria. Thus, balancing engagement and cognitive load is a fundamental challenge when designing the user experience in serious games.

Cognitive Load Theories Informing the Pacing of Serious Games

In psychology and especially educational psychology, cognitive load has been a core theoretical approach to inform the instructional design of multimedia learning material such as animations and instructional videos (Paas et al., 2003; Paas & Sweller, 2014; Sweller et al., 1998). The basic idea behind the cognitive load concept is that there are two distinct cognitive systems working together, the long-term memory and the working memory. While the capacity of the long-term memory is considered to be unlimited, the capacity of the human working memory is limited to processing only a few chunks of information at a time (Baddeley, 1986; Chandler & Sweller, 1991; Paas & Ayres, 2014). In addition, it is assumed – especially in the cognitive theory of multimedia learning – that there are separate subsystems of the working memory for auditory and visual information, which are able to process information simultaneously. These can be beneficial for learning when this leads to rich and at the same time coherent mental representations (R. E. Mayer, 2001; Paivio, 1986). With regard to the capacity of the working memory, learners have been found to differ depending on the size and complexity of the chunks of information they are able to actively process. Experts are able to employ complex schemas to summarize information and thus process more complex information, while novices need more assistance to build up meaningful schemas to do so (Sweller et al., 1998, 2011). Thus, in order to promote efficient learning, the limited capacity should be used for cognitive activities that are beneficial and not for activities that are detrimental for learning. Cognitive load theory assumes that three types of cognitive load need to be accounted for. Intrinsic cognitive load is linked to the difficulty of the content to be learned or the learning task to be performed. Germane cognitive load is attached to the efforts to understand and make sense of the learning material or the learning task. Extraneous cognitive load relates to the amount of cognitive activities that are detrimental for the learning efforts, especially distractions. These three types of cognitive load are assumed to be interdependent. As the overall capacity of cognitive activities is considered a limited resource, it seems desirable to have a good balance of intrinsic and germane cognitive load while keeping extraneous cognitive load to a minimum. Despite the large body of research, the concept of cognitive load, as well as its measurement in past studies, has been criticized for employing unfalsifiable assumptions, and unsuitable self-report measures, while ignoring findings from other conceptual strands of research (De Jong, 2010). The problem of employing only single-item self-report scales has recently been addressed by the construction of more differentiated retrospective self-report instruments (Leppink et al., 2013). Furthermore, different non-self-report approaches to measure cognitive load (e.g. dual-task-designs or physiological measures) have been developed (DeLeeuw & Mayer, 2008). Also, new theoretical models emerge. Recently, it has been proposed that there might be other, perhaps even simpler, models of cognitive load, such as reducing the types of load to intrinsic and extraneous (Kalyuga, 2011), or distinguishing between essential processing, incidental processing and representational holding (R. E. Mayer & Moreno, 2003). An especially promising alternative model of cognitive load has been the time-based resource-sharing model (Barrouillet et al., 2004, 2007; Puma et al., 2018). This model also assumes that cognitive capacity for information processing is determined by limited attention-spans, especially when juggling different attention-processes at the same time (e.g. a processing task and a remembering task). In consequence, it takes active cognitive effort to refresh information in the working memory for more than the natural time of decay. In this model, the pacing of the switch between tasks and information is central for controlling cognitive load. Summing up the state of art with regard to cognitive load theories, there seems to be consensus that the cognitive capacity of working memory is limited, indeed, and that instruction should be aware of these limitations in order to provide suitable stimuli for learning.

Recommendations to Reduce Extraneous Cognitive Load Related to Serious Game Pacing

There are a number of recommendations for the design of multimedia instructional material to achieve this, that align with older principles of instructional design (De Jong, 2010, p. 126): aligning material with prior knowledge of learners (reducing intrinsic load), avoiding irrelevant or distracting information (reducing extraneous load) and stimulating processes for conceptual processing (increasing germane load). Furthermore, instructional designers should use all channels for delivering information coherently and without redundancy, segment learning material, and provide signals highlighting more important aspects of the information and to give learners pre-training and further options to individualize the presentation (R. E. Mayer & Moreno, 2003). In light of these findings, pacing has been described as a method for controlling extraneous cognitive load, where slower pacing is typically assumed to result in lower cognitive load (Merkt & Schwan, 2016 ; Scheiter & Gerjets, 2007 ). In some studies, findings provide a more incoherent picture. Fischer et al. (2008) reported learners to grasp essential ideas better in a faster than in a slower instructional animation. A study by Meyer et al. (2010) showed evidence that faster pacing makes students focus on rather global aspects and less details when compared to a slower pacing condition. De Koning et al. (2011) have found no difference in learning gains between faster and slower pacing, both with or without cues. Many studies agree, however, that learner-controlled pacing seems to be beneficial when compared to system-controlled pacing (Hasler et al., 2007; Tabbers & de Koeijer, 2010; Wouters et al., 2008). Few studies have focused on serious games however – although it has been proposed that the guidelines and concepts of cognitive load theory are equally relevant for the design and application of educational games (Huang & Johnson, 2009; Kalyuga & Plass, 2009). In serious games, which can be considered as rich and highly complex multimedia environments, superfluous elements, and system-controlled pacing are all parts of the playful environment. As these aspects have been difficult to balance, previous studies on cognitive load in serious games have focused on instructional guidance and not on altering the content, the presentation, nor the rules of the controls of educational games (Nelson & Erlandson, 2008; Schrader & Bastiaens, 2012). Nevertheless, looking at the effects of game pacing – which is one of the more easily controlled settings – can provide insights into the validity of the application of cognitive load theory in serious game design.

Engagement Variables as Criteria for the Pacing of Serious Games

In educational research, engagement has served as an umbrella term to describe the level of behavioral, cognitive and emotional involvement of students in learning activities (Bouvier et al., 2014; Boyle et al., 2012; O’Brien & Toms, 2008). In studies on games and learning, behavioral engagement encompasses aspects such as time spent on playing, persistence of overcoming challenges, and replay cycles. Emotional engagement means enjoyment, interest and enthusiasm. Cognitive engagement relates to aspects such as the level of focused attention, memorization, application of knowledge and strategic thinking (Filsecker & Kerres, 2014; O’Brien & Toms, 2008; Phillips et al., 2014). In addition, environmental and social aspects of engagement have been proposed (Bouvier et al., 2014). Some of the most widely used measures of engagement encompass even more aspects, including aesthetic engagement and usability (Fu et al., 2009; Wiebe et al., 2014). Nevertheless, all aspects of engagement seem to converge in the concept of “flow”, where all aspects of engagement lead to a state of full immersion and non-distraction (Czikszentmihalyi, 1990). Flow is likely to happen when an activity provides an optimal level of challenge, i.e. when it is neither too difficult (which may lead to anxiety) nor too simple (resulting in boredom). Theories of engagement and cognitive load have similarities, especially with regard to cognitive engagement and mental effort. However, theories of engagement tend to have a stronger focus on the affective and motivational state of learners. Meta-analyses of research on serious games have consistently shown that game-based learning is associated with higher levels of motivational engagement than learning with comparable non-gamified instructional material (Sitzmann, 2011; Vogel et al., 2006). However, in some of the more recent studies and even meta-analysis, this effect seems to be less pronounced or even disappearing (Imlig-Iten & Petko, 2018 ; Iten & Petko, 2016; Wouters et al., 2013). In a comprehensive review of studies on factors influencing player engagement, Jabbar and Felicia (2015) have grouped the findings according to four elements – motivational elements, interactive elements, fun elements, multimedia elements – all having a positive impact on both cognitive or emotional learner engagement. However, it is still unclear how these elements need to be combined in order to yield the desired effects. As engagement in serious games cannot be taken for granted, research needs to explore the mechanisms and variables influencing engagement in serious games such as graphical representations, rules, scoring, controls and pacing more closely. Especially pacing has not been studied with regard to engagement. Additionally, research on serious games has a long history of studies comparing learning with serious games to learning with traditional learning materials, such as textbooks or non-gamified multimedia material. Because games and more conventional learning material differ in many aspects, this kind of research has struggled to isolate factors that are responsible for higher student engagement. Instead, a different kind of study has been proposed, comparing variants of the same game (Girard et al., 2013; R. E. Mayer, 2011; I. Mayer et al., 2014). Furthermore, it seems reasonable to combine different research traditions such as cognitive load theories and engagement theories to advance research on serious game design.

Intervention: The “FRESH FOOD RUNNER ” Game

To test the effect of different speed settings on different types of cognitive load and cognitive engagement, an experimental study was conducted, using the FRESH FOOD RUNNER game that we developed. FRESH FOOD RUNNER is a 3D action game in the style of SUBWAY SURFERS or TEMPLE RUN (Figure 1). It teaches declarative knowledge with regard to the seasonality of fruit and vegetables. The game has been developed in cooperation with Museum Alimentarium in Vevey, Switzerland and Kobold Games Studio in Olten, Switzerland. The information about fruit- and vegetable-seasonality presented in this game was adapted from WWF guidelines for Switzerland. The game is available in the iOS and Android AppStores and as a downloadable version for both PC and MAC (https://academy.alimentarium.org/en/games). Employing a third person perspective, players need to run through a front-scrolling market street, collecting fruits and vegetables that are in season while avoiding those that are not (Figure 2). Collecting fruits and vegetables that are in-season adds to the health bar of the player, collecting those that are out-of-season subtracts from the health bar. The game passes through the months of the year and provides short instructional sequences after fixed periods (Figure 3). Successful players reaching more than 50% of the health bar can activate an additional boost for more speed, which facilitates getting additional points. With each level, more and more fruits and vegetables are introduced in short instructional sequences followed by a new and extended round of gameplay until a final number of 10 types of fruits and nuts and 18 types of vegetables is reached. As the game progresses, obstacles are introduced that players need to dodge. The game only ends when the health-bar reaches 0 points or when the player runs into an obstacle. Collecting fruits and vegetables also adds to an open-ended highscore, so that the game can be played for an indefinite length of time. Although players have the option to pause the game and to look through the food library in order to check the seasonality of all fruits and vegetables included in the game, this option was not encouraged for the purposes of this study.

Figure 1.

Game dashboard, character selection.

Figure 2.

Ingame screenshot, collecting seasonal fruits and vegetables.

Figure 3.

Instructional in-game-sequence, explaining the seasonality of different fruits and vegetables.

Methods

To test the effects of different game speeds on student cognitive load, engagement and learning, we conducted a randomized field experiment with a pretest/posttest design.

Hypotheses

Based on our theoretical considerations, we expect the following hypotheses to be true with regard to cognitive load related variables:

H1: With increased game pacing, the intrinsic cognitive load will remain constant.

H2: With increased game pacing, the extraneous load will rise.

H3: With increased game pacing, the germane will drop.

These hypotheses are based on the assumption that with higher game speeds, controlling the game and avoiding obstacles will become more difficult (i.e. extraneous cognitive load), which reduces the amount of cognitive resources available for learning activities (i.e. germane cognitive load), while the inherent difficulty of the learning material stays the same (i.e. intrinsic cognitive load).

With regard to game engagement aspects, we will test the following hypotheses:

H4: With increased game pacing, perceived focused attention will develop as a nonlinear curve, with highest ratings at medium settings.

H5: With increased game pacing, perceived usability will develop as a nonlinear curve, with highest ratings at medium settings.

H6: With increased game pacing, perceived aesthetics will develop as a nonlinear curve, with highest ratings at medium settings.

H7: With increased game pacing, perceived satisfaction will develop as a nonlinear curve, with highest ratings at medium settings.

These hypotheses are justified by the assumptions of flow theory, which states that highest levels of engagement are present at medium settings, which are neither too easy nor too difficult. In combination, this leads to the following additional hypothesis:

H8: Highest learning gains are expected at medium speed settings, where engagement ratings are expected to be high and extraneous load to be still relatively low.

Sample

In this study, N=58 6^th grade primary school students from 4 classes (32 female and 26 male with a median age of 11 years) are individually and randomly assigned to six experimental groups, playing the game FRESH FOOD RUNNER at one of six different speed settings for a period of 30 minutes. The speed settings can be described as deviations from the normal speed of the game, ranging from −67% slower (n = 9), −50% slower (n = 9), − 34% slower (n = 10), 0% normal speed, as available in the app-stores (n = 9), +32% faster (n = 11) and +75 faster (n = 10). Initially, the study also included an extreme setting of +163% faster speed, which was dropped from the analysis as it proofed to be unplayable. The unequal steps in speed decreases and increases are due to constraints of the software where speed settings can only be adjusted by integers. The unequal steps are accounted for in the statistical analysis. Standardized pre- and posttests are used to assess learning gains. Self-reported cognitive load and self-reported engagement are measured exclusively in the posttest. This research has been conducted following the ethical requirements established by the Swiss Academies of Arts and Sciences. The study was conducted at the project school of the local teacher education university with the consent of students, parents and teachers.

Instruments

To assess prior and posterior knowledge, a test asking for the harvest seasons of selected fruit and vegetables was developed. In the test items, students are asked to identify the correct months of the harvest seasons of different fruits and vegetables (sample item, with multiple checkbox answers, where all correct months should be indicated: What is the harvest season of pumpkins? 1 January, 2 February, . . .., 12 December). As the students are asked to mark time intervals for harvest seasons, this requires a special kind of scoring. The test showed adequate levels of difficulty and internal consistency (TA.pre, TA.post, 13 items, with single item difficulties between .26 and .49 in the pretest and between .22 and .66 in the posttest, Cα^pre=.83, Cα^post=.81). The scores are computed as the product of the length of the intersection (of the true interval and pupil’s answer interval) relative to the length of pupil’s answer interval and the length of the intersection relative to the length of the true interval. This definition guarantees that longer pupil intervals do not lead to better scores when the true interval is shorter than full length. The resulting scores range from 0 to 1. For example, if the true interval spans 4 months, the answer interval spans 6 month, and they have an overlap of 2 month, then the score is 2/6 * 2/4 = 1/6.

In addition, existing self-report rating scales for intrinsic cognitive load (CLIL, which is an abbreviation for “Cognitive Load: Intrinsic Load”: 3 items, Cα=.96, sample item: “The topic covered in the game was very complex.”), extraneous cognitive load (CLEL, 3 items, Cα=.77, sample item: “The explanations during the game were very unclear”.) and germane cognitive load (CLGL, 3 items, Cα=.95, “The game really enhanced my understanding of fruit/vegetable seasonality.”) have been adapted for this study, also showing good reliability (Leppink et al., 2013). The rating scale for these items range from 0 (“not at all the case”) to 10 (“completely the case”).

User engagement was rated by students with regard to focused attention (UESzA, which stands for “User Engagement Scale aspect A”, 8 items, Cα=.90, sample item: “When I played the game, I forgot the world around me.”), perceived usability (UESzB, 8 items, Cα=.72, sample item: “I found the gameplay confusing.”, positively recoded for further analysis), aesthetics (UESzC, 5 items, Cα=.87, sample item: “I liked the graphics and the visuals in the game.”) and satisfaction (UESzD, 8 items, Cα=.93, sample item: “It was rewarding to play this game.”) (Wiebe et al., 2014). The ratings were carried out using 5-point Likert scales ranging from 1 (“I totally disagree”) to 5 (“I totally agree”). For student ratings on cognitive load and engagement, mean scores for each composite scale were used for further analysis.

Table 1 provides an overview of measures used in this study. The test of harvest seasons was used in pretest and posttest. All other scales were used in the posttest only.

Table 1.

Measures Used in This Study.

Measure	Source	No of items	PretestCronbach’s α	Posttest Cronbach’s α
Test of harvest seasons (TA)	Own development	13	.83	.81
Intrinsic cognitive load (CLIL)	Leppink et al. (2013)	3		.96
Extraneous cognitive load (CLEL)	Leppink et al. (2013)	3		.77
Germane cognitive load (CLGL)	Leppink et al. (2013)	3		.95
Focused attention (UESzA)	Wiebe et al. (2014)	8		.90
Perceived usability (UESzB)	Wiebe et al. (2014)	8		.72
Aesthetics (UESzC)	Wiebe et al. (2014)	5		.87
Satisfaction (UESzD)	Wiebe et al. (2014)	8		.93

Research Protocol

The study was conducted within a double lesson of 6^th grade science. After a short introduction (5 minutes), students were asked to work on the pretest (20 minutes). This was followed by 45 minutes of individual gameplay. For this period, each student randomly received an individual notebook with the pre-installed game, each with a predetermined speed setting. Each notebook was marked with a cryptic code that was used as an anonymized personal identifier to match experimental conditions, pretests and posttests. The posttest (20 minutes) was conducted immediately after completing the gameplay session, followed by closing remarks (5 minutes). Pretests and Posttests were administered using pen and paper.

Statistical Analysis

To provide a descriptive overview, means and standard deviations for all variables are reported for each treatment group. As sample sizes in each group are too small to test for mean score differences, the hypotheses are answered by employing polynomial regression models. These models examine the interrelation between game speed as the independent variable and learning gains, cognitive load and engagement as dependent variables, testing both linear and nonlinear (i.e. quadratic “bell curve”) effects. In these models, speed is treated as a continuous predictor for the different dependent variables. The sample size is sufficient to detect effects of simple linear regression models with an effect size greater than .30 with an assumed significance alpha level of p = .05 and a statistical power of .80 (Cohen, 1988). All analyses were conducted using R 3.6 along with the packages psych, ggplot2 and pwr.

Results

The descriptive statistics for the experimental groups are provided in Table 2. As a general observation, it can be noted that on average, the serious game-based learning experience received medium difficulty ratings (CLIL), rather low ratings on extraneous cognitive load (CLEL) and positive germane cognitive load ratings in most groups. Mean engagement scores are all positive. The results also indicate that the treatment resulted in an overall increase in pretest- to posttest-knowledge by a substantial margin (ranging from an effect size of Cohen’s d=.40 in the −34% speed-condition to d = 1.17 in the +32% speed condition; Table 2).

Table 2.

Means and Standard Deviations of Measurement Variables Across Treatment Conditions.

	−67% speed (N = 9)		−50% speed (N = 9)		−34% speed (N=10)		normal speed (N = 9)		+32% speed (N=11)		+76% speed (N=10)
	M	(SD)	M	(SD)	M	(SD)	M	(SD)	M	(SD)	M	(SD)
Intrinsic cognitive load (CLIL)	4.11	(2.69)	4.26	(3.53)	4	(3.08)	5.74	(2.23)	5.15	(1.83)	6.37	(2.70)
Extraneous cognit. load (CLEL)	2.81	(2.84)	1.52	(1.41)	2.75	(2.14)	1.00	(0.88)	2.53	(1.53)	3.9	(3.23)
Germane cognitive lad (CLGL)	6.11	(2.27)	4.96	(3.90)	6.10	(2.72)	6.63	(2.46)	6.91	(1.69)	6.17	(3.00)
Focused attenttion (UESzA)	3.46	(1.02)	3.33	(1.24)	4.31	(0.71)	3.93	(0.74)	4.04	(0.75)	3.74	(0.97)
Perceived usability (UESzB)	3.63	(0.85)	4.00	(0.38)	4.10	(0.53)	4.29	(0.53)	3.94	(0.59)	3.84	(1.00)
Aesthetics (UESzC)	3.80	(0.79)	3.96	(1.02)	3.91	(0.97)	4.42	(0.66)	4.48	(0.39)	3.48	(1.19)
Satisfaction (UESzD)	4.06	(0.80)	4.21	(1.07)	4.53	(0.60)	4.56	(0.53)	4.55	(0.63)	4.11	(1.14)
Pretest Knowledge (TA.pre)	0.31	(0.11)	0.27	(0.11)	0.35	(0.11)	0.29	(0.16)	0.41	(0.13)	0.31	(0.15)
Posttest Knowledge (TA.post)	0.37	(0.13)	0.35	(0.12)	0.39	(0.09)	0.43	(0.16)	0.58	(0.16)	0.38	(0.16)

Although there is substantial variation between students, regression analyses show clear tendencies, some of which are in line with the hypotheses stated in chapter 2 while some are not. With regard to cognitive load, results show an unexpected positive and significant effect of speed on intrinsic cognitive load (Figure 4, Table 3; H1 rejected). Although the topic and the task stays the same at all speed settings, students tend to regard the task as more difficult as speed increases. Also unexpectedly, the analysis results in a significant quadratic effect for speed on extraneous cognitive load (Figure 5, Table 3; H2 rejected). Not only higher speeds tend to lead to higher levels of extraneous cognitive load but also lower speeds. With regard to germane load, we found no effect of speed, in contrast to our hypotheses, (Figure 6, Table 3; H3 rejected). Germane load – i. e. the effort addressing the learning content – seems to stay rather constant, with high variability between student ratings.

Figure 4.

Scatterplot and polynomial regression line predicting intrinsic cognitive load (CLIL) by game pacing (Speed).

Table 3.

Polynomial Regression Coefficients Predicting Intrinsic (CLIL), Extraneous (CLEL) and Germane (CLGL) Cognitive Load by Game Pacing (Speed).

	CLIL	CLEL	CLGL
Intercept	4.954 ***	2.449 ***	6.172 ***
(Standard Error Intercept)	(0.350)	(0.291)	(0.354)
Speed	6.075 *	3.221	2.283
(Standard Error Speed)	(2.665)	(2.209)	(2.695)
Speed^2	0.232	4.702 *	−1.963
(Standard Error Speed^2)	(2.665)	(2.214)	(2.695)
N	58	57	58
R2	0.086	0.111	0.022
logLik	−137.599	−124.236	−138.257
AIC	283.199	256.471	284.514

***

p < 0.001; ** p < 0.01; * p < 0.05; † p < 0.1.

Figure 5.

Scatterplot and polynomial regression line predicting extraneous cognitive load (CLEL) by game pacing (Speed).

Figure 6.

Scatterplot and polynomial regression line predicting germane cognitive load (CLGL) by game pacing (Speed).

With regard to engagement, we found the highest student ratings at medium speed settings. However, with regard to focused attention (Figure 7), perceived usability (Figure 8) and satisfaction (Figure 9), these effects are only marginally significant (p <.10; Table 4). This leads to a rejection of the hypotheses H4, H5 and H7. With regard to aesthetics, the quadratic effect is more pronounced and significant (Figure 10; Table 4; H6 accepted).

Figure 7.

Scatterplot and polynomial regression line predicting focused attention (UESzA) by game pacing (Speed).

Figure 8.

Scatterplot and polynomial regression line predicting perceived usability (UESzB) by game pacing (Speed).

Figure 9.

Scatterplot and polynomial regression line predicting satisfaction (UESzD) by game pacing (Speed).

Table 4.

Polynomial Regression Coefficients Predicting Focused Attention (UESzA), Perceived Usability (UESzB), Aesthetics (UESzC) and Satisfaction (UESzD) by Game Pacing (Speed).

	UESzA	UESzB	UESzC	UESzD
Intercept	3.819 ***	3.964 ***	4.014 ***	4.341 ***
(Standard Error Intercept)	(0.121)	(0.088)	(0.114)	(0.105)
Speed	0.753	0.094	−0.126	0.180
(Standard Error Speed)	(0.923)	(0.673)	(0.865)	(0.803)
Speed^2	−1.598 †	−1.266 †	−2.374 **	−1.566 †
(Standard Error Speed^2)	(0.923)	(0.673)	(0.865)	(0.803)
N	58	58	58	58
R2	0.062	0.061	0.121	0.065
logLik	−76.129	−57.820	−72.361	−68.044
AIC	160.258	123.640	152.722	144.088

***

p < 0.001; ** p < 0.01; * p < 0.05; † p < 0.1.

Figure 10.

Scatterplot and polynomial regression line predicting aesthetics (UESzC) by game pacing (Speed).

With regard to learning gains, medium speed settings result in slightly higher values than slower or faster speeds when controlling for pretest knowledge (Figure 11). The quadratic effect is significant (Table 5; H8 accepted).

Figure 11.

Scatterplot and polynomial regression line predicting posttest knowledge (TA.post) by pretest knowledge (TA.pre) and game pacing (Speed).

Table 5.

Polynomial Regression Coefficients Predicting Posttest Knowledge (TA.post) by Pretest Knowledge (TA.pre) and Game Pacing (Speed).

	Posttest-Knowledge (TA.post)
Intercept	0.235 ***
(Standard Error Intercept)	(−0.044)
Pretest Knowledge (TA.pre)	0.561 ***
(Standard Error TA.pre)	(−0.123)
Speed	0.192
(Standard Error Speed)	(−0.125)
Speed^2	−0.270 *
(Standard Error Speed^2)	(−0.125)
N	58
R2	0.38
logLik	41.013
AIC	−72.027

***

p < 0.001; ** p < 0.01; * p < 0.05.

Summing up, H1, H2, H3, H4, H5 and H7 were rejected. H6 and H8 were confirmed.

Discussion

Summing up, the study shows a clear tendency that the most favorable learning experiences and learning gains are found at medium speed settings, that are neither too fast nor too slow. At medium speed settings, students show not only the highest learning gains but also report the lowest levels of distraction (i. e. extraneous cognitive load) and the highest levels of aesthetical engagement – which might be seen as emotional response to the serious gaming experience. While other engagement variables tend to show a similar pattern, the results with regard to focused attention, usability and satisfaction are only marginally significant. Rather unexpectedly, higher speed settings correspond to a higher perceived task difficulty (intrinsic cognitive load) while the self-reported effort dedicated to learning (germane cognitive load) seems to be rather constant across speed settings. In contrast to typical assumptions about cognitive load and learning, where slower speeds are assumed to reduce extraneous cognitive load and possibly increase germane cognitive load, this seems to be questionable when learning with serious games. Here, slower speeds seem to be as distracting as faster speeds. The reason for this finding might be due to the complexities of serious games design. The pacing of serious games is tightly interwoven with all other aspects of game design, with visual metaphors, controls, interrelated variables, rules, goals and success measures. When changing the pacing, the entire interplay of aspects may be affected and this might lead to complex interactions and new styles of play, changing the interaction with the game in unexpected ways. In contrast to more conventional media, where changes in pacing result in rather predicable outcomes (Wouters et al., 2008), games are more complex and changes seem to be more difficult to predict. In a similar way, research has struggled to find serious game design patterns leading to consistently higher levels of behavioral, cognitive and emotional engagement. Instead, many aspects have been found to be important, without a clear indication of which aspects matter most and in which combination aspects can be considered as optimal (Jabbar & Felicia, 2015). For example, aspects increasing extraneous load might be essential to the increase of emotional engagement, which might influence cognitive engagement, and which in turn might have an impact on germane load. The interplay of these factors need to be explored in further detail in future studies.

Limitations and Suggestions for Further Future Research

The study has various limitations. As the sample size is rather small and effective post-hoc power is lower than expected for some models due to high variance in the data, some effects of the linear models turn out to be only marginally significant. Cognitive load and game engagement variables have been measured by subjective ratings alone, which are known to be valid for intrinsic cognitive load but should be supplemented by other measures for other types of load (DeLeeuw & Mayer, 2008). The duration of the treatment was rather short. The interplay of cognitive load variables and engagement variables has not been taken into accout due to the limitations of the small sample size. Furthermore, additional moderator variables such as technology acceptance and gaming experience need to be explored in larger studies.

Conclusion

Even considering these limitations – and from the practical point of view of serious game designers – the study shows that learning gains, cognitive load and engagement variables can serve as combined criteria for determining optimal speed settings for serious games.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Author Biographies

Dominik Petko is full professor for technology and teacher education at the University of Zurich. He focuses on digital change in teaching and teacher education, in particular in connection with mobile technologies and serious games.

Contact: dominik.petko@uzh.ch

Regina Schmid is a research associate at the Schwyz University of Teacher Education. Her research focuses on technology integration in schools as well as in teacher education.

Contact: regina.schmid@phsz.ch

Andrea Cantieni works as a senior researcher at the Schwyz University of Teacher Education. He specializes in educational statistics.

Contact: andrea.cantieni@phsz.ch

References

Baddeley

A. D.

(1986). Working memory. Oxford University Press.

Barrouillet

Bernardin

Camos

(2004). Time constraints and resource sharing in adults’ working memory spans. Journal of Experimental Psychology, 133(1), 83–100. https://doi.org/10.1037/0096-3445.133.1.83.

Barrouillet

Bernardin

Portrat

Vergauwe

Camos

(2007). Time and cognitive load in working memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 33(3), 570–585. https://doi.org/10.1037/0278-7393.33.3.570.

Bergeron

(2006). Developing serious games. Charles River Media.

Bouvier

Lavoué

Sehaba

(2014). Defining engagement and characterizing engaged-behaviors in digital gaming. Simulation & Gaming, 45(4–5), 491–507. https://doi.org/10.1177/1046878114553571

Boyle

E. A.

Connolly

T. M.

Hainey

Boyle

J. M.

(2012). Engagement in digital entertainment games: A systematic review. Computers in Human Behavior, 28(3), 771–780. https://doi.org/10.1016/j.chb.2011.11.020.

Boyle

E. A.

Hainey

Connolly

T. M.

Gray

Earp

Ott

Lim

Ninaus

Ribeiro

Pereira

(2016). An update to the systematic literature review of empirical evidence of the impacts and outcomes of computer games and serious games. Computers & Education, 94, 178–192. https://doi.org/10.1016/j.compedu.2015.11.003

Chandler

Sweller

(1991). Cognitive load theory and the format of instruction. Cognition and Instruction, 8, 293–332. https://doi.org/10.1207/s1532690xci0804_2

Cohen

(1988). Statistical power analysis for the behavioral sciences. Lawrence Erlbaum.

10.

Connolly

T. M.

Boyle

E. A.

MacArthur

Hainey

Boyle

(2012). A systematic literature review of empirical evidence on computer games and serious games. Computers & Education, 59, 661–686. https://doi.org/10.1016/j.compedu.2012.03.004.

11.

Czikszentmihalyi

(1990). Flow: The psychology of optimal experience. Harper & Row.

12.

De Jong

(2010). Cognitive load theory, educational research, and instructional design: Some food for thought. Instructional Science, 38(2), 105–134. https://doi.org/10.1007/s11251-009-9110-0

13.

De Koning

B. B.

Tabbers

H. K.

Rikers

R. M.

Paas

(2011). Attention cueing in an instructional animation: The role of presentation speed. Computers in Human Behavior, 27(1), 41–45. https://doi.org/10.1016/j.chb.2010.05.010.

14.

DeLeeuw

K. E.

Mayer

R. E.

(2008). A comparison of three measures of cognitive load: Evidence for separable measures of intrinsic, extraneous, and germane load. Journal of Educational Psychology, 100(1), 223–234.

15.

Filsecker

Kerres

(2014). Engagement as a volitional construct: A framework for evidence-based research on educational games. Simulation & Gaming, 45(4–5), 450–470. https://doi.org/10.1177/1046878114553569

16.

Fischer

Lowe

R. K.

Schwan

(2008). Effects of presentation speed of a dynamic visualization on the understanding of a mechanical system. Applied Cognitive Psychology, 22(8), 1126–1141. https://doi.org/10.1002/acp.1426

17.

F. L.

R. C.

S. C.

(2009). EGameFlow: A scale to measure learners’ enjoyment of e-learning games. Computers & Education, 52(1), 101–112. https://doi.org/10.1016/j.compedu.2008.07.004.

18.

Girard

Ecalle

Magnan

(2013). Serious games as new educational tools: How effective are they? A meta-analysis of recent studies. Journal of Computer Assisted Learning, 29(3), 207–219. https://doi.org/10.1111/j.1365-2729.2012.00489.x

19.

Granic

Lobel

Engels

R. C.

(2014). The benefits of playing video games. American Psychologist, 2014(1), 66–78. https://doi.org/10.1037/a0034857

20.

Hasler

B. S.

Kersten

Sweller

(2007). Learner control, cognitive load and instructional animation. Applied Cognitive Psychology: The Official Journal of the Society for Applied Research in Memory and Cognition, 21(6), 713–729. https://doi.org/10.1002/acp.1345

21.

Huang

W. D.

Johnson

(2009). Instructional game design using cognitive load theory. In Ferdig

R. E.

(Ed.), Handbook of research on effective electronic gaming in education (pp. 1143–1165). IGI Global. https://doi.org/10.4018/978-1-60960-503-2.ch707

22.

Imlig-Iten

Petko

(2018). Comparing serious games and educational simulations: Effects on enjoyment, deep-thinking, interest and cognitive learning gains. Simulation & Gaming, 49(4), 401–422. https://doi.org/10.1177/1046878118779088

23.

Iten

Petko

(2016). Learning with serious games: Is fun playing the game a predictor of learning success? British Journal of Educational Technology, 47, 151–163. https://doi.org/10.1111/bjet.12226.

24.

Jabbar

A. I. A.

Felicia

(2015). Gameplay engagement and learning in game-based learning a systematic review. Review of Educational Research, 85(4), 740–779. https://doi.org/10.3102/0034654315577210

25.

Juul

(2004). Introduction to game time. In Wardrip-Fruin

Harrigan

(Eds.), First person: New media as story, performance, and game (pp. 131–142). MIT Press.

26.

Kalyuga

(2011). Cognitive load theory: How many types of load does it really need? Educational Psychology Review, 23(1), 1–19. https://doi.org/10.1007/s10648-010-9150-7

27.

Kalyuga

Plass

J. L.

(2009). Evaluating and managing cognitive load in games. In Ferdig

R. E.

(Ed.), Handbook of research on effective electronic gaming in education (pp. 719–737). IGI Global. https://doi.org/10.4018/978-1-59904-808-6.ch041

28.

Karich

A. C.

Burns

M. K.

Maki

K. E.

(2014). Updated meta-analysis of learner control within educational technology. Review of Educational Research, 84(3), 392–410. https://doi.org/10.3102/0034654314526064

29.

Landers

R. N.

(2014). Developing a theory of gamified learning: Linking serious games and gamification of learning. Simulation & Gaming, 45(6), 752–768. https://doi.org/10.1177/1046878114563660

30.

Leppink

Paas

Vleuten

C. P.

Van der Van Gog

Van Merriënboer

J. J.

(2013). Development of an instrument for measuring different types of cognitive load. Behavior Research Methods, 45(4), 1058–1072. https://doi.org/10.3758/s13428-013-0334-1

31.

Mayer

Bekebrede

Harteveld

Warmelink

Zhou

Van Ruijven

Kortmann

Wenzler

(2014). The research and evaluation of serious games: Toward a comprehensive methodology. British Journal of Educational Technology, 45(3), 502–527. https://doi.org/10.1111/bjet.12067

32.

Mayer

R. E.

(2001). Multimedia learning. Cambridge University Press. https://doi.org/10.1017/CBO9781139164603

33.

Mayer

R. E.

(2011). Multimedia learning and games. In Tobias

Fletcher

J. D.

(Eds.), Computer Games and Instruction (pp. 281–305). IAP Information Age.

34.

Mayer

R. E.

Moreno

(2003). Nine ways to reduce cognitive load in multimedia learning. Educational Psychologist, 38(1), 43–52. https://doi.org/10.1207/s15326985ep3801_6

35.

Merkt

Schwan

(2016). Lernen mit digitalen Videos. Der Einfluss einfacher interaktiver Kontrollmöglichkeiten [Learning with digital videos: The influence of simple interactive control possibilities]. Psychologische Rundschau, 64, 94–101. https://doi.org/10.1026/0033-3042/a000301

36.

Meyer

Rasch

Schnotz

(2010). Effects of animation’s speed of presentation on perceptual processing and learning. Learning and Instruction, 20(2), 136–145. https://doi.org/10.1016/j.learninstruc.2009.02.016.

37.

Nelson

B. C.

Erlandson

B. E.

(2008). Managing cognitive load in educational multi-user virtual environments: Reflection on design practice. Educational Technology Research and Development, 56(5–6), 619–641. https://doi.org/10.1007/s11423-007-9082-1

38.

O’Brien

H. L.

Toms

E. G.

(2008). What is user engagement? A conceptual framework for defining user engagement with technology. Journal of the American Society for Information Science and Technology, 59(6), 938–955. https://doi.org/10.1002/asi.20801.

39.

Paas

Ayres

(2014). Cognitive load theory: A broader view on the role of memory in learning and education. Educational Psychology Review, 26(2), 191–195. https://doi.org/10.1007/s10648-014-9263-5

40.

Paas

Renkl

Sweller

(2003). Cognitive load theory and instructional design: Recent developments. Educational Psychologist, 38(1), 1–4. https://doi.org/10.1207/S15326985EP3801_1

41.

Paas

Sweller

(2014). Implications of cognitive load theory for multimedia learning. In Mayer

R. E.

(Ed.), The Cambridge handbook of multimedia learning (2nd ed., pp. 27–42). Cambridge University Press. https://doi.org/10.1017/cbo9781139547369.004.

42.

Paivio

(1986). Mental representations. A dual coding approach. Oxford University Press. https://doi.org/10.1093/acprof:oso/9780195066661.001.0001.

43.

Phillips

R. S.

Horstman

Vye

Bransford

(2014). Engagement and games for learning: Expanding definitions and methodologies. Simulation & Gaming, 45(4–5), 548–568. https://doi.org/10.1177/1046878114553576

44.

Puma

Matton

Paubel

P.-V.

Tricot

(2018). Cognitive load theory and time considerations: Using the time-based resource sharing model. Educational Psychology Review, 30, 1199–1214. https://doi.org/10.1007/s10648-018-9438-6

45.

Scheiter

Gerjets

(2007). Learner control in hypermedia environments. Educational Psychology Review, 19(3), 285–307.

46.

Schrader

Bastiaens

(2012). Learning in educational computer games for novices: The impact of support provision types on virtual presence, cognitive load, and learning outcomes. The International Review of Research in Open and Distributed Learning, 13(3), 206–227. https://doi.org/10.19173/irrodl.v13i3.1166

47.

Sitzmann

(2011). A meta-analytic examination of the instructional effectiveness of computer-based simulation games. Personnel Psychology, 64(2), 489–528. https://doi.org/10.1111/j.1744-6570.2011.01190.x

48.

Sweller

Ayres

Kalyuga

(2011). Cognitive load theory. Springer. https://doi.org/10.1007/978-1-4419-8126-4

49.

Sweller

Van Merrienboer

J. J.

Paas

F. G.

(1998). Cognitive architecture and instructional design. Educational Psychology Review, 10(3), 251–296. https://doi.org/10.1023/A:1022193728205.

50.

Tabbers

H. K.

de Koeijer

(2010). Learner control in animated multimedia instructions. Instructional Science, 38(5), 441–453. https://doi.org/10.1007/s11251-009-9119-4

51.

Vogel

J. J.

Vogel

D. S.

Cannon-Bowers

Bowers

C. A.

Muse

Wright

(2006). Computer gaming and interactive simulations for learning: A meta analysis. Journal for Educational Computing Research, 34(3), 229–243. https://doi.org/10.2190/FLHV-K4WA-WPVQ-H0YM

52.

Wiebe

E. N.

Lamb

Hardy

Sharek

(2014). Measuring engagement in video game-based environments: Investigation of the User Engagement Scale. Computers in Human Behavior, 32, 123–132.

53.

Wouters

Paas

Van Merriënboer

J. J.

(2008). How to optimize learning from animated models: A review of guidelines based on cognitive load. Review of Educational Research, 78(3), 645–675. https://doi.org/10.3102/0034654308320320

54.

Wouters

van Nimwegen

Oostendorp

van der Spek

E. D.

(2013). A meta-analysis of the cognitive and motivational effects of serious games. Journal of Educational Psychology, 105(2), 249–265. https://doi.org/10.1037/a0031311