Learner-Controlled Practice Difficulty and Task Exploration in an Active-Learning Gaming Environment

Abstract

Background. With the use of computer-based simulations and games becoming increasingly common in education and organizational contexts for delivering training and instruction, learners are being granted unprecedented control over their learning. This increase in learner control leaves large differences in how learners challenge themselves and explore their learning environments. Learners can benefit from task difficulty, an aspect of challenge, when they are neither overwhelmed nor underwhelmed. The purpose of the present lab study was to test a model of how encouraging learners to challenge themselves with greater task difficulty when practicing a video game could be a “double-edged sword” in terms of their skill-based learning.

Method. Using a first-person shooter computer game as the criterion task, 120 undergraduate males were randomly assigned to one of two groups. Half of the participants were encouraged to practice at a difficulty level that matched their skill level, while the other half were encouraged to practice at a difficulty that greatly exceeded their skill. Exploratory behavior was reflected in the variety of weapons used.

Results. Findings supported a model of inconsistent mediation predicting that difficulty encouragement instructions would lead to higher selected practice difficulty, which in turn would have positive direct effects on skill transfer yet negative indirect effects through exploratory behavior.

Discussion. The present study demonstrated that encouraging learners to practice at high levels of task difficulty is a “double-edged sword.” Although high levels of task difficulty can help in the preparation for future difficulties, it can also undermine exploratory behavior which is an important aspect of the learning process. This research speaks to the potential of encouraging learners to practice under difficult conditions without undermining their learning.

Keywords

active learning difficulty exploratory behavior gaming environment training

The use of computer games and web-based tools for the administration of education and training programs is a topic of growing interest for both researchers and practitioners (Anderson, Huttenlocher, Kleinberg, & Leskovec, 2013; Armstrong & Landers, 2018; Hamari, Koivisto, & Sarsa, 2014). Within these computer-based learning environments, learners are often given a relatively high degree of control over various aspects of their instruction, especially in terms of their opportunities to explore and practice to-be-learned skills. Despite the increased use of computer-based learning and support for its effectiveness across a variety of affective (motivational and attitudinal), cognitive, and skill-based learning outcomes (Sitzmann, 2011; Wilson et al., 2009), relatively little empirical research has addressed how different aspects of learner control over practice influence learning and how to support learners’ decisions regarding their practice (Landers & Reddock, 2017; Wilson et al., 2009). Given the increased application of gaming elements in training, more research on learner control is needed.

In this vein, the present study focused specifically on learner-controlled practice difficulty and the effects of difficulty encouragement instructions on skill-based learning outcomes in a gaming environment. The present study extends previous work (i.e., Hughes et al., 2013) that showed how selecting higher levels of task difficulty in practice was associated with higher scores on post-training tests of learning, including adaptive transfer performance. Research on desirable difficulties and curiosity theory are integrated to test the proposition that encouraging learners to practice at high levels of task difficulty is a “double-edged sword,” both beneficial and harmful to the learning process. In doing so, we seek to contribute to the literature on active-learning training by focusing on practice difficulty and exploration as behavioral components of self-regulation.

Active Learning

Active learning is a framework for instruction and training that reconciles discovery-based and proceduralized approaches by granting learners a relatively large degree of control over their learning. Specifically, with active learning, “learners assume primary responsibility for important learning decisions (e.g., choosing learning activities, monitoring and judging progress; p. 297)” while training design elements are incorporated to support their self-regulation as they engage training content and practice opportunities (Bell & Kozlowski, 2008). Active learning largely reflects the constructivist perspective to learning, arguing that learning is an inductive process in which learners explore a task and infer rules, principles, and strategies to effective performance. Although a growing body of research supports active-learning training (Keith & Wolff, 2015), especially with respect to computer simulations, games, and web-based instruction (Sitzmann, 2011), the vast majority of the research has focused on the cognitive, motivational, and affective components of self-regulation with very little attention to the behavioral components.

Put another way, little is known about the decisions learners make with regard to the actions they take when practicing a task or developing a skill. Building on the notion of desirable difficulties, Hughes et al. (2013) advocated devoting research to self-selected practice difficulty, operationalized in terms of task complexity (e.g., in missions or scenarios), given that such learner control is inherent in many gaming and virtual environments (Hussain et al., 2010). Hughes et al. (2013) showed how individual differences in self-selected practice difficulty were positively related to post-practice task knowledge, analogical transfer, and adaptive transfer performance.

Learner Control

In contrast to traditional instructional methods, computer simulations, games, and web-based instruction often grant individual learners considerable control over various instructional elements such as pace of instruction, content to engage (or skip), and time devoted to studying or practicing (Landers & Reddock, 2017). In general, learner control refers to the extent to which learners can alter their own learning environment, and typically involves computer-based tools (Landers & Reddock, 2017). Researchers should be cognizant of how learner control should be examined in terms of the specific elements controlled more so than as an overarching variable that falls on a single continuum ranging from low to high control. Likewise, designers of instructional programs typically make decisions about which elements of instruction learners should be given control over rather than simply how much control to give to learners. With this conceptualization of learner control in mind, recent research on the relationship between learner control and learning outcomes has yielded positive, negative, and null effects.

In a recent meta-analysis of web-based instruction, Landers and Reddock (2017) compared the effectiveness of learner control versus program control for each of five different instructional design elements. Overall, they found mixed support for the effect of learner control, which they attributed to the confounding of multiple learner control elements in single programs. However, their results did show learner control was generally more beneficial for skill-based versus knowledge-based training. One commonality among learner-controlled design elements is that each can add difficulty or complexity to the learner experience (Hughes et al., 2013). However, very few studies have directly investigated learner control of practice/task difficulty itself, which is a common element of many gaming environments (Bedwell, Pavlas, Heyne, Lazzara, & Salas, 2012; Hughes et al., 2013). Those studies that have directly investigated learner control of practice/task difficulty (e.g., Ford, Smith, Weissbein, Gully, & Salas, 1998) focused on its association with cognitive processes and not learning outcomes per se (Hughes et al., 2013). The present study seeks to further examine the role of learner-controlled task difficulty with a special focus on how self-selected difficulty during practice is related to skill transfer via exploratory behavior during practice.

Difficulty, Exploration, and Learning

A large body of empirical research indicates that adding certain difficulties to practice, while undermining initial performance (i.e., downgrading speed of acquisition), can ultimately enhance learning with respect to tests of retention and adaptive transfer (Schmidt & Bjork, 1992; Soderstrom & Bjork, 2015). Such desirable difficulties are beneficial in as much as they promote retrieval and transfer appropriate processes, and they include such elements as spacing practice, varying the nature of practice, and delaying or reducing feedback (R. A. Bjork, 1994). Hughes et al. (2013) likened task complexity to this notion of desirable difficulties, arguing and showing that learners who selected more complex task scenarios during practice achieved higher scores on post-practice learning outcomes despite obtaining lower scores during practice than learners who selected simpler task scenarios to practice. Although one might conclude from Hughes et al. findings that learners should be encouraged to attempt more difficult and complex task scenarios in virtual, gaming, and web-based learning environments that are amenable to active learning, the authors cautioned against taking such a conclusion too far as learners could become overwhelmed if they take on content that involves processing demands that are beyond their current skill level (Berlyne, 1960). In other words, there is most likely a fine line between a difficulty that is beneficial (i.e., desirable) and a difficulty that is harmful, which is recognized with scaffolding instruction (Azevedo & Hadwin, 2005; D. Wood, Bruner, & Ross, 1976).

Curiosity theory, and its various extensions like flow theory, offer explanations for how difficulty—task complexity in particular—can be both beneficial and harmful to the learning process. Research with animals and humans alike supports the basic tenets of curiosity research, demonstrating a preference for stimuli that are slightly more complex than those which are more familiar or understood (Berlyne, 1960; Earl, 1957). With respect to training and learning contexts in general, as skill is acquired learners naturally prefer increases in task complexity. However, too much complexity is associated with anxiety, which is detrimental to learning because anxiety taxes cognitive resources and undermines exploratory behavior (Csikszentmihalyi, 1990; Loewenstein, 1994). Indeed, empirical research demonstrates that exploratory behavior, defined as an active interaction on the part of the trainee with the training environment through attempts at multiple solutions to the problem at hand (Dormann & Frese, 1994), is fundamental to the learning process (Bell & Kozlowski, 2008; Hardy, Day, Hughes, Wang, & Schuelke, 2014; Keith & Wolff, 2015).

Stemming from epistemic curiosity, which is the desire for knowledge that motivates individuals to learn new ideas, eliminate information-knowledge gaps, and solve intellectual problems (Litman, 2008), exploration is intentional behavior that minimizes uncertainty by building upon already-possessed competencies (Hardy et al., 2014; Kashdan et al., 2009; Koo & Choi, 2010; Schneider, Von Kroh, & Jäger, 2013). Thus, exploration is not random behavior. It is a systematic process of identifying, searching for, and resolving complexity and novelty in a task environment (Berlyne, 1960, 1966; Loewenstein, 1994). Those who engage in more exploratory behavior have a broader focus, are less likely to settle on sub-optimal strategies, and are better equipped to respond to a variety of context-specific demands because they acquire more diverse repertoires of skills and strategies (Hardy, Day, & Arthur, 2019; Loewenstein, 1994). The intentional, systematic nature of exploratory behavior is reflected in how fluctuations in exploration in practice are positively associated with changes in episodic practice performance, which cumulate over practice and benefit both analogical and adaptive transfer (Hardy et al., 2014).

Exploratory behavior is particularly important to learning in complex, dynamic, and open task environments. Within such environments, there is not a clearly defined problem solution or optimal approach for performance. In contrast, exploratory behavior is less suitable and may even be counterproductive for simple tasks, which are better addressed by more proceduralized instruction that places limits on exploration and instead emphasizes the repeated practice of a well-defined solution or performance strategy.

In general, difficulties and exploratory behavior are important components to learning. Learners prefer a certain degree of challenge when practicing new tasks and developing skill proficiency, but only as long as they are not outmatched by the challenge (i.e., task demands relative to skill proficiency; Csikszentmihalyi, 1990; Wilson et al., 2009). Conversely, without challenge, learners become bored and are likely to withdraw from practice or become less engaged during practice (Csikszentmihalyi, 1990; Wilson et al., 2009). In this way, increases in task complexity are desirable if not instrumental to the learning process. However, increases in task complexity that outpace the learner’s skill development are also likely to overwhelm cognitive resources and in turn undermine exploratory behaviors as learners devote resources to handling the excessive demands placed on them by using the strategies with which they are most familiar and proficient (Döerner, 1980).

Method

This research was approved by our institution’s Institutional Review Board or research ethics committee. To examine the effects of practice difficulty (i.e., task complexity) on task exploration and learning outcomes in an active-learning gaming environment, we randomly assigned participants to one of two difficulty encouragement conditions: even-matched versus outmatched. Specifically, half of the participants were encouraged to practice the criterion task at a difficulty level that matched their skill level, while the other half were encouraged to practice at a difficulty that greatly exceeded their skill. The logic of the outmatched condition was to provide the benefits of practicing at higher levels of difficulty while simultaneously promoting self-regulatory processes via learner control. That is, practicing at higher difficulty levels promotes analogical and adaptive transfer (i.e., Hughes et al., 2013), and providing learners with control over their own task difficulty should prompt them to monitor their performance and adjust task difficulty in a way they believe is most beneficial. Although learners were prompted to select task difficulty that either matched or outmatched their current proficiency, the manipulation is consistent with the active-learning approach to training in that learners were ultimately responsible for deciding which difficulty to select for each trial, which is a common yet under-examined feature of many gaming and synthetic learning environments (Hughes et al., 2013). Moreover, because control over difficulty levels is a common feature of these environments (Hussain et al., 2010; Sadagic, 2010), this manipulation is an ecologically valid way of inducing selection of higher difficulty.

We tested a model of inconsistent mediation (MacKinnon & Fairchild, 2009), hypothesizing that outmatched difficulty encouragement instructions would lead to higher levels of selected practice difficulty, which in turn would have positive direct effects on learning outcomes but would also have negative indirect effects by undermining exploratory behaviors during practice. Figure 1 shows the model and specific hypotheses examined. We examined this model across tests of both analogical and adaptive transfer. Analogical transfer refers to the capability to perform effectively in familiar situations after training, whereas adaptive transfer refers to the capability to perform effectively in response to novel (e.g., more complex) demands (Ivancic & Hesketh, 2000). By testing this model, we sought to inform theory on how to help learners better challenge themselves in practice inasmuch as challenge (e.g., learning from mistakes) is often associated with positive outcomes (E. L. Bjork & Bjork, 2011; Clark & Bjork, 2014; Frese & Keith, 2015).

Figure 1.

The proposed research model, including hypothesized relationships.

Sample and Task

In exchange for credits to fulfill a psychology course research requirement, 120 undergraduate males (M_age = 19.34, SD = 1.74) attending the University of Oklahoma completed this study. Because of the substantial gender differences in enjoyment of, performance in (Hopp & Fisher, 2017), and likelihood to play first-person shooter video games (Hartmann & Klimmt, 2006), this study utilized an all-male sample. As exhibited in their responses to a four-item survey administered near the end of their participation as part of a short demographics questionnaire, participants varied in their video game experience. For the first two items, participants responded using a 5-point Likert scale ranging from 1 (not at all) to 5 (daily) to the following questions: (a) ‘‘Over the last 12 months, how frequently have you typically played video/computer games?’’ (M = 3.39, SD = 1.31) and (b) ‘‘Over the last 12 months, how frequently have you typically played first-person shooter video/computer games (e.g., Call of Duty, Half-Life, Halo, Unreal Tournament)?’’ (M = 2.85, SD = 1.26). For the second two items, participants indicated how many hours per week they typically play video/computer games (M = 6.37, SD = 8.34, min. = 0.00, max. = 40.00) and how many hours per week they typically play first-person shooter video/computer games (M = 3.34, SD = 5.12, min. = 0.00, max. = 25.00). Despite this variability, it is important to note that none of the relationships tested in this study were moderated by prior video game experience.

We used the same performance task as Hughes et al. (2013), specifically Unreal Tournament 2004 (UT2004), a commercially available first-person-shooter computer game. Using a variety of weapons, the objective of the game is to destroy computer-controlled opponents (i.e., bots) while minimizing the destruction of one’s own character. Participants receive detailed performance feedback both during and at the end of every game. During the game, participants must find and collect weapons and other resources that affect their health, offensive, and defensive capabilities. When an opponent or participant’s character is destroyed, it reappears in a new location on the map with basic weapons and capabilities. Participants must use a mouse and keyboard simultaneously to move and control their character. Additionally, participants must learn how each weapon works, as well as weapon strengths and weaknesses, and be able to decide quickly which to use given the circumstances. Participants must learn and remember weapon and resource locations and, in some cases, use problem-solving skills to access those items. Monitoring of game aspects such as character health, opponent health, and game strategies is critical for effective performance. Each game lasted 5 minutes.

Protocol and Manipulation

Figure 2 provides a summary of the study protocol. After signing an informed consent form, participants watched a 15-minute training video that explained basic game controls, rules, weapons, and resources. Next, participants had 3 minutes of practice and familiarization without any computer-controlled bots present. Then, participants performed two baseline games against two computer-controlled bots for which they were instructed to “do their best” by maximizing their kills while simultaneously minimizing their own character’s deaths. These games were set at a medium level of difficulty (4 on a 1-to-7 scale; see Hughes et al., 2013 for details). Following these games, participants underwent two more games. However, these games were set at Difficulty Levels 2 and 6, respectively, and participants were told that these games were meant to familiarize them with some of the other difficulty settings.

Figure 2.

Chronological list of study procedures. The order of the adaptive transfer tests was counterbalanced.

Next, participants were randomly assigned to one of the two different encouragement conditions: even-matched (n = 63) or outmatched (n = 57). In the outmatched condition, participants were instructed to select a difficulty at which they had a zero percent chance of finishing the game in first place. In the even-matched condition, participants were instructed to select a difficulty at which they had a fifty percent chance to finish the game in first place. See Figure 3 for more details on the even-matched and outmatched instructions. Participants then performed three practice sessions each consisting of five games against two bots. Participants were permitted to select the level of difficulty (1 to 7) before each of their practice games using a drop-down menu on the computer. Difficulty is reflected in the skill proficiency (i.e., quickness, elusiveness, accuracy, and unpredictability) of the computer-controlled bots, which adds complexity to the game consistent with R. E. Wood’s (1986) three dimensions of task complexity: component, coordinative, and dynamic (see Hughes et al., 2013 for details). Participants were instructed to advance at their own pace and were able to view detailed onscreen performance feedback at the conclusion of each game. Encouragement refresher instructions were repeated before each of the last two practice sessions.

Figure 3.

Difficulty encouragement instructions.

After completing the last practice game, participants performed two test games at a medium level of difficulty (4) assessing their post-training analogical transfer performance (the same scenario as baseline). Finally, participants underwent two games each of three adaptive transfer scenarios for a total of six more test games. In one transfer scenario, the map and weapons remained the same, but the difficulty level was increased to 6. In another scenario, the map and difficulty remained the same, but the weapons (except one) were different. In yet another transfer scenario, the weapons and difficulty remained the same, but participants encountered a geographic layout (i.e., map) that they previously had not encountered. The order of the pairs of transfer games was counterbalanced. For both the analogical and adaptive transfer games, participants were instructed to “do their best.”

Measures

Exploratory behavior

Given the central importance of using a variety of weapons as a function of the situational dynamics of the game, exploratory behavior was scored in terms of the variety of weapon use within each five-game practice session, and the session scores were then averaged to create a single index of exploratory behavior in practice. In this study, five weapons were available during each game, four of which had both primary and secondary capabilities and one having just a single capability (see Figure 4 for details). The same five weapons were common across all the games except for the weapons transfer scenario as noted above. Scores for each weapon use across each five-game session were binary: 0 kills = 0, 1 or more kills = 1.

Figure 4.

Weapon uses and availability.

For each session, exploratory behavior was the sum of the weapon-use scores. Thus, across each five-game session, exploration scores could range from 0 to 9. Using a variety of weapons reflects attempts at multiple combat strategies as well as larger coverage of the game map, which also translates into using a variety of different resources given their distribution in various map locations.

Selected practice difficulty

Selected difficulty levels for every practice game were recorded by the Unreal Tournament 2004 software. For each session, the selected difficulty levels were averaged across the five practice games to create a session-level index of practice difficulty. Then, the average of these session-level scores was computed to create an overall index of practice difficulty.

Learning outcomes

Baseline, analogical transfer, and adaptive transfer performance scores were computed by dividing participant kills (i.e., number of times a participant destroyed an opponent) by the quantity of kills plus deaths (i.e., number of times a participant’s own character was destroyed) plus participant rank (i.e., finishing in first, second, or third place). Scores for each of these components were displayed on-screen during and at the end of every game. Scores could range from 0 to approximately 1. This formula is similar to the one used by the creators of UT2004 to create an index of efficiency and was used in this study because it accounts for multiple aspects of performance.

Results

Table 1 shows means, standard deviations, and correlations for all the variables. Independent samples t-tests were conducted to examine differences between the even-matched and outmatched difficulty conditions across scores for selected practice difficulty, exploratory behavior during practice, and the four learning outcomes: analogical transfer, adaptive weapon transfer, adaptive map transfer, and adaptive difficulty transfer. As expected, the results showed statistically significant higher selected difficulty in the outmatched condition (M = 5.53, SD = 0.91) versus the even-matched condition (M = 4.54, SD = 1.03; t(118) = 5.53, p < .0001, d = 1.02). Also as expected, there was significantly less exploratory behavior during practice in the outmatched condition (M = 5.19, SD = 0.94) versus the even-matched condition (M = 5.65, SD = 0.84; t(117) = −2.80, p < .01, d = − 0.52). However, there were no significant differences between the two groups on any of the learning outcomes (analogical transfer: t(118) = 0.55, p = 0.58, d = − 0.10; adaptive weapon transfer: t(118) = −1.20, p = 0.23, d = − 0.22; adaptive map transfer: t(118) = − 0.05, p = 0.96, d = − 0.01; adaptive difficulty transfer: t(118) = − 1.07, p = 0.29, d = − 0.20).

Table 1.

Means, Standard Deviations, and Correlations.

Variable	M	SD	1	2	3	4	5	6	7	8	9	10	11	12	13

1. Dif. encour.^a	0.48	0.50
2. Base. perf.	0.30	0.12	-.07
3. Prac. dif. 1	4.63	1.20	.32	.59
4. Prac. dif. 2	5.14	1.29	.46	.43	.74
5. Prac. dif. 3	5.25	1.29	.40	.30	.48	.66
6. Prac. dif.	5.01	1.09	.45	.51	.85	.92	.83
7. Prac. expl. 1	5.67	1.21	-.15	-.01	-.17	-.15	.01	-.12
8. Prac. expl. 2	5.34	1.30	-.16	.05	-.13	-.39	-.14	-.26	.28
9. Prac. expl. 3	5.30	1.34	-.22	.00	-.07	-.18	-.34	-.29	.25	.25
10. Prac. expl.	5.43	0.91	-.25	.02	-.17	-.34	-.23	-.28	.69	.72	.72
11. Analogical transfer	0.45	0.14	-.05	.69	.58	.47	.36	.54	.15	.04	-.01	.08
12. Adapt-dif. transfer	0.26	0.12	-.10	.62	.50	.44	.22	.44	.12	-.04	.13	.10	.67
13. Adapt-map transfer	0.33	0.13	-.01	.66	.53	.50	.44	.56	.07	.11	.05	.11	.64	.60
14. Adapt-wea. transfer	0.33	0.10	-.11	.46	.48	.40	.27	.44	.17	-.04	.11	.11	.53	.62	.48

Note. Dif. encour.^a = difficulty encouragement (0 = even-matched, 1 = outmatched). Base. perf. = baseline performance. Prac. dif. 1 = Session 1 difficulty. Prac. dif. 2 = Session 2 difficulty. Prac. dif. 3 = Session 3 difficulty. Prac. dif. = mean selected practice difficulty across all sessions. Prac. expl. 1= Session 1 exploration. Prac. expl. 2 = Session 2 exploration. Prac. expl. 3 = Session 3 exploration. Prac. expl.= mean practice exploration across all sessions. Analogical= analogical transfer performance. Adapt-dif.= adaptive transfer difficulty performance. Adapt-map = adaptive transfer map performance. Adapt-wea.= adaptive transfer weapon performance.

N = 120. r > |.17| = p < .10; r > |.20| = p < .05; r > |.25| = p < .01. All tests are two-tailed.

To test the hypotheses that outmatched difficulty encouragement instructions would lead to higher levels of selected practice difficulty, which in turn would have positive and direct effects on learning outcomes but would also have negative and indirect effects by undermining exploratory behaviors during practice, two sets of mediation analyses (two paths for each of four transfer outcomes) were conducted. A bootstrapping approach (Preacher & Hayes, 2008) was used in which point estimates of the indirect effect (ab) and bias corrected 95% confidence intervals were derived from the mean of 5,000 estimates.

The first set of mediation analyses tested the positive indirect effect of the difficulty encouragement manipulation on each of the four learning outcomes mediated by selected practice difficulty (controlling for baseline performance). As seen in Table 2, and in support of the hypothesized relationship, the indirect effect of difficulty encouragement was statistically significant and positive for each of the four learning outcomes (analogical transfer: ab = .37, 95% CI = .21, .55; adaptive weapons transfer: ab = .49, 95% CI = .26, .81; adaptive map transfer: ab = .40, 95% CI = .25, .58; adaptive difficulty transfer: ab = .29, 95% CI = .10, .52).

Table 2.

Direct and Indirect Effects of Difficulty Manipulation on Transfer Outcomes via Practice Difficulty.

	Indirect effect (ab)	Direct effects
	Indirect effect (ab)	Predictor (c’)	Mediator (a)	Control
	Difficulty encouragement	Difficulty encouragement	Practice difficulty	Baseline performance
DV: Analogical Tran.
Coefficient	.37**	−.36**	.37**	.49**
BC 95% CI Lower	.21
BC 95% CI Upper	.55
DV: Weapon Tran.
Coefficient	.49**	−.64**	.50**	.17†
BC 95% CI Lower	.26
BC 95% CI Upper	.81
DV: Map Tran.
Coefficient	.40**	−.30**	.40**	.45**
BC 95% CI Lower	.25
BC 95% CI Upper	.58
DV: Difficulty Tran.
Coefficient	.29**	−.39**	.29**	.47**
BC 95% CI Lower	.10
BC 95% CI Upper	.52

Note. DV = dependent variable. BC = bias corrected. CI = confidence interval. †p <.10, **p < .05, **p < .01 (two-tailed). N = 120.

The second set of mediation analyses tested the negative indirect effect of selected practice difficulty on each of the four learning outcomes mediated by exploratory behavior (controlling for baseline performance and difficulty encouragement). As seen in Table 3, and in support of the hypothesized relationship, the indirect effect of practice difficulty was statistically significant and negative for each of the four learning outcomes (analogical transfer: ab = –.05, 95% CI = –.12, –.01; adaptive weapons transfer: ab = –.07, 95% CI = –.16, –.02; adaptive map transfer: ab = –.07, 95% CI = –.15, –.02; adaptive difficulty transfer: ab = –.05, 95% CI = –.12, –.002). Practice difficulty was negatively related to exploratory behavior, which in turn was positively related to each of the four learning outcomes. Also, the direct effect of practice difficulty was statistically significant and positive for each of the four learning outcomes. It is also important to acknowledge that in every analysis the difficulty encouragement manipulation yielded a significant and negative direct effect.

Table 3.

Direct and Indirect Effects of Practice Difficulty on Transfer Outcomes via Exploratory Behavior.

	Indirect effect (ab)	Direct effects
	Indirect effect (ab)	Predictor (c’)	Mediator (b)	Controls
	Practice difficulty	Practice difficulty	Explanatory behavior	Baseline performance	Difficulty Encouragement
DV: Analogical Tran.
Coefficient	–.05*	.39**	.14*	.47**	–.28†
BC 95% CI Lower	–.12
BC 95% CI Upper	–.01
DV: Weapon Tran.
Coefficient	–.07*	.56**	.19*	.14	–.61**
BC 95% CI Lower	–.16
BC 95% CI Upper	–.02
DV: Map Tran.
Coefficient	–.07*	.48**	.20*	.41**	–.28†
BC 95% CI Lower	–.15
BC 95% CI Upper	–.02
DV: Difficulty Tran.
Coefficient	–.05*	.33**	.13†	.45**	–.34*
BC 95% CI Lower	–.12
BC 95% CI Upper	–.002

Note. DV = dependent variable. BC = bias corrected. CI = confidence interval. †p < .10, *p < .05, **p < .01 (two-tailed). N = 120.

Discussion

Overall, the results supported the proposed model of inconsistent mediation for the effects of selected practice difficulty on skill transfer performance in an active-learning gaming environment. Before discussing any implications, it is important to acknowledge several key limitations. One such limitation concerns the generalizability of the findings. This study used a computer game involving both cognitive and perceptual-motor demands with a young-adult male sample. Future research should examine whether the observed relationships generalize to other active-learning gaming and web-based environments across a range of demographics (gender, age, and experience) and operationalizations of task difficulty, exploratory behavior, and skill transfer. Related, our focus was limited to tests of immediate skill-based learning. Given that computer-based learning can be used to promote a variety of learning outcomes in training and education settings, it would be interesting to see future research address how self-selected difficulty relates to broader and more distal learning outcomes.

The lack of measures and manipulations of established self-regulatory variables covering cognitive, motivation, and emotion processes is another important limitation (see Sitzmann & Ely, 2011 for a meta-analytic review of self-regulatory learning processes). In particular, the present study did not examine the role of arousal and emotion control strategies. Indeed, too much complexity in a learning context is associated with anxiety, which in turn taxes cognitive resources (Berlyne, 1960; Mayer, 2009). Therefore, further research is needed to investigate the role of emotion control strategies in combination with difficulty encouragement instructions as a means of mitigating the undermining effects of task difficulty (i.e., task complexity) on exploratory behaviors. In a similar vein, future research could examine the influence of interventions that target metacognition, self-efficacy, or exploration in combination with difficulty encouragement instructions (see Keith & Wolff, 2015 for a cogent review of active training interventions). Additionally, our study focused on learning outcomes attained from self-selected practice difficulty. Future research should examine whether these findings are a function of self-selection per se, or if assigning learners to higher task difficulty levels based on their initial performance achieves similar results. With such a study it would still be important to include measures of self-regulatory processes (e.g., emotion control, metacognition, and exploratory behavior) to better understand the linkages between active-learning interventions and skill-based transfer.

Ultimately, the results of the present study demonstrated that increased task difficulty may indeed be a “double-edged sword” in the learning process. Specifically, increases in selected practice difficulty were found to be positively associated with skill transfer, lending support to the notion that difficulty in the form of task complexity can be beneficial to learning. However, given the negative indirect effects of selected difficulty via lowered exploratory behavior coupled with the negative direct effects of the difficulty encouragement instructions, the relationship between practice difficulty and learning outcomes may be more complex than simply stating that more difficulty is better. These negative effects suggest that although self-selected increases in task difficulty can be beneficial to learning, an outmatched individual will not necessarily learn as much as those who are evenly matched. The previous findings of exploratory behavior’s positive impact on learning outcomes (Hardy et al., 2014), paired with the direct and positive effect found in this study, suggest that it may be unwise to simply increase task difficulty in complex, dynamic, and open gaming environments. Instead, task difficulty and exploratory behavior should balance one another in such a way that learners are sufficiently challenged but remain able to explore in a way that ultimately maximizes learning outcomes. More broadly, the research surrounding desirable difficulties has suggested that adding certain difficulties to practice can enhance retention and adaptive transfer (Schmidt & Bjork, 1992; Soderstrom & Bjork, 2015). Although practice task difficulty was found to positively relate to skill transfer, the current research suggests that there may indeed be a fine line between difficulty that is beneficial and difficulty that is harmful.

When task difficulty becomes overwhelming, learners will often rely on strategies with which they are more familiar and proficient. Because of this reliance on familiarity, learners will engage in less exploratory behaviors. It may in fact be the case that learners will self-select lower task difficulty when exploration is the goal, and will select higher difficulty when they seek to test the skills they have developed through exploration. In this way, granting learners the capability to select different difficulty levels during training may help them better manage the tradeoff between exploring options and developing diverse repertoires versus refining and maximizing the potential of a preferred solution (Hardy et al., 2019). As the current study’s findings support, exploratory behavior is positively related to both analogical and adaptive skill transfer (Hardy et al., 2014). Although some of the negative links between overwhelming task difficulty and transfer outcomes may be due to lowered exploratory behavior, the observed negative direct effect of the difficulty encouragement instructions suggests that other self-regulatory processes could be involved. We believe that future research is needed to examine other self-regulatory processes that might mediate the effects of training and task difficulties to be better distinguish desirable from undesirable difficulties (Arthur & Day, 2013). In this respect, it might be theoretically and practically useful to examine if the effects of training and task difficulties are meaningfully distinct from those involved in setting difficult performance goals during practice (Kanfer & Ackerman, 1989), and whether task difficulty is meaningfully distinct from other learner-controlled training design elements (Landers & Reddock, 2017).

In conclusion, games and simulations are an ever-growing and effective means for training and education (Sitzmann, 2011; Wilson et al., 2009). Task difficulty should be a primary consideration in the design of games, simulations, and web-based learning environments that lend themselves to active learning and where exploration is critical to success. Granting users control over different levels of task difficulty to practice represents an important gaming attribute that can provide learners with challenge, interest, and perhaps better learning outcomes (Hussain et al., 2010; Sadagic, 2010). However, there is currently a lack of empirical research on this gaming attribute and consequently there is a lack of theory regarding its benefits and limitations. The present study extends previous work on self-selected task difficulty in gaming environments, Hughes et al.’s (2013) in particular, by showing that learner-controlled increases in task difficulty come with a downside in the form of lowered exploratory behavior, which in the long-run could inhibit the development of a diverse repertoire of component skills and strategies (Hardy et al., 2019). Training developers should consider the benefits of encouraging learners to challenge themselves in training and when practicing to-be-learned material, while simultaneously recognizing that too much difficulty can undermine exploratory behaviors and perhaps other self-regulatory processes that promote learning. Consistent with the need for more empirical research on learner control that targets specific design elements in computer-based learning environments (Landers & Reddock, 2017), we hope this study spurs more research on granting users the discretion to select different difficulty levels and its role in self-regulated learning.

Footnotes

A previous version of part of this manuscript was presented at the 32^nd Annual Conference of the Society for Industrial and Organizational Psychology.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Author Biographies

Joseph Westlin, PhD, is an Assistant Professor of Management in the Huether School of Business at the College of Saint Rose. His research interests include complex skill acquisition, employee absenteeism, and leadership. This research was conducted while Joseph was a doctoral student in the Industrial/Organizational Psychology program at the University of Oklahoma, Department of Psychology, Norman, OK, 73071.

Eric Anthony Day, PhD, is Professor and Chair of the Department of Psychology at the University of Oklahoma, USA. His research interests span personnel psychology and organizational behavior, including topics in training and development, complex skill acquisition and adaptability, group dynamics, and leadership.

Michael G. Hughes, PhD, is a Senior Scientist at the Human Resources Research Organization (HumRRO). He received his PhD in Industrial/Organizational Psychology from the University of Oklahoma. His research interests include psychological assessment and employment selection, and he has previously published research on complex skill acquisition in a gaming environment.

References

Anderson

Huttenlocher

Kleinberg

Leskovec

(2013, May). Steering user behavior with badges. In Proceedings of the 22nd International Conference on World Wide Web (pp. 95-106). Rio de Janeiro, Brazil: ACM.

Armstrong

M. B.

Landers

R. N.

(2018). Gamification of employee training and development. International Journal of Training and Development, 22, 162-169. doi:10.1111/ijtd.12124

Arthur

Jr. Day

E. A.

Jr. (2013). A look from aFarr (1987): The past, present, and future of applied skill decay research. In Arthur

Jr. Day

E. A.

Bennett

Jr. Portrey

(Eds.), Individual and team skill decay: State of the science and implications for practice (pp. 405-427). New York, NY: Taylor & Francis.

Azevedo

Hadwin

A. F.

(2005). Scaffolding self-regulated learning and metacognition—Implications for the design of computer-based scaffolds. Instructional Science, 33, 367-379. doi:10.1007/s11251-005-1272-9

Bedwell

W. L.

Pavlas

Heyne

Lazzara

E. H.

Salas

(2012). Toward a taxonomy linking game attributes to learning: An empirical study. Simulation & Gaming, 43(6), 729-760. doi:10.1177%2F1046878112439444

Bell

B. S.

Kozlowski

S. W.

(2008). Active learning: Effects of core training design elements on self-regulatory processes, learning, and adaptability. Journal of Applied Psychology, 93, 296-316. doi:10.1037/0021-9010.93.2.296

Berlyne

D. E.

(1960). Conflict, arousal, and curiosity. New York, NY: McGraw-Hill. doi:10.1037/11164-000

Berlyne

D. E.

(1966). Curiosity and exploration. Science, 153(3731), 25-33. doi:10.1126/science.153.3731.25

Bjork

E. L.

Bjork

(2011). Making things hard on yourself, but in a good way. In Gernsbacjer

M. A.

Pew

R. W.

Hough

L. M.

Pomerantz

J. R.

(Eds.), 59-68. Psychology in the real world (pp. 5). New York, NY: Worth.

10.

Bjork

R. A.

(1994). Memory and metamemory considerations in the training of human beings. In Metcalfe

J. E.

Shimamura

A. P.

(Eds.), Metacognition: Knowing about knowing (pp. 185-205). Cambridge, MA: The MIT Press.

11.

Clark

C. M.

Bjork

R. A.

(2014). When and why introducing difficulties and errors can enhance instruction. In Benassi

V. A.

Overson

C. E.

Hakala

C. M.

(Eds.), Applying science of learning in education: Infusing psychological science into the curriculum (pp. 20-30). Retrieved from http://teachpsych.org/ebooks/asle2014/index.php

12.

Csikszentmihalyi

(1990). The domain of creativity. In Runco

M. A.

Albert

R. S.

(Eds.), Theories of creativity (pp. 190-215). Newbury Park, CA: SAGE.

13.

Döerner

(1980). On the difficulties people have in dealing with complexity. Simulation & Games, 11, 87-106.

14.

Dormann

Frese

(1994). Error training: Replication and the function of exploratory behavior. International Journal of Human-Computer Interaction, 6, 365-372. doi:10.1080/10447319409526101

15.

Ford

J. K.

Smith

E. M.

Weissbein

D. A.

Gully

S. M.

Salas

(1998). Relationships of goal orientation, metacognitive activity, and practice strategies with learning outcomes and transfer. Journal of Applied Psychology, 83, 218-233. Retrieved from http://psycnet.apa.org/doi/10.1037/0021-9010.83.2.218

16.

Frese

Keith

(2015). Action errors, error management, and learning in organizations. Annual Review of Psychology, 66, 661-687. doi:10.1146/annurev-psych-010814-015205

17.

Hamari

Koivisto

Sarsa

(2014, January). Does gamification work? A literature review of empirical studies on gamification. In Hawaii, USA. Proceedings of the 47th Hawaii International Conference on System Sciences (HICSS) (pp. 3025-3034). IEEE.

18.

Hardy

J. H.

Day

E. A.

Arthur

(2019). Exploration-exploitation tradeoffs and information-knowledge gaps in self-regulated learning: Implications for learner-controlled training and development. Human Resource Management Review, 29, 196-217. doi:10.1016/j.hrmr.2018.07.004

19.

Hardy

J. H.

Day

E. A.

Hughes

M. G.

Wang

Schuelke

M. J.

(2014). Exploratory behavior in active learning: A between-and within-person examination. Organizational Behavior and Human Decision Processes, 125, 98-112. doi:10.1016/j.obhdp.2014.06.005

20.

Hartmann

Klimmt

(2006). Gender and computer games: Exploring females’ dislikes. Journal of Computer-Mediated Communication, 11, 910-931. doi:10.1111/j.1083-6101.2006.00301.x

21.

Hopp

Fisher

(2017). Examination of the relationship between gender, performance, and enjoyment of a first-person shooter game. Simulation & Gaming, 48, 338-362. doi:10.1177%2F1046878117693397

22.

Hughes

M. G.

Day

E. A.

Wang

Schuelke

M. J.

Arsenault

M. L.

Harkrider

L. N.

Cooper

O. D.

(2013). Learner-controlled practice difficulty in the training of a complex task: Cognitive and motivational mechanisms. Journal of Applied Psychology, 98, 80-98. doi:10.1037/a0029821

23.

Hussain

Feurzeig

Cannon-Bowers

Coleman

Koenig

Lee

. . . Wainess

(2010). Development of game-based training systems: Lessons learned in an inter-disciplinary field in the making. In Cannon-Bowers

Bowers

(Eds.), Serious game design and development: Technologies for training and learning (pp. 47-80). Hershey, PA: IGI Global Information Science Reference.

24.

Ivancic IV

Hesketh

(2000). Learning from errors in a driving simulation: Effects on driving skill and self-confidence. Ergonomics, 43, 1966-1984. doi:10.1080/001401300 50201427

25.

Kanfer

Ackerman

P. L.

(1989). Motivation and cognitive abilities: An integrative/aptitude-treatment interaction approach to skill acquisition. Journal of Applied Psychology, 74, 657-690. Retrieved from http://psycnet.apa.org/doi/10.1037/0021-9010.74.4.657

26.

Kashdan

T. B.

Gallagher

M. W.

Silvia

P. J.

Winterstein

B. P.

Breen

W. E.

Terhar

Steger

M. F.

(2009). The curiosity and exploration inventory-II: Development, factor structure, and psychometrics. Journal of Research in Personality, 43, 987-998. doi:10.1016/j.jrp.2009.04.011

27.

Keith

Wolff

(2015). Encouraging active learning. In Kraiger

Passmore

dos Santos

N. R.

Malvezzi (Eds.), The Wiley Blackwell handbook of training, development, and performance improvement (pp. 92-116). London, England: Wiley-Blackwell.

28.

Koo

D.-M.

Choi

Y.-Y.

(2010). Knowledge search and people with high epistemic curiosity. Computers in Human Behavior, 26, 12-22. doi:10.1016/j.chb.2009.08.013

29.

Landers

R. N.

Reddock

C. M.

(2017). A meta-analytic investigation of objective learner control in web-based instruction. Journal of Business and Psychology, 32, 455-478. doi:10.1007/s10869-016-9452-y

30.

Litman

J. A.

(2008). Interest and deprivation factors of epistemic curiosity. Personality and Individual Differences, 44, 1585-1595. doi:10.1016/j.paid.2008.01.014

31.

Loewenstein

(1994). The psychology of curiosity: A review and reinterpretation. Psychological Bulletin, 116, 75-98. Retrieved from http://psycnet.apa.org/doi/10.1037/0033-2909.116.1.75

32.

MacKinnon

D. P.

Fairchild

A. J.

(2009). Current directions in mediation analysis. Current Directions in Psychological Science, 18, 16-20. doi:10.1111%2Fj.1467-8721.2009.01598.x

33.

Mayer

R. E.

(2009). Research-based solutions to three problems in web-based training. In Kozlowski

S. W. J.

Salas

(Eds.), Learning, training, and development in organizations (pp. 203-228). New York, NY: Routledge.

34.

Preacher

K. J.

Hayes

A. F.

(2008). Asymptotic and resampling strategies for assessing and comparing indirect effects in multiple mediator models. Behavior Research Methods, 40, 879-891. doi:10.3758/BRM.40.3.879

35.

Sadagic

(December, 2010). Deconstructing game-based systems: What are they? What really matters? Tutorial presented at the 2010 Interservice/Industry Training, Simulation and Education Conference, Orlando, FL.

36.

Schmidt

R. A.

Bjork

R. A.

(1992). New conceptualizations of practice: Common principles in three paradigms suggest new concepts for training. Psychological Science, 3, 207-217. doi:10.1111%2Fj.1467-9280.1992.tb00029.x

37.

Schneider

Von Krogh

Jäger

(2013). “What’s coming next?” Epistemic curiosity and lurking behavior in online communities. Computers in Human Behavior, 29, 293-303. doi:10.1016/j.chb.2012.09.008

38.

Sitzmann

(2011). A meta-analytic examination of the instructional effectiveness of computer-based simulation games. Personnel Psychology, 64, 489-528. doi:10.1111/j.1744-6570.2011.01190.x

39.

Sitzmann

Ely

(2011). A meta-analysis of self-regulated learning in work-related training and educational attainment: What we know and where we need to go. Psychological Bulletin, 137, 421-442. Retrieved from http://psycnet.apa.org/doi/10.1037/a0022777

40.

Sitzmann

Kraiger

Stewart

Wisher

(2006). The comparative effectiveness of web-based and classroom instruction: A meta-analysis. Personnel Psychology, 59, 623-664. doi:10.1111/j.1744-6570.2006.00049.x

41.

Soderstrom

N. C.

Bjork

R. A.

(2015). Learning versus performance: An integrative review. Perspectives on Psychological Science, 10, 176-199. doi:10.1177%2F1745691615569000

42.

Wilson

K. A.

Bedwell

W. L.

Lazzara

E. H.

Salas

Burke

C. S.

Estock

J. L.

. . . Conkey

(2009). Relationships between game attributes and learning outcomes: Review and research proposals. Simulation & Gaming, 40, 217-266. doi:10.1177%2F1046878108321866

43.

Wood

Bruner

Ross

(1976). The role of tutoring in problem solving. Journal of Child Psychology and Psychiatry and Allied Disciplines, 17, 89-100. doi:10.1111/j.1469-7610.1976.tb00381.x

44.

Wood

R. E.

(1986). Task complexity: Definition of the construct. Organizational Behavior and Human Decision Processes, 37, 60-82. doi:10.1016/0749-5978(86)90044-0