Strategic Resource Use for Learning: A Self-Administered Intervention That Guides Self-Reflection on Effective Resource Use Enhances Academic Performance

Abstract

Many educational policies provide learners with more resources (e.g., new learning activities, study materials, or technologies), but less often do they address whether students are using these resources effectively. We hypothesized that making students more self-reflective about how they should approach their learning with the resources available to them would improve their class performance. We designed a novel Strategic Resource Use intervention that students could self-administer online and tested its effects in two cohorts of a college-level introductory statistics class. Before each exam, students randomly assigned to the treatment condition strategized about which academic resources they would use for studying, why each resource would be useful, and how they would use their resources. Students randomly assigned to the treatment condition reported being more self-reflective about their learning throughout the class, used their resources more effectively, and outperformed students in the control condition by an average of one third of a letter grade in the class.

Keywords

strategic resource use self-regulation psychological intervention learning performance

Educational policies encourage the provision of ample, high-quality academic resources and support for students, be it investing more money per student, introducing new class activities, or improving technology in schools (e.g., Barkley, Cross, & Major, 2014; Cuban & Cuban, 2009; Dunlosky, Rawson, Marsh, Nathan, & Willingham, 2013; Hanushek, 1997). All this makes sense from an educational perspective—after all, the effectiveness of many student-centered interventions relies on the availability of rich learning resources (Cohen, Garcia, Apfel, & Master, 2006; Paunesku et al., 2015).

However, providing students with all these resources hinges on the assumption that they know how to select and use their resources wisely. Yet empirical research suggests that many students do not tend to proactively or strategically self-regulate their learning on their own (Zimmerman, 2011; Zimmerman & Martinez-Pons, 1988). Oftentimes, many of them are passive consumers of information and lack tactical awareness when studying. This may substantially limit what students achieve in their classes, rather than allowing them to perform to their potential.

How can we help people to regulate their own learning more effectively? Effectual use of metacognitive self-regulation, which involves the proactive and tactical direction of mental processes toward one’s goals, has been shown to predict better learning, motivation, and academic performance among learners (Pintrich & De Groot, 1990; Pintrich, Smith, Garcia, & McKeachie, 1993). This mental process can be applied to students’ management of the resources available to them, such as allocating study time effectively, reviewing their class notes before an exam, and seeking help when necessary (Boekaerts, 1999; Pintrich, Smith, Garcia, & McKeachie, 1991). Previous research has found that students who manage their resources more effectively tend to perform better in their classes (Karabenick, 2003; Pintrich et al., 1993). Although greater engagement in self-reported resource-management behaviors has been associated with better academic performance in the literature, an important question remains: Would an experimental intervention that specifically addresses strategies for resource use causally contribute to performance?

In the literature, a number of educational interventions have focused on teaching students a variety of self-regulatory skills at the same time, including various learning techniques, setting goals, organizing their class material, and reflecting on their study approaches (Bembenutty, 2013; Diamond, Barnett, Thomas, & Munro, 2007; Pape, Bell, & Yetkin-Ozdemir, 2013; Weinstein & Acee, 2013). These are often instructor-facilitated, multifaceted, multisession practices that target a host of skills. However, they do not focus specifically on improving students’ strategic use of resources for learning. These intervention designs also tend to be less amenable to large-scale distribution to students in the absence of instructor facilitation.

For these reasons, we designed a novel, self-administered intervention targeting learners’ strategic use of their resources for learning. Our Strategic Resource Use intervention prompts students to think deliberately about how to approach their learning effectively with the resources available to them (e.g., their lecture notes, homework problems, and instructors’ office hours). This involves strategizing about how to approach their learning effectively, deliberately choosing the specific resources that would foster their mastery of the learning content, and then planning how they would use these resources to study the class material. Our intervention design combined theory from previous academic self-regulation interventions (e.g., Bembenutty, 2013; Pape et al., 2013; Weinstein & Acee, 2013) with the precision and scalability of brief social-psychological interventions (Walton, 2014; Yeager & Walton, 2011). Students were able to take our online intervention on their own.

We tested this intervention in two randomized controlled trials among college students. Our goal was to investigate whether this key component of self-regulated learning—strategically reflecting on how to use one’s resources effectively for learning—causally contributes to students’ performance and, if so, how it does. We expected that, relative to students in the control condition, students in the intervention treatment condition would perform better in the class by practicing greater self-reflection about their learning, and thereby use their resources more effectively while studying. Because we are not aware of any evidence to suggest that these behaviors are more or less commonly practiced among students of different demographic backgrounds and performance levels, we had no directional hypotheses about whether this intervention would preferentially advantage one particular group of students over another.

Method

We conducted two randomized field experiments in a large Midwestern public university. The participants were undergraduate students enrolled in two separate cohorts of a spring-semester introductory statistics class. Performance in introductory statistics is central to many students’ college careers: It is a prerequisite course for a number of majors in the social sciences, natural sciences, premedicine track, and business school. It also satisfies a quantitative skill requirement for undergraduates. All students had the same instructor, who was blind to individual students’ randomly assigned condition, thus con-trolling for instructional style and content. The two experiments were almost identical, but the second had minor improvements in wording and additional survey measures to test for mechanisms.

We conducted an a priori power analysis using previous cohorts’ performance in the class to estimate the standard deviation in the planned sample. We used a significance level of .05 (i.e., a false-positive rate of .05) and a power criterion of above .80 (1 – β > .80, where β is the false-negative rate) to plot a power-analysis graph (see Fig. S1 in the Supplemental Material available online). We estimated that sample sizes of approximately 200, 100, and 50 would be sufficient to detect differences between conditions of 1.5, 2.0, and 3.0 percentage points, respectively, in students’ final course performance. The instructor projected an enrollment of 200 students per class. Thus, even with a conservative forecast of a 50% participation rate, we were confident that each of our planned studies would be adequately powered to detect an effect size of at least 2.0 percentage points.

Participants in the two class cohorts had similar demographic backgrounds (Study 1: mean grade point average, or GPA = 3.11; 39.9% male, 57.9% female, 2.2% gender unknown; 63.6% White, 6.4% African American, 20.2% Asian, 2.9% Hispanic; Study 2: mean GPA = 3.17; 32.9% male, 62.3% female, 0.5% other gender, 4.3% gender unknown; 57.6% White, 10.1% African American, 20.2% Asian, 2.0% Hispanic). All students in the class were given the opportunity to participate in our surveys for homework extra credit points before and after each of their two exams. Table S1 in the Supplemental Material shows the breakdown of participation for each survey in each cohort. Our main outcome measures of student performance were students’ final course grades and their performance on their two class exams.

Individual students were randomly assigned to the intervention treatment condition or the control condition. In Study 1, there were 84 students in the treatment group and 87 in the control group. In Study 2, there were 95 in the treatment group and 95 in the control group. Random assignment occurred automatically when students started their first online preexam survey. Each survey took about 10 to 15 min to complete. For any subsequent surveys, students were always in the condition to which they had initially been assigned.

We administered the preexam surveys, which contained either the treatment or control messages, about 10 days before each of the class exams, and closed the surveys about 7 days before the exam date. In consultation with the course instructor, we deemed this timing to be the most likely to affect students’ exam preparation because it gave students enough time to study for the exam, but it also was not so far in advance as to seem irrelevant. The majority of students in our two studies took a preexam survey before each of their two exams (Study 1: 73.0%; Study 2: 69.1%; for full details of response rates, see Table S1 in the Supplemental Material).

Postexam surveys were distributed immediately after students received their exam grades in class and were open for 2 to 4 days afterward. The majority of students in each study took part in the two available postexam surveys after each of their class exams (Study 1: 68.0%; Study 2: 71.0%). The postexam surveys primarily measured which resources students had used to study for their exams, how useful they had found each resource, and the degree to which they had self-reflected on their learning throughout the class. These postexam survey measures were identical across all students regardless of condition. Our pre- and postexam surveys comprised multiple questions, including measures other than those reported here, for the purposes of research that is not the focus of this article.

Preexam treatment and control messages

At the start of each preexam survey, all students were reminded that their upcoming exam was worth 100 points. They were asked to write down their desired grade on the upcoming exam and to answer three survey questions about how motivated they were to get that grade, how important it was to them to achieve that grade, and how confident they were in achieving that grade.

After this, students in the control condition received a regular exam reminder that their exam was coming up in a week and that they should start preparing for it.

Students in the treatment condition received this same exam reminder and then a brief Strategic Resource Use exercise. In a nutshell, the Strategic Resource Use exercise prompted students to deliberately consider the upcoming exam format, which resources would facilitate their studying, why each resource would be useful, and how they were planning to use each resource. In the first part of the intervention, students in the treatment condition read a message telling them that successful high achievers use resources strategically when preparing for exams. After considering the types of questions that they expected to be tested on in their upcoming exam, students then indicated which class resources they wanted to use (from a list of 15 available) to maximize the effectiveness of their learning. The checklist of class resources included lecture notes, practice exam questions, textbook readings, instructor office hours, peer discussions, private tutoring, and many others (see Appendix S1 in the Supplemental Material). We collaborated with the course instructor to design this comprehensive class resource checklist. When students actively choose their learning resources while anticipating the kinds of questions that they will get on the upcoming exam, they think strategically about which resources they should channel their efforts toward in order to make their learning effective.

After filling out the checklist, students in the treatment condition then answered two open-ended response questions. First, they described why they thought each chosen resource would be useful for their exam preparation. This elaborative process is important because it helps students articulate exactly why each resource will contribute to their learning and primes them to think about how they would make use of the resource effectively. Second, students described specific, realistic, and concrete plans for when, where, and how they would study with the resources they had chosen. Forming such goal-directed plans for action makes it more likely that students will translate their resource-use intentions into actual behavior (Gollwitzer, 1999; Gollwitzer & Brandstätter, 1997). After all, strategies are important, but they would be no better than castles in the air if not executed. We supplemented our instructions for these two open-ended questions with concrete examples to guide students through their explanations.

This Strategic Resource Use exercise guided students to think strategically about how to approach their learning by considering which learning resources to use, why each resource would be useful, when to schedule studying, where to study, and how (or what steps to take) to study effectively. The first two of these questions constituted the strategic component, whereas the latter three formed the planning component of our Strategic Re-source Use intervention—thus addressing both tactical and implemental parts of learning.

Assessing causal mechanisms

The goal of the Strategic Resource Use intervention was to have students reflect on how they would learn most effectively with the resources available in their environment. We predicted that this strategic reflection would make students’ resource use more effective during learning and, therefore, help them perform better in the class.

Self-reflections on how to learn effectively

At the end of the class in Study 2, we administered an eight-item Self-Reflection on Learning scale. This scale assessed the extent to which students adjusted their studying to the class, thought about how effectively they were learning, changed the way they were studying when their ap-proaches were ineffective, and reflected on their performance. It included questions such as “I actively tried to find out what was expected of me to get good grades in this class,” “As I studied for the class, I kept monitoring whether or not the way I was studying was effective,” and “After each exam, I thought about how my performance in class was a result of how I had been doing things” (see Appendix S2 in the Supplemental Material; α = .80). These measures were adapted from the metacognitive self-regulation subscale of the Motivated Strategies for Learning Questionnaire from Pintrich et al. (1991).

Self-reported effectiveness of resource use

In each postexam survey in both conditions, we asked students to indicate which class resources they had actually used in their exam preparation. The students chose from a comprehensive list of resources—the same list that students in the treatment condition saw in their preexam surveys. Students rated how useful they had found each resource that they had used in their exam preparation (1 = not useful, 5 = extremely useful). These measures were adapted from the Resource Questionnaire in Brown, Doughty, Draper, Henderson, and McAteer (1996). We averaged the students’ usefulness ratings across their two exams to obtain a proxy of how effectively students were using their resources for learning in the class.

Secondary emotional and motivational effects

To examine other psychological processes that may also have benefited from our intervention, we measured students’ preexam negative affect, their perceived control over their performance, the extent to which they had planned their studying ahead of time, and how well they had kept to their plans (Gollwitzer & Brandstätter, 1997; Locke & Latham, 1990; Pham & Taylor, 1999). Although these processes were not the primary goals of our intervention, they are also plausible effects of the intervention that are relevant both to the psychology of effective learning and to students’ performance.

On their preexam surveys, students rated the negative affect that they were experiencing with regard to their upcoming exams. For example, they reported how anxious, nervous, fearful, and stressed they were about their upcoming exam on a scale from 1 (not at all) to 7 (extremely). Our negative-affect measure was adapted from Smith and Ellsworth (1987). We averaged students’ responses on the negative-affect questions to calculate a composite score of negative affect for each exam (αs ranged from .82 to .91). In their postexam surveys, students’ rated the degree of control that they perceived they had over their exam performance. For example, they rated how much they agreed with the statements “I believe that how well I do in this class is mostly under my control” and “My exam grades are affected by the way I choose to study for this course,” on a scale from 1 (strongly disagree) to 6 (strongly agree). In our postexam surveys, we asked students to rate how much planning they had done ahead of time (e.g., “How much planning did you do to prepare yourself for the Exam 2?” 1 = none, 7 = a great deal) and how well they had followed through with their plans (e.g., “To be honest, how well did you follow through with your plans?” 1 = not at all, 7 = extremely well).

Students’ exam and class performance data

At the end of the class, we obtained students’ class performance data from the instructor, along with their demographic and prior performance data from the registrar.

Results

Across our two studies, there were no statistically significant differences between conditions in students’ prior performance and preexam motivation levels (see Table S2 in the Supplemental Material for descriptive statistics for these variables). Regression analyses showed that there were no statistically significant differences between conditions in students’ high school GPAs (Study 1: p = .939; Study 2: p = .393) and college GPAs before the intervention (Study 1: p = .577; Study 2: p = .557). Across both cohorts, there were also no statistically significant differences between conditions in students’ desired grades on each of their two exams (all ps > .160), their motivation to achieve their desired grades (all ps > .267), the personal importance of these desired grades (all ps > .181), and their confidence in attaining their desired grades (all ps > .161).

Treatment effects

We conducted our analyses using three main approaches: First, we conducted an intent-to-treat analysis (Gupta, 2011; Wertz, 1995) by comparing the performance of all students randomly assigned to a condition, regardless of how many surveys they took. This avoided the self-selection bias potentially introduced by analyzing only students who completed all the surveys in either condition. Second, we compared the performance of students in the treatment and control conditions who took a survey before each of their two exams (i.e., all treatment preexam surveys vs. all control preexam surveys). Third, we considered whether treatment dosage (i.e., the number of preexam surveys taken) resulted in differential benefits among those treated.

In both studies, our intent-to-treat analyses found that students in the treatment condition outperformed those in the control condition on their final course grades by an average of one third of a letter grade. In Study 1, students in the treatment condition performed an average of 3.64 percentage points (95% confidence interval, or CI = [0.28, 7.00]) higher on their final course grades than students in the control condition (treatment condition: M = 83.90%; control condition: M = 80.26%), Cohen’s d = 0.33, Welch’s two-sample¹ t(162) = 2.14, p = .034. This performance advantage was replicated in Study 2, where students in the treatment condition scored an average of 4.21 percentage points (95% CI = [0.97, 7.44]) higher in the class than did the students in the control condition (treatment condition: M = 83.44%; control condition: M = 79.23%), d = 0.37, t(183) = 2.56, p = .011. In both studies, performance differences between conditions were significant on every exam except Exam 1 in Study 1; in that case, the difference was in the same predicted direction but not statistically significant at the .05 level (Fig. 1).

Fig. 1.

Students’ average performances on Exam 1, Exam 2, and the entire course, presented separately for the control and treatment conditions. Error bars represent 95% confidence intervals of the means for each condition.

We found the same results when we compared the final course performances of students who received the full intervention (i.e., a survey before each of their two exams) against the performance of students in the control condition who received the same number of control messages. In both studies, the average between-groups difference in final course performance was one third of a letter grade. Compared with students in the control condition, students in the treatment condition attained final course grades that were, on average, 3.45 percentage points higher (95% CI = [0.26, 6.65]) in Study 1 (treatment condition: M = 86.35%; control condition: M = 82.90%), d = 0.38, t(127) = 2.14, p = .034, and 4.65 percentage points higher (95% CI = [1.45, 7.85]) in Study 2 (treatment condition: M = 85.77%; control condition: M = 81.12%), d = 0.47, t(139) = 2.87, p = .005. Significant performance differences were also observed on students’ exams, with the exception of Exam 1 in Study 1; the results for that exam were in the predicted direction but not statistically significant (see Fig. S1 in the Supplemental Material).

We found a treatment-dosage effect among students who had taken the intervention. The majority of treated students in each study took the treatment twice rather than once (Study 1: 75.9% twice vs. 24.1% once; Study 2: 70.5% twice vs. 29.5% once). Students in the treatment condition who took the intervention twice (rather than once) scored significantly higher on their final course grades in both Study 1 (mean difference = 10.16 percentage points, 95% CI = [5.30, 15.03]), d = 1.11, t(32) = 4.26, p < .001, and Study 2 (mean difference = 7.90 percentage points, 95% CI = [2.77, 13.04]), d = 0.81, t(38) = 3.12, p = .003.

To rule out the possibility that these performance differences were driven primarily by differences in students’ motivation, we tested whether there were significant differences in self-reported motivation between the students who took the treatment survey before only one exam and the students who took the treatment survey before each of their two exams. These motivation variables were assessed in our preexam surveys: students’ desired grades on each exam, their self-reported motivation to achieve their desired grades, the personal importance of their grades in the course, and their confidence in attaining their desired grades. There were no statistically significant between-groups differences in any of these motivation variables (all ps > .05). In addition, we found that the effect of the number of treatment dosages received remained statistically significant even when we controlled for students’ GPA at the beginning of the class in Study 1, b = 6.24, 95% CI = [1.85, 10.63], SE = 2.21, t(81) = 2.83, p = .006, and in Study 2, b = 6.17, 95% CI = [2.35, 9.99], SE = 1.92, t(90) = 3.21, p = .002. GPA is a performance index that is often associated with students’ motivation to learn and do well academically.

We repeated our three analytical approaches by excluding the homework extra-credit points that students attained for participating in our surveys. We obtained the same between-conditions differences in students’ exam and final course performances in all three analyses.

In summary, we concluded that students benefited from doing the intervention exercise compared with getting a regular exam reminder, and that greater exposure to the intervention was associated with higher performance in the class.

Treatment homogeneity

We found that the Strategic Resource Use intervention was academically advantageous for different types of college students across the demographic and performance variables that we had collected (i.e., gender, race, class standing, and preintervention performance levels). Moderation analyses showed that there were no statistically significant differences in the treatment effect between males and females, among students of different racial groups, among students of different class standings, and between low- and high-performing students in both cohorts (all interaction ps > .188). Model comparisons further reinforced these results: We pooled data across both studies and compared one model specifying all the interactions between condition and individual difference variables (gender, race, class standing, preintervention GPA, and cohort) with another model without the interactions (i.e., with only the main effects). Results from the two models were not statistically different (p = .369), which implied that the more parsimonious model without interactions was sufficient to explain the data. These results support our inference that the intervention did not provide greater benefit to one kind of student compared with another.

Causal process

We tested our prediction that, compared with the business-as-usual control message, our Strategic Resource Use intervention would affect students’ performance through greater self-reflection on their learning and more effective resource-use behaviors, in that order. Aggregating the available data in our two studies, we first ran regression analyses to test for the predicted relationships among our variables. Students who had received the treatment reported practicing significantly more self-reflection on their learning in class, b = 0.21, 95% CI = [0.03, 0.38], SE = 0.09, t(163) = 2.38, p = .019. The more students thought strategically about how to effectively approach their learning, the more useful they found the resources they had used for studying, b = 0.22, 95% CI = [0.08, 0.36], SE = 0.07, t(145) = 3.12, p = .002, and this predicted how well they performed in the class, b = 2.71, 95% CI = [0.44, 4.98], SE = 1.15, t(265) = 2.35, p = .019. There was no direct effect of condition on students’ resource-use behaviors (p = .418).

The treatment effect was not driven by students in the treatment condition using a greater number of resources than students in the control condition. If anything, students in the treatment condition used fewer learning resources on average (treatment: M = 11.76; control: M = 13.42; difference between means = 1.66, 95% CI = [0.44, 2.88]), d = 0.33, t(261) = 2.67, p = .008. This result suggests that the intervention made students use their resources more effectively—by getting them to self-reflect more about how they were approaching their learning, rather than just getting them to use a greater number of resources.

We tested our serial mediation model, aggregating across the data in both studies, using Mplus (Version 7.4; Muthén & Muthén, 2015) with 10,000 bias-corrected bootstrap resamples to estimate the indirect effect. This bias-corrected bootstrap method is preferable to the Sobel test because it corrects for any nonnormality in the distributions of the variables and their product term when computing the indirect effect (MacKinnon, Lockwood, & Williams, 2004). Our serial mediation model is represented in Figure 2.

Fig. 2.

Serial mediation model showing the effect of the treatment condition on students’ final course performance, mediated by their self-reflections on learning and reported effectiveness of their resource use. Condition is coded as follows: control = 0, treatment = 1. CI = confidence interval. Residual error terms are not included in this figure.

There was a significant indirect effect through students’ self-reflection about their learning and the reported effectiveness of their resource use (in that order), which explained how our intervention affected students’ performance, indirect effect b = 0.20, bias-corrected bootstrap 95% CI = [0.02, 0.69]. Goodness-of-fit statistics showed that our predicted model was an excellent fit to the data, χ²(2) = 1.02, p = .601, root-mean-square error of approximation (RMSEA) = 0.00, comparative fit index (CFI) = 1.00, standardized root-mean-square residual (SRMR) = 0.020 (Hu & Bentler, 1999).

We compared our predicted model with a more complex, saturated model that included two additional pathways—one with students’ condition predicting their reported resource-use effectiveness and another using students’ self-reflections about learning to predict their final grades. A χ² difference test showed that the more complex, saturated model did not do a better job of explaining the data, Δχ² = 1.02, Δdf = 2, p = .601. Therefore, according to Occam’s razor and the parsimony principle in structural equation modeling (Kelloway, 1998; Kline, 2016), our simpler predicted model is preferable to the more complex, saturated model. Moreover, neither of the two additional pathways in the saturated model was statistically significant (both ps > .290), further supporting our rationale for excluding them. We also ruled out alternative models that did not fit our data well, such as a serial mediation model with the mediators in the opposite order (students’ reported resource-use effectiveness preceding their self-reflections about learning), and a parallel mediation model with students’ reported resource-use effectiveness and their self-reflections about learning as parallel mediators of the treatment effect. Goodness-of-fit statistics for these alternative models are presented in Table S3 in the Supplemental Material.

Although a single mediator model revealed a weak indirect effect of students’ self-reflections about learning, b = 0.02, 95% CI = [0.001, 1.65], our predicted serial mediation model is a more theoretically accurate representation of the process that the intervention targeted. The intervention was designed to guide students to strategize how they could learn effectively with the resources that they had and thereby change how effectively they used their resources to study. Thus, of these plausible process models, the model proposed in Figure 2 best captured the causal process by which our Strategic Resource Use intervention benefited students’ performance.

Exam-focused resource selection and follow-through in the treatment condition

To further understand how the intervention translated into benefits for students in the treatment condition, we asked the following questions: What were the performance benefits of using resources that students had strategically selected ahead of time rather than those that they had not selected in advance but ended up using? How much did the benefits of planning resource use depend on actually following through with these plans? To address these questions, we matched the resources that every student had selected and planned to use before their exams with their postexam resource-use responses. We aggregated across all exams in both studies and used mixed-effects models with exam number, individual student, and cohort included as random effects.

Importance of strategic selection in resource use

We tested the contribution of strategic selection to students’ grades by comparing the degree to which treatment group students’ exam performance was explained by the number of resources that they had strategically selected and used versus the number of resources they had not selected a priori but ended up using. Both of these variables were added as fixed-effects predictors in our mixed-effects model. Only the number of resources that students had strategically selected in advance and used was positively related to their exam performance, b = 0.77, 95% CI = [0.33, 1.21], SE = 0.22, t(241) = 3.48, p < .001; the number of resources that they used but had not selected in the intervention ahead of time did not significantly predict their exam performance (p = .382). Thus, within the same model, the resources that students had strategically selected through our intervention exercise predicted students’ exam performance, but not those that they used without such strategic forethought.

Follow-through with plans

Were students’ exam performances influenced by their degree of follow-through with their resource-use plans? Note that individual students differed in the total number of resources that they planned to use, as well as in the number of those planned resources that they ended up using. We ran a mixed-effects model predicting students’ exam performance with three fixed-effects predictors: the total number of resources that treatment-condition students had planned to use, the number of those resources that they actually used, and the interaction between these two regressors. Note that the total number of resources that students planned to use included the number of resources that they had planned to use and actually did use, as well as the number of resources that they had planned to use but did not end up using. The interaction allowed us to model the effect of follow-through across different numbers of resources that students had planned to use.

There was a significant negative interaction between the total number of resources that treatment-condition students had planned to use and the number of those resources that they actually used, b = −0.13, 95% CI = [−0.24, −0.02], SE = 0.05, t(219) = −2.35, p = .020. In addition, the number of planned resources that students actually used had a significant simple effect on their exam performance, b = 1.82, 95% CI = [0.71, 2.94], SE = 0.56, t(217) = 3.25, p = .001; however, there was no significant simple effect of the total number of resources that they had planned to use (p = .367). Our results suggest that merely strategically planning which resources would be useful did not, by itself, automatically boost students’ grades—improvements in performance also required putting these strategic plans into practice (Gollwitzer, 1999). In addition, the significant interaction indicates that students’ use of resources conferred decreasing marginal benefit as the total number of resources that students planned to use increased (i.e., planning to use one additional resource conferred greater benefit when it was the 4th resource than when it was the 14th). In other words, planning to use more resources conferred performance benefits to the extent that (a) individuals followed through on using those resources and (b) the scope of planning stayed within reasonably practical bounds rather than being indiscriminate.

Additional emotional and motivational benefits

We examined additional consequences of participation in the intervention, including its effects on students’ preexam negative affect, students’ perceived control over their exam performance, students’ self-reported preparatory preexam planning, and the degree to which students followed through with their plans. We aggregated the data across all exams in both studies and used mixed-effects models to test for differences on each of these variables by condition, including exam number, individual student, and cohort as random effects. Relative to students in the control condition, students in the treatment condition experienced lower negative affect toward their upcoming exams, b = −0.43, 95% CI = [−0.73, −0.14], SE = 0.15, t(353) = −2.88, p = .004, and perceived greater control over their own performance in the class (although this effect was marginally significant), b = 0.16, 95% CI = [−0.01, 0.33], SE = 0.09, t(347) = 1.86, p = .064. Neither students’ subjective degree of prior planning (p = .492) nor the degree to which they felt that they had followed through with their plans (p = .381) significantly differed between conditions.

Students’ open-ended responses

To understand which psychological elements of the intervention predicted students’ class performance, we coded and analyzed students’ open-ended responses about why each resource they had chosen would be useful to them and their exam-preparation plans. Note that this was done only for students in the treatment condition who had answered these open-ended questions; students in the control condition were not exposed to these questions. Examples of students’ open-ended responses are provided in Appendix S3 of the Supplemental Material.

Students’ explanations about why their chosen resources would be useful were coded into five main psychological processes that are consistent with self-regulation theory: (a) explicit consideration of the exam format, (b) leveraging multiple resources in a synergistic manner, (c) fostering learning and understanding of the class material, (d) illustrating an understanding of personal strengths and weaknesses, and (e) recognizing that learning is a social process (as opposed to an individual’s isolated endeavor). Two independent coders categorized students’ open-ended responses into these five categories (interrater κ ranged from .88 to 1.00), and any disagreements were resolved through discussion. Students’ plans were similarly coded into the following three planning categories: when, where, and how the resources were going to be used (interrater κ ranged from .94 to 1.00). We created a measure of the extent to which students engaged in each category of psychological processes across their two exams (0 = the student did not mention it at all, 1 = the student wrote about it only before one exam, and 2 = the student wrote about it before both exams).

We regressed the final course performance of students in the treatment condition on this measure of engagement separately for each of these eight categories (for results, see Table 1). Four psychological elements of the intervention significantly and consistently related to students’ final course performance across our two studies: explicitly tailoring one’s choice of resources to the exam questions anticipated, focusing resource use on building better learning and understanding of the content, planning when to use the resources, and planning how to use their resources to study (Table 1). For example, as students chose their resources, those in the treatment condition who were more engaged in reflecting on what was expected of them on their exams tended to perform better in the class. These results emphasize that strategic self-reflection and planning are both important psychological processes that are activated by the intervention and contribute to the benefits learners derive from it.

Table 1.

Results From Regression Analyses Testing the Extent to Which Engagement in Each Psychological Process Was Associated With Students’ Final Course Grades

Study and psychological process	b	95% CI	t	p
Self-regulation categories
Study 1
Consideration of exam format	3.76	[1.07, 6.45]	2.78	.007
Synergistic use with other resources	1.36	[−1.57, 4.29]	0.92	.359
Learning and understanding the class material	4.98	[1.04, 8.92]	2.51	.014
Understanding personal strengths and weaknesses	1.38	[−1.55, 4.30]	0.94	.352
Learning as a social process	0.16	[−2.08, 2.41]	0.14	.886
Study 2
Consideration of exam format	3.22	[0.62, 5.83]	2.46	.016
Synergistic use with other resources	1.79	[−1.03, 4.61]	1.26	.211
Learning and understanding the class material	8.18	[4.85, 11.50]	4.88	< .001
Understanding personal strengths and weaknesses	2.44	[−1.52, 6.40]	1.22	.224
Learning as a social process	−0.41	[−2.57, 1.74]	−0.38	.706
Planning categories
Study 1
When	4.33	[0.79, 7.87]	2.43	.017
Where	3.16	[0.31, 6.00]	2.20	.030
How	5.67	[2.50, 8.83]	3.56	< .001
Study 2
When	4.97	[1.66, 8.29]	2.98	.004
Where	−0.19	[−3.40, 3.01]	−0.12	.906
How	6.71	[3.53, 9.89]	4.19	< .001

Note: CI = confidence interval. The degrees of freedom for all Welch’s two-sample t tests was 85 in Study 1 and 93 in Study 2.

General Discussion

Goal achievement is not always about having more resources; it is also about how effectively people use their resources. Regardless of how richly we endow students with study materials, support, and environments conducive to learning, many of these resources will be wasted on students who do not thoughtfully use them in a productive manner. Encouraging students to be strategic in their use of class resources to master the class material enables them to leverage more of their potential during performance.

We showed that a brief, self-administered intervention that guided students to make strategic use of their available resources had a significant impact on their grades. Across two studies, our intervention produced a difference of one third of a letter grade, on average, in a college class that is a prerequisite to many college majors. Our intervention promoted students’ performance by fostering greater self-reflection about how best to approach their learning in class, which directed more effective resource use while studying. In addition to performing better, students in the treatment condition also reaped other psychological benefits: They experienced lower negative affect toward their upcoming exams and perceived greater control over their performance, relative to students in the control condition. These secondary benefits add to the value in offering this brief, online intervention to students.

It was unlikely that the between-conditions performance differences that we observed were due simply to increased awareness of the resources available. In this class, the nature of instruction involved multiple reminders about what students should be doing each week, including the resources they could use for learning. For example, all students, regardless of condition, received a “Get Things Done” list (see Appendix S4 in the Supplemental Material) in their e-mail inbox every week. It is therefore likely that students in the treatment condition benefited from greater self-reflection, and thereby more effective resource use, rather than simply receiving more reminders of the resources available.

Our results suggest that the process that the intervention sets in motion goes beyond just planning (e.g., Gollwitzer & Brandstätter, 1997; Kirschenbaum, Humphrey, & Malett, 1981). It triggers general self-reflection about how effectively students are approaching their learning, such as thinking about how productive their learning approaches are and reflecting on how they have been learning. This self-reflection directs learners’ efforts, which makes their resource use more effective during learning, rather than just strengthening the likelihood that they will enact their resource-use intentions. Students’ open-ended responses also showed that it was more than planning the use of one’s resources that related to better class performance: The self-regulatory processes of selecting resources strategically in light of the anticipated exam format, and doing so in a manner that would maximize content mastery, also significantly contributed to students’ performance.

In our studies, the benefits of this Strategic Resource Use intervention were not limited to students of a particular demographic background or performance level. Although this relatively homogeneous benefit may seem somewhat surprising in light of other interventions that specifically target a particular group of students (e.g., Cohen et al., 2006; Hulleman & Harackiewicz, 2009; Walton & Cohen, 2011), our intervention seemed to foster a general approach to learning that many students were generally either unaware of or not practicing optimally.

Because it changes the way that students strategize about how to use existing resources, our intervention may provide the greatest benefit for motivated students in resource-rich learning environments. In learning environments with scarce resources, it may be more pertinent to ensure that a basic repertoire of resources is available for learners to use, even before confronting the problem of how effectively they are making use of what is available. But in the many learning contexts with an already-existing assortment of resources, it is valuable for students to self-reflect about how they should effectively use their resources to learn, rather than doing so inefficiently. Moreover, this intervention may confer performance benefits to students to the extent that they are not already practicing these skills effectively on their own. Its benefits may not be as large for students who are already very self-reflective about how they use their resources for learning. Future research should continue to address whether there are conditions under which the intervention leads to more versus less benefits for different kinds of learners.

We showed that the intervention brought about other psychological benefits in addition to improvements in class grades. However, the intervention may have had other effects on students that we did not measure. For instance, it may have influenced how much time students spent studying. Past literature suggests that the relationship between academic performance and the time students spend studying is tenuous (Plant, Ericsson, Hill, & Asberg, 2005; Schuman, Walsh, Olson, & Etheridge, 1985). This relation tends to be qualified by how effectively they spend their study time (Plant et al., 2005; Schuman et al., 1985). Here, we focused on measuring how effectively students reported using their resources for learning, rather than just their sheer amount of studying. Nonetheless, it is plausible that the intervention may increase students’ study duration or even the way they distribute their study time (Cepeda, Pashler, Vul, Wixted, & Rohrer, 2006)—questions that future research could look into testing conclusively.

In this research, we faced the challenge of assessing students’ self-reflections about their learning and the effectiveness of their resource use without affecting the learning process as it was taking place. For instance, asking students to evaluate the effectiveness of their resource use as they are studying might influence them to change their own study practices in the moment, thereby rendering such measurements invalid. This also potentially undermines the intervention itself. To avoid these complications, we chose to measure students’ retrospective self-reports about these learning behaviors. However, this self-report approach has inherent weaknesses, too—such as potential discrepancies between students’ reports and their actual behaviors. Assessing students’ self-reflections about their learning and their resource-use effectiveness in the moment, without influencing the learning process itself, should be a goal of future research. This may be possible with less invasive measurement methods than those used in the current study.

Our Strategic Resource Use intervention combines psychologically precise design with an easily scalable self-administration format (e.g., Paunesku et al., 2015; Walton, 2014). We made the design self-explanatory, concise, and easily accessible via the Internet so that learners can autonomously initiate the intervention to improve the way they approach their learning. Thus, the intervention is amenable to convenient, large-scale application in schools, and even potentially for online learners taking massive open online courses.

Beyond education, there are many other situations in real life in which people engage in goal pursuit ineffectively, especially when they are not aware of how unproductive their strategies are or how to make the most of the resources around them. Encouraging self-reflection in people about how to approach their goals strategically with the resources that are available to them can go a long way in helping them to achieve their goals. Through psychologically precise, self-administered interventions, such as the Strategic Resource Use intervention, people can be empowered to take control of their goal pursuit in a strategic and effective manner.

Footnotes

Acknowledgements

We thank the following people, whose feedback and assistance have made this project possible: Stuart A. Karabenick, Jared Tritz, Carol S. Dweck, Gregory M. Walton, Thomas N. Robinson, Teck Sheng Tan, Michelle S. Y. Lee, Patty Kuo, Amiram Vinokur, and all the students who supported our research.

Action Editor

D. Stephen Lindsay served as action editor for this article.

Declaration of Conflicting Interests

The authors declared that they had no conflicts of interest with respect to their authorship or the publication of this article.

Supplemental Material

Additional supporting information can be found at

Open Practices

We support open, collaborative science conducted in a mutually respectful and responsible manner. We have posted our data, code, and materials at http://osf.io/5fr3d. These materials are accessible by researchers and practitioners upon request. To request access, please complete and submit the request form at http://www.patriciachen.com/requestmaterials. We want to work with researchers and practitioners to improve student learning and are happy to answer any questions about the materials. The complete Open Practices Disclosure for this article can be found at .

Notes

References

Barkley

E. F.

Cross

K. P.

Major

C. H.

(2014). Collaborative learning techniques: A handbook for college faculty. San Francisco, CA: John Wiley & Sons.

Bembenutty

(2013). The triumph of homework completion through a learning academy of self-regulation. In Bembenutty

Cleary

T. J.

Kitsantas

(Eds.), Applications of self-regulated learning across diverse disciplines (pp. 153–196). Charlotte, NC: Information Age.

Boekaerts

(1999). Self-regulated learning: Where we are today. International Journal of Educational Research, 31, 445–457.

Brown

M. I.

Doughty

G. F.

Draper

S. W.

Henderson

F. P.

McAteer

(1996). Measuring learning resource use. Computers & Education, 27, 103–113.

Cepeda

N. J.

Pashler

Vul

Wixted

J. T.

Rohrer

(2006). Distributed practice in verbal recall tasks: A review and quantitative synthesis. Psychological Bulletin, 132, 354–380.

Cohen

G. L.

Garcia

Apfel

Master

(2006). Reducing the racial achievement gap: A social-psychological intervention. Science, 313, 1307–1310.

Cuban

(2009). Oversold and underused: Computers in the classroom. Cambridge, MA: Harvard University Press.

Diamond

Barnett

W. S.

Thomas

Munro

(2007). Preschool program improves cognitive control. Science, 318, 1387–1388.

Dunlosky

Rawson

K. A.

Marsh

E. J.

Nathan

M. J.

Willingham

D. T.

(2013). Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the Public Interest, 14, 4–58.

10.

Gollwitzer

P. M.

(1999). Implementation intentions: Strong effects of simple plans. American Psychologist, 54, 493–503.

11.

Gollwitzer

P. M.

Brandstätter

(1997). Implementation intentions and effective goal pursuit. Journal of Personality and Social Psychology, 73, 186–199.

12.

Gupta

S. K.

(2011). Intention-to-treat concept: A review. Perspectives in Clinical Research, 2, 109–112.

13.

Hanushek

E. A.

(1997). Assessing the effects of school resources on student performance: An update. Educational Evaluation and Policy Analysis, 19, 141–164.

14.

L. T.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1–55.

15.

Hulleman

C. S.

Harackiewicz

J. M.

(2009). Promoting interest and performance in high school science classes. Science, 326, 1410–1412.

16.

Karabenick

S. A.

(2003). Seeking help in large college classes: A person-centered approach. Contemporary Educational Psychology, 28, 37–58.

17.

Kelloway

E. K.

(1998). Using LISERL for structural equation modeling: A researcher’s guide. Thousand Oaks, CA: Sage.

18.

Kirschenbaum

D. S.

Humphrey

L. L.

Malett

S. D.

(1981). Specificity of planning in adult self-control: An applied investigation. Journal of Personality and Social Psychology, 40, 941–950.

19.

Kline

R. B.

(2016). Principles and practice of structural equation modeling. New York, NY: Guilford Press.

20.

Locke

E. A.

Latham

G. P.

(1990). A theory of goal setting and task performance. Englewood Cliffs, NJ: Prentice Hall.

21.

MacKinnon

D. P.

Lockwood

C. M.

Williams

(2004). Confidence limits for the indirect effect: Distribution of the product and resampling methods. Multivariate Behavioral Research, 39, 99–128.

22.

Muthén

L. K.

Muthén

B. O.

(2015). Mplus user’s guide (7th ed.). Los Angeles, CA: Author.

23.

Pape

S. J.

Bell

C. V.

Yetkin-Ozdemir

I. E.

(2013). Sequencing components of mathematics lessons to maximize development of self-regulation: Theory, practice, and intervention. In Bembenutty

Cleary

T. J.

Kitsantas

(Eds.), Applications of self-regulated learning across diverse disciplines (pp. 29–58). Charlotte, NC: Information Age.

24.

Paunesku

Walton

G. M.

Romero

Smith

E. N.

Yeager

D. S.

Dweck

C. S.

(2015). Mind-set interventions are a scalable treatment for academic underachievement. Psychological Science, 26, 784–793.

25.

Pham

L. B.

Taylor

S. E.

(1999). From thought to action: Effects of process-versus outcome-based mental simulations on performance. Personality and Social Psychology Bulletin, 25, 250–260.

26.

Pintrich

P. R.

De Groot

E. V.

(1990). Motivational and self-regulated learning components of classroom academic performance. Journal of Educational Psychology, 82, 33–40.

27.

Pintrich

P. R.

Smith

D. A. F.

Garcia

McKeachie

W. J.

(1991). A manual for the use of the Motivated Strategies for Learning Questionnaire (MSLQ). Retrieved from http://eric.ed.gov/?id=ED338122

28.

Pintrich

P. R.

Smith

D. A. F.

Garcia

McKeachie

W. J.

(1993). Reliability and predictive validity of the Motivated Strategies for Learning Questionnaire (MSLQ). Educational and Psychological Measurement, 53, 801–813.

29.

Plant

E. A.

Ericsson

K. A.

Hill

Asberg

(2005). Why study time does not predict grade point average across college students: Implications of deliberate practice for academic performance. Contemporary Educational Psychology, 30, 96–116.

30.

Schuman

Walsh

Olson

Etheridge

(1985). Effort and reward: The assumption that college grades are affected by quantity of study. Social Forces, 63, 945–966.

31.

Smith

C. A.

Ellsworth

P. C.

(1987). Patterns of appraisal and emotion related to taking an exam. Journal of Personality and Social Psychology, 52, 475–488.

32.

Walton

G. M.

(2014). The new science of wise psychological interventions. Current Directions in Psychological Science, 23, 73–82.

33.

Walton

G. M.

Cohen

G. L.

(2011). A brief social-belonging intervention improves academic and health outcomes of minority students. Science, 331, 1447–1451.

34.

Weinstein

C. E.

Acee

T. W.

(2013). Helping college students become more strategic and self-regulated learners. In Bembenutty

Cleary

T. J.

Kitsantas

(Eds.), Applications of self-regulated learning across diverse disciplines (pp. 197–236). Charlotte, NC: Information Age.

35.

Wertz

R. T.

(1995). Intention to treat: Once randomized, always analyzed. Clinical Aphasiology, 23, 57–64.

36.

Yeager

D. S.

Walton

G. M.

(2011). Social-psychological interventions in education: They’re not magic. Review of Educational Research, 81, 267–301.

37.

Zimmerman

B. J.

(2011). Motivational sources and outcomes of self-regulated learning and performance. In Zimmerman

B. J.

Schunk

D. H.

(Eds.), Handbook of self-regulation of learning and performance (pp. 49–64). New York, NY: Routledge.

38.

Zimmerman

B. J.

Martinez-Pons

(1988). Construct validation of a strategy model of student self-regulated learning. Journal of Educational Psychology, 80, 284–290.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.58 MB