Abstract
This research assessed the impact of combining small-group cognitive reading intervention with a motivational program targeting students’ goals, emotions, and self-efficacy beliefs on the reading performance and motivation of third-grade students at risk for reading difficulties (n = 25, Mage = 8.99, SD = 0.38). Using a quasi-experimental pre-test/post-test efficacy trial design, effects of the intervention on phonological awareness, listening comprehension, accuracy, fluency, reading comprehension, and motivation were assessed. Results indicate that compared with students who received Cognitive-Only reading intervention, students who received the combined Cognitive plus Motivational reading intervention showed greater gains in reading comprehension and phonological awareness. Findings provide preliminary evidence that supplementing cognitive reading intervention with the proposed motivational program can improve the reading performance of students at risk for reading difficulties.
Research has suggested that reading motivation contributes to reading acquisition (e.g., Bates et al., 2016). Positive correlations have been found between reading motivation and both strategy use and achievement in reading (Guthrie & Wigfield, 2000), while it has been suggested that reading motivation partly mediates responsiveness to intensive reading instruction (Bates et al., 2016). Motivation may operate to improve different components of reading achievement through multiple pathways, including increasing time-on-task, frequency of reading, systematic use of reading strategies, and engagement (Vollmeyer & Rheinberg, 2006). For example, reading comprehension is thought to depend in part on activating central processes, including fluent reading, activation of background knowledge, application of reading strategies, and self-regulation (Ahmadi & Gilakjani, 2012). Potentially, motivation may improve reading comprehension by increasing strategy use and cognitive engagement during reading. Development of fluency, on the contrary, may depend in large part upon practice (National Reading Panel, 2000). Motivation may possibly improve fluency in part by increasing the frequency and duration of children’s reading. Notably, the impact of reading motivation on reading achievement has been shown to be strongest for students at risk for reading difficulties (Logan et al., 2011).
Findings linking motivation to reading achievement suggest it may be important to address motivation when designing reading instruction, especially for students atrisk for developing reading difficulties. Intervention research on motivational reading instruction further suggests that targeting motivation alongside reading skills during instruction may lead to improvements in both reading achievement and motivation. For example, a meta-analysis of interventions assessing the impact of Concept-Oriented Reading Instruction (CORI), a multi-component program that supports motivation by providing relevant tasks, choice, social motivation, and promoting self-efficacy, yielded positive effects on both reading comprehension (d = 0.91) and motivation (d = 0.30) (Guthrie et al., 2007). However, it should be noted that other meta-analyses suggest effect sizes are much smaller, namely, g = .25 for reading achievement outcomes and g = .21 for reading motivation outcomes (McBreen & Savage, in press).
While findings from intervention research offer support for motivational approaches to reading instruction, notable gaps in the literature remain to be addressed. First, most interventions are not solidly grounded in motivational theory. In a recent systematic review and meta-analysis of 33 intervention studies on motivational reading instruction for students in Grades K-6 selected on methodological quality, only 10 studies made any direct mention of motivational theory (McBreen & Savage, in press). It has been noted that to comprehensively address the needs of learners, the design of effective instruction should draw upon solid theoretical grounding on one hand and empirically supported teaching practices on the other (Snowling & Hulme, 2011). Second, existing intervention research has mainly assessed the impact of motivational instruction on reading motivation and comprehension, but not on other aspects of reading such as phonological awareness, accuracy, and fluency. Examining how motivational instruction impacts a range of reading abilities may provide further insight into the precise mechanisms through which motivation improves reading, for example, through increasing strategy use and engagement during reading or by increasing frequency or duration of reading. Third, there exist few high-quality intervention studies of motivational reading instruction with elementary-age students, which limits researchers’ ability to draw strong conclusions concerning the effectiveness of such an approach. In the above-mentioned systematic review and meta-analysis of motivational reading instruction, only two studies were rated as high quality (McBreen & Savage, in press). The lack of high-quality intervention research into motivational reading instruction in Grades K–6 highlights the need for additional well-designed studies.
The present research sought to address these gaps through a well-designed quasi-experimental pre-test/post-test efficacy intervention trial, which evaluated the impact of a theoretically and empirically driven Cognitive plus Motivational reading intervention on the reading outcomes of students at risk for reading difficulties. The intervention was developed in a principled manner, by combining recommendations from motivational theory with practice recommendations extracted from an extensive search for, and an analysis of, existing intervention research. A brief overview of the program’s theoretical and empirical background is provided, followed by an outline of the program’s main components. Results from an intervention study evaluating the program’s impact on reading achievement and motivation are then presented.
Theoretical Framework: Motivational Systems Theory (MST)
MST was used as the theoretical framework guiding this intervention. This framework was chosen because it is argued that MST most thoroughly accounts for the multifaceted nature of reading motivation. MST arose in response to calls from motivational theorists that the field lacked consensus, cohesion, and integration (Ford, 1992). To address this, MST incorporates recommendations from 32 theories of motivation within a coherent and organized framework of motivation. This is in line with (a) recommendations that multiple theories of motivation be integrated to fully describe the motivational components involved in learning and (b) the view that different theories build systematically on each other to explain achievement situations (Anderman & Wolters, 2006).
MST identifies three main subcomponents of motivation—goals (i.e., the direction of effort), emotions (i.e., affective reactions to learning tasks), and self-efficacy beliefs (i.e., beliefs about one’s abilities)—and argues that each of these must be addressed to support optimal learning (Ford, 1992). To foster motivation, MST proposes that classrooms should (a) support students’ progress toward concrete goals by, for example, providing relevant tasks and encouraging students to set clear, attainable, and realistic goals; (b) promote “positive learning emotions” such as value for reading and pride by, for example, drawing upon students’ interests and providing opportunities for success, and should minimize “negative learning emotions” that can arise from, for example, competition; and (c) build students’ self-efficacy beliefs by, for example, providing opportunities for students to feel both independent and competent (Ford, 1992). This theoretical framework was complemented with empirically supported instructional practices drawn from a recent meta-analysis and systematic review of intervention research on motivational reading interventions in students in Grades K–6.
Empirical Framework: Motivational Reading Instruction
Current best practices in motivational instruction were identified based on a systematic review and meta-analysis of motivational reading interventions in students in Grades K–6 (McBreen & Savage, in press). The review sought to identify the most recent studies (up to 2019) and was restricted to intervention studies comparing the impact of motivational reading interventions with a non-motivational control on reading achievement and/or reading motivation on students in Grades K–6. It was further restricted to studies with a randomized control trial or quasi-experimental design, which provided data allowing effect sizes to be calculated for pre- to post-test gains across intervention and control conditions. Synthesis of effect sizes identified a significant overall effect of motivational reading interventions on both reading achievement (g = .25) and reading motivation (g = .21) (McBreen & Savage, in press). In addition, a review of content approaches to motivational instruction identified five main categories of motivational reading instruction that have gained support: (a) interest-enhancing programs, (b) self-regulatory instruction, (c) autonomy-supportive practices, (d) attribution/goal orientation training, and (e) multi-component motivational interventions, such as CORI (Guthrie et al., 2007). Self-regulatory instruction aims to help students become self-directed in their learning. This includes teaching them to set learning goals for themselves, monitoring and evaluating their learning, and regulating their emotions and self-efficacy beliefs (Dignath & Büttner, 2008). Autonomy-supportive instruction aims to develop students’ independence. For example, instructional practices that invite students to give input on classroom targets and tasks have been found to promote value for reading compared with traditional instruction (Marinak, 2013). Interest-based instruction builds upon students’ interests to generate intrinsic motivation and foster value for reading. For instance, the CORI (Guthrie et al., 2004) framework first exposes students to tasks they find intrinsically interesting, and then coherently links these tasks to reading. Attribution and goal orientation training both help students make attributions for their performance focused on factors within their control and adopt learning goals that are realistic and progress-oriented. It has been found that students who received strategy instruction with attribution training (i.e., self-reflection, positive self-talk, recognition of negative beliefs) showed greater gains in comprehension than students who received only strategy instruction or traditional instruction (Toste et al., 2017). To develop the intervention, effective practices were incorporated within MST’s theoretical framework.
Building a Motivational Reading Intervention
MST proposes that to foster motivation, it is necessary to target students’ goals, emotions, and self-efficacy beliefs, while a review of intervention research identified five broad categories of empirically supported motivational reading practices. Recommendations from theory and practice were integrated to create a comprehensive motivational intervention designed to address the needs of students at-risk for reading difficulties. The resulting program is subdivided into three components, each of which addresses one of the subcomponents of motivation identified by MST. In the first component, focused on goal setting, students and teachers collaborate to set learning goals for the classroom and students are instructed to set personal goals focused on progress, to support autonomy. Self-regulatory instruction is used to guide students in setting and planning goals, while attribution/goal training instruction is used to help students adopt goals centered on progress, effort, and personal mastery. In the second component, focused on learning emotions, students receive motivational self-regulatory instruction to develop affective self-regulation, and interest-based practices are used to support interest and enjoyment. In the third component, focused on promoting self-efficacy beliefs, students are instructed to track their progress and retrain their performance attributions, using self-regulatory instruction and attribution/goal orientation training. Autonomy-supportive practices were used to promote students’ independence, further contributing to their self-efficacy beliefs.
It has been noted that both cognitive and motivational components of reading should be addressed during instruction to promote optimal outcomes (Morgan et al., 2008). Indeed, targeting motivation alone is unlikely to yield gains in reading development. Thus, the program evaluated here is designed to be used in conjunction with evidence-based cognitive reading intervention. In this intervention, the motivational program was implemented in conjunction with the following cognitive instructional components: differentiated reading instruction including direct systematic instruction in synthetic and analogic phonics following, game-based practice of phonics concepts, and shared book reading with direct mapping (i.e., new phonological concepts learned in a session were applied to decode novel words during shared book reading in that same session). These practices were chosen as they have been found to yield positive effects on the outcomes of at-risk students in previous intervention research (e.g., Savage et al., 2018). Further detail on delivery of instructional components, as well as a template lesson plan, is provided in online supplemental Appendix A. The program was informally validated with experienced teachers.
Research Aims and Questions
The present research sought to contribute to the field of reading motivation research by assessing a cognitive and motivational reading program based on the principles outlined above. The aim of this research was to determine the impact of supplementing cognitive reading intervention with the proposed motivational program on the reading outcomes of students at risk for reading difficulties. A secondary aim of this research was to examine the mechanisms through which motivational instruction leads to gains in reading achievement. It was guided by the following research question and subquestions:
The theoretical framework adopted here suggests that to comprehensively support learning, both cognitive and motivational components of learning should be addressed during instruction. Thus, it was hypothesized that supplementing small-group cognitive reading intervention with the present motivational program would lead to greater growth in reading achievement and motivation than Cognitive-Only reading intervention. It was further hypothesized that gains in achievement would be mediated by gains in motivation.
Method
Sample
Participant selection
Prior to data collection, the study was reviewed and accepted by a research ethics board, and written parental consent was obtained for all participants. Participants were selected from Grade 3 classrooms of participating schools. This age group was selected because by this age, many students at risk for reading difficulties experience (a) a decline in reading motivation (Nelson & Manset-Williamson, 2006) and (b) difficulties with multiple aspects of reading (e.g., accuracy, fluency, comprehension). Choosing this age group thus allowed us to evaluate the intervention’s impact both on reading motivation and a range of reading skills. Students at risk for reading difficulties were identified through universal screening on four measures of reading ability: (a) phonological awareness, (b) reading accuracy, (c) reading fluency, and (d) reading comprehension. Selection criterion was scoring below the 30th percentile on at least one of these four measures. A 30th percentile cutoff point has been used as an operational definition of at-risk status in previous literacy research (e.g., Savage et al., 2018).
Sample description
Students were selected from nine English-speaking third-grade classrooms in six elementary schools in a large urban Canadian city. Written consent was obtained from the parents or guardians of 50 students, 26 of whom met study eligibility criteria. One student withdrew from the study during pre-test assessment. The final sample of eligible participants consisted of 25 students (Mage = 8.99, SD = 0.38, 36% female). Of these 25 students, all were typically developing, and slightly more than half of the sample was Caucasian (56%). Based on parents who responded to a demographic parent survey, the primary language spoken at home by the final sample of 25 students was either English or English and French, and the median household income was CAN$100,000 to CAN$124,999.
Procedure
The study took place over the course of 8 weeks during the winter semester of the 2018 school year. Children in both conditions were divided into groups of two to five students, and received two to three lessons per week, each lasting 45 min, for a total of 1.5 to 2.25 hr per week. If a student was absent for a session, makeup sessions were offered. All students received between 9 and 11 hr of total instructional time, which is consistent with previous research finding a measurable effect on standardized reading measures for students at risk for reading difficulties (e.g., on word reading and comprehension; Savage et al., 2018). A quasi-experimental pre-test/post-test efficacy intervention trial design, in which participants or groups are allocated to one out of two or more experimental conditions, was used to assess the impact of the intervention on students’ reading achievement and reading motivation. Due to the small number of students in each school, students were grouped according to school. Schools were randomly allocated to one of two treatment conditions by creating a number code (1 or 2) for each condition and using an online random number generator (random.org; Haahr & Haahr, 2005) to allocate schools to conditions. Using this approach, two schools (11 students, three groups) were allocated to the Cognitive-Only condition, and four schools (14 students, four groups) were allocated to the Cognitive plus Motivational condition. Following randomization, the quality of matching between conditions was assessed on control measures of receptive vocabulary, non-verbal IQ, and classroom quality, and pre-test primary outcome measures of reading comprehension and motivation. No significant differences were found across conditions, indicating high quality of match. Descriptive data for control, pre-, and post-test measures are presented in Tables 1 and 2.
Means for Control Measure Variables Across Conditions at Pre-Test.
Note. PPVT = Peabody Picture Vocabulary Test; ELLCO = Early Literacy and Language Classroom Observation.
Standard scores. bRaw scores.
Pre- and Post-Test Means for Main Outcomes and Effect Sizes Across Conditions.
Standard scores. bScaled scores. cStanine scores. dRaw scores.
Assessment
Pre-test assessment consisted of two 30-min sessions, during which students’ performance on outcome measures and control measures were assessed. Post-test assessment consisted of one 30-min session, during which outcome measures were assessed. Instructions were delivered orally by the primary researcher or by a research assistant (RA). RAs were trained on how to administer the tests, observed the primary researcher administer the tests, and finally were observed administering the tests by the primary researcher. Once they were deemed able to administer the tests correctly, RAs tested independently.
Conditions
In both conditions, the primary researcher taught some groups, and RAs taught some groups. RAs were undergraduate students in psychology or education. RAs were trained on how to deliver the intervention by the primary researcher and received an intervention booklet providing a detailed overview of lesson plans. In both conditions, students received the same cognitive instructional components: differentiated reading instruction including direct, systematic instruction in synthetic and analogic phonics following the simplicity principle, game-based practice of phonics concepts, and shared book reading. These practices have been found to yield positive effects on the outcomes of at-risk students in previous intervention research (e.g., Savage et al., 2018). In the motivational and cognitive condition, autonomy-supportive and interest-based practices were integrated throughout lessons. Students were asked to select the books and reading games used, to foster feelings of autonomy and allow them to select books/activities based on their interests. In addition, in the Cognitive plus Motivational condition, 25% of each lesson was allocated to self-regulatory instruction and/or attribution training, which did not include any direct reading instruction. During the first two lessons, which centered on goal setting, the instructor introduced group goals for the intervention and asked students to provide input. Students were then asked to set personal goals. The instructor guided students to set goals that were realistic, clear, challenging, and attainable. The subsequent eight lessons included socio-emotional coaching, that aimed to make students more aware of their strengths and interests, and to identify ways to cope with challenging learning situations (e.g., identifying a support team, visualizing progress made and next steps). Finally, all lessons included training in self-regulatory instruction and positive attribution beliefs. Students were given progress charts to self-track their progress toward goals, and specific, goal-directed feedback was provided by the instructor to help students identify successful strategies and plan next steps. To ensure the amount of direct reading instruction received across conditions was balanced while maintaining the length of intervention constant across conditions, 25% of lesson time in the Cognitive-Only condition was allocated to non-verbal mathematic exercises. Further detail and template lesson plans for both conditions are provided in Supplemental Appendix A.
Implementation fidelity
Implementation fidelity was assessed for a randomly selected subset of 20% of all lessons. Raters, who were either the primary investigator or RAs, were provided with a fidelity of implementation rubric for each condition that assessed delivery of instructional components, quality of teaching, and quality of the teaching environment (see online supplemental Appendix B), and trained on how to complete them. A series of 2 × 2 chi-square analyses were then conducted to compare implementation fidelity between conditions. No significant differences were found between groups on delivery of instructional components (χ2 = 1.92, p = .15), quality of teaching (χ2 = 2.92, p = .088), or quality of the teaching environment (χ2 = 0.83, p = .36). Instructional components were rated as delivered fully in 87.9% of cases and partly in 12.1% of cases; teacher quality was rated as excellent in 77.6% of cases and generally good in 22.4% of cases; and environment quality was rated as excellent in 60% of cases and adequate in 40% of cases.
Student response
Raters were also asked to rate students’ engagement for 20% of lessons in both conditions. Engagement was defined as time students spent on-task, application of strategies during reading, and enthusiasm (interest and excitement) during the instructional period. Significant differences emerged across groups, χ2 = 4.21, p = .04. Frequencies indicate that students were more engaged in the Cognitive plus Motivational condition than in the Cognitive-Only condition; the response of students in the Cognitive plus Motivational condition was rated as excellent in 100% of cases, the response of students in the Cognitive-Only condition was rated as excellent in 70.37% of cases, and generally good in 29.62% of cases.
Classroom observations
To ensure results were not due to differences in the quality of the classroom literacy environment, raters observed 1 hr of a regular English Language Arts class for each classroom. Observations were made using the Early Literacy and Language Classroom Observation (ELLCO K-3; Smith et al., 2008), which assesses Classroom Structure and Climate, Language Environment, Books and Reading, and Print and Writing. For each observation, two raters first filled out the rubric independently and then consulted until agreement was reached on all measures. A t test conducted on global ELLCO scores indicated no significant differences in regular classroom climate and quality of the literacy environment across conditions, t(23) = −1.562, p = .132.
Measures
Guthrie et al.’s (2007) meta-analysis reports a small effect size of motivational interventions on reading motivation outcomes and a large effect size on reading comprehension outcomes. Both reading comprehension and reading motivation were therefore included as primary outcome measures. To assess the impact of the intervention on a broader range of reading abilities, phonological awareness, accuracy, fluency, and listening comprehension were included as secondary outcome measures. For all outcome measures, internal reliability of outcome measures was assessed in this sample. Internal reliability, an index of how consistently different elements on a test measure a construct, was calculated by correlating students’ score on even numbered test items to their score on odd numbered test items, using the Spearman–Brown correction for internal reliability.
Primary outcome measures
Reading comprehension
Reading comprehension was assessed using the Sentence Comprehension subtest of the Group Reading Assessment and Diagnostic Evaluation (GRADE; Williams, 2001). During the test, students are asked to read sentences in which one word is missing, and to choose the missing word from four choices (e.g., “Cars and buses run on the ____”: told, road, goat, roar). Criterion validity ranges between .69 and .90 (American Institutes for Research, 2017). Internal reliability for this sample wasr = .68.
Reading motivation
Reading motivation was assessed using the Motivation to Read Profile–Revised (MRP-R; Marinak et al., 2016). The test was designed for use with students in second to sixth grade. It includes a value for Reading subscale and a Self-Efficacy for Reading subscale composed of 10 Likert-type-scale questions each. The value for Reading subscale was used as an index of emotions toward reading, while the Self-Efficacy for Reading subscale was used as an index of self-efficacy beliefs. As items are rated on an ordinal scale, nonparametric analysis using root mean square error of approximation (RMSEA) has been used to assess validity (Malloy et al., 2013). Malloy et al. (2013) reported an RMSEA estimate of .089, p < .001. Internal reliability in this sample was r = .66 for the Self-Efficacy subtest and r = .65 for the Value subtest.
Secondary outcome measures
Phonological awareness
The Comprehensive Test of Phonological Processing–2 (CTOPP-2; Wagner et al., 1999) “Blending Words” and “Segmenting Non-Words” subtests were used to assess phonological awareness. The “Blending Words” subtest consists of 33 items that assess students’ ability to combine orally presented speech sounds into words (e.g., /dʒ/-/ʌ/-/m/-/p/ = “jump”). The “Segmenting Non-Words” subtest consists of 31 items that assess students’ ability to segment orally presented non-words into speech sounds (e.g., “seb” = /s/-/e/-/b/). The CTOPP manual reports validity correlations ranging between .49 and .84 for all subtests (Wagner et al., 1999). Internal reliability for this sample was r = .80 for the Blending subtest and r = .98 for the Segmenting subtest.
Reading accuracy
Reading accuracy was assessed using the Wide Range Achievement Test–4 (WRAT-4; Wilkinson & Robertson, 2006) Word Reading subtest. In the test, students are asked to name 15 letters and read from a list of 55 typed, lowercase, progressively more difficult words (e.g., “see,” “wrap,” “rancid”). The test is discontinued following 10 consecutive errors. Students’ scores are obtained by calculating the number of words read correctly. Internal validity ranges from .60 to .63 (Wilkinson & Robertson, 2006). Internal reliability for this sample was r = .57.
Reading fluency
Reading fluency was assessed using the Dynamic Indicators of Basic Early Literacy Skills–Oral Reading Fluency (DIBELS ORF, 6th edition; Good & Kaminski, 2007) subtest. In the test, students are asked to read three grade-level passages (“Finding a nest,” “A famous food: The history of pizza,” and “Living in Singapore”) out loud for 1 min, and the number of correctly read words is calculated. Students read the same passages at pre-test and post-test. Omissions, substitutions, and hesitations longer than 3 s are scored as errors. An average score is calculated based on performance on the three passages. The DIBELS ORF alternate-form criterion validity is .79 (Good et al., 2001). As the DIBELS uses text rather than text items, published reliability measures were used for this measure. Good et al. (2001) reported alternate-form reliability of .94
Listening comprehension
The GRADE (Williams, 2001) Third-Grade Listening Comprehension subtest was used to assess listening comprehension. During the test, students are asked to choose a picture out of four choices that visually depicts a sentence orally presented by the experimenter (e.g., “The horse is jumping over the fence”). Criterion validity ranges between .69 and .90 (American Institutes for Research, 2017). Internal reliability for this sample was r = .53.
Control measures
Non-verbal intelligence
The Raven’s Colored Progressive Matrices was used to assess non-verbal intelligence (Raven, 1998). The test is made up of 36 untimed perceptual problems in three sets of 12 items. The manual reports reliability of .88 (Raven, 1998). Criterion-referenced validity for children ages 3 to 9 ranges from .643 to .703 (Bildiren, 2017).
Receptive vocabulary
English receptive vocabulary was assessed using the Peabody Picture Vocabulary Test–IIIA (PPVT-IIIA; Dunn & Dunn, 1997). In the test, participants are asked to choose a picture out of four choices that visually depicts a word orally presented by the experimenter (e.g., “plumber,” “vase”). The manual reports reliability of .94 and criterion validity ranging from .69 to .91 (Dunn & Dunn, 1997).
Results
Preliminary Data Analysis
To reduce the potential influence of extreme scores, outliers were identified by searching for data points that fell outside of the interquartile range for all outcome measures at pre- and post-test. Through this process, 22 potential outliers were identified (11% of the entire data set) and adjusted either to the lower or upper boundary of the interquartile range or to 1.5 SD from the mean. Analyses with outliers adjusted by either method yielded the same results. The final analyses were conducted with 22 outliers adjusted to either the lower or upper boundary, respectively. There were no missing data. Finally, to assess generalizability of results to the wider population, the sample was compared with average norms for non-verbal intelligence and family characteristics. The sample’s average raw score on Raven’s Colored Progressive Matrices was in the 50th percentile for its age group (M = 26, SD = 4.71), indicating average levels of non-verbal intelligence. Parents were given a short survey assessing household income and mother’s education. Parents answers to the survey were compared with national norms from the 2016 Canadian census (Statistics Canada, 2016) using chi-square analyses. No significant differences between the sample and population were observed for household income, χ2(18) = 12.03, p = .85, or mother’s education level, χ2(9) = 8.01, p = .53. Means and standard deviations for control measures are reported in Table 1. Means, standard deviations, and effect sizes across conditions are reported in Table 2.
Inferential Analyses
To ensure that the control and treatment groups did not differ significantly prior to receiving intervention, independent-samples t tests were conducted on pre-test scores with condition as the independent variable for all outcome and control measures. At pre-test, no significant differences across conditions were found for control measures (see Table 1). The groups differed significantly on two secondary outcome measures, fluency, t(23) = 3.287, p < .001, and phonological blending, t(23) = −2.591, p = .016 (see Table 2). No other significant differences were present for either achievement or motivation measures at pre-test.
To facilitate interpretation and comparison across outcomes, data for outcome measures at pre- and post-test were first converted to z scores and then grand-mean centered. Due to the nested nature of the data, outcomes were analyzed using hierarchical linear modeling (HLM; for example, Hayes, 2006). The final HLM models were built sequentially from preliminary analyses. Students were grouped according to school, and schools were randomized into either condition. In some schools, students within a single group came from multiple classrooms. To control for possible variations in baseline activity due to regular teacher effects within individual classes, classroom-level variance was assessed. In Model 1, an unconditional one-way analysis of variance (ANOVA) indicated there was significant classroom-level variance at pre-test and post-test on primary and secondary outcome measures beyond the variance attributable to students (i.e., intra-class correlations ranged from .06 to .57). Thus, it was deemed that HLM with clustering at the classroom level was appropriate. The intra-class correlation at the classroom level was below .05 for four outcome measures: accuracy at both pre-test and post-test, segmenting at pre-test, and listening comprehension at pre-test.
The final two-level hierarchical model examined whether variance on post-test reading achievement and motivation for students (Level 1) at the class level (Level 2) was explained by condition (Cognitive plus Motivational vs. Cognitive-Only, at Level 2), after controlling for children’s pre-test achievement on the outcome measure (Level 2).
An analysis of covariance (ANCOVA) model was deemed appropriate because controlling for pre-test levels of attainment improves the power of analyses even when the covariate is not statistically significant (e.g., Raudenbush et al., 2011). Equations 1 and 2 describe this final model at the student and classroom levels, for student i in classroom j, respectively.
Equation for Student Level 1 Model:
Equation for Classroom Level 2 Model:
where Υij = reading achievement or motivation outcome,
As intervention groups were variable in composition (i.e., students within groups had variability in pre-test levels of achievement and motivation, and variable demographic characteristics), individual scores rather than group means were the unit of analysis used in the analyses, both for predictor and outcome variables. For all primary and secondary outcome measures, identical two-level models were run, with pre-test attainment on the outcome measure included as a covariate. Results of the HLM analysis are presented in Table 3. For clarity of interpretation, only the effects of condition are reported within this section. However, coefficients representing fixed effects of intercept, condition, and pre-test, and random effects of child- and classroom-level variance on post-test are reported in Table 3.
Hierarchical Linear Modeling Results for the Effect of the Intervention Condition on Post-Test Attainment.
Significant at <.05. **Significant at <.01.
Primary outcomes were analyzed at conventional significance (α = .05). Secondary outcomes were analyzed in the same manner but with adjustments for multiple contrasts (α = .05/4 = .0125). Effect sizes (Cohen’s d) were calculated for all dependent measures by comparing gains in pre- to post-test scores across conditions and dividing them by the pooled standard deviation (Busk & Serlin, 1992). Cohen’s (1988) standards were used to determine strength of effects, where .2 indicates a small effect, .5 a medium effect, and .8 a large effect. Effect sizes of pre- to post-test gains across conditions are summarized in Table 2.
The results revealed a significant effect of condition on Sentence Comprehension (p = .024, d = 0.55), Blending (p < .01, d = 0.62), and Segmenting (p < .01, d = 1.76) at post-test. In all cases, this favored the Cognitive plus Motivational condition over the Cognitive-Only condition. Other effects did not reach significance. However, analysis of non-significant effect sizes suggested small effects of the Cognitive plus Motivational intervention compared with the Cognitive-Only intervention on value for reading (d = 0.40), self-efficacy for reading (d = 0.28), and fluency for reading (d = 0.40), as well as a medium effect on listening comprehension (d = 0.57). As no significant differences in pre- to post-test gains in motivation were observed across conditions, a mediation analysis examining whether gains in reading achievement were mediated by gains in motivation was not possible.
Discussion
The present research sought to assess the impact of a motivational program on the reading achievement and motivation of third-grade students at risk for reading difficulties. To limit the impact of extraneous variables, conditions were matched on pre-test performance on main outcome variables, receptive vocabulary, non-verbal IQ, and quality of the classroom literacy environment. It was first hypothesized that, compared with Cognitive-Only reading intervention, supplementing cognitive reading intervention with the motivational program proposed here would lead to greater improvements in reading achievement and motivation. Results partly support this. Comparison of pre- to post-test gains across conditions indicated that the Cognitive plus Motivational intervention had a medium effect on sentence comprehension and a medium to large effect on phonological awareness for students at risk for reading difficulties. These findings are consistent with some previous findings that motivational instruction contributes to gains in reading comprehension (e.g., Guthrie et al., 2007), and extend upon these by showing gains on a wider range of reading skills, including phonological awareness. No significant differences between groups were observed for gains in reading accuracy, fluency, or listening comprehension. Several factors may have contributed to this. First, given the study’s modest sample size, it is possible that it was not sufficiently powered to detect significant effects on secondary outcomes. Analysis of non-significant effect sizes suggested a small effect of the Cognitive plus Motivational intervention compared with the Cognitive-Only intervention on fluency, as well as a medium effect on listening comprehension. Second, the cognitive components of the intervention focused on direct phonics instruction, combined with shared book reading. As the intervention lasted only 8 weeks, it is possible that gains in phonological awareness were more readily observable, as this was directly trained. Conversely, gains in reading skills such as accuracy and fluency are thought to cascade from phonological awareness and may also depend upon repeated exposure and practice (e.g., National Reading Panel, 2000), and thus may not have been observable within the time frame of this study. Larger and longer term studies that include a delayed post-test are needed to properly assess this.
Second, it was hypothesized that gains in reading ability would be mediated by gains in motivation. No significant differences in gains across conditions were observed for either value for reading or self-efficacy for reading. Thus, it was not possible to examine whether gains in motivation mediated gains in achievement. However, analysis of non-significant effect sizes for motivation measures indicated a small effect of condition on both subcomponents. In addition, it is worth noting student response ratings indicate that students in the Cognitive plus Motivational condition were significantly more engaged during lessons than those in the Cognitive-Only condition. Engagement was operationalized as time spent on-task, interest, and application of reading strategies during instruction. It is possible that gains in performance were mediated by levels of engagement, which would be in line with findings linking engagement to learning (Reeve, 2013).
These findings have several implications. First, these results support the general view of motivational theorists that when students learn in environments that support motivation, their learning improves (Ford, 1992). It further provides tentative support for a model of motivation advanced by MST theorists, which proposes that targeting goals, emotions, and self-efficacy beliefs within a responsive environment contributes to gains in achievement (Ford, 1992). Second, the finding that the program improved both sentence comprehension and phonological awareness, which are thought to involve different cognitive processes, may provide some insight into the mechanisms through which motivational instruction may impact achievement. Reading comprehension is thought to depend not only on successful word reading but also on central processes, such as activating background knowledge, applying reading strategies, and self-regulation (Ahmadi & Gilakjani, 2012). Similarly, phonological awareness may involve not only on successfully linking letters to sounds but also other cognitive processes such as working memory (i.e., to hold and manipulate sounds) and application of reading strategies (e.g., blending, segmenting). It is possible that the motivational components of the intervention assessed here contributed to greater gains in both reading comprehension and phonological awareness through increasing the activation or efficient use of cognitive processes involved in both skills, including application of reading strategies and self-regulation. One pathway through which this may have occurred is by higher engagement levels during learning tasks.
Greater engagement during lessons was observed for students in the Cognitive plus Motivational condition compared with the Cognitive-Only condition, tentatively supporting this hypothesis. Higher levels of engagement have been argued to lead to gains in achievement in part by operating directly on learning behaviors, for example, by increasing goal-directed behaviors and strategy use during reading tasks (Guthrie et al., 2004). In addition, analysis of non-significant effect sizes indicated that while value for reading underwent a small increase for students in the Cognitive plus Motivational condition, it showed a small decrease for students in the Cognitive-Only condition. This is consistent with previous research showing that intensive reading interventions that focus only on cognitive components of reading may in some cases lead to more negative attitudes toward reading (Wanzek et al., 2006). Conversely, self-efficacy for reading showed a slight decrease for students in the Cognitive plus Motivational condition, compared with a slight increase for students in the Cognitive-Only condition. This is in line with previous findings that self-regulatory instruction can lead to more negative self-efficacy beliefs for students with learning difficulties (Nelson & Manset-Williamson, 2006). One explanation proposed for this is that self-regulatory instruction leads students to recalibrate their self-efficacy beliefs, as they gain a better understanding of their actual skill level (Kruger & Dunning, 1999). While slightly elevated self-efficacy beliefs are generally thought to be adaptive, as they may increase persistence (Bandura, 1997), self-efficacy beliefs that are mismatched with actual ability have in some cases been linked to incorrect strategy use (Schraw et al., 1993) and lower engagement (Linnenbrink & Pintrich, 2003). It is possible that the greater value for reading observed for students in the Cognitive plus Motivational condition, combined with more accurate self-efficacy beliefs, contributed to greater engagement for students in this condition, which in turn contributed to greater effort and more effective strategy use during reading tasks. However, as differences in subcomponents of motivation did not reach significance, and engagement was only assessed for 20% of the lessons, was assessed on a group level, and was not assessed using a formal measure, extreme caution is needed in interpreting these results.
Implications for Practice
The results presented here have several implications for teaching practice. First, as the only difference between both treatment conditions was the presence or absence of motivational instructional components, these results tentatively support the hypothesis that supplementing high-quality reading instruction with supports for motivation can improve reading achievement. In addition, the intervention proposed and assessed here provides a set of instructional practices with which to target motivation during reading instruction, including goal-directed, progress-oriented, and autonomy-supportive practices. Findings that incorporating these during reading intervention supported greater gains in some reading skills (i.e., phonological awareness and reading comprehension) provide support for the benefits of a motivational program that incorporates these practices for students at risk for reading difficulties. It should be noted that the research design used here examined the impact of supplementing cognitive reading instruction with the proposed motivational program during small-group, intensive reading intervention. Findings may underestimate the effects of the intervention compared with regular classroom instruction. Conversely, findings may not be generalizable to the context of whole-class reading instruction or to the outcomes of typically developing readers. Nonetheless, the results presented here provide tentative support for the benefits of a reading intervention driven by MST that addresses both cognitive and motivational components of reading and provide specific practical guidelines to target the motivation of students at risk for reading difficulties.
Limitations
Several limitations should be addressed when interpreting results. First, the small sample size may have limited generalizability. However, conditions were carefully matched prior to beginning the intervention, and the intervention was tightly controlled to increase validity. Second, only 32% of parents responded to the parent survey. While the socio-demographic characteristics of families for whom parents responded were representative of the general population, due to the low response rate it is possible this is not the case for the overall sample. Third, the relatively short duration of intervention, while sufficient to substantially improve reading comprehension and phonological awareness, may have limited the potential for measurable effects on other aspects of students’ reading motivation and achievement. Longer term studies are needed to properly assess this. Nevertheless, the study was sufficient to measurably improve standardized outcome measures, including reading comprehension. Fourth, some of the measures (GRADE, CTOPP) used to assess outcomes were standardized some time ago. However, all measures included were standardized and have been widely used in previous reading research, and generic effects of tests would not account for the specific effects of intervention reported here. In addition, measures of reading comprehension, reading motivation, reading accuracy, and listening comprehension had somewhat lower internal reliability (r < .80) as assessed in this sample, which may have compromised reliability of the data. This could in part be due to the modest sample size used here, as the published reliabilities of most tests used here are high. Fifth, raters for treatment fidelity and post-test assessment were not blind to condition, which may have introduced bias into the coding and testing procedure. It may, however, be hard to disguise the visible differences in these interventions to a treatment integrity team. In addition, both treatment fidelity for each lesson and pre- and post-test measures were rated by a single coder, which may have compromised reliability. Finally, because the intervention contained multiple components, it was not possible to identify which of these were effective, or whether the efficacy of the intervention would be improved by removing or adding components. Follow-up investigations using other designs such as factorial experimental interventions are needed to assess this.
Conclusions, Contributions, and Further Directions
The results presented here tentatively suggest that remedial reading instruction has a greater impact on various aspects of reading performance and motivation when it is combined with evidence- and theory-driven instruction that aims to improve motivation. A review of the intervention research of motivational reading interventions in students in Grades K-6 highlighted a lack of well-designed studies, as well as the need for research that examines the impact of motivational reading instruction on a range of reading outcomes. The present study contributes to the literature on motivational reading instruction by proposing a high-quality quasi-experimental trial of one approach to motivational reading instruction, and examining its impact on an array of reading skills. Furthermore, the effect of the program on reading performance provides support for a theoretical framework of motivation based on MST to address goals, emotions, and self-efficacy beliefs within a single comprehensive program. To draw stronger conclusions about the efficacy of this approach, replication of these findings is needed. A larger sample and the inclusion of a delayed post-test to evaluate the sustained effects of the interventions would extend the present findings. In addition, to more fully assess the program’s impact on affective components of learning, longitudinal studies that assess the potentially complex and inter-related impact of intervention on attainment, motivation, and engagement are needed, with candidate mediation effects of motivation modeled. If supported by future research, the results observed here have important implications for teaching practice and intervention with students at-risk for reading difficulties. Results suggest that designing reading programs that complement skills-based instruction with motivation support by offering goal-directed, progress-oriented, and autonomy-supportive teaching can lead to better outcomes than those which focus only on the cognitive aspects of reading acquisition. In addition, if validated by future research, the results presented here provide tentative support for a model of reading development in which reading motivation operates to improve reading skills through its effects on task-related behaviors, including increased engagement and effective strategy use during reading.
Supplemental Material
AppendixA – Supplemental material for The Impact of a Cognitive and Motivational Reading Intervention on the Reading Achievement and Motivation of Students At-Risk for Reading Difficulties
Supplemental material, AppendixA for The Impact of a Cognitive and Motivational Reading Intervention on the Reading Achievement and Motivation of Students At-Risk for Reading Difficulties by Miriam McBreen and Robert Savage in Learning Disability Quarterly
Supplemental Material
AppendixB – Supplemental material for The Impact of a Cognitive and Motivational Reading Intervention on the Reading Achievement and Motivation of Students At-Risk for Reading Difficulties
Supplemental material, AppendixB for The Impact of a Cognitive and Motivational Reading Intervention on the Reading Achievement and Motivation of Students At-Risk for Reading Difficulties by Miriam McBreen and Robert Savage in Learning Disability Quarterly
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
This research received funding from the Fonds de Recherche du Québec - Société et Culture (Reference number: 136078).
Supplemental Material
Supplemental material for this article is available on the Learning Disability Quarterly website with the online version of this article.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
