A Preliminary Component Analysis of Self-Regulated Strategy Development for Persuasive Writing in Grades 5 to 7 in British Columbia

Abstract

Self-regulated strategy development (SRSD) has extensive evidence of improving student writing; however, few studies have investigated the relative importance of specific self-regulation components in SRSD. Twelve students in Grades 5 to 7 were assigned to one of two, 5-week SRSD conditions for persuasive writing that differed in instruction on the use of self-statements. No differences in outcomes were found between SRSD conditions, but large gains in writing quality, composition duration, and writing self-efficacy were found in both conditions. Although limited by not including a no-intervention comparison condition, the observed gains were typical of effect sizes reported in meta-analyses of SRSD and provide some additional evidence of replication in independent research teams and in a Canadian context.

Keywords

written expression writing intervention strategy instruction self-regulated learning

Skillful writing is important for both academic and employment success (National Commission on Writing, 2004); yet, many students struggle to reach adequate levels of writing proficiency. For example, the School Achievement Indicators Program for Writing (Council of Ministers of Education Canada, 2003) indicated that 40% of 16-year-old Canadian students had not reached expected levels in English writing ability. Consequently, researchers must investigate writing instructional practices to better support students.

One well-researched approach is self-regulated strategy development (SRSD; Graham, Harris, & Mason, 2005), with findings demonstrating improved student writing across genres and grade levels (see Graham, Harris, & McKeown, 2013). Although the overall model is effective, a clearer understanding of the relative importance of individual SRSD components would inform educators how to focus instructional time when implementing the model. Accordingly, the purpose of this study is to determine whether SRSD is more effective when instruction emphasizes the use of self-statements by students.

Writing Task Strategy Instruction

SRSD includes writing task strategy instruction while encouraging self-regulation during collaborative practice (Harris, Graham, Mason, & Friedlander, 2008). Writing task strategy instruction is explicit and includes (a) explaining the type of writing and strategy to be used, (b) presenting an acronym to help students remember the steps, (c) ensuring students memorize the strategy, (d) demonstrating strategy use, (e) scaffolding students while they use the strategy, (f) providing feedback about student strategy use, and (g) encouraging independent strategy use (Harris et al., 2008). Various writing task strategies can be taught depending on student age and writing genre.

For example, POW is a general strategy used across genres (Harris et al., 2008). Students Pick an idea, Organize notes, Write and say more. By contrast, the five-Part TREE strategy is genre specific and used to plan and compose persuasive writing (Harris et al., 2008). Students develop a Topic sentence that states their opinion on the subject, provide three Reasons for their opinion, develop an Ending or concluding sentence, and Examine their writing to ensure they have all the necessary parts.

Compared with writing task strategy instruction in SRSD, self-regulation instruction is less explicit. Acronyms are not used for the self-regulation strategies, nor are strategy steps memorized by students. To promote self-regulation, teachers can encourage students to engage in goal setting, self-monitoring, and self-instruction through self-statements (Graham & Harris, 1987; Mason, Harris, & Graham, 2011). Self-statements may help students to focus attention, define problems, generate ideas, remember and perform procedural steps, cope with emotion, and reinforce positive behaviors (Harris et al., 2008; Meichenbaum, 1977; Schunk & Cox, 1986). In addition, self-statements may encourage students to adopt attributional beliefs that further encourage student motivation, that is, that they can improve their writing by using specific strategies and working hard. If students attribute success or failure to factors over which they have control, such as use of effective strategies and effort, they will be more likely to work harder and persist (Schunk, 2008). In SRSD, use of self-statements can be modeled when introducing new writing strategies; also, students can be encouraged to create and use their own self-statements and verbalize their own self-instructions (Graham & Harris, 1989a; Sawyer, Graham, & Harris, 1992). When SRSD is implemented, however, the breadth and depth of self-regulation instruction can vary greatly depending on available instructional time, student needs, and teacher preferences (Harris et al., 2008).

Effectiveness of SRSD

Several meta-analyses and systematic reviews have demonstrated that SRSD is an effective method of improving students’ writing quality, duration of planning and writing, composition length, and incorporation of functional writing elements (Baker, Chard, Ketterlin-Geller, Apichatabutra, & Doabler, 2009; Graham et al., 2013). Effects on writing quality typically are of large magnitude—Graham et al. (2013) reported a weighted effect size of 1.75 based on 28 studies; and, analysis of outcomes and design quality across group and single-case experimental studies led Baker et al. (2009) to characterize SRSD as an evidence-based practice. Although the overall body of evidence for SRSD is impressive, additional research is needed to determine the extent to which (a) results can be replicated by independent research teams, (b) comparable results can be obtained in Canadian schools, (c) individual SRSD components contribute to student outcomes, and (d) SRSD improves writing self-efficacy.

Independent Replications

The majority of studies on SRSD have been conducted by research teams that included the SRSD authors or close collaborators (Baker et al., 2009). Additional systematic replications of SRSD by independent research teams are needed—the odds of replicating research findings in special education are significantly higher when the replicating research team includes one or more authors from original studies (Cook et al., 2016).

Studies in Canada

Although SRSD has been widely researched, the vast majority of studies have been conducted in the United States. To our knowledge, only two published randomized controlled trials (RCTs; Reynolds & Perin, 2009; Wong, Hoskyn, Jai, Ellis, & Watson, 2008), one unpublished thesis using a single-case experimental design (Sinclair, 2014), and one published case study (Milford & Harrison, 2010) have been conducted in Canada. In an RCT including 57 Grade 6 students in British Columbia, Wong et al. (2008) found that instruction on a persuasive writing strategy delivered consistently with the SRSD model resulted in improvements in writing clarity (posttest d = 0.87, 2-week maintenance d = 0.93), organization (posttest d = 0.54, 2-week maintenance d = 0.77), and cogency (i.e., persuasiveness of arguments; post-test d = 0.33, 2-week maintenance d = 0.55). In another RCT of 121 Grade 7 students in British Columbia, Reynolds and Perin (2009) found that not only SRSD instruction for summarizing expository text, compared with typical instruction, improved students’ writing quality (d = 0.26) and inclusion of main ideas in the summaries (d = 0.77) at posttest, but also explicit text strategy instruction (non-SRSD) that incorporated graphic organizers yielded greater gains on writing quality (d = 0.96).

The unpublished thesis (Sinclair, 2014) and published case study (Milford & Harrison, 2010) also were conducted in British Columbia and focused on the PLEASE strategy (Pick the paragraph students want to write, List ideas that might be included, Evaluate the list for relevant ideas, Activate by choosing a topic sentence, Supply the list of ideas to generate sentences to support the topic sentence, End with a concluding sentence) for expository paragraph writing (Welch, 1992). Using a multiple baseline design across three, 8- to 11-year-old students with epilepsy, Sinclair (2014) found that students learned to use the specific elements of the strategy in their compositions, composition length increased, and paragraphs became better organized after seven lessons. Similarly, Milford and Harrison (2010) described improvements in use of strategy elements, composition length, and paragraph quality for an 11-year-old student with a chronic health condition. Both studies provide rich operational details of SRSD implementation as well as evidence that SRSD is effective for expository writing in Grades 3 to 5.

In sum, studies in Canada provide evidence that SRSD is effective in Grades 3 to 7 for improving student persuasive and expository writing; however, Wong et al. (2008) did not evaluate the TREE persuasive writing strategy that has the most extensive research evidence, and the effect sizes on writing quality in both RCTs are below the average weighted effect size of d = 1.7 (Graham et al., 2013). Effect sizes for the multiple baseline study (Sinclair, 2014) were strong, for example, percentage of nonoverlapping data (PND) of 90% to 100% for writing organization; however, these effect sizes are not directly comparable with those from group-design studies that were included in the Graham et al. (2013) meta-analysis. Notably, the student participants in the two Canadian RCTs were more linguistically diverse than the U.S. average of 9.4% of students considered English language learners (ELLs), with 77% of these students speaking Spanish at home (National Center for Education Statistics, 2017). In Reynolds and Perin’s (2009), 17% of participants spoke an unspecified language other than English at home, and 84% of participants in Wong et al.’s (2008) spoke Mandarin, Cantonese, Punjabi, Polish, Russian, Vietnamese, or Tagalog at home. For these reasons, additional studies of SRSD in Canada are needed to inform Canadian educational practice.

Component Analyses

Although the overall effectiveness of SRSD has been extensively studied, few component analyses have been conducted to determine the most critical components of SRSD in achieving student outcomes. Component analyses are helpful in evaluating the causal mechanisms of interventions—by systematically adding or removing key components of interventions and determining whether intervention effects change, the theoretical rationale for specific intervention components, that is, how each component leads to outcomes, can be tested (Mercer, Idler, & Bartfai, 2014). Two prior U.S. studies found similar student writing outcomes whether or not explicit instruction in goal setting and self-monitoring was included (Graham & Harris, 1989a; Sawyer et al., 1992); however, two more recent studies in Germany found greater improvements in writing quality when writing task instruction was combined with instruction in goal setting, self-monitoring, and self-evaluation (Brunstein & Glaser, 2011; Glaser & Brunstein, 2007). None of these studies, however, directly examined the effect of teaching students to use self-statements because self-instruction was targeted, to some extent, in all SRSD intervention conditions.

Effects on Writing Self-Efficacy

Despite the emphasis on self-regulation in SRSD, research has primarily focused on writing skill outcomes. Theory suggests that as self-regulation improves, student self-efficacy should increase in tandem with writing skills (Zimmerman & Risemberg, 1997); however, the few studies investigating effects on writing self-efficacy have yielded mixed findings. Some studies found that student self-efficacy increased postinstruction (Brunstein & Glaser, 2011; Graham & Harris, 1989a, 1989b), but other studies have found no changes in writing self-efficacy (García-Sánchez & Fidalgo-Redondo, 2006; Graham et al., 2005; Sawyer et al., 1992). All but one of these studies used a 10-item writing self efficacy scale developed by Graham and Harris (1989a), with a four-item researcher-developed measure used in García-Sánchez and Fidalgo-Redondo (2006). Although most studies scored the Graham and Harris’ (1989a) measure as unidimensional, scoring was subsequently revised to two factors (with α = .69 and .73) based on findings that the scale is not unidimensional in Graham et al. (2005); consequently, measurement concerns may be, in part, contributing to inconsistency of findings. A measure with better evidence of reliability and validity (Pajares & Valiante, 1997; Shell, Murphy, & Bruning, 1989) was used in the Sinclair’s (2014) multiple baseline study; however, because the measure was only administered once preintervention and again postintervention, there were not enough repeated assessments to establish experimental control in a single-case design study (Horner et al., 2005).

Current Study

The primary purpose of this study is to provide preliminary data on the differential effects of explicitly teaching students to use self-statements within SRSD. Two research questions are addressed:

Research Question 1: Does explicit instruction on self-statements improve student writing beyond regular SRSD instruction?

Research Question 2: Does explicit instruction on self-statements increase student self-efficacy compared with regular SRSD instruction?

Because the research questions involve comparisons of two SRSD intervention conditions and there were research site considerations precluding the delay of a likely effective intervention, we did not include a no-intervention comparison condition. Despite this limitation, the use of repeated measures of writing skill and self-efficacy provides some ability to evaluate the extent to which obtained effect sizes are consistent with those reported in the research literature, potentially providing additional evidence of independent replication in a Canadian context.

Method

Participants

Twelve students in Grades 5 to 7 from an independent elementary school in Vancouver, British Columbia, participated in the study. The school had an enrollment of approximately 200 students from kindergarten to Grade 7 with 63% of students identified as ELLs (British Columbia Ministry of Education, 2014). All participants spoke and received instruction in English at school, and all identified that a language other than English was spoken in their home: Filipino (five students), Vietnamese (five students), Polish (one student), and Cantonese (one student). Classroom teachers nominated students for participation who could write good sentences, but had difficulty with paragraph-level writing; none of the participating students received special education or ELL services at the time of the study.

Study inclusion criteria required that participants have scores within 1.5 SD of the mean for the student’s age group on standardized assessments of sentence writing skill and cognitive ability. Three screening measures were used to ensure that students had the cognitive and sentence writing skill needed to benefit from the interventions. These measures included the Writing Samples Test from the Woodcock-Johnson Tests of Achievement, Third Edition (WJ-III ACH; Woodcock, McGrew, & Mather, 2007) and the Verbal Knowledge and the Matrices subtests of the Kaufman Brief Intelligence Test, Second Edition (KBIT-2; Kaufman & Kaufman, 2004). Student demographics and selected screening results are presented in Table 1. Presented student names are pseudonyms.

Table 1.

Student Demographic Information and Screening Results.

Student number	Grade	Age	Gender	Verbal knowledge KBIT-2	Matrices KBIT-2	Writing samples WJ-III ACH
1	7	12	F	85	93	102
2	7	14	F	80	83	93
3	7	12	F	90	98	110
4	7	12	M	95	85	116
5	6	11	M	95	81	102
6	6	12	M	85	97	85
7	6	12	M	100	105	115
8	6	11	M	110	111	104
9	5	10	M	105	81	105
10	5	10	M	85	98	100
11	5	11	M	85	82	114
12	5	11	M	90	82	104

Note. KBIT-2 = Kaufman Brief Intelligence Test, Second Edition; WJ-III ACH = Woodcock-Johnson Tests of Achievement, Third Edition. Reported scores have an M = 100 and SD = 15.

Materials

To obtain student writing samples for program evaluation, the first author adapted 12 persuasive writing prompts presented in Harris et al. (2008). The prompts were short statements or questions that encouraged students to convince their audience to agree with their position on an issue. A sample prompt is “Should students be allowed to eat snacks in the classroom?” Before the study, prompts were evaluated by teachers of participating students for difficulty, appropriateness, and potential interest to students. All were evaluated as acceptable, so six of the prompts were randomly selected for the study.

Measures

Basic writing elements

For this measure, writing samples were scored for the number of TREE elements used. One point was given for each element present (i.e., topic sentence, reason one, reason two, reason three, and ending sentence).

Written expression curriculum-based measures (WE-CBM)

Three WE-CBM metrics were used: total words written (TWW), correct word sequences (CWS), and percent correct word sequences (%CWS). TWW is a count of the number of words (i.e., one or more grouped letters), including incorrectly spelled and nonsense words (Hosp, Hosp, & Howell, 2016). CWS is a count of the number of adjacent, correctly spelled words that make sense in context (Hosp et al., 2016), with %CWS scored as CWS divided by the total number of word sequences.

Analytic writing rubric

Samples were also scored using the Grade 6 British Columbia Performance Standards for Writing (BCPS-W) Quick Scale Rubric (British Columbia Ministry of Education, 2009). The rubric is used to evaluate four areas of writing quality, including (a) ideas and meaning, (b) clarity and style, (c) form and organization, and (d) mechanics and conventions. Within each area, samples are evaluated as (a) not yet within expectations, (b) meets expectations (minimal level), (c) fully meets expectations, or (d) exceeds expectations. Values of 1 to 4 were assigned within each area, yielding a maximum total score of 16.

Writing duration

The time students spent planning and composing their writing samples was recorded. Timing began when students were told to begin writing and ended when the students returned the completed writing sample.

Writing self-efficacy

The Self-Efficacy for Writing Scale (SEWS) for middle school students (Bruning, Dempsey, Kauffman, McKim, & Zumbrunn, 2013) contains 16 items, and students are asked to rate how confident they are that they can perform writing-related skills (e.g., “I can think of many ideas for my writing.”) on a visual analog scale from 0 (no confidence) to 100 (complete confidence). A factor analysis indicated support for a three-factor solution (ideation, conventions, and self-regulation), with estimates of reliability at or above α = .84 for each factor (Bruning et al., 2013). Because the factors are moderately to strongly correlated (Bruning et al., 2013), we used an average of all SEWS items in analyses.

Procedure

Following university ethics review, we obtained parental consent for 12 students. All students met eligibility criteria and were randomly assigned to a within-grade pair, and each pair was randomly assigned to SRSD– (no explicit instruction on self-statements) or SRSD+ (explicit instruction on self-statements) conditions. The first author, a graduate student in school psychology at the time of the study and certified teacher, served as interventionist.

Assessment procedures

At baseline, all students completed the SEWS and a persuasive writing sample. All writing prompts were presented orally and in writing. Students were told they had 15 min to handwrite the sample. During intervention, students continued to complete persuasive writing samples, administered with the same procedures as baseline, once per week. The interventionist did not assist students during the writing samples; also, the students did not have access to any SRSD handouts or materials used in intervention sessions. The order of administered writing prompts across student groups was counterbalanced. The SEWS was administered again after the final intervention session.

Instructional procedures

Students received 30 min of persuasive writing instruction in same-condition pairs, two times a week for 5 weeks. Lesson plans were based on the POW + TREE plans in Harris et al.’s (2008). Participating students’ classroom teachers indicated that there was no direct instruction on paragraph writing beyond the intervention sessions during the time of the study, and all participating students indicated that they were unfamiliar with the POW + TREE strategy at study outset.

The SRSD + pairs were explicitly taught to use self-statements throughout all stages of the writing instruction. Self-statements were presented and described to the SRSD+ pairs as statements we make to ourselves as we write to help us remember what to do, calm ourselves down, and reinforce ourselves for doing something. The interventionist modeled self-statements throughout the instructional process and the students filled out worksheets identifying the self-statements they used, practiced using self-statements, and collaborated with each other in developing their self-statements. Examples of self-statements modeled by the interventionist are “If I follow the TREE strategy and work hard, I can write a good paragraph”; “First I need to remember to read the prompt a few times”; “The last E stands for Examine, so I need to re-read my work and look for errors”; and “I am getting upset; I need to calm down; just breathe and ideas will come.” Differences in instruction between the SRSD+ and SRSD– conditions are highlighted in Table 2.

Table 2.

Differences Between Instructional Conditions.

Stage of SRSD	SRSD+	SRSD–
Background knowledge	Self-statements were explicitly explained and encouraged Self-statement handout provided	Self-statements were not explained nor encouraged Self-statement handout was not provided
Discussing the strategy	Discussed the self-statements students used to help them remember POW + TREE	Self-statements were not discussed
Modeling	Think aloud modeling strategy was demonstrated Interventionist modeled self-statement use, including self-statements used to aid student memory, coping, and behavior reinforcement Students identified self-statements used by interventionist Students created and used their own self-statements	A descriptive modeling strategy was demonstrated Self-statements were not modeled Self-statements were not identified or created
Collaborative practice	Students used the self-statements they created during practice	Students practiced without using self-statements

Note. SRSD+ = self-regulated strategy development with explicit self-statement instruction; SRSD– = self-regulated strategy development without explicit self-statement instruction.

Fidelity of implementation

To ensure that the intervention was completed with fidelity, the interventionist completed a checklist of key features for each condition after each lesson. Average self-reported fidelity for the SRSD+ and SRSD– conditions was 99% and 98%, respectively. In addition, each lesson was audio recorded, and an independent evaluator checked adherence to lesson plans. Average treatment fidelity for the conditions was 100% and 98%.

Interscorer agreement

Two raters independently scored all writing samples. The primary rater did not implement any intervention sessions or know student assignments to experimental conditions. The first author (interventionist) served as the secondary rater, whose scores were only used to calculate interscorer agreement. To determine absolute agreement on continuous measures (i.e., WE-CBM, BCPS-W, and duration), Lin’s (1989) concordance correlation coefficient (r_c) was calculated. All continuous measures showed good interscorer agreement (r_c ≥ .85). For the ordinal basic writing elements scale, Cohen’s weighted kappa (κ_w), was calculated. Initial agreement was acceptable (κ_w = .74), but lower than desired; thus, the criteria were rediscussed and all samples were rescored, yielding a final agreement of κ_w = .95.

Data analysis

Because pairs of students were randomly assigned to experimental conditions and completed intervention sessions in pairs, differences in change on writing measures between SRSD+ and SRSD– conditions were evaluated using three-level multilevel models (time nested in students nested in pairs) with random intercepts at the student and pair levels. Multilevel models are often used and perform well in situations with repeated measures from a small number of students, for example, in single-case design studies (Ferron, Bell, Hess, Rendina-Gobioff, & Hibbard, 2009; Moeyaert, Ferron, Beretvas, & Van den Noortgate, 2014). All models included fixed effects for time (0 = preintervention, 1 = postintervention for SEWS; for writing elements, WE-CBM, BCPS-W, and writing duration, 0 = baseline, 1 = first week of intervention, 2 = second week of intervention, etc.), group (0 = SRSD–, 1 = SRSD+), and group × time. The fixed effect for group × time tested differences in change by SRSD condition, thereby evaluating the study’s primary research questions. To account for the ordinal scaling of the writing elements measure, a cumulative logit model was used.

Results

Means and standard deviations across time for all measures by intervention group are presented in Table 3. Results of models testing differences in change by SRSD condition are presented in Table 4. Overall, there were no statistically significant differences in change between the SRSD groups; however, there were statistically significant improvements (p < .01) on all measures other than TWW that did not differ by SRSD condition. These gains are described in more detail below. To aid in the interpretation of gains, standardized mean difference (d) effect sizes are reported for continuous dependent variables based on the overall model-predicted change during the study (i.e., b for time multiplied by 5 weeks for weekly measures) divided by the overall raw baseline SD (Feingold, 2009)

Table 3.

Means and Standard Deviations by Instructional Group, Measure, and Week of Intervention.

Measure	Group	Week 0	Week 1	Week 2	Week 3	Week 4	Week 5
Measure	Group	M (SD)	M (SD)	M (SD)	M (SD)	M (SD)	M (SD)
Elements	SRSD+	3.00 (0.89)	4.00 (0.89)	4.83 (0.41)	4.83 (0.41)	4.83 (0.41)	5.00 (0.00)
	SRSD–	2.00 (1.26)	3.33 (0.82)	4.67 (0.52)	4.17 (1.17)	4.83 (0.41)	4.83 (0.41)
TWW	SRSD+	83.17 (46.96)	79.83 (30.87)	75.00 (29.83)	76.83 (20.54)	88.67 (26.84)	88.50 (25.97)
	SRSD–	84.33 (59.46)	54.67 (47.00)	70.67 (20.18)	66.33 (19.31)	85.17 (10.23)	92.50 (33.28)
CWS	SRSD+	68.00 (39.00)	68.50 (32.40)	69.50 (29.83)	69.83 (16.56)	77.83 (28.17)	83.83 (25.93)
	SRSD–	55.00 (33.42)	63.33 (33.00)	62.67 (25.42)	55.33 (17.32)	67.00 (10.71)	87.00 (36.88)
%CWS	SRSD+	76.55 (4.66)	77.44 (16.53)	85.99 (9.56)	86.01 (8.13)	81.46 (11.18)	90.56 (3.74)
	SRSD–	66.25 (19.75)	75.20 (21.13)	82.28 (14.61)	79.98 (14.91)	74.36 (4.95)	86.52 (11.48)
BCPS-W	SRSD+	6.67 (1.21)	7.67 (2.58)	8.83 (1.47)	9.00 (0.89)	9.33 (1.51)	10.67 (1.51)
	SRSD–	5.83 (1.60)	6.00 (1.26)	8.83 (1.60)	8.33 (1.37)	8.83 (0.98)	9.83 (1.72)
Duration (m)	SRSD+	8.56 (1.62)	8.17 (3.28)	8.81 (2.35)	11.99 (2.94)	10.22 (2.57)	12.77 (3.90)
	SRSD–	6.76 (2.00)	8.33 (4.10)	6.43 (1.89)	9.58 (3.13)	11.78 (2.24)	13.24 (5.91)
SEWS	SRSD+	66.49 (12.84)	—	—	—	—	84.70 (8.34)
	SRSD–	55.25 (18.89)	—	—	—	—	80.35 (8.05)

Note. For each group, n = 6. SRSD+ = self-regulated strategy development with explicit self-statement instruction; SRSD– = self-regulated strategy development without explicit self-statement instruction; Elements = number of TREE strategy writing elements in composition. TWW = total words written; CWS = correct word sequences; %CWS = percent correct word sequences; BCPS-W = British Columbia Performance Standards for Writing; SEWS = Self-Efficacy for Writing Scale. —indicates that the measure was not administered in that week.

Table 4.

Model Results Testing Differential Change by Intervention Condition.

	Elements	TWW	CWS	%CWS	BCPS-W	Duration	SEWS
	Estimate (SE)	Estimate (SE)	Estimate (SE)	Estimate (SE)	Estimate (SE)	Estimate (SE)	Estimate (SE)
Fixed effects
Intercept	—_a	77.90*** (14.81)	53.37*** (11.75)	70.54*** (4.51)	5.94*** (0.51)	6.07*** (1.24)	55.25*** (5.39)
Group	1.23 (0.96)	0.17 (20.94)	11.87 (16.62)	6.60 (6.38)	0.95 (0.72)	1.84 (1.75)	11.24 (7.63)
Time	1.38*** (0.32)	1.09 (2.08)	4.68** (1.69)	2.76** (0.82)	0.80*** (0.12)	1.31*** (0.25)	25.10*** (4.56)
Group × time	0.09 (0.43)	0.49 (2.95)	−1.61 (2.40)	−0.41 (1.16)	−0.09 (0.16)	−0.44 (0.36)	−6.90 (6.46)
Random
Intercept (student)	0.73	—_c	12.25	9.09	0.85	0.46	6.03
Intercept (pair)	0.20	23.21	16.14	1.07	0.22	1.65	6.15
Residual	—_b	21.35	17.36	8.42	1.19	2.59	7.91

Note. n = 12. Random effects are presented as standard deviations. Four thresholds (SE) were estimated instead of a model intercept: 1|2 = −1.25 (0.81), 2|3 = 0.21 (0.71), 3|4 = 1.41 (0.73), 4|5 = 3.70 (0.91). Elements = number of TREE strategy writing elements in composition. TWW = total words written; CWS = correct word sequences; %CWS = percent correct word sequences; BCPS-W = British Columbia Performance Standards for Writing; SEWS = Self-Efficacy for Writing Scale. Group was coded as 0 = SRSD– or 1 = SRSD+. –_a = A cumulative logit model was used to address ordinal response variable. –_b = no residual variance was estimated due to use of cumulative logit model. –_c = this variance component was removed from the model because it was approximately zero.

p < .01. ***p < .001.

On TREE writing elements, there was statistically significant weekly change (p < .001), improving from M = 2.50 at baseline to M = 4.92 at study end, indicating that students could write compositions with nearly all TREE elements. Students’ odds of a one-element gain increased by 3.97 per week (e^1.38 = 3.97). On TWW, scores increased from M = 83.75 at baseline to M = 90.50 at study end, but the change was not statistically significant (p = .60) and of small magnitude (d = 0.07). Despite no statistically significant change in TWW, there were statistically significant gains in other WE-CBM indicators more reflective of composition quality, CWS (p = .008) and %CWS (p = .001). On CWS, students’ scores improved from M = 61.50 at baseline to M = 85.42 at study end (d = 0.66). On %CWS, scores improved from M = 71.40% to M = 88.54% at study end (d = 0.94). Similar results were found on BCPS-W (p < .001), with scores improving from M = 6.25 to M = 10.25 (d = 2.82) on the 16-point scale reflecting provincial writing standards. Students’ duration of composition also increased (p < .001) from M = 7.66 to M = 13.00 min at study end (d = 3.32). Regarding self-efficacy for writing, students’ SEWS scores increased (p < .001) from M = 60.87 to M = 82.53 at study end (d = 1.52).

Discussion

The current study examined the importance of the self-statement component of SRSD with two primary objectives. First, we investigated whether explicit instruction on the use of self-statements (SRSD+) would result in improved persuasive writing skill when compared with instruction that did not provide these instructional activities (SRSD–). Second, we determined whether explicit instruction on self-statements would differentially affect student self-efficacy for writing.

Overall, both SRSD groups showed medium to large improvements on most outcomes over time; however, results indicated that there were no statistically significant differences in improvements between the SRSD+ and SRSD– groups. Because the length of instructional time for each group was held constant, these results suggest that explicit instruction on the use of self-statements may not be an essential component of the SRSD model.

Several possibilities may explain the similar improvements in writing outcomes across the SRSD conditions. First, prior component analyses examining goal setting and self-monitoring in SRSD have yielded mixed findings (Brunstein & Glaser, 2011; Glaser & Brunstein, 2007; Graham & Harris, 1989a; Sawyer et al., 1992), despite general research evidence for their effectiveness in improving student writing (e.g., Alitto, Malecki, Coyle, & Santuzzi, 2016; Glaser, Palm, & Brunstein, 2012). Thus, it is possible that the writing task instructional strategies, more so than individual self-regulation instructional components such as goal setting, self-monitoring, and self-statement instruction, may be the main driver of SRSD outcomes. Second, due to finite instructional time in each session, students in SRSD+ may not have received enough direct instruction in either writing task strategies or use of self-statements to outperform students who received only direct instruction in persuasive writing strategies. Third, it is possible that some of the SRSD– students did use self-statements effectively prior to the intervention. Instruction on the use of self-statements may be more effective on improving student writing for students who were not already using self-statements to regulate their behavior.

Limitations

Results of this study should be interpreted considering several limitations. First, the sample size of the study was small, with six students in each SRSD condition, limiting both statistical power and the generalizability of results. Second, the lead author was the interventionist for both groups. Ideally, the interventionist would not have been aware of the research questions or conditions to reduce potential bias. This concern is reduced somewhat by independent review of lessons for fidelity and interscorer agreement; however, it is possible that the interventionist behaved differently across groups based on knowledge of the study.

Research Contributions and Future Directions

The current study is one of few studies that have conducted component analyses of SRSD. We found that similar outcomes were obtained, regardless of the extent to which self-statements were emphasized, and these results are similar to some prior component analyses finding that other self-regulation components do not appear to be essential components of SRSD (Graham & Harris, 1989a; Sawyer et al., 1992). Considering that other studies have found differential effects when self-regulation components are included in SRSD (Brunstein & Glaser, 2011; Glaser & Brunstein, 2007), additional studies are needed before making strong conclusions given the few component analyses in the 100+ studies on SRSD.

Although examining overall change during intervention across groups is limited by no comparison condition, and thus, observed changes cannot be conclusively attributed to SRSD, the observed gains were typical of reported effect sizes for SRSD. Graham et al. (2013) reported a weighted effect size of 1.75 for writing quality outcomes; in the current study, we found an effect of d = 2.82 based on the BCPS-W rubric that aligns with provincial standards; d = 0.94 on %CWS, a WE-CBM metric that reflects writing accuracy (i.e., percentage of word sequences that are spelled correctly and make sense in context); and d = 0.66 on CWS, a WE-CBM metric that considers accurate writing production. Relative to gains in writing quality, gains in writing length are typically smaller for SRSD (0.47 in Graham et al., 2013). In the current study, we had separate measures of composition length (TWW) and writing duration; although length did not increase (d = 0.07), students spent more time on compositions as intervention progressed (d = 3.32). In sum, these results provide additional evidence of replicated outcomes in research teams independent of the SRSD developers and in Canada. The current study is the second study of SRSD to demonstrate gains in quality when evaluated on BCPS-W (see Reynolds & Perin, 2009) and the third study (also see Wong et al., 2008) demonstrating outcomes for students in Grades 5 to 7 in British Columbia. Based on these findings, in conjunction with Milford and Harrison’s (2010) and Sinclair’s (2014) results demonstrating improvements in expository writing for British Columbia students in Grades 3 to 5, teachers in Canada should be encouraged to utilize SRSD with students struggling with persuasive and expository writing in their classrooms. Some SRSD strategies focus on narrative writing (Harris, Graham, & Adkins, 2015); however, these have yet to be evaluated in Canada.

Although SRSD has been extensively evaluated, few studies have examined changes in writing self-efficacy. The observed large change in writing self-efficacy (d = 1.52) is in line with the few SRSD studies reporting self-efficacy gains (Brunstein & Glaser, 2011; Graham & Harris, 1989a, 1989b); however, other studies have found no change in self-efficacy (García-Sánchez & Fidalgo-Redondo, 2006; Graham et al., 2005; Sawyer et al., 1992). Given the few SRSD studies examining self-efficacy, it is difficult to determine whether variable findings are true differences in outcome versus an artifact of differences in sampling or outcome measures; thus, inclusion of self-efficacy measures in future SRSD is recommended. Determining the specific SRSD conditions that yield changes in self-efficacy is important given the prominent position of self-efficacy in self-regulated learning theory (Zimmerman & Risemberg, 1997).

Conclusion

Although limited, overall results are consistent with prior research demonstrating that SRSD improves student writing, providing additional evidence of replication in an independent research team and in Canada. There were no differences found between the SRSD groups that differed in instruction on self-statements, indicating that this feature may not be a critical component. Future component analyses of SRSD are encouraged, so that practitioners can determine how best to allocate instructional and intervention time.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported, in part, by a graduate student research grant from the National Association of School Psychologists to the first author, scholarships from the Social Sciences and Humanities Research Council of Canada to the third and fourth authors, and a research grant by the Social Sciences and Humanities Research Council of Canada to the second author.

Author Biographies

Rhonda Geres-Smith, MA, MEd, completed her MEd degree in curriculum development at the University of Victoria and her MA degree in school psychology at the University of British Columbia. She is currently a PhD student in curriculum and instruction at University of Victoria and a school psychologist in the Cowichan Valley School District in Duncan, BC.

Sterett H. Mercer, PhD, is an associate professor in special education and school psychology at the University of British Columbia. His research interests include methods to improve judgment and decision making using curriculum-based academic measures and academic intervention.

Catherine Archambault, MA, completed her MA degree in school psychology at the University of British Columbia. She is currently a PhD student in School/Applied Child Psychology at McGill University. Her research interests include bilingualism and reading achievement.

Jamie M. Bartfai, MA, completed her MA degree in school psychology at the University of British Columbia. She is currently a school psychologist for the Greater Victoria School District in Victoria, BC. Her research interests include examining the educational implications of embodied cognition through the investigation of multisensory methods in early literacy programs.

References

Alitto

Malecki

C. K.

Coyle

Santuzzi

(2016). Examining the effects of adult and peer mediated goal setting and feedback interventions for writing: Two studies. Journal of School Psychology, 56, 89-109. doi:10.1016/j.jsp.2016.03.002

Baker

S. K.

Chard

D. J.

Ketterlin-Geller

L. R.

Apichatabutra

Doabler

(2009). Teaching writing to at-risk students: The quality of evidence for self-regulated strategy development. Exceptional Children, 75, 303-318. doi:10.1177/001440290907500303

British Columbia Ministry of Education. (2009). BC performance standards: Writing. Retrieved from http://www.bced.gov.bc.ca/perf_stands/writing_intro.pdf

British Columbia Ministry of Education. (2014). Independent school reports: Student statistics—2013/14. Retrieved from http://www.bced.gov.bc.ca/reports/pdfs/student_stats/03996145.pdf

Bruning

Dempsey

Kauffman

D. F.

McKim

Zumbrunn

(2013). Examining dimensions of self-efficacy for writing. Journal of Educational Psychology, 105, 25-28. doi:10.1037/a0029692

Brunstein

J. C.

Glaser

(2011). Testing a path-analytic mediation model of how self-regulated writing strategies improve fourth graders’ composition skills: A randomized controlled trial. Journal of Educational Psychology, 103, 922-938. doi:10.1037/a0024622

Cook

B. G.

Therrien

W. J.

Coyne

M. D.

Makel

M. C.

Plucker

J. A.

Freeman

Coyne

(2016). Replication of special education research: Necessary but far too rare. Remedial and Special Education, 37, 205-212. doi:10.1177/0741932516646083

Council of Ministers of Education Canada. (2003). School achievement indicators program: Report on writing assessment III. Retrieved from http://www.cmec.ca/Publications/Lists/Publications/Attachments/7/saip2002.en.pdf

Feingold

(2009). Effect sizes for growth-modeling analysis for controlled clinical trials in the same metric as for classical analysis. Psychological Methods, 14, 43-53. doi:10.1037/a0014699

10.

Ferron

J. M.

Bell

B. A.

Hess

M. R.

Rendina-Gobioff

Hibbard

S. T.

(2009). Making treatment effect inferences from multiple-baseline data: The utility of multilevel modeling approaches. Behavior Research Methods, 41, 372-384.

11.

García-Sánchez

J.-N.

Fidalgo-Redondo

(2006). Effects of two types of self-regulatory instruction programs on students with learning disabilities in writing products, processes, and self-efficacy. Learning Disability Quarterly, 29, 181-211. doi:10.2307/30035506

12.

Glaser

Brunstein

J. C.

(2007). Improving fourth-grade students’ composition skills: Effects of strategy instruction and self-regulation procedures. Journal of Educational Psychology, 99, 297-310. doi:10.1037/0022-0663.99.2.297

13.

Glaser

Palm

Brunstein

J. C.

(2012). Writing strategies instruction for fourth graders with and without problem behavior: Effects of self-monitoring and operant procedures on compositional achievements and on-task behavior. Zeitschrift für Pädagogische Psychologie, 26, 19-30. doi:10.1024/1010-0652/a000057

14.

Graham

Harris

K. R.

(1987). Improving composition skills of inefficient learners with self-instructional strategy training. Topics in Language Disorders, 7, 66-77. doi:10.1097/00011363-198709000-00008

15.

Graham

Harris

K. R.

(1989a). Components analysis of cognitive strategy instruction: Effects on learning disabled students’ compositions and self-efficacy. Journal of Educational Psychology, 81, 353-361. doi:10.1037/0022-0663.81.3.353

16.

Graham

Harris

K. R.

(1989b). Improving learning disabled students’ skills at composing essays: Self-instructional strategy training. Exceptional Children, 56, 201-214.

17.

Graham

Harris

K. R.

Mason

(2005). Improving the writing performance, knowledge, and self-efficacy of struggling young writers: The effects of self-regulated strategy development. Contemporary Educational Psychology, 30, 207-241. doi:10.1016/j.cedpsych.2004.08.001

18.

Graham

Harris

K. R.

McKeown

(2013). The writing of students with learning disabilities, meta-analysis of self-regulated strategy development writing intervention studies: Redux. In Swanson

H. L.

Harris

K. R.

Graham

(Eds.), Handbook of learning disabilities (2nd ed., pp. 405-438). New York, NY: Guilford Press.

19.

Harris

K. R.

Graham

Adkins

(2015). Practice-based professional development and self-regulated strategy development for tier 2, at-risk writers in second grade. Contemporary Educational Psychology, 40, 5-16. doi:10.1016/j.cedpsych.2014.02.003

20.

Harris

K. R.

Graham

Mason

L. H.

Friedlander

(2008). Powerful writing strategies for all students. Baltimore, MD: Brooks Publishing.

21.

Horner

R. H.

Carr

E. G.

Halle

McGee

Odom

Wolery

(2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71, 165-179. doi:10.1177/001440290507100203

22.

Hosp

M. K.

Hosp

J. L.

Howell

K. W.

(2016). The ABCs of CBM: A practical guide to curriculum-based measurement. New York, NY: Guilford Press.

23.

Kaufman

A. S.

Kaufman

N. L.

(2004). Kaufman brief intelligence test (2nd ed.). Bloomington, MN: Pearson.

24.

Lin

L. I.

(1989). A concordance correlation coefficient to evaluate reproducibility. Biometrics, 45, 255-268. doi:10.2307/2532051

25.

Mason

L. H.

Harris

K. R.

Graham

(2011). Self-regulated strategy development for students with writing difficulties. Theory into Practice, 50, 20-27. doi:10.1080/00405841.2011.534922

26.

Meichenbaum

(1977). Cognitive-behavior modification: An integrative approach. New York, NY: Plenum Press.

27.

Mercer

S. H.

Idler

A. M.

Bartfai

J. M.

(2014). Theory-driven evaluation in school psychology intervention research: 2007-2012. School Psychology Review, 43, 119-131.

28.

Milford

Harrison

G. L.

(2010). Using the PLEASE strategy with a struggling middle school writer with a disability. Intervention in School and Clinic, 45, 326-332. doi:10.1177/1053451209359080

29.

Moeyaert

Ferron

J. M.

Beretvas

S. N.

Van den Noortgate

(2014). From a single-level analysis to a multilevel analysis of single-case experimental designs. Journal of School Psychology, 52, 191-211. doi:10.1016/j.jsp.2013.11.003

30.

National Center for Education Statistics. (2017). The condition of education 2017: English language learners in public schools. Retrieved from https://nces.ed.gov/programs/coe/indicator_cgf.asp

31.

National Commission on Writing. (2004). Writing: A ticket to work . . . or a ticket out. New York, NY: College Board.

32.

Pajares

Valiante

(1997). Influence of self-efficacy on elementary students’ writing. The Journal of Educational Research, 90, 353-360. doi:10.1080/00220671.1997.10544593

33.

Reynolds

G. A.

Perin

(2009). A comparison of text structure and self-regulated writing strategies for composing from sources by middle school students. Reading Psychology, 30, 265-300. doi:10.1080/02702710802411547

34.

Sawyer

R. J.

Graham

Harris

K. R.

(1992). Direct teaching, strategy instruction, and strategy instruction with explicit self-regulation: Effects on the composition skills and self-efficacy of students with learning disabilities. Journal of Educational Psychology, 84, 340-352. doi:10.1037/0022-0663.84.3.340

35.

Schunk

D. H.

(2008). Attributions as motivators of self-regulated learning. In Schunk

D. H.

Zimmerman

B. J.

(Eds.), Motivation and self-regulated learning: Theory, research, and applications (pp. 245-266). New York, NY: Routledge.

36.

Schunk

D. H.

Cox

P. D.

(1986). Strategy training and attributional feedback with learning disabled students. Journal of Educational Psychology, 78, 201-209. doi:10.1037/0022-0663.78.3.201

37.

Shell

D. F.

Murphy

C. C.

Bruning

R. H.

(1989). Self-efficacy and outcome expectancy mechanisms in reading and writing achievement. Journal of Educational Psychology, 81, 91-100. doi:10.1037/0022-0663.81.1.91

38.

Sinclair

(2014). The effectiveness of an SRSD writing intervention for students with epilepsy (Master’s thesis, University of Victoria, British Columbia, Canada). Retrieved from https://dspace.library.uvic.ca/bitstream/handle/1828/5222/Sinclair_Kristin_MA_2014.pdf

39.

Welch

(1992). The PLEASE strategy: A metacognitive learning strategy for improving the paragraph writing of students with mild learning disabilities. Learning Disability Quarterly, 15, 119-128. doi:10.2307/1511013

40.

Wong

B. Y. L.

Hoskyn

Jai

Ellis

Watson

(2008). The comparative efficacy of two approaches to teaching sixth graders opinion essay writing. Contemporary Educational Psychology, 33, 757-784. doi:10.1016/j.cedpsych.2007.12.004

41.

Woodcock

McGrew

Mather

(2007). Woodcock-Johnson tests of achievement III, form B, with normative update. Toronto, Ontario, Canada: Nelson Education.

42.

Zimmerman

B. J.

Risemberg

(1997). Becoming a self-regulated writer: A social cognitive perspective. Contemporary Educational Psychology, 22, 73-101. doi:10.1006/ceps.1997.0919

Student number	Grade	Age	Gender	Verbal knowledge KBIT-2	Matrices KBIT-2	Writing samples WJ-III ACH
1	7	12	F	85	93	102
2	7	14	F	80	83	93
3	7	12	F	90	98	110
4	7	12	M	95	85	116
5	6	11	M	95	81	102
6	6	12	M	85	97	85
7	6	12	M	100	105	115
8	6	11	M	110	111	104
9	5	10	M	105	81	105
10	5	10	M	85	98	100
11	5	11	M	85	82	114
12	5	11	M	90	82	104

Student number	Grade	Age	Gender	Verbal knowledge KBIT-2	Matrices KBIT-2	Writing samples WJ-III ACH
1	7	12	F	85	93	102
2	7	14	F	80	83	93
3	7	12	F	90	98	110
4	7	12	M	95	85	116
5	6	11	M	95	81	102
6	6	12	M	85	97	85
7	6	12	M	100	105	115
8	6	11	M	110	111	104
9	5	10	M	105	81	105
10	5	10	M	85	98	100
11	5	11	M	85	82	114
12	5	11	M	90	82	104

Student number	Grade	Age	Gender	Verbal knowledge KBIT-2	Matrices KBIT-2	Writing samples WJ-III ACH
1	7	12	F	85	93	102
2	7	14	F	80	83	93
3	7	12	F	90	98	110
4	7	12	M	95	85	116
5	6	11	M	95	81	102
6	6	12	M	85	97	85
7	6	12	M	100	105	115
8	6	11	M	110	111	104
9	5	10	M	105	81	105
10	5	10	M	85	98	100
11	5	11	M	85	82	114
12	5	11	M	90	82	104