Abstract
Theories of writing development posit several component skills as necessary to the writing process. This meta-analysis synthesizes the literature on the correlation between these proposed component skills and writing outcomes. Specifically, in this study, we examine the bivariate relationships between handwriting fluency, spelling, reading, and oral language and students’ quality of writing and writing production. Additionally, the extent to which such relationships are moderated by student grade level and type of learner is also investigated. The findings document that each of the component skills demonstrates a weak to moderate positive relationship to outcomes assessing writing quality (rs = .33−.49) and the amount students write (rs = .20−.48). Moderator analyses were generally not significant with the exception that the relationship between reading and writing production was significantly higher for students in the primary grades. The implications of these findings to current theories and future research are discussed.
The very act of putting language into words on paper, or writing, is a complex task that requires considerable knowledge and cognitive effort. Flower and Hayes (1980) described the writing process as follows: “As a dynamic process, writing is the act of dealing with an excessive number of simultaneous demands or constraints. Viewed this way, a writer in the act is a thinker on full-time cognitive overload” (p. 33). Learning to write effectively in the early grades often involves a focus on letter formation, basic spelling, and composing of simple descriptive and narrative texts, whereas in the secondary grades, more complex writing activities such as conveying arguments with evidence are required. Although the quality of one’s writing certainly progresses over time, hallmarks of compositional quality include the production of text that is structured and coherent, demonstration of a logical and sequential development of ideas and details in text, inclusion of a variety of sentence structures with appropriate use of grammar conventions, purposeful use of words and phrases, and correct application of writing conventions and mechanics (National Center for Education Statistics [NCES], 2012). Given these expectations, one can hypothesize that writers across the developmental span must call on different skills and cognitive resources to meet increasing demands of the writing task.
The sheer complexity of the writing process is highlighted by the relatively poor writing achievement of students in the United States. Only one quarter of students in 8th and 12th grades demonstrated writing proficiency on the most recent National Assessment of Educational Progress (NCES, 2012), and one in five students performed at a below basic level. Data for the nation’s fourth-grade students are remarkably similar, with only 28% scoring at or above the proficient level (Persky, Daane, & Jin, 2003). Additional research supports the high prevalence of writing difficulties across grade levels (Hooper, Swartz, Montgomery, & Reed, 1993; Katusic, Colligan, Weaver, & Barbaresi, 2009; Mayes & Calhoun, 2007). In an epidemiological study, Katusic et al. (2009) identified 14% of a birth cohort as meeting one of three criteria for written language disorder, and 53% of all students with learning problems exhibited a written language disorder. Hooper et al. (1993) found that the percentage of students whose writing proficiency fell at least 1 standard deviation (SD) below the mean was 19% to 52% across three geographic samples of middle school students. In a study of learning disability subtypes, Mayes and Calhoun (2007) found that 92% of students showed evidence of a learning disability in written expression, with one-half of these students demonstrating a learning disability in written expression alone. These data further bring to light the need for greater understanding of writing development. The purpose of this meta-analysis is to examine the extant literature on correlates of writing skill across the grade levels with the goal of better understanding writing development over time. Additionally, a review of existing evidence for component skills of writing may be beneficial for informing writing instruction with an ultimate goal of improving students’ writing ability.
Component Processes in Writing
Prior to the 1980s, there existed a stronger focus on the written products produced by students rather than the actual process involved when composing (Berninger, Garcia, & Abbott, 2009). However, coinciding with the influential work of Flower and Hayes (1980; Hayes & Flower, 1980), this paradigm began to shift toward a focus on both product and process. Conceptualizing writing from a problem-solving approach, they attempted to describe the processes that a writer goes through when generating text—that is, planning (to include idea development, organization, and goal setting), translating/generating text, reviewing (revising/proofreading), and self-monitoring during writing. Since this early work on proficient adult writers, several models of the component cognitive skills and processes involved in writing during childhood and adolescence have been proposed.
Recognizing that lower level skills may be necessary for individuals to engage in the high-order cognitive processes required in writing, Juel and colleagues sought to describe and provide empirical support for their simple view of writing (Juel, 1988; Juel, Griffith, & Gough, 1986). Akin to Gough and Tunmer’s (1986) simple view of reading that described reading as primarily influenced by decoding skill and listening comprehension, the simple view of writing theorized writing development as guided by two components, ideation and spelling. Ideation was thought to involve the origination of thoughts and ideas and subsequent generation of these ideas into written text (similar to processes described by Hayes & Flower, 1980). Although early work appeared to support the influence of these processes (Juel, 1988), evidence also suggested that the relative impact of these skills changed over time, with spelling demonstrating greater influence than ideation in first grade and vice versa in fourth grade.
Berninger, Abbott, Abbott, Graham, and Richards (2002) presented their own simple view of the functional writing system with text generation at the word, sentence, and discourse levels influenced by (a) transcription skills such as spelling and handwriting fluency (or keyboarding) and (b) executive functions that assist in self-regulation during the writing process (Berninger et al., 2002). In this model, executive functions regulate processes such as those described by Hayes and Flower (1980): planning, reviewing, and revising, as well as attentional processes. Recognizing the higher level demands placed on writers beyond early childhood, Berninger and Winn (2006) expanded this model to highlight how the role of working memory during planning, reviewing, and revising increases in importance, as does supervisory attention that regulates attention, inhibition, and mental shifting (Berninger & Winn, 2006).
McCutchen (2000) also emphasized the role of transcription skills and working memory and described how these memory processes are coordinated during writing. McCutchen posited that novice writers face constraints from a lack of automaticity in transcription skills (see Bourdin & Fayol, 2000; Graham & Harris, 2000, for a review of evidence), as well as limited writing-relevant knowledge. The lack of fluent transcription and limited knowledge taxes short-term working memory, and novice writers are overwhelmed with the processing demands of the writing task. McCutchen argued that once text generation becomes more fluent, as occurs across development, and genre and topic knowledge increase, individuals become more skilled writers who are no longer constrained by working memory and can better access long-term memory during writing.
Despite the extant theoretical models of writing, there is still much to learn about the development of writing (Graham, Gillespie, & McKeown, 2013). As can be seen from the aforementioned summary of models of writing processes and development, several factors merge for an individual to generate text. Furthermore, although overlapping in some areas, the different models place emphasis on different skills/processes at different points of development. Transcription skills such as handwriting fluency and spelling are foundational skills that, when not as fluent, may serve to constrain individual’s access to higher level processes (Berninger et al., 2002; Berninger & Winn, 2006; Juel et al., 1986). As described by Juel and colleagues, oral language has been thought to influence writing, as one must be able to generate and translate language (ideas) into written words, and semantic and syntactic knowledge may assist in the fluency of text at the sentence and discourse levels. Several of the models of writing highlight executive functions and self-regulation as higher order processes that allow strategic writing to take place through the coordination of working memory, attention, inhibition, and attention shifting (Berninger et al., 2002; Berninger & Winn, 2006; McCutchen, 2000).
Another important component involved in the writing process is reading. Researchers (Fitzgerald & Shanahan, 2000; Shanahan, 2006) have hypothesized that reading and writing processes have a reciprocal relationship, most likely due to common cognitive processes such as phonological and orthographic systems and short- and long-term memory, as well as shared knowledge (e.g., metaknowledge, knowledge of text attributes). Evidence from neuroimaging studies have also demonstrated that during reading and writing activities there is an overlap in the activation of specific brain regions (Pugh et al., 2006). It has also been posited that the process of self-monitoring during planning and revision relies in part on reading skill—that is, being able to decode and comprehend what has been written to assess and plan for revisions (Hayes, 1996; McCutchen, Francis, & Kerr, 1997). However, although studies have shown that reading and writing each have a positive relation with the other, much less is known about the developmental nature of the relationship (Ahmed, Wagner, & Lopez, 2014).
Two areas of research, subtype analysis and instructional studies, also provide valuable insight into the potential influence of these skills on students’ writing. Berninger, Hart, Abbott, and Karovsky (1992) found that for students with writing disorders, the prevalence of difficulties in handwriting or spelling alone was 1% to 3% and 3% to 4%, respectively; and Mayes and Calhoun (2007) also found 4% of writing problems were marked by spelling deficits alone while another 28% were comorbid with deficits in written expression. Although often students demonstrate a combination of writing difficulties such as grammatical errors, poor paragraph organization, multiple spelling errors, and poor handwriting, Katusic et al. (2009) found that almost one in five students had a sole deficit in either handwriting (5%) or spelling (13%). Wakely, Hooper, de Kruif, Renee, and Swartz (2006) identified six subtypes of writers in fourth and fifth grades. Two clusters included students considered either average or expert writers, while the other four showed evidence of deficits in either language-based skills, such as semantics or grammar, spelling and reading performance, or discourse-level text quality. Students in the subgroup marked by poor text quality also demonstrated relative weakness with writing self-regulation and metacognition during writing. Finally, the overlap between disorders of written language and reading presents further support for a reading–writing connection. Rates of comorbidity have ranged from 30% (Mayes & Calhoun, 2007) to 75% (Berninger, Hart, et al., 1992) of students with a learning disability in written language also exhibiting a reading disability.
In a recent meta-analysis of writing instruction in the elementary grades (Grades 1–3), the explicit teaching of transcription skills, including spelling, handwriting, and word processing/keyboarding, was found to enhance writing quality with an effect size (ES) of .55 (Graham, McKeown, Kiuhara, & Harris, 2012). The teaching of self-regulatory strategies has also been shown to have strong effects on writing performance across both elementary (Graham, McKeown et al., 2012) and secondary (Graham & Perin, 2007) grades. With regard to reading, meta-analyses (Graham & Hebert, 2011; Hebert, Gillespie, & Graham, 2013) have shown that having students write about what they read improves reading comprehension and that writing instruction results in improved outcomes in reading comprehension (Grades 4–12), reading fluency (Grades 1–7), and word recognition (Grades 1–5). Of note, there is a dearth of empirical instructional research that investigates the effects of oral language instruction on student writing outcomes.
It is not surprising, given this previous research, that a recently released practice guide for writing instruction in the elementary grades recommends the allocation of specific time for the teaching of basic writing skills, including spelling and handwriting or word processing (Graham, Bollinger, et al., 2012). Moats (2004) has also suggested teachers address these early foundational skills in order that children may engage in the higher order processes of writing. The focus on component skills should not be viewed as a call for the isolated teaching of such skills at the expense of instruction on the process of writing, but instead these component skills may be viewed as developmental skills that influence, or supplement, the process and product of writing (Berninger, Yates, et al., 1992). Furthermore, although recent surveys suggested a heavy reliance on the process approach to writing instruction (Cutler & Graham, 2008; Kiuhara, Graham, & Hawken, 2009), Langer (2001) found that the majority of the most successful middle and high school English teachers explicitly used a combination of instruction focused on teaching isolated skills, application and practice of skills taught, and integration of knowledge and skills within an embedded activity.
Study Rationale and Research Questions
Although research in the area of writing appears to have proliferated in recent years, there remain questions to be answered. Graham, McKeown, et al. (2012) identified a crucial need for studies in the area of writing theory that specifically address the impact of various skills to writing across the developmental spectrum. Much of the extant theoretical and empirical research has focused on a specific developmental level (e.g., elementary, secondary grades), component skill or process (e.g., transcription and writing, reading and writing), or type of learner (e.g., typically developing, struggling writers). Miller and McCardle (2011) also highlighted the need for continued research on the development of writing and, more specifically, the relationship of writing to reading and oral language. Furthermore, they advocated for research not just at the earliest level but also for subgroups of students with academic difficulties in reading and writing. To date, there exist no empirical studies that specifically examine the relationship of multiple potential component skills of writing to quality of writing and production/fluency of text generation across the developmental span (i.e., lower elementary through secondary grades), and for students with and without disabilities. Although the sheer magnitude of resources that would be required to carry out such an endeavor makes such a study unlikely, meta-analysis provides a method by which this important information from many individual studies can be summarized and quantified.
In summary, this meta-analysis is intended to further clarify and elaborate models of writing development, namely, improving the understanding of how component skills and processes are related to writing quality and writing production. In recognition of the wide variation in how writing is both learned and used across grade levels, we also seek to examine influences of specific components on writing across development. Finally, with an understanding that there exist individual differences in student’s writing ability, this study investigates whether the nature of relations between component processes and writing is different for students exhibiting learning difficulties. This information has important implications for instruction and remediation of writing in the school setting. Given the complex and multifaceted nature of self-regulation and executive functioning, it was determined that it would be difficult to effectively examine all the elements of these processes along with all the other components; therefore, we chose not to include them in the current synthesis.
The following research questions were addressed:
Method
Search Terms and Procedures
Prior to conducting the systematic search for potential studies, a list of search terms was generated for component skill constructs and for writing outcomes. The following set of keywords related to component skills was employed: grammar, language, listening comprehension, morphology, semantics, syntax, vocabulary, speech, decoding, phonemic awareness, phonics, word recognition, word study, reading comprehension, beginning reading, early reading, independent reading, reading fluency, reading, spelling, encoding, handwriting, letter writing fluency, and transcription. To identify the broadest range of studies related to writing outcomes, these keywords were linked to the keywords writing, composition, basic writing, and written language using an “AND” statement.
The search for relevant studies was conducted using the Education Resources Information Center (ERIC) and PsycINFO databases. Final search parameters included limiting the search to articles published in the last 25 years (January 1, 1988, through May 2013) as well as in peer-reviewed, English language journals. We chose 1988 as the year to begin our search given that this year marked the emergence of initial studies examining component processes of writing as posited in popular theories of writing development (e.g., Juel, 1988). The initial search yielded 13,864 potential articles.
Inclusion and Exclusion Criteria
A list of screening criteria was formulated to identify the most relevant studies to answer the research questions. First, only studies with samples of school-age children were included; studies with samples of students in preschool or postsecondary were excluded unless they also included disaggregated data for students in kindergarten through high school. Second, studies had to be published in a peer-reviewed journal and printed in English to be selected. We limited the search to studies that had gone through the peer review process in order to assist in controlling for the quality of the research in our review. This same inclusion criteria has been used in several recent meta-analyses (García & Cain, 2014; National Early Literacy Panel, 2008; Scammacca, Roberts, Vaughn, & Stuebing, 2013; Stuebing et al., 2014). Although recognizing the concerns that arise with the potential for publication bias when including only published research, it has been noted that in this specific type of research (i.e., correlational), authors frequently evaluate hypotheses both in support of and against significant and substantial relationships among constructs; thus, published studies would not be as susceptible to bias as in some other areas of research (García & Cain, 2014).
Third, included studies had to measure component skills and writing outcomes within the English language. Relevant studies in non-English language systems were excluded. Fourth, only studies that involved quantitative, empirical research on groups of students with no disabilities and/or students with academic difficulties and high-incidence disabilities (e.g., learning disabilities, speech/language impairments) were included. Studies with data provided only for students with cognitive or sensory disabilities were excluded. Mixed-methods studies were eligible for inclusion as long as they specifically included the requisite quantitative data.
Fifth, studies that sought to specifically examine the concurrent relationship between a component skill (or multiple component skills) and writing quality or writing production were chosen; studies that only provided longitudinal data were excluded, as were instruction/intervention studies. Intervention studies were excluded due to concerns about identifying all such studies that include correlations between the component skills of interest and writing outcomes. Namely, there could be many studies of academic interventions, including those not specific to writing, that include a writing measure and where correlations with other skills (e.g., reading) are provided (e.g., Wanzek et al., 2015). Thus, there was a strong likelihood that obtained search results would include only a partial representation of these studies, making replication of search results difficult. Sixth, to be selected for this synthesis, the study had to include a direct student measure of constructs (e.g., standardized or researcher-created assessment), so studies that only measured reading, writing, oral language, or transcription skill via teacher or parent report were excluded. Finally, studies had to include bivariate correlations (r) between at least one component skill and one writing outcome or contain all necessary statistics for computing the correlation coefficient.
To further ensure the comprehensiveness of the search procedures, additional searching of the reference lists of existing, related meta-analyses and of studies identified for the current synthesis, as well as hand searches of relevant journals for 2010 to 2013 were conducted to identify any additional potential studies. The specific journals searched were Exceptional Children, Journal of Educational Research, Journal of Educational Psychology, Journal of Learning Disabilities, Journal of Literacy, Journal of School Psychology, Learning and Individual Differences, Reading and Writing: An Interdisciplinary Journal, Reading and Writing Quarterly, School Psychology Review, and Written Communication. These specific journals were selected as they represent common mediums for studies involving writing and have been included in previous meta-analyses in the area of writing (e.g., Graham, McKeown et al., 2012).
After a review of potential articles identified from the initial electronic and hand searches, 43 studies met criteria for the present meta-analysis. Two studies (Abbott & Berninger, 1993; Graham, Berninger, Abbott, Abbott, & Whitaker, 1997) used overlapping samples, but examined different component skills; thus, both were included in this meta-analysis. However, five studies (Berninger & Abbott, 2010; Berninger, Cartwright, Yates, Swanson, & Abbott, 1994; Berninger, Yates, et al., 1992; Puranik & Al Otaiba, 2012; Whitaker, Berninger, Johnston, & Swanson, 1994) were subsequently excluded because they reported correlations from the same sample and construct measures as another included study. Thus, 38 separate studies were included. A flowchart outlining the stages and results of the search process is provided in Figure 1.
Flowchart of search process to determine study eligibility.
Coding Process
A codesheet was developed specifically for this study in order to code relevant variables of interest to address the research questions. Five open-ended items captured information about each published study (e.g., author names, title of article, journal information). Sample demographics for the total sample and/or disaggregated groups (e.g., by grade level or disability status) were recorded via seven forced choice items (school setting, socioeconomic status, grade levels, ethnicity/race, disabilities). For each of the component skills/processes and writing outcomes, any relevant measures included in each study were coded to include the measure name, type of measure (standardized or researcher developed), and reliability and validity data, when available. In the area of reading, instruments included measures of phonological/phonemic awareness, decoding or word recognition, or reading comprehension. Measures of oral language were generally expressive, receptive, or grammar/semantic assessments.
Table 1 provides examples of measures used to assess component skills in the primary studies. Each writing outcome measure was coded as either writing quality or writing production, and furthermore, if coded under writing quality, the type of scoring used (holistic, analytic, standard score) was coded. In general, measures of writing production included counts of the number of words, sentences, and ideas produced by students when given a specific prompt. Writing quality was most often assessed by having students write based on a researcher-created narrative and/or expository prompt and then scoring the composition analytically for separate elements such as idea development, organization, word choice, sentence fluency, and writing convention, analogous with the 6 + 1 Traits of Writing Rubric (Northwest Regional Educational Laboratory, 2011), or holistically, which includes a single rating that takes into account many or all of these elements. In addition, some common standardized measures were also used in the primary studies to assess student’s quality of writing (e.g., Test of Written Language–3rd Edition; Woodcock-Johnson Tests of Achievement–3rd Edition). Last, each rater coded the ES (i.e., correlation coefficient) for all bivariate relationships reported between component skills/processes and writing outcomes. For studies where data were disaggregated by grade level or disability status, the ES for each group was coded. The significance value and sample size corresponding to the correlation analysis was also recorded.
Example measures used in primary studies
Each study was coded by the first author and a random sample of 30% of these studies was double-coded by a trained research assistant. The research assistant had a master’s degree in Reading and Language Arts and was familiar with research in the area of literacy. Initial reliability of the two raters was established through training on the coding process provided by the first author and then independent coding of a sample article. Interrater agreement of at least 90% was required before coding could begin; an actual agreement during training of 93.3% was attained. For studies that were double-coded, interrater agreement was 91.5%. Any discrepancies between coders were discussed until consensus was reached.
Analytic Procedures
The ES index of interest in this meta-analysis was Pearson’s product–moment coefficient, or Pearson’s r, which represented the bivariate correlation between handwriting fluency, spelling, reading, and/or oral language with each writing outcome. For purposes of this study, the unit of analysis was defined as ESs obtained from individual, independent samples. That is, for studies with multiple samples, each sample was treated as independent and each was included. For example, Graham et al. (1997) reported correlations of both handwriting fluency and spelling with writing quality disaggregated for students in Grades 1 to 3 and Grades 4 to 6, resulting in a total of four ES (two for handwriting fluency, two for spelling) estimates for the analyses. As recommended by Cooper (2009), the ES from each sample was weighted independently based on the respective sample size. The use of independent groups or samples as the unit of analysis has been used in previous meta-analyses such as the National Early Literacy Panel study (National Early Literacy Panel, 2008).
Although a particular sample could produce ESs for multiple constructs of interest, to handle further potential issues of nonindependence of data in the analyses, only one ES per individual construct (e.g., relationship between spelling and writing quality) was included for each independent sample. Following the recommendation of Lipsey and Wilson (2001), when correlations between multiple measures of a particular component skill and writing outcomes for a sample were reported—particularly evident when analytic scoring of writing samples was used—a simple average of the correlations was calculated to create a single ES for inclusion in the analyses. This process was used in several recent meta-analyses involving writing outcomes (Graham, McKeown, et al., 2012; Graham & Perin, 2007; Graham & Sandmel, 2011). Furthermore, as suggested in the work of Graham, Hebert, and Harris (2011), a single, cumulative score appears to capture writing quality given that ratings of student’s writing across several different traits are typically moderately to highly related. The data were also examined for the presence of potential outliers, which was defined as any ES estimate more than 2 SD above or below the simple mean effect for each component skill.
The statistical package used to conduct all analyses was SPSS for Windows (Version 20; IBM Corp., 2011) along with macros from Lipsey and Wilson (2001). To address the dependency of variances on the obtained correlation coefficients, Fisher’s z transformation was used, and all analyses were conducted using these z transformed correlations (Borenstein, Hedges, Higgins, & Rothstein, 2009). Variance estimates were calculated for each transformed ES, and the inverse of the variance was used to weight each z score for the analyses. Given the variation in instruments used to measure the constructs and the range of populations sampled within the primary studies, it was hypothesized that much of the variation in effects would be attributed to factors beyond just sampling error; thus, a random effects model was initially employed to examine the main effects. Tau, or the measure of between studies’ variance for the random effects model, was calculated using the method of moments estimate (Lipsey & Wilson, 2001). Heterogeneity of the effects was examined using a Q test to determine whether variance exceeded that which would be expected by sampling error (Cooper, 2009); the Q statistic tests the null hypothesis that ESs are homogeneous and is tested using a chi-square test with k − 1 degrees of freedom. Additionally, an I2 statistic, representing the proportion of total variance between studies that can be attributed to true heterogeneity among ESs, was also calculated.
The effects of the two moderators were tested using a mixed-effects analog to analysis of variance (Lipsey & Wilson, 2001). The mixed-effects model, advocated by Borenstein et al. (2009) when conducting moderator analyses, assumes that some of the variance in ESs are beyond sampling error and can be attributed to systematic between-subject components (i.e., moderators), while also allowing for a random within-subjects component, or residual variance. The original intent of the moderator analyses for grade level was to examine potential differences in correlations for students in Grades K to 3, Grades 4 to 6, and Grade 7 and above. However, only three studies were identified that involved samples of students in seventh grade and beyond, resulting in an insufficient number of ESs for this analysis. Thus, the decision was made to combine samples of students in Grade 4 and above for moderator analyses; this grade level range has been used in several recent meta-analyses in the area of reading and writing (e.g., Graham & Perin, 2007; Scammacca et al., 2007; Wanzek et al., 2013). Second, the effect of learner type was examined to determine if ESs differed significantly for students with academic difficulties, including those with high-incidence disabilities, in comparison to a full range sample. Comparisons were only conducted when a minimum of three studies were available per group (e.g., per grade category, student level). Finally, all ES estimates and confidence intervals (CIs) were transformed from z scores to Pearson’s r for reporting and interpretation.
Results
Study Characteristics
Table 2 presents study characteristics, including sample size, grade/age level, type of learners in the sample (i.e., full range or students with academic difficulties), and component skills examined. The final corpus of literature for this review contained 38 studies ranging in publication date from 1988 to 2013. As some of these primary studies provided disaggregated data (i.e., ESs) by grade levels and student ability level, 54 independent samples were included. The respective samples across the studies covered the full range of students from kindergarten through high school, and students from 5 to 18 years of age; 17 (31%) samples included students in kindergarten through third grade, 17 (31%) were comprised of students in Grade 4 to 6, four samples (7%) involved students above Grade 7, and 16 (30%) included samples aggregated across primary and secondary grades. Only nine samples were specifically comprised of students with academic difficulties, that is, students with high-incidence disabilities or students with identified reading or writing deficits. Because individual samples frequently contributed an ES for more than one component skill, there were a total of 146 aggregated ESs coded across these studies: 24 for handwriting fluency, 26 for spelling, 55 for reading, and 41 for oral language and writing outcomes.
Study characteristics
Note. RDG = reading; OL = oral language; HW = handwriting fluency; SP = spelling.
A majority of the ESs were correlations of component skills with a writing quality outcome (68%, k = 100). More specifically, of the 35 studies that measured students’ composition quality, the use of analytic rubric/scoring was most prevalent (n = 16), followed by administration of norm-referenced assessments (n = 12) and application of holistic scoring to students’ writing samples (n = 7). When analytic scoring was used, multiple indices of writing quality were evaluated simultaneously with evaluation of idea development (n = 9) and students’ vocabulary and word choice occurring most frequently (n = 9). Writing organization was evaluated in eight studies, sentence fluency and/or grammar usage in seven studies, and conventions/mechanics was assessed in six studies. Again, correlations between a particular component skill and multiple analytic scores for a particular sample were averaged to create a single ES for analysis purposes.
The Relationship Between Component Skills and Writing Outcomes
Separate analyses were conducted to investigate the mean correlation between each component skill and both writing outcomes, quality of writing and writing production, as well as the potential moderating effects of grade level and student ability level. Results from these analyses are summarized in Table 3.
Mean effect sizes and moderator analyses for the relationship between component skills and writing outcomes
Note. K = number of studies; ES = mean effect size; CI = 95% confidence interval.
Handwriting Fluency
Of the 24 ESs coded for handwriting fluency as a component skill, 17 examined the correlation to writing quality (N = 3,226) and the remaining seven (N = 1,222) examined writing production. All correlations between fluency in handwriting and quality of student’s writing were positive (rs = .07–.82). Under a random-effects model, the mean ES for the relationship was positive and moderate in magnitude though there was significant heterogeneity evident among these results. There was a nonsignificant between-group difference (Qbetween = 2.66, p = .10) for grade-level effects on the correlation between handwriting fluency and quality of writing; the mean ES for samples in the primary grades and for samples in Grade 4 and above were both in the moderate range. Although for purposes of moderator analysis we were not able to separate students in fourth through sixth grades from students in seventh grade and above, further examination indicated that the mean ES was .29 (95% CI [.20, .53]) for Grades 4 to 6 compared with .54 for the single correlation for older students (Dockrell et al., 2009). Similarly, results from the mixed-effects ANOVA procedure revealed that student ability level did not appear to moderate the correlation between student’s handwriting fluency and the quality of their writing (Qbetween = 0.01, p = .92). The mean ES from the 14 samples representing the full range of abilities and for samples of students with academic difficulties ranged from .49 to .50.
All seven correlations between handwriting fluency and the amount of writing produced were positive and ranged from .25 to .64. A significant and moderate, positive relationship was evident, and results suggested significant heterogeneity among these ES estimates; the I2 statistic indicated approximately 90% of the variance could be attributed to between-study differences beyond sampling error alone. However, there were insufficient numbers of ESs between handwriting fluency and the amount of writing produced at each grade category and student ability category to conduct moderator analyses. The ESs for the two samples beyond Grade 3 were .30 (Graham et al., 1997) and .47 (Wagner et al., 2011), respectively. Further examination revealed that the ESs for the relationship between handwriting fluency and writing production for samples of students with academic difficulties were .54 (Berninger et al., 2001) and .46 (Thomson et al., 2005), respectively, both within the confidence interval for the mean ES across all studies.
Spelling
A total of 18 ESs (N = 3,684) were obtained for the correlation between students’ spelling ability and writing quality, with individual ES estimates generally positive (rs = −.09 to .67). The mean ES indicated a moderate relationship between these constructs. There was significant between-study variation in the ES estimates beyond sampling error. Student grade level did not appear to moderate the relationship between spelling ability and writing quality. The mean ESs for students in the primary grades and for students in Grade 4 and above were both moderate in strength and not statistically different (Qbetween = 0.05, p = .83). We further examined correlations within the group of students in Grade 4 and beyond. The reported correlation for the single study involving older students was .65 (Dockrell et al., 2009). In comparison, the mean ES for the correlation between spelling and writing quality for students in Grades 4 to 6 was .41 (95% CI [.27, .54]). No statistically significant differences in the mean ES between spelling and writing quality were evident based on student ability level (Qbetween = 1.76, p = .19). The mean ESs for both samples representing the full range of ability and those comprised of students with academic difficulties were positive and moderate in nature.
The correlation between spelling ability and the amount of writing students produced was examined in eight samples (N = 1,120). All correlations were positive (rs = .02–.73) and the mean ES was small, but statistically significant. Variability in the ESs was statistically greater than sampling error alone with between-study variance estimated to be 87.1%. The correlation between spelling and writing production was not statistically different (Qbetween = 1.34, p = .24) for students in the primary grades and those in fourth grade and beyond. For the three samples with primary grade students, the mean ES was suggestive of a weak, yet positive, correlation. Meanwhile, across the four samples of older students, the mean ES was not statistically significant from zero (p = .28). It should be noted that all these studies with older students involved samples between fourth and sixth grades. There were no samples examining the correlation between spelling and writing production involving students with academic difficulties; thus, no moderator analyses were conducted.
Reading
Of the 55 ESs coded, a total of 39 (N = 558,203) were available for the correlation between reading achievement and the quality of student’s written composition. All the correlations in the primary studies were positive (rs = .02–.81), with the exception of Mackie and Dockrell (2004) reporting a correlation of −.56 between these variables for one of the samples in their study; this negative correlation represented a potential outlier that may influence the mean ES estimate. A sensitivity analysis was conducted to examine the mean ES both with and without inclusion of this study, which resulted in no change in the ES. Thus, this ES was retained in the final analysis. Examining the correlation between reading achievement and writing quality, the mean ES indicated a positive relationship that was moderate in strength. The Q statistic was significant and the I2 value indicated that almost all (99.1%) the between-study variance was due to true heterogeneity beyond sampling error.
It should be noted that three ESs originated from the same primary study (Shermis & Long, 2009), which included extremely large samples of students in 4th (n = 185,868), 8th (n = 193,128), and 10th grades (n = 174,838). Data were also analyzed without these ESs to examine the potential influence of these large samples, which resulted in similar results (r = .47 without the samples). Therefore, no ESs were removed from the final analysis. With regard to moderators, no statistically significant differences were found in the correlation between student’s reading ability and writing quality based on grade level (Qbetween = 0.00, p = .98). Across both students in the primary grades and those in Grade 4 and above, the mean ES suggested a moderate relationship between these variables that was consistent across grade levels. With one exception, correlations between reading and writing quality for students in Grades 4 to 6 ranged from .25 to .70 (M = .45, 95% CI [.42, .48]). In contrast, reported correlations in samples of students in Grade 7 and above were .32 to .58 (M = .56, 95% CI [.54, .59]). Student ability level was not found to be a moderator of the relationship between reading ability and the quality of students’ writing (Qbetween = .08, p = .77). The mean ESs for the full range of ability levels and for students with academic difficulties were .48 and .47, respectively.
The relationship between reading and writing production was examined across 16 samples (N = 1,306), with correlations ranging from −.08 to .69. Results indicated a generally weak yet positive relationship with significant heterogeneity evident across the studies suggesting large between-study variance not attributable to sampling error alone. However, the relationship between reading ability and the amount of writing students produce does appear to be moderated by grade level. Statistically significant differences were evident (Qbetween = 4.43, p = .04) between groups. The mean ES for students in the primary grades was positive and generally moderate in strength, while the mean ES for older students was not significantly different from zero and suggested a negligible relation between reading achievement and writing production; none of these studies with older students included samples beyond Grade 6. As there was only one sample that reported the correlation for reading and writing production for students with academic difficulties, no further moderator analyses were conducted.
Oral Language
ESs from the 26 samples (N = 3,049) providing the correlation between oral language measures and writing quality were primarily positive, with one exception, and ranged from −.23 to .55. The mean ES indicated that oral language skills have a weak to moderate relationship with students’ quality of written composition. Variance among ESs was not due to sampling error alone as nearly two thirds of the variance could be attributed to true between-study differences in the sample of ESs. The reported correlation of −.23 in one study (Mackie & Dockrell, 2004) was more than 2 SD below the simple mean, requiring follow-up analysis to identify the potential effect on the mean ES; results were unchanged and the study effect remained in the analysis. Grade level was not a moderator of the correlation between oral language and writing quality (Qbetween = 0.22, p = .64); the mean ESs for primary grade students and for Grade 4 and above ranged from .30 to .32. Of note, the mean ES for samples in Grades 4 to 6 (M = .32, 95% CI [.24, .40]) and Grade 7 and above (M = .32, 95% CI [.14, .48] were identical. Results did not indicate a moderator effect for student ability level on the correlation between oral language skills and writing quality (Qbetween = .00, p = .98). For the 22 samples including the full range of abilities, the mean ES represented a moderate relationship. Similarly, a moderate relationship was evident between oral language and writing quality for students with academic difficulties.
A total of 15 ESs (N = 1,204) examined the relationship between oral language and the amount of writing students produced, with all correlations being positive (rs = .01–.52). The mean ES showed a significant, yet weak correlation between these two variables. The Q statistic of homogeneity was significant, and more than one-half of the total variance could be attributed to between-study differences. However, grade level did not appear to account for this between-study variance (Qbetween = 1.39, p = .24). The mean ESs for oral language and writing production for students in kindergarten through third grade and for students in Grade 4 and above (more specifically, Grades 4–6) ranged from .09 to .17. No moderator analyses could be conducted to examine the impact of student ability level on the correlation between oral language and writing production due to insufficient ESs to carry out such analyses.
Publication Bias
To address issues of potential publication bias, funnel plots of the individual ESs for each analysis were created and visually inspected. Given the difficulty in interpretation of funnel plots with relatively few ESs, only plots for analyses involving at least 10 ESs are provided in Supplementary Figures S1 to S6 (available in the online version of the journal). Evidence of publication bias would generally be reflected by areas of asymmetry in the plots, particularly a gap in the plot where ESs that would be considered nonsignificant would be if they were able to be located. On visual analysis, it was noted that in all but one case, the ESs from the individual samples were distributed symmetrically around the mean effect from each analysis. The one plot suggestive of potential bias was the funnel plot of ESs between oral language and writing fluency. Thus, although not definitive evidence, our funnel plot analyses would suggest minimal evidence for the presence of publication bias.
Discussion
This meta-analysis investigated the relationship between multiple proposed component skills (i.e., handwriting fluency, spelling, reading, and oral language) and writing outcomes assessing the quality of student’s written composition and their amount of writing produced. Additionally, this study sought to examine whether such relationships might be explained by the developmental (i.e., grade) level of students and/or the academic level of students. The primary findings confirmed moderate, significant correlations between individual component skills and students’ writing quality and generally weak relationships with the amount of writing produced. Although the majority of ESs were heterogeneous, only one statistically significant moderator effect was found for either grade level or student ability/achievement level; the relationship between reading achievement and writing production was stronger for students in the primary grades.
Relative to all component skills examined, the quality of writing was most strongly correlated with transcription skills, that is, handwriting fluency and spelling, as well as with reading achievement (rs = .48–.49). The ESs suggest that when examined independently, these component skills account for approximately 25% of the variance in the quality of student’s writing. The finding that automaticity and fluency in handwriting and spelling demonstrated moderate bivariate relationships to quality of composition supports the perspective regarding their importance as foundational skills for the development of writing (Berninger & Winn, 2006; Juel et al., 1986). As has been posited by these theories as well as other researchers (e.g., Graham & Harris, 2000; McCutchen, 1996, 2000), the development of proficient transcription skills may free up cognitive resources for higher level processes required for writing tasks. However, because ESs represented bivariate relationships between handwriting fluency and spelling, and each writing outcome separately, they do not take into account the shared variance between handwriting fluency and spelling. The relationship between transcription skills and the amount of writing produced by students were more mixed. Whereas students’ ability to fluently and accurately write letters demonstrated a moderate correlation with the total amount of writing produced (r = .48), spelling skills were not as strongly related (r = .25).
Although the relationship between handwriting fluency and writing quality was not moderated by grade level, there was a trend toward a relatively stronger correlation for students in the primary grades. The strength of the relationship in the earlier grades may highlight the particular importance of handwriting automaticity for emerging writers in the earliest grades. In fact, explicit instruction in handwriting has been recommended, beginning in kindergarten, and research supports the efficacy of such practices (Graham, Bollinger, et al., 2012; Graham, McKeown, et al., 2012). These findings also suggest that the ability to accurately and fluently generate text appears to have continuing influence on the writing quality of students in Grades 4 and above (r = .34). Similarly, a moderate correlation was evident between spelling ability and students’ writing quality for students beyond the primary grades. These results appear to support the contention that in addition to the potential constraints placed on cognitive resources by inefficient handwriting and spelling, for older students, poor transcription skills may also influence assessment of writing quality via presentation effects. That is, composition has shown to be judged more harshly, regardless of content, when legibility is poor and spelling errors are frequent (Graham & Hebert, 2011).
The finding of similar relationships between transcription skills and writing quality across student ability levels is also noteworthy. Research has shown that teachers often spend a substantial amount of time on these skills with lower functioning students (Graham, Harris, Fink-Chorzempa, & MacArthur, 2003); however, they only account for, individually, at most 25% of the variance in writing outcomes based on the present synthesis. Thus, improvement in these skills alone may not be sufficient for improving the writing performance of students with academic difficulties and exclusive focus on transcription as a target for instruction may not be beneficial for students regardless of achievement level.
The present findings also seem to support the presence of a moderate reading–writing relationship, which aligns with previous research summarizing this relationship (Tierney & Shanahan, 1991). Across the studies included, reading proficiency accounted for approximately 25% of the variance in students’ quality of writing and just under 10% of the variance in writing production. Fitzgerald and Shanahan (2000) have argued that reading and writing are connected due to shared knowledge and cognitive processes required in both domains. It is important to note, however, that in correlational research, the obtained effects do not provide specific evidence of the directionality of the relationship between reading and writing. Previously, Shanahan and Lomax’s (1986) research supported an interactive model of reading and writing skill. The interactive model posits that reading knowledge directly effects writing skills within each level of discourse (i.e., word analysis directly influences spelling; comprehension influences story structure) while knowledge of writing can influence reading across levels of discourse. Specifically, lower level writing skills can directly affect higher level reading skills such as vocabulary diversity in writing, influencing students’ reading comprehension.
Meanwhile, Berninger et al. (2002) demonstrated a bidirectional relationship between reading and writing in later elementary grades, with reading comprehension ability having a direct effect on writing quality and vice versa. However, Berninger and colleagues’ findings also suggested that in the earliest grades, reading had a unidirectional influence on writing quality and production. Most recently, research by Ahmed et al. (2014) demonstrated stronger reading-to-writing effects in comparison to writing-to-reading and bidirectional models in a longitudinal study from first to fourth grades. In general, these findings, as well as the current meta-analysis, reinforce that reading and writing are related, yet not parallel in development, as no more than one quarter of the variance in reading and writing can be accounted for by the other.
Although students’ reading achievement appeared to be less related to how much writing students actually produced (r = .26), grade level was a significant moderator of this relationship; a moderate correlation was evident in the primary grades, while essentially no relationship was found in Grade 4 and above. The stronger correlation in the primary grades may be attributed to the strong relationship between reading and other early literacy skills in these early grades. Prior research (Kent, Wanzek, Petscher, Al Otaiba, & Kim, 2014; Mehta et al., 2005) with students in the primary grades suggests that literacy, including reading, spelling, and language skills, may be unidimensional. Thus, reading skills would be expected to represent overall literacy ability and have a stronger relationship to writing. Furthermore, the direct effect of word recognition skills on handwriting fluency in the primary grades has been previously demonstrated in the earliest grades (Berninger et al., 2002). This moderator effect should be interpreted cautiously, as there were only four samples involving students in Grade 4 and above and only one study that included secondary students (i.e., Grades 7–12).
Relative to other outcomes, oral language showed the weakest relationship to writing quality, accounting for just over 10% of the variance. The correlation of oral language with the amount of writing student’s produced was also small in magnitude (r = .20). As language skills can be thought of as the vehicle by which ideas for text generation are formulated and developed, and influence both semantic and syntactic aspects of writing, all of which affect the quality of written language, these findings were somewhat unexpected. Furthermore, the results stand in contrast with the importance placed on oral language (as it relates to the development of task-relevant ideas) by Juel et al. (1986) in their simple view of writing, as one of two key components for proficient writing in the elementary grades.
There is an important caveat however; oral language, as represented by both expressive and receptive language ability as well as knowledge of grammatical conventions of language, may represent only one aspect of ideation noted by Juel et al. (1986). Ideation in a broader sense may involve a more active and recursive process of planning, ongoing evaluation, and revision (Bereiter & Scardamalia, 1987). The relationship between oral language and writing quality did not differ based on grade level. It should be noted that of the 11 samples in Grade 4 and above, 9 involved students at the upper elementary level (i.e., 4th-6th grade); only 2 included students in 7th through 12th grades (DeBono et al., 2012; Dockrell et al., 2009), and both samples comprised students with attention-deficit hyperactivity disorder or documented language impairments. Thus, it is possible that the relationship between oral language and writing may be stronger in typically developing students in the secondary grades where writing tasks typically require increased lexical and grammatical knowledge.
Limitations
Although we sought to summarize the extant literature from the past quarter-century, inherently, all meta-analyses must be interpreted in light of the inclusion/exclusion criteria used to obtain the corpus of studies analyzed. In this study, the decision was made to include only ESs obtained from studies that had been published in refereed journals. The decision to include only published research, however, is not without controversy. On the one hand, the establishment of this inclusionary criterion can help insure the synthesis of high-quality, rigorous research. That is, the peer review process can be thought of as a proxy for a study’s quality, and thus, we made the determination that focusing on the published literature would strengthen the confidence in the obtained results. As noted, several recent meta-analyses have made a similar contention (e.g., García & Cain, 2014; Scammacca et al., 2013; Stuebing et al., 2014). Furthermore, we concurred with García and Cain’s (2014) assessment that research focused on the relationships (i.e., correlations) among variables is less susceptible to publication bias than those studies employing treatment-control designs.
On the other hand, it has been argued that the exclusion of unpublished research does introduce the possibility of publication bias into meta-analyses (e.g., Borenstein et al., 2009; Rothstein, Sutton, & Borenstein, 2005). Namely, by excluding the unpublished literature, the current corpus may not be representative of the population of completed studies in this area and may ultimately compromise the validity of the findings. Publication bias was originally and most often referred to as the publication or nonpublication of studies depending on the statistical significance (e.g., Dickersin & Min, 1993; Rothstein et al., 2005). With this in mind, we did undertake a more detailed examination of the individual correlations used to compute the ES estimate from individual studies. Overall, when significant levels were reported, approximately 20% of the correlations between component skills and writing quality were nonsignificant while 37% of correlations between component skills and measures of writing production were nonsignificant.
This information suggests that the current corpus of published studies was not limited solely to findings that reached statistical significance. Furthermore, visual inspection of funnel plots revealed in all but one case symmetry around the mean ES, suggesting limited publication bias in the obtained studies. Finally, to address potential concerns regarding the representativeness of the included studies, we did conduct a secondary search for unpublished dissertations and theses. We located six additional studies (Ahmed, 2011; Altemeier, 2006; Mulligan, 2002; Nathan, 2009; Nelson, 2003; Smith, 2011). Except for the relationship between reading and writing quality (six additional ESs), these unpublished studies would have contributed two or fewer ES estimates to each analysis. Ultimately, the decision was made to not include these studies in further analyses as the reported ESs, with a few exceptions, were all within the confidence intervals obtained in the original analyses. Nonetheless, the potential omission of unpublished studies remains a limitation in the present study.
Studies that examined the relationship between particular component skills and written outcomes in languages other than English were also excluded. It is quite possible that in other alphabetic as well as nonalphabetic writing systems such correlations may be different. However, the vast differences across language systems, including various degrees of transparency in orthographies, would make comparisons untenable. Nonetheless, these relationships are certainly worthy of examination and should be explored in future research.
This study sought to investigate whether the correlations between component skills and writing outcomes were moderated by grade level, including comparisons across the primary, upper elementary, and secondary grades. However, we were limited by the dearth of existing literature focused on students in the secondary grades. Specifically, we were able to locate only three studies, with four total samples, that specifically measured and reported correlational data on the relationship between one or more component skill and writing quality and/or production for students in Grades 7 to 12 (DeBono et al., 2012; Dockrell et al., 2009; Shermis & Long, 2009). The absence of a sufficient number of secondary studies, specifically with regard to the components analyzed in this study, resulted in our having to aggregate studies for samples in Grade 4 and above for purposes of moderator analysis. Furthermore, the relative absence of studies involving secondary grade samples meant that the results for older students were primarily influenced by upper elementary samples.
Future Research
There are several implications from the current study with regard to future research. As just noted, it is clear that future research examining these relationships between component skills and writing outcomes for students at the secondary level is warranted and would help create a clearer picture of influences on writing quality and production and identify potential targets for instruction. For example, a certain level of reading proficiency has been posited as essential for older students during self-monitoring of writing (Deane et al., 2008; Hayes, 1996; McCutchen et al., 1997), and similarly, the integration of reading and writing instruction has been advocated as providing practice across both domains (Abbott et al., 2010; Shanahan, 2006). However, we still know relatively little about such relationships beyond the elementary grades.
Additionally, although specific and explicit instruction in the areas of transcription for improving and/or remediating writing in elementary level students has significant support in the literature (see Graham, McKeown, et al., 2012), additional evidence is needed to determine whether these relationships at the secondary level warrant further instructional research. This need for focused instructional practices is particularly important in light of the continued difficultly exhibited by adolescents in the United States on large-scale writing assessments (NCES, 2012), as well as increasing standards for writing across the curriculum and the importance of writing in postsecondary settings (Common Core State Standards Initiative, 2010; National Commission on Writing, 2004).
Another area that warrants further investigation is the relationship between component skills and writing quality and production for students with academic difficulties, including those students with high-incidence disabilities. Of the 146 ESs examined, approximately 13% (k = 19) involved students with academic difficulties; these effects came from only nine primary studies. Furthermore, although two of these studies (Berninger et al., 2001; Thomson et al., 2005) involved students with identified reading difficulties and one study examined students identified as at-risk for writing difficulties (Nagy et al., 2003), no studies specifically examined students with specific learning disabilities. Previous meta-analyses of writing instruction continue to highlight the need for research with struggling writers (Graham, McKeown, et al., 2012; Graham & Perin, 2007; Graham & Sandmel, 2011). If we are to learn more about effectively addressing the difficulties in writing for students with academic difficulties, we must begin with identifying those skills and processes that are related to writing development in order to devise appropriate instructional programs. For example, although process approaches to writing instruction have become prevalent in schools, and have been shown to have positive effects on writing for typical learners (Graham & Sandmel, 2011), it has been argued that such approaches should also incorporate direct instruction in specific skills and strategies, specifically for struggling learners (Troia, Lin, Monroe, & Cohen, 2009).
Last, while the present findings point to small to moderate correlations between component skills and writing quality and fluency, interpretation is limited to individual relationships. Although not specifically the focus of this study, several of the included studies did include multivariate analyses in primary grades (e.g., Hooper et al., 2011; Kim et al., 2011) and beyond (e.g., Abbott & Berninger, 1993; DeBono et al., 2012; Graham et al., 1997). Multivariate studies have specific implications as to how individual differences in component skills covary and interact to influence writing outcomes. Continued research in this area may prove fruitful for specifically identifying the relative importance of specific skills, individually and collectively, for the development of proficient writing skills across age/grade levels. Furthermore, examining multiple skills concurrently is necessary in light of evidence that students do not develop skills across domains in parallel fashion. Berninger and Abbott (2010) have found that about one quarter to one third of typically developing students exhibit a relative strength and/or weakness within language domains (i.e., oral language, reading, and writing). They further discovered that language profiles changed over time with only 7% of students retaining a similar profile from mid- to late-elementary school. Despite the knowledge that each of these component skills has a relation to writing development, it is possible that specific profiles of skills may be more strongly related to the development of writing difficulties. Thus, studies that use multivariate techniques, such as latent profile analysis, may assist in identifying patterns of skills within these component processes that correlate with writing difficulty.
In conclusion, this study demonstrated that several component skills have small to moderate correlations with the quality of student writing and amount of writing produced and that such relationships were generally stable across grade levels and for different types of learners. Findings highlight the potential of each of these component processes as instructional targets that warrant further investigation. Finally, it is clear that additional research with students at the secondary level and students with academic difficulties, including students with learning disabilities, is needed to clarify the role of each of these component processes in the development of writing.
Footnotes
Notes
Authors
SHAWN C. KENT, PhD, is an assistant professor with the Special Populations program in the Department of Educational Leadership and Policy Studies at the University of Houston, Farish Hall, Suite 112, Houston, TX 77204-5023; e-mail:
JEANNE WANZEK, PhD, is an associate professor in the Department of Special Education at Vanderbilt University, 110 Magnolia Circle, Nashville, TN 37203; e-mail:
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
