Early Grade Writing Assessment: An Instrument Model

Abstract

The United Nations Educational, Scientific, and Cultural Organization promoted the creation of a model instrument for individual assessment of students’ foundational writing skills in the Spanish language that was based on a literature review and existing writing tools and assessments. The purpose of the Early Grade Writing Assessment (EGWA) is to document learners’ basic writing skills, mapped in composing units of increasing complexity to communicate meaning. Validation and standardization of EGWA was conducted in the Canary Islands (Spain) in 12 schools using a cross-sectional design with a sample of 1,653 Spanish-speaking students in Grades 1 through 3. The author describes EGWA’s internal structure, along with the prevalence of learning disabilities (LD) in transcription and developmental differences in writing between Spanish-speaking children with LD and typical peers. Findings suggest that EGWA’s psychometric characteristics are satisfactory, and its internal structure can be attributed to four factors responsible for a high percentage of the variance. The odds ratio indicated that 2 Spanish-speaking children with LD in transcription are identified out of every 100. A comparison between students with and without LD in transcription revealed statistically significant differences concerning sentence and text production across grades. Results are interpreted within current theoretical accounts of writing models.

Keywords

handwriting spelling sentence production text production writing disabilities factor structure validation

Learning to write is an issue that worries parents, teachers, students, and researchers. In 2001, the World Health Organization included writing difficulties as one of the problems considered to constitute an impediment to school participation, a significant element in the normal developmental process of a child. Writing is an essential activity that enables students to express their knowledge and thoughts and to participate in most types of academic activities (Berninger, 1994). This complex activity involves multiple processes, and students have to invest a great amount of time for learning and development. Writing models have been proposed for developing child writers (e.g., the Simple View of Writing, Juel, 1988; Juel, Griffith, & Gough, 1986; and the Not-So-Simple View of Writing, Berninger & Winn, 2006), and it is evident that the nature of writing changes as they develop. The attention that young writers pay to forming individual letters and spelling words shifts to constructing phrases, sentences, and paragraphs. As their proficiency develops, the texts become increasingly complex, progressing from random-word combinations to extended narratives and organized texts (Berninger, Abbott, Whitaker, Sylvester, & Nolen, 1995).

That having been said, the task of assessing writing ability is just as important as the task of assessing reading and math. Writing assessment can be performed for different purposes, such as (a) diagnostic evaluation to help teachers plan instruction, (b) screening assessment to aid in identifying students who are at risk of academic failure and in need of additional writing instruction, (c) progress monitoring to determine whether students are making adequate progress or need further intervention to achieve grade-level writing outcomes, and (d) outcome assessment: providing information about the effectiveness of the writing program in relation to established performance levels (NRP, 2000). Assessment is key to decision making at the policy level as well: specifically, regarding how to allocate resources more efficiently so as to improve education quality and reduce inequality in education. Policy makers and administrators need information about the overall performance of the education system (diagnostic), the existing gaps between schools and student subpopulations (screening), the progress being made by schools or school districts with the resources available (monitoring), and the impact of government-mandated and alternative programs (outcome evaluation) so that they can make decisions (Kudo & Bazan, 2009).

Writing is inherently more difficult to assess, as it is a more “productive” rather than “receptive” activity. For that reason, it may be difficult to produce summative or highly comparable, policy-relevant results across a whole country. The United Nations Educational, Scientific and Cultural Organization (UNESCO) adopted a working method similar to the one successfully used in the development of other basic, formative, early-grade tools for reading (e.g., Early Grade Reading Assessment) and math (e.g., Early-Grade Mathematics Assessment). This method consists of several steps for developing a practical, but rigorous, research-based measurement tool for writing skills early on in writing literacy. The main aspects of these measures that make them useful are that (a) they can provide information that is immediately actionable at the classroom level, (b) they are often based on measures whose direct relationship to the skill being measured is fairly obvious, and (c) they can be scored fairly easily. The Early Grade Writing Assessment (EGWA) has been designed not only for Spanish teachers’ needs but also for those of other teachers from diverse linguistic backgrounds.¹

Alphabetical Systems and Learning to Write

Alphabetical written-language systems vary widely in the consistency of correspondence between grapheme-phoneme correspondence (GPC, used in reading) and phoneme-grapheme correspondence (PGC, used in spelling). According to Wydell and Butterworth (1999), any orthography can be described in two dimensions: the transparency of print-to-sound translation and the size of the smallest orthographic unit that represents sound (i.e., granularity). In a continuum concerning the predictability of grapheme-phoneme relationships, at one extreme are languages such as Serbo-Croatian, Spanish, or Finnish that have a high degree of orthographic transparency, in which the mapping between graphemes and phonemes is largely consistent. At the other extreme are languages such as English that possess a somewhat opaque orthography, as there are many ways of sounding out graphemes; subsequently, many of the correspondences cannot be predicted from context-dependent grapheme-phoneme rules.

Spanish orthography shows significant asymmetry when dealing with spelling and reading transparency; for example, mapping orthography to phonology (reading) can be more reliable than when mapping phonology to orthography (spelling). Some phonemes can be represented by more than one grapheme, and some graphemes can represent more than one phoneme (i.e., inconsistent graphemes, as has been stated above). In this sense, it has been suggested that Spanish is shallower in the grapheme-phoneme direction than in the phoneme-grapheme direction.

There are inconsistent phoneme-grapheme relationships because a phoneme can be represented by several graphemes; however, there is no phonological rule that specifies the appropriate grapheme for the correct word spelling. For example, the phoneme /ʎ/ can be transcribed as Y or LL (e.g., pollo = chicken and poyo = kitchen top), and both are pronounced /poʎo/; the phoneme /b/ can be represented by V, B, or W (e.g., vaca = cow and baca = roof rack), and both are pronounced /baka/; the phoneme /x/ accompanied by /e/, /i/ can be represented as G or J (e.g., genio/Jerusalén = genius/Jerusalem; jirafa/jilguero = giraffe/goldfinch). Hence, simple phonological transcription is not enough to spell words containing these sounds correctly. On the other hand, the transcription of a consonantal phoneme depends on the sound of the accompanying vowel (e.g., with the phonemes /x/, /g/, and /k/).

Other languages, such as English and French, also show orthographies characterized by inconsistent relationships (e.g., the vowel /i/ is spelled in many different ways in English words, such as in eel, tea, theme, thief, people, and me). There are many irregular and some almost arbitrarily spelled words in English (e.g., pint, yacht) and—but to a lesser extent—in French (e.g., monsieur, femme) because for GPC, the number of irregular words is low in French and GPC in “fraise” are regular, but not PGC. These linguistic differences across alphabetical written language systems have an influence on prevalence of reading and spelling disabilities (Wydell & Butterworth, 1999).

Prevalence of Writing Disabilities and Developmental and Individual Differences

A significant proportion of school children (approximately 10–20%) has been reported to suffer from writing difficulties (Smits-Engelsman, Schomaker, Van Galen, & Michels, 1995), while the prevalence of handwriting difficulties among school-age children varies between 10% and 34% (Rubin & Henderson, 1982; Smits-Engelsman et al., 1995; Smits-Engelsman, Niemeijer, & Van Galen, 2001). Handwriting difficulties are especially prevalent among children who are diagnosed with Developmental Coordination Disorder (Diagnostic and Statistical Manual of Mental Disorders, fifth edition; American Psychiatric Association, 2013) and learning disabilities (LD) (Waber & Bernstein, 1994) or who are defined as “clumsy” by their teachers (Laszlo, 1990; Laszlo, Bairstow, & Bartip, 1988). Katusic, Colligan, Weaver, and Barbaresi (2009) conducted an epidemiologic longitudinal study on the incidence of written language disorder (WLD) in the United States. They used a population-based birth cohort, and the essential features of writing problems from the Diagnostic and Statistical Manual of Mental Disorders, fourth edition, text revision, were included in the operationalized definition of WLD. WLD incident cases were established using research criteria based on three formulas (regression-based discrepancy, non-regression-based discrepancy, and low achievement). Cumulative incidence rates of WLD varied from 6.9% to 14.7%, depending on the formula.

Nonetheless, linguistic differences with respect to phoneme-grapheme consistency are responsible for the ease with which children in shallow orthographies, in comparison to opaque orthographies, learn to spell (Wimmer & Landerl, 1997). For instance, Wydell and Butterworth (1999) suggested that orthographies with fine granularity and opaque print-to-sound translation would have a high incidence of phonological dyslexia. In contrast, orthographies with fine granularity and shallow print-to-sound translation would have a low incidence of phonological dyslexia (e.g., Serbo-Croatian, Spanish, and Finnish, to name just a few). On the other hand, the fact that there are phonemes that have more than one graphical representation makes writing and correctly spelling certain words a difficult task for many students. Further cross-linguistic research is necessary because the study of spelling has long been dominated by studies of one writing system, the alphabet, and one orthography, that of English (Perfetti, 1997).

Despite this, the results of many studies have sustained the conclusion that specific LD exist in transcription skills, handwriting, and/or spelling that interfere with typical writing development (Berninger, Abbott, Augsburger, & García, 2009). Low-level developmental skills, as well as component skills such as handwriting and spelling, may provide a critical foundation in the early stages of writing that influence the degree to which a child may subsequently develop higher level composition skills (for a review, see Jones & Christensen, 1999).

To my knowledge, there is no such research on the prevalence of writing disabilities and developmental and individual differences in the Spanish language; to this I would add the lack of assessment instruments that allow us to evaluate early writing skills together with the prompt identification of Spanish-speaking school children who are struggling to learn to write. Therefore, the instrument model presented here attempts to fill this gap.

EGWA: An Instrument Model

EGWA (Jiménez, in press) was developed to document the foundational levels of student learning, including assessment of the first steps pupils take when learning to write, whether it be the alphabet, words, sentences, or a story. The objective is to help UNESCO member countries begin the process of a systematic measurement of how well children in the early grades of primary school acquire writing skills—or help them improve existing processes—and ultimately, to foster more effective efforts to improve performance in this core learning skill. Based on a literature review and existing writing tools and assessments, EGWA was developed as a model instrument for individual assessment of a student’s foundational writing skills, and it has been presented at different international conferences (Jiménez, 2012a, 2012b, 2012c, 2013a, 2013b). The aim is to document learners’ basic writing skills, mapped in composing units of increasing complexity: letters, words, sentences, and finally stories, so as to be able to communicate meaning. This study therefore was conceived for two purposes:

To analyze the internal structure of EGWA and its construct validity by looking at the correlations between EGWA and the Test Estandarizado para la Evaluación de la Escritura con Teclado (“Spanish Keyboarding Writing Test” [TEVET]; Jiménez, 2012d); see also Jiménez, Marcos, González, and Suárez (2016) in this issue for more information about this assessment tool.

To identify the effectiveness of EGWA tasks in enabling users to differentiate between typically achieving students and students receiving educational support. I also analyzed the prevalence of LD in transcription and developmental differences in text generation between the Spanish-speaking children who had writing LD in transcription and those who did not.

Method

Participants

A total of 1,653 students were selected from 12 schools in the Canary Islands. Under Spanish law, the Education Act (Ley Orgánica para la Mejora de la Calidad Educativa 8/2013, December 9) defines students with specific educational support needs (SESN) as individuals who require additional educational support due to their special educational needs (i.e., they require support and specific educational attention due to a sensory impairment or acquired neurological problems, among others; these are traditionally used as exclusionary criteria for LD or severe behavioral disorders, either temporarily or throughout the whole of their schooling). The Education Act also covers individuals who have specific LD, are gifted, or enter the education system late (i.e., students coming from other countries or who enter the education system late for any other reason or due to personal circumstances or past school records).

In the present study, children with special educational needs were excluded (i.e., children who had sensory, acquired neurological, or other problems traditionally used as exclusionary criteria for LD). This information was provided by the Dirección General de Ordenación, Promoción e Innovación Educativa belonging to the Canary Islands government (BOC 250, December 22, 2010; BOC 40, February 24, 2011). Accordingly, the final sample consisted of 1,333 children (692 boys, 641 girls) as follows: (a) Grade 1 (age M = 81.9 months, SD = 4.28; 218 boys, 220 girls); (b) Grade 2 (age M = 93.3 months, SD = 4.97; 242 boys, 224 girls); and (c) Grade 3 (age M = 105.0 months, SD = 6.13; 232 boys, 197 girls). The children came from urban areas and from average socioeconomic backgrounds, and they attended state and nonstate schools.

Materials

EGWA contains 10 tasks ranging in difficulty from copying letters to writing stories. The first four tasks consist of copying letters, selecting allographs, copying words, and copying sentences. The next four tasks consist of writing from dictation activities: writing dictated words with inconsistent spellings, dictated words that comply with spelling rules, pseudowords, and sentences. Tasks 9 and 10 are free-writing tasks, which consist of making sentences or writing a story.

Writing the alphabet in order from memory

The student is asked to write all the letters of the alphabet in the right order. The student is told to write as fast and as accurately as possible. Timing begins when the student begins to write the first letter, and the timer stops when he or she finishes writing the last letter or after 5 min have elapsed since the beginning of the task. The examiner records the total amount of time taken on the data collection form, the total number of letters written in the time used, and the number of letters written correctly in the correct order in 1 full min together with the number of letters omitted in the right order, again, in 1 full min.

Alphabet copying and allograph selection

The purpose of this task is twofold: (a) to see if the student has acquired the motor patterns for writing letters and (b) to see if the student is able to select allographs (i.e., a lowercase letter) for each capital letter. For the Alphabet Copying subtask, the examiner has two types of answer sheets: one with the alphabet written in manuscript and one with the alphabet written in cursive. The examiner selects the manuscript or cursive letter template depending on the type of format that children are learning at school. In this subtask the student must make an exact copy of each of the letters. For the Allograph Selection subtask, the student gets an answer sheet containing all the letters of the alphabet in capital letters and is instructed to write the correct lowercase letter for each capital letter. The student is able to look at the capital letters of the alphabet and write the corresponding lowercase allograph. For each subtask, the student must write the letters as fast and as accurately as possible while staying within the lines of the ruled paper (i.e., separate lines of handwritten text) and paying attention to the limits. The examiner indicates the time taken on the data collection form, the number of letters written in 1 full min, and the number of letters written correctly in the given time. Furthermore, alignment, reversals, added strokes, and missing strokes are recorded.

Word copying

The objective of this task is to determine whether the student has acquired the motor patterns for writing words. The examiner shows the student an answer sheet where words must be copied on the baseline below the word to be copied. The student should copy the words as fast and as accurately as possible staying within the lines of the ruled paper and paying attention to the limits. The examiner indicates the time taken on the data collection form, the number of letters written in 1 full min, and the number of letters written correctly in the time used. As with the previous subtask, alignment, reversals, added strokes, missing strokes, and missing letters are recorded.

Sentence copying

The objective of this task is to determine whether the student has acquired the motor patterns for writing sentences. The examiner shows the student an answer sheet where sentences must be copied between the baselines. The examiner indicates the time taken on the data collection form, the number of letters written in 1 full min, and the number of letters written correctly in the time used. Alignment, reversals, added strokes, missing strokes, missing letters, word spacing, and missing words are also recorded.

Writing dictated words with inconsistent spelling

The objective is to determine whether the student is able to write words that are spelled differently from the way they are pronounced, that is, words that do not conform to any spelling rules. If the student is able to write these words, it indicates that he or she has memorized its orthographic representation. The examiner indicates on the data collection form the number of words spelled correctly, together with the substitutions, deletions, added letters, translations, and pronunciation for each word (i.e., if the student pronounces the word or part of a word while writing).

Writing words that follow spelling rules from dictation

The objective is to assess whether the student is able to write words that follow the spelling rules; this indicates that he or she has memorized these rules, recalling its orthographic representation (e.g., m before p and b [sombrero = hat/ombligo = navel]; the letter r is written at the beginning of a word [radio = radio] and after l, n, s [sonreír = to smile]). The examiner indicates on the data collection form the number of words for which the student has used the correct spelling rule.

Writing pseudowords from dictation

The objective is to assess whether the student is able to write the graphemes that correspond to the phonemes belonging to a word. The spelling of pseudowords indicates knowledge of the rules of PGC. All the words that sound the same, whether they do or do not conform to spelling rules, are valid (e.g., if the student writes “hiefe,” “iefe,” or “yefe,” all of them are considered correct since their reading sounds like the pseudoword itself). The examiner indicates on the data collection form the number of pseudowords written correctly.

Writing a sentence from dictation

The student is given an answer sheet where he or she proceeds to write a dictated sentence on the baseline. The examiner should indicate on the data collection form the number of words spelled correctly, correct use of capital letters, use of stress marks, number of spaces between words, and punctuation marks.

Writing an independently composed sentence

The main target of this task is to assess whether the student is able to write an independently composed sentence. The student is given an answer sheet where the phrases should be written on the baseline. The examiner encourages a conversation with the student about his or her interests, favorite games, hobbies, and so forth. After the student has described one or two situations, he or she is asked to write down two sentences concerning what was just talked about. The examiner indicates on the data collection form the time taken; the number of words written in each period of time: in 1 min, 2 min, the entire task (up to 5 min); number of words spelled correctly in each period of time; use of capital letters; use of stress marks; spaces between words; and punctuation marks.

Writing a story

The aim of this task is to assess whether the student has acquired narrative writing ability. The examiner gives the student an answer sheet where the text should be written on the baseline. The examiner indicates on the data collection form the amount of time spent and the following sections for each time interval (first time interval: 5 min, second time interval: the following 5 min, and the total amount of time), the number of written words, the number of words with correct spellings, the use of capital letters and stress marks, word sequence, use of punctuation marks, and the total global structure (i.e., the narrative categories used in the written text, such as when the story took place and the characters; initial problems or causes; execution of actions; and what happens at the end of the story).

Internal Consistency

The most widely used measure of test-score reliability is Cronbach’s alpha, which is a measure of the internal consistency of a test. However, Cronbach’s alpha may not be the most approriate measure of reliability of EGWA scores because portions of the EGWA instrument are timed. For the internal reliability of some tasks of EGWA, Cronbach’s alpha coefficients were .76, .90, .70, .71, .71, and .72, for word copying, sentence copying, writing dictated words with inconsistent spelling, writing words that comply with spelling rules, writing pseudowords, and writing sentences from dictation, respectively.

Interrater Reliability

Two psychology students were recruited as raters. A training manual, EGWA—Administrator Instructions and Protocol (see Jiménez, in press), was compiled, and a 5-hr-long workshop was set up to teach assessors how to score EGWA. Raters then assessed 70 writing samples (Grades 1–3) independently. The technique utilized to assess the relationship between the scores provided by multiple raters has been interrater reliability. This shows stability of scores a student receives from different raters. The evaluation index for interrater reliability is based on the comparison of the score variances among different raters. The most popular method used for testing interrater reliability is correlation. Correlation tests the relationship between the scores of two raters, which can be achieved by reporting the following coefficients: Pearson, Kendall’s tau, and Spearman’s rho. Results were also analysed using intraclass correlation coefficients (i.e., the value obtained for the correlation coefficient of absolute agreement [random effects model] between the two examiners). Overall, the interrater reliability of EGWA ranged from high to very high (see Jiménez, in press).

Results

Analyzing the Correlations Between EGWA and TEVET

The concurrent validity of EGWA with TEVET (Jiménez, 2012d) was established by using the Pearson product-moment to intercorrelate the scores of the participants in EGWA and TEVET. TEVET is composed of six tasks ranging in difficulty from writing letters to writing sentences. The first two tasks consist of typing the alphabet in order from memory and allograph selection. The third task consists of copying words. The next two tasks consist of dictation activities (dictating words and pseudowords). Task 6 is a free-writing task (typing sentences). We hypothesized that there would be a positive, significant correlation between the scores of the participants on EGWA and on TEVET. The results showed that EGWA has a positive and significant concurrent validity with TEVET (r = .56, p < .01).

Data analyses were conducted by means of the SPSS 15.0 for Windows while the Kaiser-Meyer-Olkin measure of sampling adequacy and the Bartlett Test of Sphericity were conducted prior to factor extraction to ensure that the characteristics of the data set were suitable for the factor analysis. The Kaiser-Meyer-Olkin analysis yielded an index of 0.862 in concert with a highly significant Bartlett Test of Sphericity, χ²(91, N = 1,333) = 16738.77, p < .001. Exploratory factor analysis for EGWA was conducted. Principal component analysis with varimax rotation identified four factors with eigenvalues greater than 1.0. A scree plot identified four factors that accounted for about 76.39% of the total variance (see Figure 1).

Figure 1.

Scree plot and extracted eigenvalue suggested a four-factor solution.

The results in Table 1 show that four factors with eigenvalues greater than 1 were extracted, and they accounted for a total of 76.39% cumulative variance. The first factor has an eigenvalue of 6.08 and a variance 43.46%, while the values for the last factor are 1.18 and 8.48%, respectively. The first factor, Text Production, contained compositional fluency (.909), compositional spelling (.890), correct word sequence (.886), and global structure total (.654) in text production; it accounted for 43.46% of the variance. The second factor, Sentence Production, contained the total number of spelled words (.949), total word spacing (.947), and the total number of correctly spelled words (.909) in sentence production; it accounted for 12.72% of the variance. A third factor, Word Production, contained the total number of correctly spelled words (.790) and the total number of correctly spelled words (.779) in word dictation tasks, the total number of correctly spelled words (.687) in sentence dictation, the total number of pseudowords correctly spelled using phoneme-grapheme rules (.626), and letters written correctly in their right order from memory per minute (.496); it accounted for 11.72% of the variance. The last factor, Letter Production, contained letters written correctly per minute (.896) in an allograph selection task and letters written correctly per minute (.893) in alphabet copying; it accounted for 8.48% of the variance. Before carrrying out this analysis we had included word copying and sentence copying tasks, but their inclusion accounted for somewhat less than a total of 72.2% cumulative variance and saturated in the latter factor.

Table 1.

Factor Loadings From Principal-Components Analysis: Eigenvalues, Percentages of Variance, and Cumulative Percentages for Early Grade Writing Assessment (EGWA) Tasks.

EGWA task	Factor loading
EGWA task	1	2	3	4
Writing a story
Compositional fluency	.90	.26	.18	.04
Compositional spelling	.89	.26	.28	.05
Correct word sequence	.88	.24	.25	−.01
Global structure total	.65	.02	.22	.07
Writing an independently composed sentence
Total number of spelled words	.19	.94	.14	.06
Total word spacing	.21	.94	.13	.04
Total number of correctly spelled words	.23	.90	.27	.06
Writing dictated words with inconsistent spelling: Total number of correctly spelled words	.19	.15	.79	.06
Writing words that fit spelling rules: Total number of correctly spelled words	.26	.15	.77	.04
Writing sentence from dictation: Total number of correctly spelled words	.19	.19	.68	−.01
Writing pseudowords: Total number of pseudowords correctly spelled using phoneme-grapheme rules	.12	.01	.62	.15
Letters written correctly in order from memory per minute	.341	.23	.49	.32
Allograph selection: Letters written correctly per minute	.045	.02	.11	.89
Alphabet copying: Letters written correctly per minute	.03	.08	.09	.89
Eigenvalues	6.08	1.78	1.64	1.18
% of variance	43.46	12.72	11.72	8.48
Cumulative %	43.46	56.18	67.91	76.39

Note. Boldface indicates highest factor loadings.

Identifying Effectiveness of EGWA Factors in Differentiating Between Students With and Without Educational Support

To be able to determine the effectiveness of EGWA factors in differentiating between students with and without educational support in the early grades, teachers were asked to identify children who were struggling with learning literacy skills. Children with special educational needs (i.e., those who require support and specific educational attention due to their sensory impairment or acquired neurological problems) were excluded. The result was a sample of 181 students indicated by teachers as receiving educational support. We also conducted a random sample for which we selected the same number of children who had been perceived as not receiving any educational support. Table 2 shows the means and standard deviations in EGWA’s components as a function of receiving educational support.

Table 2.

Means and Standard Deviations in Early Grade Writing Assessment’s (EGWA’s) Components as a Function of Educational Support Identified by Teachers.

EGWA component	Educational support			Noneducational support
EGWA component	M	SD	n	M	SD	n
Transcription—handwriting	8.1	5.5	181	11.6	6.5	181
Transcription—spelling	66.6	22.7	181	81.7	17.8	181
Text generation—sentence	42.8	23.0	170	50.6	24.4	178
Text generation—text	108.3	63.7	164	112.3	63.7	177

A linear discriminant analysis was performed to determine the capability of the different components to correctly classify children with and without educational support using the scores for the EGWA main components. Table 3 shows the results obtained in the linear discriminant analysis using the EGWA main components. Standardized coefficients demonstrated that spelling (.80) and handwriting (.65) were the variables with the highest load in this discriminant function, meaning that transcription skills play an important role in early-grade writing.

Table 3.

Correlation of Predictor Variables With Discriminant Functions (Function Structure Matrix) and Standardized Discriminant Function Coefficients.

Variable	Correlation with discriminant functions	Standardized discriminant function coefficients
	Function 1
Letter production (handwriting fluency)	.80	.44
Word production (spelling)	.66	.86
Sentence production	.37	.23
Text production	.15	−.51

Group classification was correctly predicted by this discriminant function in 65.4% of children with or without educational support.

Prevalence of Writing Disabilities and Developmental/Individual Differences in Spanish

We operationalized LD in transcription (handwriting and spelling) (LD-TD) based on the following criterion: a percentile score less than 25 on Letter Production and Word Production from the Spanish Standardized EGWA (n = 125). Children without LD-TD (non-LD) were identified by means of the following criterion: a percentile score greater than 25 on Letter Production and Word Production from EGWA (n = 1,208).

A total of 125 Spanish children were identified with LD in transcription (81 boys and 44 girls). This represents 8% of the total sample of 1,653 students. The odds ratio indicated that Spanish children with LD were 0.08 times more likely to be detected than Spanish children without LD (n = 1,528). On the other hand, the odds ratio indicated that Spanish children without SESN (n = 1,208) were 2.7 times more likely to be detected than Spanish children with SESN (n = 445). This means that 2 out of every 100 children are identified with LD in transcription. If we take the total population identified with SESN into account, the odds ratio indicated that Spanish children with LD were 0.39 times more likely to be detected than Spanish children assigned to the other SESN categories. On the other hand, the odds ratio indicated that Spanish children with SESN (i.e., excluding LD) were 2.56 times more likely to be detected than Spanish children with LD, meaning that 15 children with LD in transcription are identified for every 100 classified in other SESN categories.

In order to analyze developmental and individual differences in writing, a 2 × 3 MANOVA was conducted using a general linear model that had (a) independent intersubject variables: group (LD vs. non-LD) and grade (1–3) and (b) dependent variables, the scores in the tasks included in the EGWA main components related with written production (i.e., sentence and text). We calculated a factor-score including only items with loading values greater than a cut-off value in the computations (i.e., > .45) (DiStefano, Zhu, & Mîndrilă, 2009). Tables 4 and 5 show the means and standard deviations for each of the EGWA tasks as a function of group and grade.

Table 4.

Means and Standard Deviations of Early Grade Writing Assessment’s Writing an Independently Composed Sentence Tasks as a Function of Group and Grade.

	Grade
Group	1	2	3	Total
Total number of spelled words task
LD
M	10.58	14.36	18.40	14.57
SD	4.79	6.18	7.77	6.99
Non-LD
M	13.32	17.02	21.31	17.16
SD	4.16	7.04	8.77	7.52
Total
M	12.07	15.69	19.87	15.90
SD	4.63	6.73	8.36	7.37
Total number of correctly spelled words task
LD
M	7.52	11.46	15.51	11.63
SD	4.02	5.40	6.85	6.30
Non-LD
M	11.27	15.08	20.50	15.52
SD	3.90	6.44	7.59	7.11
Total
M	9.56	13.27	18.04	13.63
SD	4.36	6.18	7.61	6.99
Total word spacing task
LD
M	8.90	12.48	16.23	12.66
SD	4.39	5.59	7.26	6.45
Non-LD
M	11.11	14.98	19.50	15.14
SD	4.05	7.00	8.32	7.41
Total
M	10.10	13.73	17.89	13.93
SD	4.32	6.43	7.93	7.06

Note. LD = learning disabilities.

Table 5.

Means and Standard Deviations for Early Grade Writing Assessment’s Writing a Story Text Tasks as a Function of Group and Grade.

	Grade
Group	1	2	3	Total
Compositional fluency task
LD
M	19.39	32.63	41.37	32.05
SD	14.02	15.47	18.20	17.91
Non-LD
M	27.32	40.96	57.06	41.33
SD	11.86	18.56	18.46	20.20
Total
M	23.95	36.84	49.21	36.91
SD	13.32	17.52	19.84	19.66
Compositional spelling task
LD
M	13.11	24.67	36.86	25.59
SD	8.66	13.73	18.20	16.76
Non-LD
M	22.39	34.76	54.06	36.43
SD	10.54	16.59	17.27	19.47
Total
M	18.45	29.77	45.46	31.26
SD	10.76	15.99	19.63	18.98
Correct word sequence task
LD
M	14.68	26.51	35.37	26.32
SD	11.83	16.59	19.22	18.06
Non-LD
M	21.71	33.52	51.23	34.91
SD	11.58	17.25	18.86	19.77
Total
M	18.73	30.05	43.30	30.82
SD	12.12	17.21	20.52	19.42
Global structure task total
LD
M	2.71	3.59	4.28	3.58
SD	1.24	1.29	1.36	1.42
Non-LD
M	3.71	4.22	4.42	4.12
SD	1.01	1.25	1.04	1.15
Total
M	3.28	3.90	4.35	3.86
SD	1.21	1.30	1.20	1.31

Note. LD = learning disabilities.

With regard to sentence production, the results showed an effect due to the group variable, ƛ = .81, F(3, 231) = 17.4, p < .01, η² = .18. In order to find in which EGWA tasks there were significant differences between groups, univariate comparisons for each of the tasks assessed were conducted; the total number of spelled words, F(1, 233) = 9.98, p < .01, η² = .04; the total number of correctly spelled words, F(1, 233) = 28.2, p < .001, η² = .10; and total word spacing, F(1, 233) = 10.2, p < .01, η² = .04. In all these tasks typically achieving writers performed significantly better than children with LD across grade levels. There was also a significant effect of grade, ƛ = .74, F(6, 462) = 12.3, p < .01, η² = .13 All post hoc comparisons were significant (F > 1).

Similar results were obtained for text production. A main effect of group was also found, ƛ = .81, F(4, 226) = 13.0, p < .01, η² = .18 Univariate comparisons were conducted for each of the assessed tasks: compositional fluency, F(1, 229) = 23.7, p < .001, η² = .09; compositional spelling, F(1, 229) = 38.3, p < .001, η² = .14; correct word sequence, F(1, 229) = 20.9, p < .001, η² = .08, and global-structure total, F(1, 229) = 13.3, p < .001, η² = .05 In all these tasks normally achieving writers performed significantly better than children with LD across grade levels. Again, there was a main effect of grade that was also significant, ƛ = .74, F(6, 462) = 12.3, p < .01, η² = .13 Pair by pair comparisons revealed also significant differences between the analyzed groups where students improve from one grade to the next (F > 1).

Discussion

The findings suggest that the psychometric characteristics of EGWA are satisfactory and its internal structure can be attributed to four factors that are responsible for a high percentage of the variance. Several components regarding the understanding of writing development in younger students or students with low-level writing skills have been identified, among which the following are included: transcription (i.e., handwriting and spelling) and text production (i.e., sentence and text; Berninger, 2000; Berninger & Amtmann, 2003). An exploratory factor analysis for EGWA was conducted in the Spanish language, and principal component analysis with varimax rotation identified four main factors: Letter Production (i.e., handwriting fluency), Word Production (i.e., spelling), Sentence Production, and Text Production. The first two components are related to transcription (i.e., handwriting and spelling), and the second two are related to text production (i.e., sentence and text production). This factorial structure seems similar to the one found in English-language studies (e.g., Crawford, Tindal, & Carpenter, 2006).

Transcription

Handwriting (letter production) and spelling (word production) are necessary tools for transcribing oral language into written text. In the EGWA protocol the handwriting fluency component (i.e., letter production) is measured calculating the number of letters written correctly per minute in alphabet copying, and letters written correctly per minute in allograph selection. Competence in handwriting is usually described in terms of legibility and speed (Graham, 1986; Graham & Weintraub, 1996). In the EGWA protocol, the spelling component (i.e., word production) is measured by calculating the number of words spelled correctly in word and sentence dictation tasks and the number of pseudowords with correct graphical representation of the sounds in pseudoword dictation tasks. For the first task, the student must have an orthographic representation of the arbitrary spelling words so as to write them correctly. The procedure is that the meaning directly activates the written representation without the need to break the word into its phonemes/graphemes. For the second task, a spelling of pseudoword indicates knowledge of the rules of PGC. If the student makes many more mistakes in writing pseudowords it will be because there are difficulties in using the phonological route. The dual-route model of spelling production (e.g., Ellis, 1988) proposes that both routes to access the orthography of a word constitute the central processes of writing.

Text Generation

Young writers’ attention shifts from forming individual letters and spelling words to constructing phrases, sentences, and paragraphs. As their proficiency develops, the texts become increasingly complex, progressing from random word combinations to extended narratives and organized texts (Berninger et al., 2006). Therefore, current measures used to assess spelling or letter writing—or that require copying and/or dictation—fail to capture the potential range of writing competencies that students in elementary grades possess. This is a school period time when composition skills are emerging.

Sentence Writing

Coker and Ritchey (2010) investigated the use of a sentence-writing assessment administered to students in kindergarten and first grade. They found that first-grade students demonstrated more developed sentence-writing abilities than kindergarten children. Students in first grade were more likely to write a sentence or multiple sentences and more likely to write complete sentences that included mechanical features such as capital letters and punctuation. In the EGWA protocol, the sentence-writing component (i.e., sentence production) is measured by the number of words written, number of words spelled correctly, and spacing word.

Written Expression

However, to become fully literate, children need to move beyond single words and sentences and learn how to deal with text. Young children’s early writings are characterized as knowledge telling, that is, writing whatever a prompt brings to their minds and lacking higher levels of processing skills, such as revision and planning. One explanation is that children rely on a simpler approach to generating and encoding ideas because knowledge telling makes fewer cognitive demands for processes such as planning and revision. Children may devote less attention to planning and revision because translation, which involves generating ideas and transcribing them onto paper, is not yet automated and demands considerable cognitive energy (Graham, 1990; Graham & Harris, 2000). Previous research with curriculum-based measures has indicated that the number of words written, number of correctly spelled words, and number of correct word sequences (defined as two adjacent words that are correct) in a 3-min response to a narrative story starter serve as a reliable and valid indicator of writing proficiency at the elementary school level (Deno, Marston, & Mirkin, 1982; Videen, Deno, & Marston, 1982). Also studied have been alternate indexes such as the number of correct punctuation marks, number of simple words in complete sentences, number of large words, T-units (i.e., the smallest word group that could be considered a grammatical sentence), and/or mature words. In the EGWA protocol the text-writing component (i.e., text production) is measured by the number of written words (compositional fluency); the number of words with correct spelling (compositional spelling); word sequence; and global structure (i.e., characters involved in the story, initial event or problem, implementation, what happens at the end of the story).

On the other hand, the evaluation of EGWA’s construct validity was performed by analyzing the correlations between EGWA and TEVET and identifying the effectiveness of EGWA tasks in differentiating between typically achieving students and those receiving educational support. We found a significant correlation between the ability to handwrite quickly and the ability to keyboard quickly, which is consistent with the results from other studies (e.g., Connelly, Gee, & Walsh, 2007; Rogers & Case-Smith, 2002). In addition, we found the EGWA factors to be effective in differentiating between students who are struggling with learning literacy skills and typically achieving students.

Prevalence of LD in Transcription in Spanish

We operationalized LD in writing based on the first two EGWA components related to transcription (i.e., handwriting and spelling). Under this criterion, a second purpose of this research was to analyze the prevalence of LD in transcription and developmental differences in writing in Spanish children with and without LD.

To our knowledge, no estimates of the prevalence of LD in writing have been reported in Spanish-speaking countries, but it is expected that the prevalence of writing disabilities among a sample of Spanish-speaking children would be lower than that of English children due to the linguistic differences between Spanish and English. Jiménez and García de la Cadena (2007) used interviews with teachers from Guatemala and Spain. Through the interviews with the Guatemalan teachers, 178 children were identified with reading and spelling disabilities. This represents 32% of the total sample of 557 students. A total of 11% were identified with reading disabilities, 9% with spelling disabilities, and 12% with reading and spelling disabilities. In the Spanish sample, 291 students—or 28% of the 1,408 children—were identified as having LD in reading and spelling. Spanish teachers reported that 6% of the children showed reading disabilities, 8% showed spelling disabilities, and 14% showed both.

However, in the present study we took only the psychometric criterion (i.e., a percentile score less than 25 on Letter Production and Word Production) from the Spanish Standardized EGWA into account. A total of 125 Spanish children were identified with LD in transcription (81 boys and 44 girls). The odds ratio indicated that 2 Spanish children with LD in transcription are identified for every 100. This finding suggests that the prevalence would be lower in shallow orthographies in comparison to opaque orthographies, but more cross-linguistic research is necessary to confirm this assumption.

Developmental Differences in Writing: Spanish Children With and Without LD

When developmental and individual differences in writing were analyzed, typically achieving writers performed significantly better than children with LD across grade levels in both sentence and text production. Puranik and AlOtaiba (2012), after studying the influence of transcription skills over text generation, concluded that “writing for young emergent writers is dependent on their ability to form and write letters fluently and spell words” (p. 1533). Transcription is a basic cognitive process in writing that enables the writer to translate internal language into external written symbols to express ideas in written language. Transcription ability may be especially important in beginning and developing writing in the elementary school years. Thus, the “simple view of writing” model predicts that if children are slow or inaccurate at transcription (e.g., slow handwriting and poor spelling), then their overall compositional quality will suffer, as they will have to devote more resources to this area than the others.

In addition, transcription ability has been found to uniquely predict composing length and quality in developing writers (see for review Berninger et al., 2009). One possible interpretation of our findings is that typically achieving writers who are more fluent or accurate in transcription skills would have more attentional resources that can be devoted to composing when writing, compared to individuals with LD who are not fluent or who are inaccurate in transcription and must devote attentional resources to this aspect of writing.

Study Limitations

Nevertheless, there are some limitations of the present study. First, the cross-sectional design limits our ability to test alternative models of relationships between transcription skills and written composition that could be tested in a longitudinal study. Second, we did not assess the contribution of other variables (e.g., working memory, vocabulary, attention) that have an important role in explaining developmental and individual differences in writing.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by Agencia Canaria de Investigación, Innovación y Sociedad de la Información, ref. ProID20100030 from the Government of the Canary Islands.

Notes

References

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Arlington, VA: Author.

Berninger

V. W.

(1994). Intraindividual differences in levels of language in comprehension of written sentences. Learning and Individual Differences, 6, 433–457. doi: 10.1016/1041-6080(94)90004-3

Berninger

V. W.

(2000). Development of language by hand and its connections with language by ear, mouth, and eye. Topics in Language Disorders, 20, 65–84.

Berninger

V. W.

Abbott

R. D.

Augsburger

García

(2009). Comparison of pen and keyboard transcription modes in children with and without learning disabilities. Learning Disability Quarterly, 32, 123–141.

Berninger

V. W.

Abbott

R. D.

Jones

Wolf

Gould

Anderson-Youngstrom

… Apel

(2006). Early development of language by hand: Composing, reading, listening and speaking connections; three letter writing modes and fast mapping in spelling. Developmental Neuropsychology, 29, 61–92.

Berninger

V. W.

Abbott

R. D.

Whitaker

Sylvester

Nolen

S. B.

(1995). Integrating low- and high level skills in instructional protocols for writing disabilities. Learning Disability Quarterly, 18, 293–309.

Berninger

V. W.

Amtmann

(2003). Preventing written expression disabilities through early and continuing assessment and intervention for handwriting and/or spelling problems: Research into practice. In Swanson

H. L.

Harris

K. R.

Graham

(Eds.), Handbook of learning disabilities (pp. 345–363). New York, NY: Guilford.

Berninger

Winn

W. D.

(2006). Implications of advancements in brain research and technology for writing development, writing instruction, and educational evolution. In MacArthur

Graham

Fitzgerald

(Eds.), Handbook of writing research (pp. 96–114). New York, NY: The Guilford Press.

Coker

D. L.

Ritchey

K. D.

(2010). Curriculum-based measurement of writing in kindergarten and first grade: An investigation of production and qualitative scores. Exceptional Children, 76, 175–193.

10.

Connelly

Gee

Walsh

(2007). A comparison of keyboarded and handwritten compositions and the relationship with transcription speed. British Journal of Educational Psychology, 77, 479–492.

11.

Crawford

Tindal

Carpenter

D. M.

(2006). Exploring the validity of the Oregon Extended Writing Assessment. The Journal of Special Education, 40, 16–27.

12.

Deno

S. L.

Marston

Mirkin

P. K.

(1982). Valid measurement procedures for continuous evaluation of written expression. Exceptional Children, 48, 368–371.

13.

DiStefano

Zhu

Mîndrilă

(2009). Understanding and using factor scores: Considerations for the applied researcher. Practical Assessment, Research & Evaluation, 14, 1–11.

14.

Ellis

A. W.

(1988). Normal writing processes and peripheral acquired dysgraphias. Language and Cognitive Processes, 3, 99–127.

15.

Graham

(1986). The reliability, validity, and utility of three handwriting measurement procedures. Journal of Educational Research, 79, 373–380.

16.

Graham

(1990). The role of production factors in learning disabled students’ compositions. Journal of Educational Psychology, 82, 781–791.

17.

Graham

Harris

K. R.

(2000). The role of self-regulation and transcription skills in writing and writing development. Educational Psychologist, 35, 3–12.

18.

Graham

Weintraub

(1996). A review of handwriting research: Progress and prospects. Educational Psychology Review, 8, 7–87.

19.

Jiménez

J. E.

(2012a, July). EGWA: A tool for writing assessment in early grades. Paper presented at the SIG—Writing Conference, Porto, Portugal.

20.

Jiménez

J. E.

(2012b, June). EGWA: Early grade writing assessment. Paper presented at the 37th annual conference for the International Academy for Research in Learning Disabilities, Boston, MA.

21.

Jiménez

J. E.

(2012c, September). Teaching practices, assessment and writing disabilities in early grades. Symposium presented at the international meeting organized by the Association Learning Disabilities Worldwide, Oviedo, Spain.

22.

Jiménez

J. E.

(2012d). Test Estandarizado para la Evaluación Inicial de la Escritura con Teclado (TEVET) [Spanish keyboarding writing test] [Computer software]. The Canary Islands, Spain: Universidad de La Laguna.

23.

Jiménez

J. E.

(2013a, June). ¿Cómo evaluar los resultados de aprendizaje de la escritura en los primeros años de Primaria? [How to assess learning-to-write stage in the early grades] Workshop presented at the International meeting Lectura y Escritura en una Sociedad Global [Reading and Writing in a Global Society], Barranquilla, Colombia.

24.

Jiménez

J. E.

(2013b, September). EGWA: Early Grade Writing Assessment. Paper presented at the Colloquium on Advancing Towards a Literate Twenty-First Century organized by United Nations, Educational, Scientific and Cultural Organization to celebrate the International Literacy Day, Literacies for the 21st Century, Paris, France.

25.

Jiménez

J. E.

(in press). Early Grade Writing Assessment (EGWA): A report and a model instrument. Paris, France: United Nations Educational, Scientific and Cultural Organization.

26.

Jiménez

J. E.

García de la Cadena

(2007). Learning disabilities in Guatemala and Spain: A cross-national study of the prevalence and cognitive processes associated to reading and spelling disabilities. Learning Disabilities Research & Practice, 22, 200–208.

27.

Jiménez

J. E.

Marcos

González

Suárez

(2016). The internal structure and development of Spanish-speaking primary children’s keyboarding skills in children with and without LD in writing. Journal of Learning Disabilities, 50, 522–533.

28.

Jones

Christensen

C. A.

(1999). Relationship between automaticity in handwriting and students’ ability to generate written text. Journal of Educational Psychology, 91, 44–49.

29.

Juel

(1988). Learning to read and write: A longitudinal study of 54 children from first through fourth grades. Journal of Educational Psychology, 80, 437–447.

30.

Juel

Griffith

Gough

(1986). Acquisition of literacy: A longitudinal study of children in first and second grade. Journal of Educational Psychology, 78, 243–255.

31.

Katusic

S. K.

Colligan

R. C.

Weaver

A. L.

Barbaresi

W. J.

(2009). The forgotten learning disability—Epidemiology of written language disorder in a population-based birth cohort (1976–1982), Rochester, Minnesota. Pediatrics, 123, 1306–1313.

32.

Kudo

Bazan

(2009). Measuring beginner reading skills: An empirical evaluation of alternative instruments and their potential use for policymaking and accountability in Peru (Policy Research Working Paper 4812). Washington, DC: World Bank, Latin America & the Caribbean Region, Human Development Department, Education Sector.

33.

Laszlo

J. I.

(1990). Child perceptuo-motor development: Normal and abnormal development of skilled behavior. In Hauert

C.A.

(Ed.), Developmental psychology: Cognitive, perceptuo-motor and neurophysiological perspective (pp. 273–308). Amsterdam, the Netherlands: North Holland.

34.

Laszlo

J. I.

Bairstow

P. J.

Bartip

(1988). A new approach to treatment of perceptuo-motor dysfunction: Previously called clumsiness. Support Learn, 3, 35–40.

35.

NRP. (2000). Teaching children to read: An evidence based assessment of the scientific research literature on reading and its implications for reading instruction. Washington, DC: National Reading Panel.

36.

Perfetti

C. A.

(1997). The psycholinguistics of spelling and reading. In Perfetti

Rieben

Fayol

(Eds.), Learning to spell: Research, theory, and practice across languages (pp. 21–38). Mahwah, NJ: Erlbaum.

37.

Puranik

C. S.

AlOtaiba

(2012). Examining the contribution of handwriting and spelling to written expression in kindergarten children. Reading and Writing, 25, 1523–1546.

38.

Rogers

Case-Smith

(2002). Relationships between handwriting and keyboarding performance of sixth-grade students. American Journal of Occupational Therapy, 56, 34–39.

39.

Rubin

Henderson

S. E.

(1982). Two sides of the same coin: Variations in teaching methods and failure to learn to write. Special Education: Forward Trends, 9, 7–24.

40.

Smits-Engelsman

Niemeijer

A. S.

Van Galen

G. P.

(2001). Fine motor deficiencies in children diagnosed as DCD based on poor grapho-motor ability. Human Movement Science, 20, 161–182.

41.

Smits-Engelsman

Schomaker

M. M.

Van Galen

G. P.

Michels

C. G. M.

(1995). Psysiotherapy for children’s writing problems: Evaluation study. In Simner

M. L.

Leedham

C. G.

Thamassen

A. J. W. M.

(Eds.), Handwriting and drawing research: Basic and applied issues (pp. 227–240), Amsterdam, the Netherlands: IOS Press.

42.

Videen

Deno

Marston

(1982). Correct word sequences: A valid indicator of proficiency in written expression (Research Report No. 84). Minneapolis: University of Minnesota, Institute of Research on Learning Disabilities.

43.

Waber

D. P.

Bernstein

J. H.

(1994). Repetitive graphomotor output in learning-disabled and nonlearning-disabled children: The repeated patterns test. Developmental Neuropsychology, 10, 51–65.

44.

Wimmer

Landerl

(1997). How learning to spell German differs from learning to spell English. In Perfetti

Rieben

Fayol

(Eds.), Learning to spell: Research, theory, and practice across languages (pp. 81–96). Mahwah, NJ: Erlbaum.

45.

Wydell

T. N.

Butterworth

(1999). A case study of an English-Japanese bilingual with monolingual dyslexia. Cognition, 70, 273–305.