Effect of the linguistic complexity of the input text on alignment,writing fluency,and writing accuracy in the continuation task

Abstract

Previous studies demonstrated that the continuation task has great language learning potential and that various task-related factors may affect the extent to which the potential can be exploited (e.g. Wang & Wang, 2015). This study investigates the effect of one understudied factor, the linguistic complexity of the input text, on English as a foreign language (EFL) learners’ alignment, writing fluency, and writing accuracy in the continuation task. Two comparable groups of Chinese undergraduate EFL learners read and continued a simplified and unsimplified version of the same incomplete story whose linguistic complexity matched and exceeded their production ability, respectively. Compared to the unsimplified version, the simplified version resulted in more automatic alignment and greater improvement in writing fluency and accuracy. The implications of these findings for writing pedagogy are discussed.

Keywords

alignment linguistic complexity the continuation task writing fluency writing accuracy

I Introduction

The continuation task, which requires learners to read and continue an incomplete text, differs substantially from other commonly used second language (L2) writing tasks (Plakans & Gebril, 2013) and offers unique opportunities for language learning. When continuing an incomplete text, L2 learners tend to repeat some of its linguistic elements (Wang & Wang, 2015), a process dubbed alignment (Nishino & Atkinson, 2015; Pickering & Garrod, 2004). Previous studies have shown that linguistic alignment facilitated by the continuation task positively affects L2 learners’ written production and that various task-related factors may affect learners’ alignment and writing performance (e.g. Wang & Wang, 2015; Zhang, 2017). One task-related factor whose role in the continuation task remains unexplored is the linguistic complexity of the input text. This factor is crucial in that learners must comprehend the input text to continue it, and linguistic complexity is known to significantly affect comprehension (e.g. Chang, 2006; Droop & Verhoeven, 1998; O’Connor, Swanson, & Geraghty, 2010; Qi & Wang, 1988). The present study fills this research gap by investigating the effect of the linguistic complexity of the input text on EFL learners’ alignment, writing fluency, and writing accuracy in the continuation task. By doing so, this study contributes to the emerging line of L2 writing research that systematically examines ways to maximize the language learning potential of the continuation task.

II Literature review

1 The continuation task, alignment, and the learn-together-use-together principle

While the continuation task is not a recent invention, systematic theorizing of this practice and research into its language learning potential came about only recently (Wang, 2012; Wang & Wang, 2015). Used as an L2 learning activity, the task requires learners to read an incomplete text and continue it coherently. The incompleteness of the input text sets learners’ mind in motion, triggering their intrinsic motivation to communicate by writing (Wang, 2016). This is non-trivial, because the lack of ‘pressing, authentic, real-world needs’ (Ortega, 2011, p. 246) for L2 learners to write is demotivating. To ensure the continuation flows coherently from the input text, the continuation needs to adequately align with the input text.

The construct of alignment has been approached from both psycholinguistic and sociocognitive perspectives. From the psycholinguistic perspective, Pickering and Garrod (2004) advanced the Interactive Alignment Model to account for the mechanisms of alignment in dialogic production, arguing that successful dialogue is characterized by alignment at various levels of representation and alignment at one level activates that at another. This argument finds support in Reitter and Moore’s (2014) ‘first empirical, large-scale test of the model’ (p. 44), in which they analysed a corpus of task-oriented dialogue and reported positive correlations of linguistic alignment and dialogic task success. From the sociocognitive perspective, Atkinson, Nishino, Churchill, and Okada (2007) argued that alignment occurs between human beings and ‘environments, situations, tools, and affordances’ and that such alignment is ‘a necessary and crucial requirement for L2 development’ (p. 169). Following this view, Nishino and Atkinson (2015) analysed a case of collaborative writing and reported multimodal alignment of writers that facilitated the negotiation of lexical and concept choices. Researchers have drawn upon insights from both perspectives in theorizing the continuation task. It is argued that alignment takes place between learners and the environment (i.e. the input text) during a continuation task, that alignment can be analysed by identifying how learners’ continuations align with the input text at different representational levels, and that alignment contributes to the language learning potential of this task (Wang & Wang, 2015; Zhang, 2017). In line with this conceptualization, we construe alignment as a complex process in which learners engage in coordinated interaction with the input text in the continuation task.

Alignment may be characterized in terms of strength and automaticity, although reliable quantitative measures for both are rare. Wang and Wang (2015) operationalized alignment strength as the number of aligned lexical elements (i.e. the number of overlapping keywords; see Section III) between learners’ continuations and the input text, and called for further research on alignment at other linguistic levels. Costa, Pickering, and Sorace (2008) proposed that alignment may range from automatic to non-automatic. Automatic alignment occurs as involuntary processes, requiring minimal or no cognitive resources. For example, in a conversation, if one interlocutor uses the word sofa, it is more likely for another interlocutor to unknowingly adopt sofa than its synonym couch (Costa et al., 2008). Non-automatic alignment happens as controlled processes, entailing more cognitive resources and greater engagement of human agency. A case in point is when an L2 speaker intentionally reuses unfamiliar words uttered by a native speaker with uncertainty, as manifested in pauses, hesitations, or multiple attempts. Quantitative operationalization of alignment automaticity, however, remains largely unexplored.

Empirical research into the language learning potential of the continuation task is emerging. Much research along this line has been informed by the learn-together-use-together (LTUT) principle proposed by Wang (2009). This principle posits that contextual variables interacting with a linguistic form being learned ‘would influence its storage, retrieval, and use,’ because they occur in the leaner as an interconnected, dynamic system in which the activation of one ‘would trigger other related variables’ and because their concomitant occurrence would prime the use of the form ‘due to alignment at different levels’ (Wang & Wang, 2015, p. 522). Wang and Wang argued that the continuation task ‘tallies with this conception in that what is learned during reading affects subsequent L2 use and the reading component serves as context for the ensuring writing, exerting an alignment effect’ (p. 522). In the learn-together or reading phase of the continuation task, learners need to adequately comprehend the input text. According to Zwaan and Radvansky (1998), comprehension of a text involves construction of a situation model, defined as an integrated mental representation of the state of affairs described in the text consisting of five key dimensions: space, time, causality, intentionality, and reference to main individuals. Meanwhile, the demand of the ensuing writing may motivate learners to focus on form (Williams, 2012) while reading, with special attention to forms they may use in the continuation. In the use-together or writing phase, what happens in the reading process, including in particular the situation models constructed and the forms attended to, would affect learners’ alignment with the input text and their writing performance.

Previous studies on the continuation task have investigated the effects of several task-related factors on learners’ alignment and individual writing performance. Significantly stronger alignment and higher writing accuracy have been reported when the input text and continuations were in the same language (vs. different languages) (Wang & Wang, 2015); when learners perceived the input text as more interesting (Xue, 2013); when explicit instructions encouraging alignment were given (vs. when not) (Yuan, 2013); and when learners interacted with each other before the revision process (vs. when they did not) (Pang, 2014). These studies showed that stronger alignment generally led to better writing performance, although alignment automaticity was left unexplored.

2 Linguistic complexity of the input text in the continuation task

Notably, previous studies on the continuation task generally assumed that learners could fully comprehend the input text, with the exception of Pang (2014). Concomitantly, they overlooked the potential effect of the linguistic complexity of the input text on learners’ alignment and writing performance. Comprehensibility of the input text cannot be assumed, and learners’ comprehension of the text will affect how well they can continue it (Pang, 2014). Furthermore, as mentioned earlier, the linguistic complexity of a text bears upon its comprehensibility. The role of the linguistic complexity of the input text therefore deserves close scrutiny.

The linguistic complexity of a text is commonly characterized as the extent to which its language is elaborate and varied (e.g. Bulté & Housen, 2014; Ellis, 2003; Lu, 2011; Norris & Ortega, 2009), and much L2 complexity research to date has focused on lexical and syntactic complexity (e.g. Bulté & Housen, 2014; Lu, 2014, 2017). As the first attempt in exploring the effect of linguistic complexity of the input text on learners’ written production in the continuation task, this study considers a measure of lexical complexity based on word frequency and a measure of syntactic complexity based on sentence length. The motivation to focus on these two straightforward measures is twofold. First, word frequency and sentence length have been found to significantly affect reading comprehension (e.g. Droop & Verhoeven, 1998; McGregor, 1989) and are used in many commonly adopted text readability formulas, such as the new Dale–Chall readability formula (Chall & Dale, 1995) and the Lexile measure (Hiebert & Mesmer, 2013). Second, they are also the most accessible measures used in text simplification (e.g. Siddharthan, 2006). This second consideration is non-trivial, because text simplification is challenging even for experienced teachers (e.g. Jin & Lu, 2018) and because our findings are intended to be directly useful for L2 writing teachers. Based on the notion that ‘more is more complex’ (Bulté & Housen, 2014, p. 45), a higher degree of linguistic complexity in this study is associated with a higher proportion of low frequency words and a higher mean sentence length.

Both child and second language acquisition studies have found that learners’ productive ability lags behind their comprehension ability (Clark, 1993; Laufer & Paribakht, 1998). In other words, learners can usually understand texts that are linguistically more complex than their written production (Wang, 2012). A relevant question for writing teachers utilizing the continuation task is then whether the linguistic complexity of the input text should match or exceed learners’ production ability in order to maximize the L2 learning potential of the task. Wang (2012) claimed that the linguistic complexity of the input text should not far exceed learners’ production ability, but this claim remains to be empirically validated.

3 Theoretical predictions

Based on the LTUT principle, several predictions can be made on the potential effect of the linguistic complexity of the input text on learners’ alignment and writing performance. In the reading phase, the linguistic complexity of the input text could affect the completeness of the situation models learners construct and the level of their attention to form. An input text whose linguistic complexity matches learners’ production ability would likely allow them to construct more complete situation models than one that exceeds their production ability, given that learners’ production ability lags behind their comprehension ability and that linguistic complexity affects reading comprehension. The validity of this prediction could be assessed by a comprehension test focusing on the dimensions of the situation model. More complete situation models may lead to better attention to form, as noticing of form occurs only after the input has been processed for meaning (VanPatten, 2007).

Learners who constructed more complete situation models and paid better attention to form in the reading phase may demonstrate stronger and more automatic alignment in the writing phase, for at least two reasons. First, more complete situation models generally lead to stronger situational alignment and alignment at one level activates alignment at another (Pickering & Garrod, 2004). Second, with sufficient attention to certain forms, learners could reuse them automatically. Meanwhile, a more complex input text may incur greater human agency, which could translate into more effort in aligning with the text, increasing the strength but reducing the automaticity of alignment, because ‘[learners] try hardest for things [they] consider challenging but not nearly impossible’ (Gass & Selinker, 2008, p. 431).

More automatic alignment is likely to bring about higher writing fluency and accuracy. These two variables have been frequently used to measure L2 writing performance (e.g. Baba & Nitta, 2014; Polio & Shea, 2014; Wolfe-Quintero, Inagaki, & Kim, 1998). Fluency has been assessed using both process-based measures such as length of translating episodes and product-based measures such as composing rate (Latif, 2013). Text length is considered a valid measure of writing fluency in timed tasks (Baba & Nitta, 2014; Wolfe-Quintero et al., 1998) and is also the measure adopted in the current study. Writing accuracy is construed as ‘the ability to be free from errors’ (Wolfe-Quintero et al., 1998, p. 33) and has been assessed using ‘holistic measures, error-free units, number of errors, number of specific error types, and measures that take error severity into account’ (Polio & Shea, 2014, p. 10). Following Wang and Wang (2015), we operationalized writing accuracy as the frequency of several specific error types commonly found among Chinese EFL learners (see Section III). In a timed continuation task, more automatic alignment gives learners more time to write longer and more attentional resources to write more accurately. As Skehan and Foster (2007) argued, when learners have inadequate attentional resources for language forms, one strategy they readily employ is to ‘forget accuracy’ (p. 197).

Finally, more complete situation models and better attention to form could lead to higher writing accuracy. The situation models constructed can serve as an internal context and constrain learners’ L2 use (Sawaki, Quinlan, & Lee, 2013). More complete situation models would impose more potent constraints and help reduce error rate. Better attention to form in the reading phase may result in stronger and more accurate alignment with the input text and consequently help reduce error rate, too.

4 Research questions and hypotheses

The current study seeks to address the following three research questions: In a continuation task, does the use of simplified and unsimplified versions of the same input text whose linguistic complexity matches and exceeds learners’ production ability, respectively, affect 1) the completeness of the situation models they construct, 2) the strength and automaticity of the linguistic alignment between their continuations and the input text, and 3) their writing fluency and accuracy?

Based on the LTUT principle and the theoretical predictions discussed above, we hypothesize that learners continuing the simplified text would construct more complete situation models, demonstrate stronger and more automatic alignment, and show greater improvement in writing fluency and accuracy than those continuing the unsimplified text.

III Methodology

1 Participants

Forty-seven freshman (second semester) English majors at a university in southern China participated in this study. They were from two parallel intact classes with 23 and 24 students, respectively. Participant age ranged from 18 to 20. No participant had been abroad for over three months. In their first semester, they took communicative English, listening, and reading courses, and in their second semester, they continued these courses along with an English writing course. Their writing experience in the first semester was limited to five essay assignments from the communicative English course. For each assignment, they were given an outline and asked to write 200 to 250 words. At data collection time, they had no experience with English continuation tasks.

Forty participants were selected based on their pre-test compositions (see Section III.2) to form two groups (each with 20 participants from one class) with comparable production ability in terms of linguistic complexity. The two groups were randomly assigned to read an unsimplified (hereafter, the Unsimplified-Text Group) and simplified (hereafter, the Simplified-Text Group) input text in the continuation task, respectively. As Table 1 shows, their pre-test compositions had comparable coverage of the most frequent 3,000 words (96.657% vs. 97.629%, t(38) = −1.651, p = .107) and similar mean length of error-free T-units (8.819 vs. 9.004, t(38) = −.285, p = .778). The participants also took the Oxford Quick Placement Test after the pre-test. No significant difference was found between the mean scores for the Simplified-Text (M = 44.350, SD = 3.543) and Unsimplified-Text Group (M = 44.700, SD = 4.780) (t(38) = −.263, p = .794). These scores indicate that the participants were at the upper intermediate level (Geranpayeh, 2003).

Table 1.

Linguistic complexity of the pre-test writings and input texts.

	Pre-test writings		Input texts
	Simplified-Text Group	Unsimplified-Text Group	Simplified	Unsimplified
Texts	20	20	1	1
Words	5,059	5,890	934	935
Coverage of the most frequent 3,000 words	96.657% (SD = 1.909)	97.629% (SD = 1.815)	95.73%	87.53%
Mean length of error-free T-units	8.819 (SD = 2.267)	9.004 (SD = 1.820)	9.34	13.75

2 Procedure

a Materials, instruments, and procedure piloting

The materials, instruments, and task procedure were piloted in several iterations before the main study. Participants of the pilot study were first-year English majors at the same university; they did not participate in the main study.

The unsimplified input text used in this study was a shortened version of the story A mother in Mannville by Marjorie Kinnan Rawlings. Based on the syllabus for Chinese English majors, this story was appropriate for first-year English majors. The simplified version was created from the unsimplified text by increasing the proportion of high frequency words and shortening the sentences. High frequency words were defined as the most frequent 3,000 words, based on the expectations specified in the syllabus of the writing course. The target proportion of such words was based on an analysis of the proportion in the participants’ pre-test writings, calculated using Paul Nation’s Range 32 program with the British National Corpus wordlist. In shortening the sentences, in order to more accurately match the simplified text to the participants’ production ability, we first calculated the mean length of error-free T-units in the participants’ pre-test writings, which was about 9 words, and then shortened the T-units in the simplified text to that level. This was done as the participants’ compositions contained errors (e.g. run-on sentences) that make direct calculation of mean length of sentence problematic. A T-unit is defined as ‘one main clause plus any subordinate clause or nonclausal structure that is attached to or embedded in it’ (Hunt, 1970, p. 4), and an error-free T-unit is one that is ‘both grammatically correct and semantically appropriate’ (Jiang, 2013, p. 8). Mean length of error-free T-unit has also been reported to be a reliable measure of L2 proficiency (Polio & Shea, 2014).

The input texts were piloted in several iterations and proofread by three native speakers to ensure their comparability in content, comprehensibility, and language quality. As summarized in Table 1, the two final versions contained 934 and 935 words, respectively. The most frequent 3,000 words covered 95.73% of the simplified text but only 87.53% of the unsimplified text.¹ The mean length of T-units was 9.34 and 13.75 words for the simplified and unsimplified text, respectively. A comparison with the participants’ pre-test writings indicates that, based on these measures, the linguistic complexity of the two texts matched and exceeded the participants’ production ability, respectively. The Lexile measure, determined by the Lexile Analyzer, was 520 L and 960 L for the two texts, respectively.

Two questionnaires were used to collect perception, comprehension, and learner background data. The first questionnaire was designed to: (1) check whether participants had read the story before, (2) gauge their perception of the linguistic complexity of the input texts, and (3) evaluate their comprehension of those texts. Participants who had read the story before were excluded. Those who participated in the study rated the linguistic complexity of the text on a 7-point Likert scale, with 1 meaning very easy and 7 very difficult. They then answered 15 true or false questions, with three on each situation model dimension (Zwaan & Radvansky, 1998). The comprehension test results thus revealed the completeness of the situation models constructed for the input text. Two measures were taken to mitigate the guessing effect. First, we added an ‘unknown’ option to each question and instructed the participants to check this box when they were uncertain. Second, the participants were unaware that their responses would be graded. The first questionnaire underwent the same piloting process as the input texts to ensure its clarity and fine-tune the questions. The second questionnaire was designed to collect information about participants’ English learning background and was tested in the procedure piloting phase. Finally, the procedure of the main study, summarized in Table 2 and detailed in the next section, underwent three iterations of piloting to ensure its smooth implementation.

Table 2.

Procedure of the main study.

Week	Task	Duration
1	Pre-test writing task	45 minutes
	Participant grouping	n/a
4	Continuation task (two participants videotaped in each group)
	Step 1: read the input text	10 minutes
	Step 2: complete questionnaire one	5 minutes
	Step 3: continue the story	45 minutes
	Step 4: complete questionnaire two	5 minutes
	Stimulated recall interviews on the same day	25–30 minutes

b The main study

The main study took place in the usual classrooms of the participants’ English writing classes. The study spanned the first four weeks of class, and all writing tasks and questionnaires were completed on paper. In the first week, all students from the two classes completed a pre-test writing task on the topic ‘My first experience of …’. The compositions were used to assess the participants’ production ability in terms of linguistic complexity, form two comparable groups, and establish fluency and accuracy baselines. In the fourth week, participants completed a continuation task in four steps: reading the simplified or unsimplified input text, completing questionnaire one, continuing the input text, and completing questionnaire two. The input text remained accessible to the participants during the writing process. Two focal participants randomly selected from each group were videotaped throughout the four steps.

On the same day after the continuation task, one researcher held semi-structured stimulated recall interviews with each focal participant and recorded the interviews. The interviews were meant to gather information about participants’ continuation writing process and were conducted in Chinese to avoid comprehension problems. Each focal participant watched the episodes of her own writing process showing notable pauses or retrieval of the input text. The input text and their own continuation remained accessible to help them recall the moments captured in those episodes. After each episode, the interviewer asked the participant to explain why she paused or retrieved the input text, using the following question: ‘You paused/went back to the input text here. What were you thinking then?’ This procedure allowed us to identify alignment-related pauses in the continuation writing process as those in which the participants attempted to find specific linguistic elements in the input text to align with.

3 Analysis

The participants’ compositions and questionnaire answers were computerized, and recordings of the interviews were transcribed. The two groups’ pre-test writings and continuations formed four small corpora, each with 20 texts. A mixed-method analysis was employed to gauge the effect of the linguistic complexity of the input text on participants’ alignment and writing performance. The two groups’ comprehension scores were quantitatively analysed for completeness of the situation models constructed, the two continuation corpora for strength of alignment, the alignment-related pauses for automaticity of alignment, and all four corpora for writing fluency and accuracy. The interview transcripts were qualitatively analysed to look into the writing process.

In assessing the effect of the linguistic complexity of the input text on alignment, we first compared the continuations with the corresponding input text to identify aligned linguistic elements; we then compared the strength and automaticity of alignment between the two groups. In assessing its effect on writing performance, we used participants’ pre-test writings as the baseline and compared the two groups’ changes in writing fluency and accuracy from their pre-test writings to their continuations.

In response to Wang and Wang’s (2015) call for research on linguistic alignment beyond the lexical level, we examined alignment at the lexical, phrasal and clausal levels. At the lexical level, we used the Keyword List functionality of AntConc 3.4.1w (Anthony, 2014) to identify overlapping words in the continuations and the input texts. This tool compares the frequencies of the words in a corpus with their frequencies in a reference corpus and determines which words are overused or underused. Words in the corpus are ranked on a keyword list by their log-likelihood ratios or chi-square values (i.e. their ‘keyness’ values). To match the genre of the input texts, we used as a reference corpus the general fiction section of the one-million-word Freiburg-Brown Corpus of American English (Hundt, Sand, & Skandera, 1999). The keyword list of each continuation corpus was compared to that of its corresponding input text to assess the extent of lexical alignment.

At the phrasal level, we identified multiword sequences in the continuations that aligned with those in the input text. A multiword sequence was defined as a sequence of words which as a whole is meaningful, such as to chop wood, to stare at, I went to, etc. Two multiword sequences were considered to align with each other if one of the following conditions was met: first, they had the same words or lemmas; second, one contained all the words or lemmas in the other and some additional elements (e.g. modifiers) that do not change its fundamental meaning; and third, one contained over half of the words or lemmas, including in particular the verbs, in the other that contribute to its fundamental meaning and differed only from the other in the use of words that do not alter its structure (e.g. pronouns). At the clausal level, we identified strings of words that form clauses in the continuations that aligned with clauses in the input text. A string of words was considered a clause if it is in one of seven basic clause types: SV, SVO, SVC, SVA, SVOO, SVOC, SVOA (S = subject, V = verb, O = object, C = complement, A = adverbial) (Yang, 2005). Criteria similar to those for phrasal alignment identification were used in clausal alignment identification. The excerpts below illustrate aligned multiword sequences (italicized) and an aligned clause (bold- faced). Additional examples are provided in Table 3.

Input text: I looked at him, carefully, for the first time.

Continuation: In a deep silence, I looked at the gloves.

Input text: She stared at me.

He should have the same gray-blue eyes and …

Continuation: Jerry stared at me, with his gray-blue eyes, tensionally.

Table 3.

Examples of aligned multiword sequences and clauses.

Element aligned	Expression in the input text	Expression in a continuation
Multiword	I went to	I go to
sequences	It was time to	It was time to
	In winter	In that cold winter
	I asked her to	I asked Miss Clark to
	Blamed himself for	Blaming me for
Clauses	I would miss him.	I missed him.
	I don’t understand.	I didn’t understand.
	He had a mother in Mannville.	He has a mother living in Mannville.
	His mother had sent him a pair of skates.	I sent him a pair of skates.
	She stared at me.	He just stared at me.

Strength of alignment was quantified by counting the number of identified aligned elements. Automaticity of alignment was measured as the frequency and length of alignment-related pauses, obtained by counting the number of such pauses identified for each focal participant through the stimulated recall interview and by recording the length of each pause.

Writing fluency was operationalized as the number of words in a composition, counted using Range 32, as all compositions were produced within 45 minutes. To assess writing accuracy, we coded the following five categories of errors in each composition: number agreement errors, misuse of articles, misuse of copula, misuse of non-finite verbs, and tense errors, illustrated in the examples below. These categories have been found common in the written production by Chinese EFL learners (e.g. Tang, 2000). In addition, Wang and Wang (2015) reported that the continuation task could help reduce these errors in learner production.

All the things was OK. (number agreement error)

They made snowman … (misuse of articles)

… he always my boy. (misuse of copula)

I couldn’t wait to opening my postbox. (misuse of non-finite verb)

… what he had said is just to comfort me. (tense error)

Table 4 summarizes the variables in this study along with their operationalizations.

Table 4.

Variables in this study.

	Variable	Operationalization
Independent	Linguistic complexity of the input text	Word frequency and sentence length
Dependent	Situation model completeness	Comprehension test score
	Alignment strength	Number of aligned linguistic elements
	Alignment automaticity	Number and length of alignment-related pauses
	Writing fluency	Number of words in a composition
	Writing accuracy	Number of errors in the five categories

Two researchers independently identified aligned multiword sequences and clauses in each continuation and coded each composition for the five categories of errors. Inter-rater reliability measured by Cronbach’s alpha reached .962 for alignment identification and .927 for error coding. All discrepancies were resolved through discussion and consultation with native speakers. The aligned elements and the errors of each category in each composition were counted and their normed frequency (per 100 words) recorded.

IV Results

1 Research question 1: Effect of the linguistic complexity of the input text on the completeness of the situation models constructed

As mentioned earlier, participants rated the linguistic complexity of the input texts immediately after reading them. The mean rating for the simplified version (M = 3.450, SD = .945) was significantly lower than that for the unsimplified version (M = 4.800, SD = .410) (t(38) = −5.863, p = .000, Cohen’s d = 1.85). Cronbach’s alpha for the comprehension test reached .926, indicating that the test was reliable. The Simplified-Text Group correctly answered significantly more comprehension questions (M = 11.400, SD = 1.142) than the Unsimplified-Text Group (M = 10.050, SD = 1.468) (t(38) = 3.245, p = .002, Cohen’s d = 1.03), indicating that, overall, the former constructed more complete situation models than the latter. Participants’ comprehension scores negatively correlated with their ratings of the linguistic complexity of the input text (r = −.399, p = .011).

2 Research question 2: Effect of the linguistic complexity of the input text on the strength and automaticity of linguistic alignment

Results of the keyword analysis revealed no significant difference in the strength of lexical alignment between the two groups. The simplified text and its continuations shared 10 (Jerry, I, mother, orphanage, Mannville, skates, boy, cabin, him, pair) of the top 30 keywords, and the unsimplified text and its continuations 13 (Jerry, I, orphanage, mother, skates, cabin, my, me, Mannville, him, boy, pair, Pat). A closer examination of the keyword lists revealed a difference in the alignment of shocked and its synonym staggered. Shocked is among the second thousand most frequent words and staggered the fourth. Their frequency in the input texts is identical: shocked occurred once in the simplified text and staggered once in the unsimplified text. However, shocked appeared 15 times in the continuations for the simplified text, while staggered only twice in the continuations for the unsimplified text. The difference suggests that the Simplified-Text Group paid better attention to shocked than the Unsimplified-Text Group to staggered.

The normed frequency of aligned phrases and clauses did not differ significantly between the two groups. The stimulated recall interview data showed that while the number of aligned phrases and clauses was comparable among the focal participants, those from the Simplified-Text Group incurred fewer and shorter alignment-related pauses. Mary (all names are pseudonymous) and Helen from the Simplified-Text Group produced 14 and 20 aligned phrases and clauses, respectively, and they each paused only once in the writing process. Mary paused 7 seconds for chop wood and Helen 8 seconds for the cabin. Lisa and Susan from the Unsimplified-Text Group each produced 16 aligned phrases and clauses with four pauses. Lisa paused 43 seconds for you look a little bit like my mother and 14, 10, and 9 seconds for the mountains, the village below, and a pair of gloves, respectively. Susan’s pauses averaged 12.2 seconds, with 14 for you look a little bit like my mother, and 12, 12, and 11 for Miss Clark, the orphanage, and chop wood, respectively. These results indicate that the focal participants from the Simplified-Text Group had higher automaticity of alignment than those from the Unsimplified-Text Group.

3 Research question 3: Effect of the linguistic complexity of the input text on writing fluency and accuracy

In the pre-test, the mean composition length for the Simplified-Text Group (M = 252.950, SD = 37.266) was shorter than that for the Unsimplified-Text Group (M = 294.500, SD = 67.326), but the difference (t(38) = −2.415, p = .022) was non-significant with the alpha value adjusted to .005 by Bonferroni correction, as 10 tests were performed. The mean continuation length for the Simplified-Text Group (M = 381.500, SD = 78.278) was longer than that for the Unsimplified-Text Group (M = 337.750, SD = 68.295), but the difference (t(38) = 1.883, p = .067) was again non-significant. Figure 1 shows the change in mean length between the two groups’ continuations and their pre-test writings. The increase for the Simplified-Text Group (M = 128.550, SD = 79.100) was significantly larger than that for the Unsimplified-Text Group (M = 43.250, SD = 56.216) (t(38) = 3.931, p = .000, Cohen’s d = 1.24). As shown in the figure, this significant difference resulted in the reversal of the pattern of fluency of the two groups relative to each other between the two tasks.

Figure 1.

Change in mean length between the two tasks.

In the pre-test, the mean error rate for the Simplified-Text Group (M = 4.193, SD = 1.391) was significantly higher than that for the Unsimplified-Text Group (M = 2.860, SD = 1.284) (t(38) = 3.148, p = .003). In the continuation task, no significant difference was found between the mean error rate for the Simplified-Text Group (M = 2.533, SD = 1.147) and the Unsimplified-Text Group (M = 2.470, SD = 1.513) (t(38) = 0.151, p = .881). Figure 2 shows the change in mean error rate between the two groups’ continuations and pre-test writings. The reduction was larger for the Simplified-Text Group (M = −1.660, SD = 1.962) than for the Unsimplified-Text Group (M = −0.391, SD = 1.623). While the difference in the change narrowly missed statistical significance (t(38) = −2.227, p = .032) with the adjusted alpha value of .005, the effect size indicated by Cohen’s d (0.70) was fairly large.

Figure 2.

Change in mean error rate between the two tasks.

V Discussion

Our findings partially confirmed our hypotheses. First, the Simplified-Text Group constructed more complete situation models than the Unsimplified-Text Group. Second, the two groups did not differ significantly in alignment strength; however, the focal participants from the Simplified-Text Group demonstrated more automatic alignment than those from the Unsimplified-Text Group. Third, the Simplified-Text Group saw significantly greater improvement in writing fluency than the Unsimplified-Text Group; the former started with significantly lower writing accuracy in the pre-test writing task but improved to a comparable level as the latter in the continuation task. Overall, these findings show that compared to the linguistically more complex unsimplified text, the simplified text whose linguistic complexity matched learners’ production ability positively affected learners’ alignment, writing fluency and accuracy. We discuss these results in light of the LTUT principle below.

In the reading or learn-together phase, the simplified text allowed learners to build more complete situation models, as evidenced by the significantly higher comprehension scores of the Simplified-Text Group. This result is consistent with prior findings on the effect of linguistic complexity on reading comprehension (e.g. Chang, 2006; Droop & Verhoeven, 1998; Qi & Wang, 1988) and supports Feng, D’Mello, and Graesser’s (2013) claim on the role of linguistic complexity in situation model construction. The simplified text likely also enabled learners to pay more attention to form, as suggested by the difference in the alignment of shocked and staggered between the two groups and by the fact that the focal participants from the Unsimplified-Text Group needed to look up a wider range of forms they wanted to align with, including, for example, the name of a major character, Miss Clark.

Following the LTUT principle, the more complete situation models constructed by the Simplified-Text Group and the more attention to form during the reading phase can explain why the focal participants from the Simplified-Text Group exhibited more automatic alignment in the continuation writing phase. However, the more challenging text may have brought about higher levels of engagement of human agency (Gass & Selinker, 2008). This was reflected in the greater effort made (i.e. more frequent pauses to look up a wider range of forms) by the focal participants from the Unsimplified-Text Group to align with the input text, facilitated by the availability of the source text in the continuation writing process. This higher level of agency may have helped increase alignment strength for the focal participants from the Unsimplified-Text Group at the cost of reduced automaticity. Finally, the more automatic alignment shown by the focal participants from the Simplified-Text Group, coupled with more complete situation models and more attention to form, could have led to the Simplified-Text Group’s greater improvement in writing fluency and helped improve its writing accuracy from a lower level than to a comparable level as the Unsimplified-Text Group. As such, our findings offer empirical support for Wang’s (2012) claim that the linguistic complexity of the input text for the continuation task should not far exceed learners’ production ability.

Our findings have important implications for L2 writing pedagogy. Different types of input should be used for writing tasks serving different learning purposes. Our results show that, in the continuation task, input texts whose linguistic complexity matches L2 learners’ production ability help learners better improve their writing fluency and accuracy than those whose linguistic complexity exceeds their production ability. As such, input text selection or adaptation should ideally be informed by an evaluation of the linguistic complexity of learners’ written production. This ensures that the input text will allow learners to construct complete situation models and notice the language forms they are expected to align with and acquire. This is important because without adequate attention to form, no input can be turned into intake (Robinson, 2003), and L2 learners can only pay attention to form when they can process input for meaning effortlessly (VanPatten, 2007). It is also advisable for writing teachers to help L2 learners become aware of the gap between their comprehension and production abilities and the importance of selecting appropriate input texts for practicing continuation writing.

Our study also has useful implications for L2 writing assessment. Continuation tasks are especially compatible with Swain and Lapkin’s (1998) notion that language learning takes place when learners are able to reuse others’ language forms. Swain and Lapkin viewed examples in which participants learn correct forms or rules from each other in collaborative dialogue and subsequently successfully re-use or re-apply them as evidence of dialogue as an occasion for L2 learning. We argue that, similarly, instances of learners’ successful alignment with the input text also constitute evidence of continuation writing as an occasion for L2 learning. In explaining the expectations of a continuation task, therefore, L2 writing teachers should encourage students to align with the language in the input text, provided that such alignment facilitates coherent development of the story. In assessing learners’ continuations, contextually appropriate linguistic alignment should be valued.

VI Conclusions

The present study reveals that the linguistic complexity of the input text significantly affects L2 learners’ alignment, writing fluency, and writing accuracy in the continuation task. Specifically, the simplified text matching learners’ production ability led to more automatic alignment and greater improvement in writing fluency and accuracy than the unsimplified text that exceeded learners’ production ability. The understanding of this effect provides critical new insights into how the language learning potential of the continuation task can be maximized.

This study has several limitations, some of which can be addressed in future research. First, we only considered word frequency and mean sentence length in assessing the linguistic complexity of the input text and learners’ written production. Future research could include other measures and dimensions of linguistic complexity that affect reading comprehension or index L2 writing quality. Second, the number of focal participants included in our stimulated recall interviews was small. The inclusion of more participants will yield richer information about the influence of the reading process on continuation writing. Third, our measurement of alignment automaticity was rudimentary, and future research could explore more rigorous measurement of this construct. Finally, the question of how the linguistic complexity of the input text may affect L2 learners’ writing development over time remains unanswered. This will also be a fruitful direction for future research.

Footnotes

Acknowledgements

The authors thank anonymous reviewers for their helpful suggestions on improvement and all the students and teachers who participated in the empirical study. Thanks also go to the GDUFS Center for Linguistics and Applied Linguistics, where the study was completed.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the Major Project of the National Social Science Foundation of China (Grant 12&ZD224).

Notes

ORCID iD

Jinfang Peng

References

Anthony

(2014). AntConc: Version 3.4.1w [computer software]. Tokyo: Waseda University. Available at: http://www.laurenceanthony.net (accessed June 2018).

Atkinson

Nishino

Churchill

Okada

(2007). Alignment and interaction in a sociocognitive approach to second language acquisition. The Modern Language Journal, 91, 169–188.

Baba

Nitta

(2014). Phase transitions in development of writing fluency from a complex dynamic systems perspective. Language Learning, 64, 1–35.

Bulté

Housen

(2014). Conceptualizing and measuring short-term changes in L2 writing complexity. Journal of Second Language Writing, 26, 42–65.

Chall

J.S.

Dale

(1995). Readability reviewed: The new Dale–Chall readability formula. Cambridge, MA: Brookline Books.

Chang

(2006). Effects of topic familiarity and linguistic difficulty on the reading strategies and mental representations of nonnative readers of Chinese. Journal of Language Learning, 4, 172–198.

Clark

E.V.

(1993). The lexicon in acquisition. New York: Cambridge University Press.

Costa

Pickering

M.J.

Sorace

(2008). Alignment in second language dialogue. Language and Cognitive Processes, 23, 528–556.

Droop

Verhoeven

(1998). Background knowledge, linguistic complexity and second-language reading comprehension. Journal of Literacy Research, 30, 253–271.

10.

Ellis

(2003). Task-based language learning and teaching. Oxford: Oxford University Press.

11.

Feng

D’Mello

Graesser

A.C.

(2013). Mind wandering while reading easy and difficulty texts. Psychonomic Bulletin Review, 20, 586–592.

12.

Gass

S.M.

Selinker

(2008). Second language acquisition: An introduction course. 3rd edition. New York: Routledge.

13.

Geranpayeh

(2003). A quick view of the English Quick Placement Test. Research Notes, 12, 8–10.

14.

Hiebert

E.H.

Mesmer

H.A.E.

(2013). Upping the ante of text complexity in the Common Core State Standards: Examining its potential impact on young readers. Educational Researcher, 42, 44–51.

15.

Hundt

Sand

Skandera

(1999). Manual of information to accompany the Freiburg-Brown Corpus of American English. Freiburg: Department of English, Albert-Ludwigs-Universität Freiburg.

16.

Hunt

K.W.

(1970). Syntactic maturity in schoolchildren and adults. Monographs of the Society for Research in Child Development, 35, 1–67.

17.

Jiang

(2013). Measurements of development in L2 written production: the case of L2 Chinese. Applied Linguistics, 34, 1–24.

18.

Jin

(2018). A data-driven approach to text adaptation in teaching material preparation: Design, implementation and teacher professional development. TESOL Quarterly, 52, 257–267.

19.

Latif

M.M.M.A

. (2013). What do we mean by writing fluency and how can it be validly measured? Applied Linguistics, 34, 99–105.

20.

Laufer

Paribakht

(1998). The relationship between passive and active vocabularies: Effects of language learning context. Language Learning, 48, 365–391.

21.

(2011). A corpus-based evaluation of syntactic complexity measures as indices of college-level ESL writers’ language development. TESOL Quarterly, 45, 36–62.

22.

(2014). Computational methods for corpus annotation and analysis. Dordrecht: Springer.

23.

(2017). Automated measurement of syntactic complexity in corpus-based L2 writing research and implications for writing assessment. Language Testing, 34, 493–511.

24.

McGregor

A.K.

(1989). The effect of word frequency and social class on children’s reading comprehension. Reading, 23, 105–115.

25.

Nishino

Atkinson

(2015). Second language writing as sociocognitive alignment. Journal of Second Language Writing, 27, 37–54.

26.

Norris

J.M.

Ortega

(2009). Towards an organic approach to investigating CAF in instructed SLA: The case of complexity. Applied Linguistics, 30, 555–578.

27.

O’Connor

R.E.

Swanson

H.L.

Geraghty

(2010). Improvement in reading rate under independent and difficult text levels: Influences on word and comprehension skills. Journal of Educational Psychology, 102, 1–19.

28.

Ortega

(2011). Reflections on the learning-to-write and writing-to-learn dimensions of second language writing. In R.M.

Manchón

(Ed.), Learning-to-write and write-to-learn in an additional language (pp. 237–250). Amsterdam: John Benjamins.

29.

Pang

(2014). The effect of peer interaction on alignment in a continuation task. Unpublished master’s thesis, Guangdong University of Foreign Studies, Guangzhou, China.

30.

Pickering

M.J.

Garrod

(2004). Toward a mechanistic psychology of dialogue. Behavioral and Brain Sciences, 27, 169–226.

31.

Plakans

Gebril

(2013). Using multiple texts in an integrated writing assessment: Source text use as a predictor of score. Journal of Second Language Writing, 22, 217–230.

32.

Polio

Shea

M.C.

(2014). An investigation into current measures of linguistic accuracy in second language writing research. Journal of Second Language Writing, 26, 10–27.

33.

Wang

(1988). Background knowledge and language complexity in reading comprehension. Foreign Language Teaching and Research, 2, 24–30.

34.

Reitter

Moore

J.D.

(2014). Alignment and task success in spoken dialogue. Journal of Memory and Language, 76, 29–46.

35.

Robinson

(2003). Attention and memory during SLA. In C.J.

Doughty

Long

M.H.

(Eds.), The handbook of second language acquisition (pp. 631–678). Oxford: Blackwell Publishing.

36.

Sawaki

Quinlan

Lee

(2013). Understanding learner weaknesses and strengths: assessing performance on an integrated writing task. Language Assessment Quarterly, 10, 73–95.

37.

Siddharthan

(2006). Syntactic simplification and text cohesion. Research on Language and Computation, 4, 77–109.

38.

Skehan

Foster

(2007). Cognition and tasks. In P.

Robinson

(Ed.), Cognition and second language instruction (pp. 287–318). Beijing: World Publishing Corporation.

39.

Swain

Lapkin

(1998). Interaction and second language learning: Two adolescent French immersion students working together. The Modern Language Journal, 82, 320–337.

40.

Tang

(2000). A preliminary study of foreign language learners’ errors. Foreign Languages and Their Teaching, 12, 56–58.

41.

VanPatten

(2007). Processing instruction. In Sanz

(Ed.), Mind and context in adult second language acquisition (pp. 267–281). Washington, DC: Georgetown University Press.

42.

Wang

(2009). A study of the interrelationship between L2 learning variables and L2 use. Foreign Languages in China, 6, 53–59.

43.

Wang

(2012). The continuation task: An effective way to facilitate L2 learning. Foreign Language World, 5, 2–7.

44.

Wang

(2016). Learning by extension. Modern Foreign Languages, 6, 784–793.

45.

Wang

(2015). Effect of alignment on L2 written production. Applied Linguistics, 36, 503–526.

46.

Williams

(2012). The potential role(s) of writing in second language development. Journal of Second Language Writing, 21, 321–331.

47.

Wolfe-Quintero

Inagaki

Kim

H.Y.

(1998). Second language development in writing: Measures of fluency, accuracy, and complexity. Honolulu, HI: University of Hawaii at Manoa.

48.

Xue

(2013). Interest-related alignment in L2 story continuation. Unpublished master’s thesis, Guangdong University of Foreign Studies, Guangzhou, China.

49.

Yang

(2005). An introduction to linguistics. Beijing: Higher Education Press.

50.

Yuan

(2013). The effect of task instructions on L2 writing from the perspective of alignment. Unpublished master’s thesis, Guangdong University of Foreign Studies, Guangzhou, China.

51.

Zhang

(2017). Reading–writing integrated tasks, comprehensive corrective feedback, and EFL writing development. Language Teaching Research, 21, 217–240.

52.

Zwaan

R.A.

Radvansky

G.A.

(1998). Situation models in language comprehension and memory. Psychological Bulletin, 123, 162–185.