Abstract
This article reports on a study in which stimulated recall data and quantitative measures of complexity, accuracy and fluency (CAF) were used to address three interconnected questions in different planning conditions: (1) how learners share their limited attentional capacity with different cognitive processes of ‘planner/proposer’, ‘translator’ and ‘evaluator/reviser’, (2) what kinds of self-repairs are more prone to be utilized by L2 writers, and (3), which condition can provide a better work plan for producing high quality narratives in terms of CAF triad. Sixty intermediate L2 writers narrated a picture story task in four types of planning conditions. The results show that while on-line planning induced the most cognitive processes of planning, translating and evaluating, pre-task planning reduced the number of processes at the time of writing. Moreover, the results reveal that pre-task planning significantly reduced the amount of self-repair when compared to on-line planning, which activated more error repairs, rephrasing repairs and different information repairs. The CAF measures disclose that enhancement of all measures at the same time could not be attained even with the provision of both pre- and on-line planning simultaneously and therefore lend support to the Overload Hypothesis and the Limited Attentional Capacity Model. The implications of these outcomes are discussed, and suggestions for further research are advanced.
Keywords
I Introduction
The impact of planning on language production has been thoroughly investigated by a number of task-based studies, especially in oral production (for a review see Ellis, 2009). However, the number of studies that investigated the effect of planning on L2 writing is very limited. Those who studied its effect focused solely on the product and neglected the actual processes involved at the time of writing (Ellis & Yuan, 2004; Johnson, Mercado, & Acevedo, 2012; Ong & Zhang, 2010) while others focused on the processes and overlooked the impact of processing on the final product (Ong, 2014; Roca de Larios, Manchón, Murphy, & Marín, 2008). To get the whole picture, as Ortega (2005, p. 78) states, future research agendas ought to ‘adopt a process-product approach and encompass both task and learner in the study of planning’.
The aim of this article is to scrutinize the planning/proposing, translating and evaluating/revising processes of L2 writers in different time constraint task conditions and their effects on the complexity, accuracy and fluency (CAF) of the final product. In order to show planning and monitoring behavior, most of the studies on writing processes used think-aloud protocols in which participants were asked to articulate what came into their mind as they wrote the task (Roca de Larios et al., 2008). Although think-aloud protocols provide rich information about the hidden processes, there are several severe shortcomings such as ‘reactivity’ and ‘overloading working memory’ that cause low reliability and validity. The reactivity of think-aloud protocols may cause the disruption of a writer’s cognitive processes due to an overload of working memory because of doing two things at the same time (for a review see Smagorinsky, 1994). Another promising line of research is based on the investigation of pauses during writing. Following oral studies, some scholars investigated pauses in writing production (Matsuhashi, 1981; Schilperoord, 2002). However, the methodology that was selected for these studies could not delve into what goes on in the mind of the writers when they are planning, translating and monitoring. Although it is based on the premise that ‘pauses, moments of physical inactivity during writing, offer observable clue to the covert cognition processes’ (Matsuhashi, 1981, p. 114), the quantitative data alone could not disclose the nature of hidden processes in the learner’s mind. Therefore, the need for the combination of qualitative and quantitative data gathering seems inevitable. Building on previous studies of stimulated recall on writing processes (Bosher, 1998; Sasaki, 2004; Schumacher, Klare, Cronin, & Moses, 1984), this study tried to combine retrospective analysis of pauses through a stimulating recall procedure to delve into the nature of the processes at the time of writing in different planning time conditions. Moreover, in order to corroborate the effects of processes in the product, quantitative measures of production, i.e. CAF were employed.
II Background
1 Planning and cognitive processes
The initiative for studying the effect of planning time on the (oral) production of learners draws on controversial theories. On the one hand, it is based on Skehan’s (1998) Limited Attentional Capacity Model which states that human attentional capacity is limited, and that to attend to one aspect of performance can cause the other dimensions to suffer. This theory is portrayed well by Trade-Off Hypothesis, which assumes competition for cognitive resources. Hence, even when providing learners with attentional resources (planning time), whether it is pre-task planning (strategic planning) or on-line planning, this may not compensate for the trade-off effect between content (fluency) and form (accuracy and complexity) and then, within form, between accuracy and complexity. On the other hand, Robinson’s (2001) Cognition Hypothesis contends that cognitive processes can rely on multiple attentional resources and increasing task complexity may, in fact, result in more elaborate processing that leads to better linguistic performance in all the areas of CAF simultaneously. Likewise, in the realm of writing, Kellogg (1990) proposed two opposing theories: Overload Hypothesis and Interaction Hypothesis. Overload Hypothesis holds that planning time should improve writing quality and the fluency of language production, as planning reduces the cognitive demands and therefore allows learners to focus on other processes such as translating. Interaction Hypothesis predicts that the planning condition reduces text quality because the writing process is typically dynamic, and planning may prevent writers from making use of the opportunities that arise during the interaction of planning, translating, and reviewing and therefore recommends free writing without pre-planning. Similarly, Galbraith’s (1999) Dual Processing Model, in which text production is assumed to be an active knowledge-constituting process, claims that novel content can be spontaneously constituted without explicit planning as thoughts emerge during text production.
Notwithstanding the existence of a number of theories, few writing studies systematically investigated the effects of different kinds of planning on cognitive processes and the effects of these processes on the final product in written narratives. As mentioned by Kormos (2011, p. 148) ‘studies on written narratives produced by foreign language learners are scarce despite the fact that this genre is an important one both in language teaching pedagogy and in the assessment of foreign language (FL) competence.’ Interestingly, except for Kellogg’s (1988) and Ortega’s (2005) studies, which focused on both cognitive processes during the writing and final text quality, the other studies simply focused on the cognitive processes or only investigated the final product. Thus, as mentioned by Roca de Larios et al. (2008, p. 44) ‘another area for future research is the analysis of the relationship between the interaction of writing processes and quality of writing.’ In other words, as astutely mentioned by Ong and Zhang (2010, p. 218), writing models ‘made no predictions with regard to the manipulation of cognitive processes (such as planning, transcribing, and revising) on writing quality’ and hence little is known about the effect of manipulation of tasks through planning time on what learners will actually do and monitor. Which cognitive processes are traded-off to prioritize the allocation of the limited attentional resources in the process of composing remains unknown to date. In the same vein, Ellis and Yuan (2004, p. 82) in the conclusion of their study pointed out the need ‘to examine more closely the relationship between ‘process’ and ‘product’ using the same experimental design, but probing more deeply what L2 writers actually do when they engage in pre-task and on-line planning’ and ‘to include a third experimental condition, in which the participants have the opportunity for both pre-task and unpressured on-line planning’. To fill the gap, we have chosen Hayes’s (2012) Revised Model of Writing Process that fits our purpose well; this model was developed out of Sentence Composition Model (Kaufer, Hayes, & Flower, 1986). Using thinking aloud study of pauses of two or more seconds and revisions, they found writers produce a sentence in parts, or ‘bursts’, instead of generating a complete sentence. Later on, a more detailed model of the processes involved in text production has been developed (Chenoweth & Hayes, 2001) which bears a close resemblance to Levelt’s (1989) Model of Speech Production. The model includes the following cognitive processes (examples are given based on the stimulated recall interviews, i.e. watching the recorded video of their writing to recall and describe what they were thinking at the moment of pauses more than three seconds and revisions. The stimulated recall comments [SRC] are translated into English):
Planner/Proposer that includes the intentions, goals, plans and ideas to be expressed.
SRC: I was thinking about the rest of the story and the way I can expand it.
Translator that converts the delivered prelinguistic idea package into an unarticulated surface linguistic string by selecting lexical items and the appropriate structure.
SRC: I was thinking about how to render my sentence ‘the boy ran and the man chased him’ to English.
Evaluator/reviser that assesses the product of the writing processes with the writer’s goals.
SRC: drop–dropped: I reread the sentence and added ‘ed’.
Memory resources that assume buffers that stores intermediate representations of messages (preverbal message, surface structure) making them available for further processing.
Transcriber that encodes the linguistic string into an orthographic plan and then produces the text.
2 Evaluation and self-repair behavior
Revision is regarded as a major process in the different models of writing. Fitzgerald (1987, p. 484) provides a comprehensive definition for revision: ‘Revision means making any change at any point in the writing process.’ Similarly, as characterized by Hayes’s (2012) model of writing, revision can intervene at any moment in the writing process. For instance, after the translator provides the output, it will be kept in an articulatory buffer of working memory to be checked by the evaluator. In case the output is considered problematic, the planner/proposer or the translator proposes a new output.
Based on the comprehensive taxonomy of self-repair developed by Kormos (1998) for L2 speech, which was developed based on qualitative analysis of retrospections, we can make inferences regarding learners’ self-monitoring processes and mechanisms in different planning conditions. To this end, we adapted this taxonomy for writing which includes the following (examples are based on the stimulated recall comments [SRC] of the learners about their evaluating/revising behavior translated into English):
Different information (D-) repair, where the writer decides to change the inappropriate information, change the order or abandon the message totally (usually at the planner/proposer stage).
John said to him hello Mr. Bean.
SRC: The content changed in my mind, so I changed the sentence and omitted why. First, I wanted to ask John about the reasons he started running; then I realized it is better to use Mr. Bean to say the reasons.
Appropriacy (A-) repair, which is employed when the writer intends to produce the originally intended idea, but modifies it for reasons such as inaccurate and ambiguous information, incoherent terminology, unsophisticated language, and pragmatically inappropriate language.
He decided to
SRC: I was thinking about using different verbs to avoid using the same words repeatedly.
When he took off the bus,
SRC: Here I wanted to tell more precisely what exactly fell on the ground. In the picture, it was more like a box than a pocket.
Error (E-) repair, which mostly happens at the translator stage and consists of lexical repair, grammatical repair, spelling, capitalization, and punctuation.
His pocket
SRC: I realized that the tense was wrong because it happened in the past.
Rephrasing (R-) repair, which mostly happens at the translator stage where the writer decides to reformulate the preverbal plan without changing the content of the original message by adding something and using paraphrase to convey the message.
He understands one person [pause]
SRC: I was searching for the word ‘chasing’ but couldn’t remember it, so I changed my English sentence and used ‘follow’.
To conclude, the present study, drawing on Hayes’s (2012) model of text production and Kormos’s (1998) taxonomy of self-correction, tries to delve into the underlying cognitive processes of L2 writing under different planning conditions.
III Research questions
This study investigated the following research questions:
Does the provision of different planning time make a difference in planning/proposing, translating and revising/evaluating processes of L2 writing?
What kinds of evaluating/revising behavior will be more drawn on by L2 writers in different planning conditions?
How do different planning conditions affect the complexity, accuracy and fluency of L2 writing?
Building on Hayes’s (2012) model of text production and the findings of the previous studies (Ellis & Yuan, 2004, 2005), it was hypothesized that the opportunity for pre-task planning will induce learners to organize and integrate more complex ideas that, in turn, produces a more rehearsed complex preverbal message. This message will reduce the number of ‘planner/proposer’ and ‘translator’ processes and thus learners can be expected to benefit in fluency and also in lexical and syntactic complexity. However, the provision of time for on-line planning will increase the number of ‘planner/proposer’, ‘translator’, and, in particular, ‘evaluator/reviser’ processes that will likely have an adverse effect on fluency, but it may boost accuracy, and also lexical and syntactic complexity.
To provide answers to these questions, we drew on the data obtained through a stimulated recall procedure, via qualitative analysis of pauses and revisions, to (1) explore what language processes (planner/proposer, translator and evaluator/revise) L2 writers invoked under different planning conditions; and (2) examine and compare the number and kinds of self-corrections that learners produced under four conditions of task performance (see Table 1). Finally, to examine the effects of planning on L2 writing production, the four groups were compared regarding the linguistic CAF measures.
Task conditions.
Notes. NP = no-planning group; PTP = pre-task planning group; OLP = on-line planning group; PTOLP = pre-task and on-line planning group.
IV Method
1 Participants
The participants of the study were 60 Iranian intermediate EFL learners selected from 120 intermediate learners in a private language center in Iran. All of the participants were junior high school students. Several measures were taken into account to ensure that participants were fairly similar in terms of their level of language proficiency, L1 writing expertise and their cognitive processing abilities. To do so, all 120 learners took the complete listening and grammar Oxford Placement Test 2 (OPT; Allan, 2004). Participants’ responses were scored on a scale of 200 points and, after calculating the results, 40 students who got below intermediate OPT scores were omitted. Moreover, because the previous studies that scrutinized writing processes found that there appears to be a writing expertise independent of L2 proficiency, affecting L2 writing (Cumming, 1989; Sasaki & Hirose, 1996), it stands to reason that students L1 writing should be checked for the sake of homogeneity at the outset of the study. Therefore, their L1 writing was assessed through a writing narration task in Persian. To do so, the remaining 80 students were given a picture story task adapted from Hill (1960) that had a sequence of strip pictures to narrate the story in Persian. The picture sets are about a man who bought a clock to be on time; however, the next morning when the alarm goes off, he is so annoyed that he throws the pillow at the clock and goes to sleep again. Afterward, their writing was holistically assessed and given numerical scores between zero to 20 by a Persian literature PhD student. This way, 20 students who got below 15 were removed from the study. The remaining participants were 16 males and 44 females with an average age of 15.5 (ranging from 14 to 17 years old). All of the participants signed the informed consent forms and were then randomly assigned to one of the four groups (see Table 1; n = 15). The results of one-way ANOVA on OPT scores revealed that there were no statistically significant differences between the four groups proficiency at the p < .05 level in either overall OPT scores [F (3, 56) = .25, p = .86] or listening scores [F(3, 56) = .86, p = .46].
Moreover, many studies have pointed out the pivotal role of working memory (WM) in L2 sub-skills and processing (Wen, Mota, & McNeill, 2015). WM generally refers to ‘the temporary maintenance, access and control of a limited number of pieces of linguistic information’ (Wen, 2016, p. 80; italics in original) in a limited capacity system while performing complex cognitive functions such as speaking and writing. To ensure that the participants in the four groups were similar in terms of their cognitive processing abilities (i.e. WM), the automated version of Persian reading span test (Shahnazari, 2013) that has 54 items was administered. A composite WM score was obtained by adding the processing and recall z-scores. Comparing the groups through ANOVA showed no significant difference at the p < .05 level between the four groups [F(3, 56) = .55, p = .64].
2 Task and procedure
In the present study, following Ellis and Yuan (2004), the participants narrated a story based on a set of six pictures from Heaton (1975). The story was about a boy who got off the bus and dropped one of the packages. A kind man found the package and chased the boy to return it, but the boy was frightened and started to run. When the man returned the package to him, he became embarrassed of his misunderstanding. The task instruction was given in Persian with a same prompt to start the task for all four groups: ‘This afternoon, John …’
In Group 1, the participants were required to carry out the narrative task under time pressured or no planning conditions (NP) (Control group). In the pilot study with no time limit, 10 intermediate students were given the story task. Based on the time noted, the fastest students completed the task in 15:30 minutes. Thus, this time was established as the time limit for pressured conditions. Moreover, they were asked to write at least 200 words in order to urge them to write quickly. Further, in order to prevent pre-task planning, they had to start the task immediately. In Group 2, they were asked to perform the narrative task under on-line planning conditions without time pressure (OLP). This group had no time limit, and they were not required to write a minimum of 200 words, but they had to start the task immediately. In Group 3, the narrative task was performed after 10 minutes of pre-planning under time-pressured conditions (PTP). Allocating 10 minutes for planning time was based on previous studies (Mehnert, 1998) that investigated the effects of different length of time for planning and showed this amount of time had significant effects on CAF of final products. After 10 minutes of planning, their notes were gathered, and they had to write 200 words in 15:30 minutes. Finally, in Group 4, they were asked to do 10 minutes pre-planning and then perform the narrative task under on-line planning conditions without time pressure (PTOLP). This group, as in the OLP group, was not required to write at least 200 words and their pre-task planning notes were gathered before the writing task.
To collect the data, the participants were asked to come individually to a quiet laboratory and narrate the story based on the task conditions they were put in. The sessions were videotaped with the camera mainly focused on their hands and pencil movement. To ensure confidentiality and reduce the anxiety of the participants, they wrote their names on the paper after completing the task.
3 Stimulated recall interview
Stimulated recall is generally referred to as providing prompts for the participants to recall thoughts they had while performing writing task with the help of video-recordings as the stimulus (Gass & Mackey, 2000). Immediately after the recording, their video-watching session was started to save the validity of the data obtained via this technique (Smagorinsky, 1994). In doing so, the participants were shown the recorded video of their writing session and were asked to say in their native language what they were thinking about and what was in their mind even if they thought those ideas were ridiculous. The videos were stopped at the moment of pauses (more than three seconds) or revising so that their thought about content and language as well as their revisions and so forth could be examined. The stimulated recall interview for each participant took about 25 minutes. The interviews were all audio recorded and subsequently transcribed for coding and analysis.
4 Transcription, coding, and analysis
To scrutinize what learners were actually doing at the time of their writing and their self-corrections, the data obtained from stimulated recall interviews were transcribed. The analysis was conducted with the software MAXQDA, Version 11 (Verbi, 1989–2016). MAXQDA is a professional software for qualitative data analysis that allows coding large amount of research materials such as interviews. These data can then be sorted and retrieved according to the categories established. The transcribed data were loaded onto the MAXQDA software and categorized into main writing processes of ‘planner/proposer’, ‘translator’ and ‘evaluator/reviser’ based on Hayes’s (2012) model of written language production. As well, 10% of the data were coded by the second researcher. The result showed the inter-coder reliability of 95% agreement.
Stimulated recall data were also used to classify different self-corrections produced by each participant. Kormos’s (1998) comprehensive taxonomy of self-repairs for L2 speech was adapted to categorize the revising behavior into four groups: (1) D-repair, (2) A- repair, (3) E-repair and (4) R- repair. For this purpose, the retrospection of the participants’ thoughts about their monitoring behavior was coded based on the taxonomy. One unique feature of this kind of analysis is the provision of the chance to investigate the instances of covert repairs. To ensure the inter-coder reliability, the second researcher coded 10% of the data. The result revealed inter-coder reliability of 93% agreement for inter-coding of four self-corrections.
5 Measurement of the CAF triad
There are many measures for estimating CAF. Based on the arguments of previous studies on T-units and C-units (Wolfe-Quintero, Inagaki, & Kim, 1998), C-units are preferable to T-units to be used in the CAF measures and also may lead to greater cross-linguistic comparability. To calculate the CAF triad, based on the work of previous researchers, the following measures were chosen:
Syntactic complexity:
The following two syntactic complexity measures were conducted via L2 Syntactic Complexity Analyzer software which is designed by Lu (2010) to automate syntactic complexity analysis of written language production. He recommended these two measures because they are progressing linearly across the three school levels and show statistically significant between-level differences:
Mean length of clause (MLC): The ratio of the number of words to the number of clauses in the participants’ production.
Coordinate phrases per clause (CP/C): The ratio of the number of coordinate phrases to the number of clauses in the participants’ narration.
Lexical complexity:
Using Lexical Complexity Analyzer (Lu, 2012) which is designed to automate lexical complexity analysis of English text, two measures of lexical variation were calculated. Among the host of measures of lexical variation, the following appears to have construct validity and are; therefore, recommended by his study:
Mean Segmental Type–Token Ratio (50) (MSTTR–50): The mean type–token ratio of all 50-word segments in the participants’ narration.
Number of Different Words (expected random 50) (NDW–ER50): The mean number of different words of 10 random 50-word samples in the participants’ narration.
Accuracy:
Building on the previous studies (Ellis & Yuan, 2004; Yuan & Ellis, 2003) accuracy was measured in two ways:
Error-free clauses: The ratio of the clauses that was not erroneous. All syntactic, morphological, and lexical errors were taken into consideration. Any error excludes a clause from being error-free; however, based on the guidelines of Wigglesworth and Storch (2009) we did not count errors of capitalization, spelling and punctuation.
Correct verb forms: The ratio of all verbs that were used correctly in terms of tense, aspect, modality, and subject-verb agreement.
Fluency:
Two measures were chosen to assess fluency:
Rate A (Syllable Per Minute): The total number of syllables produced divided by the total number of minutes a participant took to complete the task (Chenoweth & Hayes, 2001).
Rate B (Clause Length W/C): The total number of words produced divided by the total number of clauses. This measure is recommended as one of the most reliable and valid ratio measures of fluency by Wolfe-Quintero, et al. (1998).
To ensure the reliability of accuracy and fluency measures, the second researcher calculated five participants from each group. The inter-rater reliability coefficient was above .89 for all measures (with a mean of .91). Then the results were entered into SPSS version 20.0, and the descriptive statistics were checked for normality of distribution regarding skewness and kurtosis.
V Results
The results of the findings of the study are presented in terms of the three research questions investigated. It should be noted that since all the research questions are related, in order to give a more process-product integrated interpretation of the findings, whenever related the findings of the third research question are also mentioned.
1 Research question 1
To answer the first research question – that is, whether or not the provision of different planning time makes a difference in planning/proposing, translating and evaluating/revising behavior of L2 writers – the frequencies of the cognitive processes were calculated.
The mean and standard deviation of the coding of cognitive processes in the stimulated recall of the pauses (more than three seconds) and revising based on the writing model of Hayes (2012) are presented in Table 2. As Table 2 indicates, the OLP group had the most ‘proposing processes’ (M = 13.73, SD = 4.30). One-way ANOVA showed a significant group difference for ‘proposer/planner’ [F (3, 56) = 3.60, p = .019]. The post-hoc Tukey’s results revealed that the OLP group (M = 13.73, SD = 4.30), which was privileged with having ample time for completing the task, significantly proposed more content at the time of writing (p = .030) than the NP group (M = 9.60, SD = 3.31). Additionally, Table 2 also indicates the significant decrease of proposing between the OLP and PTP that had not access to ample time at the time of their writing. Nevertheless, the significant decrease of proposing between the PTP and OLP groups did not lead to the increase of lexical complexity, which was mostly influenced by the elaborateness of the pre-planned message, for the PTP group. Unexpectedly, the PTOLP group did not have as many proposing processes, even though they had ample time at their disposal. This may point to the fact that pre-task planning decreased the need for searching for content to form a preverbal message at the time of writing which led to significant increase of fluency (p = .007), as indicated by Tukey test, in the PTP group (M = 19.22, SD = 5.42) (see Table 4 below). Regarding translating processes, again the OLP group outperformed the other groups (M = 14.07, SD = 5.70) which indicates having enough time at the time of writing helps learners to search for better lexical and grammatical features as shown by one-way ANOVA [F(3, 56) = 4.69, p = .005]. Therefore, having time to focus on translation combined with revising processes may be the prime reason for the significant increase of accuracy (M = .69, SD = .15) at the expense of losing syntactic complexity (M = 6.31, SD = .74) in the OLP group. The results of the post-hoc Tukey between the pre-task groups (PTP and PTOLP) and OLP group showed that pre-task planning significantly decreased ‘translating’, which may be due to the fact that 10 minutes pre-planning helped the learners to search for appropriate lexical and grammatical features in advance, prepare syntactic frames, develop sentence fragments or even complete sentences (Ortega, 2005). This way freed up their limited attentional capacity at the time of writing and directed it toward syntactic operations that led to significant increase in both syntactic complexity measures between the OLP and pre-task groups (see Table 4 below). With the ‘revising’ process, again the same picture emerged [F(3, 56) = 5.17, p = .003]. The results of Tukey test showed that pre-planning significantly reduced the number of the revising processes in the PTP and PTOLP groups (M = 9.40, SD = 4.31; M = 9.27, SD = 4.20, respectively) and as such indicative of the fact that pre-planning significantly decreased accuracy between the pre-task groups (PTP and PTOLP) and the OLP group in both accuracy measures (see Table 4 below).
Descriptive statistics, results of ANOVA and post-hoc Tukey test for cognitive processes.
Notes. * The mean difference is significant at the 0.05 level. SD = Standard Deviation; NP = no-planning group; PTP = pre-task planning group; OLP = on-line planning group; PTOLP = pre-task and on-line planning group.
2 Research question 2
The second research question scrutinized the frequency and types of repairs that L2 writers drew on in four planning conditions. The means and standard deviations of the number of different types of self-repairs are shown in Table 3. Overall, language learners produced more E-repairs than the other repairs. Moreover, the OLP group produced the least A-repairs (M = 1.53, SD = 1.60) but the highest E-repairs (M = 8.00, SD = 3.70), D-repairs (M = 3.87, SD = 1.81) and R-repairs (M = 3.67, SD = 1.80). As one-way ANOVA indicated, the participants’ repairs were statistically significant between planning conditions for R-repairs [F(3, 56) = 13.18, p = .000], E-repairs [F(3, 56) = 7.05, p = .000], A-repairs [F(3, 56) = 4.52, p = .007] and D-repairs [F(3, 56) = 8.69, p = .000]. As it is shown in Table 3, the results of Tukey test revealed that the pre-task (PTP and PTOLP) significantly decreased the number of R-repairs and E-repairs. Although the PTOLP group had also enjoyed the availability of ample time for completing the task, they did not employ the same repair strategies as the OLP group and represented the behavior of the PTP group. Furthermore, as the Tukey test results showed, whereas the OLP group traded-off A-repairs for E-repairs by producing significantly fewer A-repairs (M = 1.53, SD = 1.60) (p = .013), the pre-task groups (PTP and PTOLP) significantly produced more A-repairs than the OLP group (p = .038; p = .018, respectively). These trade-off findings demonstrated that while learners in the OLP group directed their attention toward grammatical appropriacy and hence generated more accurate sentences, the pre-task groups (PTP and PTOLP) focused toward message conveyance and hence achieved more fluency and syntactic complexity (see Table 4).
Descriptive statistics, results of ANOVA and post-hoc Tukey test for different types of self-repairs.
Notes. A-repairs = Appropriacy repairs; D-repairs = Different information repairs; E-repairs = Error repairs; R-repairs = Rephrasing repairs; * The mean difference is significant at the 0.05 level. SD = Standard Deviation. NP = no-planning group; OLP = on-line planning group; PTOLP = pre-task and on-line planning group; PTP = pre-task planning group.
Descriptive statistics, results of ANOVA and post-hoc Tukey test for CAF measures.
Notes. Dashes show the Tukey procedure was not conducted. * The mean difference is significant at the 0.05 level. SD = Standard Deviation; NP = no-planning group; PTP = pre-task planning group; OLP = on-line planning group; PTOLP = pre-task and on-line planning group; MLC = Mean length of clause; CP/C = Coordinate phrases per clause; MSTTR–50 = Mean Segmental Type–Token Ratio (50); NDW–ER50 = Number of Different Words (expected random 50).
3 Research question 3
The third research question sought to examine the effect of different planning conditions on CAF of the final product. Table 4 presents the results for the CAF variables. With regard to complexity, the OLP group obtained the lowest mean length of clause (MLC) (M = 6.31, SD = .74). Thus, the OLP group wrote less complex sentences than the other groups, and as one-way ANOVA showed the difference was statistically significant [F(3, 56) = 4.16, p = .010]. In the case of the second variable of complexity, as it is shown in Table 4, the difference between planning conditions reached the level of significance [F(3, 56) = 4.57, p = .006]. The results of Tukey test showed the pre-task groups obtained (PTP and PTOLP) greater mean (M = .38, SD = .15; M = .40, SD = .26, respectively) than the OLP group and this difference reached the level of significance between the OLP and pre-task groups (p = .040, p = .025, respectively). Regarding lexical complexity, the result revealed that the PTOLP group produced the lowest number of different words (M = 32.62, SD = 2.11) and the one-way ANOVA indicated that the differences reached the level of significance at the p < .05 level [F (3, 56) = 3.87, p = .014]. This unpredicted result may be indicative of the fact that providing pre- and on-line planning time help the learners develop their ideas in a more complex way and propose a preverbal message, which is more demanding. However, because of the fact that the L2 learner’s lexicon is not as organized and elaborated as their L1, the learners are unable to find an appropriate lemma for the concept (Skehan, 2009; Temple, 2000). Overall, these results indicated that pre-task increased the level of syntactic complexity and on-line planning had a detrimental effect on sentence complexity.
Two variables were assessed to measure the accuracy of the participants’ narratives. In the case of error-free clauses, as the one-way ANOVA presented, the OLP group (M = .69, SD = .15) outperformed the other groups [F(3, 56) = 6.55, p = .001]. Post-hoc comparisons using the Tukey test indicated that the difference reached the level of significance in comparison with the NP (M = .46, SD = .24) (p = .020), PTP (M = .44, SD = .25) (p = .010) and PTOLP (M = .37, SD = .18) (p = .001). The same ranking for the correct verb forms can also be observed in Table 4 as calculated by the one-way ANOVA [F(3, 56) = 4.46, p = .007]. In this case, the post-hoc comparisons using the Tukey test indicated that the pre-task groups gained the lowest mean and the difference between the pre-task groups (PTP and PTOLP) and the OLP reached the level of significance. The PTOLP group, although it had ample time like the OLP group, did not gain high mean in accuracy that may point to the fact that pre-task directs attentional resources toward syntactic complexity rather than accuracy. Overall, these results showed that on-line planning resulted in more accuracy in the final product rather than pre-task planning.
Fluency was measured in two ways: syllable per minute and word per clauses. As Table 4 presents, the PTP group (M = 19.22, SD = 5.42) had the highest mean for Rate A, and this difference reached the level of significance at the p < .01 level [F (3, 56) = 4.54, p = .006]. Furthermore, as the Tukey test results demonstrated, the difference between the PTP and NP and also the PTP and OLP reached the level of significance that signified the effect of pre-task on fluency (p = .007, p = .031, respectively). With the Rate B, the OLP group gained the least level of fluency (M = 6.33, SD = .75) and this difference reached the level of significance as exhibited by the one-way ANOVA [F(3, 56) = 4.09, p = .011]. Post-hoc comparisons using the Tukey test disclosed that the mean score for the NP group (M = 8.01, SD = 1.41) was significantly different from the OLP (M = 6.33, SD = .75). Further, the pre-task groups (PTP and PTOLP) significantly differ from the OLP group that showed the detrimental effect of on-line planning on fluency (p = .032, p = .024, respectively). In summary, the results indicated that on-line planning had a detrimental effect on fluency but pre-task improved fluency.
VI Discussion
Using qualitative and quantitative analysis, this study contributes and expands our current knowledge regarding the effect of different planning conditions on EFL students L2 writing. One of the core findings pertinent to the first research question was that different planning conditions provoked different kinds of behavior and processes. Except for the PTOLP group that did not reveal the effect of planning conditions as anticipated and expected, the other groups, based on the instruction and time limitation, showed that on-line and pre-task planning were operationalized successfully. Therefore, the argument that ‘if the goal is to ensure that L2 writers produce their highest quality work, they need time for both types of planning’ (Ellis & Yuan, 2004, p. 81) is not a valid one. Consequently, it seems that providing both pre-planning and on-line planning at the same time will not bring about the most fruitful conditions for the narratives and may even act counter-productively. This might be due to the fact that the availability of ample time for pre- and on-line planning cannot be a panacea for lack of automaticity and smaller, less organized and elaborated lexicon that leads to serial performance rather than parallel composing processes in L2 (Kormos, 2006).
In terms of Hayes’s (2012) model of text processing, the availability of ample time for the OLP group led to more pauses to think about and ‘propose’ new content for their narratives. However, this involvement did not show itself in the lexical and syntactic complexity of the product. One possible explanation can be that the proposed complex preverbal message is not materialized in the ‘translator’ stage because of a lack of lexicon or appropriate structure (Temple, 2000). As has been pointed out by Skehan (2009, p. 204), a non-native mental lexicon is ‘smaller’, ‘incomplete’, ‘less organized’ and ‘less redundantly structured’, and as such ‘the preverbal message, however impressive, encounters considerable difficulty.’ In addition, another speculation maybe that the outputs of the ‘translator’ stage omitted before or after transcribing at the ‘evaluator’ stage because the translated linguistic strings failed to convey the writer’s intentions or had structural errors. Likewise, the OLP group revealed more instances of translation and revising processes by spending more time to find better lexical and grammatical correspondents and also revising and evaluating what they wrote. Regarding the high frequency of these two processes, it can be surmised that the accuracy and complexity of their final product will outperform the other groups. Possibly, if the translator has the capability of finding appropriate language to encode complex ideas which are dependent, at least in part, on the mental lexicon and proceduralized linguistic knowledge, the text would enjoy higher complexity and accuracy without trade-off. Therefore, it seems that learners in the OLP group trade off complexity for accuracy. Besides, their repair behavior shows that they direct their attention to form, which leads to the revision of the output of translator extensively before and after execution, which benefits their accuracy.
Considering the time pressure, the PTP group had restricted time for completing their writing and thus significantly produced fewer ‘proposing’ and ‘translating’ compared to the OLP group. This shows that 10 minutes of pre-planning facilitates the process of planning for the propositional content, retrieving the necessary lexicons, making the lexicons more salient, and activating required grammatical structures. Hence, pre-planning decreases the pressure on working memory and compensates for the lack of L2 proficiency at the time they commence their writing. This, in turn, resulted in better fluency and syntactic complexity (but not lexical complexity) in their narrative. However, the pressure of a lack of time induced less ‘revising’ behavior compared to the OLP group. This is in line with Skehan’s (1998) dual-mode system that states to access the rule-based system, more time and attention is required. Consequently, lack of monitoring and rule-based grammatical knowledge to revise the output of the translator might be the prime reason of decay in their final product accuracy.
The second research question addressed the number and types of self-corrections in different time restricted conditions. The number of E-repairs outnumbered all the other kinds of revising, which is in line with the previous findings (Ahmadian & Tavakoli, 2014; Mojavezi & Ahmadian, 2014) in L2 speech production. Moreover, the results revealed that the availability of pre-task planning reduced the number of E-repairs, R-repairs and D-repairs. One possible explanation would be that as they planned the prepositional and grammatical structure of their narratives, naturally the need for revision decreased and led to less ‘evaluating’ at the time of writing. Besides, the trade-off between A-repairs and E-repairs showed that the PTP and PTOLP groups had focused more on fluency and appropriacy of their message conveyance rather than accuracy.
The third research question concerned the effects of different planning conditions on CAF in written narratives. The findings of the present study offer further support to the main results of the previous studies regarding linguistic measures. The PTP group outperformed the NP group in terms of fluency (Rate A), which showed they had a higher production speed but no significant effect on accuracy and complexity when compared with the NP. These results lend support to previous studies regarding the positive effects of pre-task planning on fluency (Ellis & Yuan, 2004, 2005; Johnson et al., 2012; Kellogg, 1988, 1990; Ong & Zhang, 2010) and no effect for complexity (Johnson et al., 2012; Ong & Zhang, 2010) and accuracy (Ellis & Yuan, 2004, 2005; Yuan & Ellis, 2003). However, the outcomes run counter to previous studies that found significant effects for pre-planning on syntactic and lexical complexity (Ellis & Yuan, 2004, 2005; Yuan & Ellis, 2003) and Kroll’s (1990) study that found no effect for pre-planning. With regard to comparing the OLP group with the NP group, the results revealed that on-line planning had a significant effect on accuracy but reduced fluency, which lent support to previous studies that found positive effects of on-line planning for accuracy (Ahmadian & Tavakoli, 2014; Ellis & Yuan, 2004, 2005; Yuan & Ellis, 2003). However, we found no positive effects for lexical and syntactic complexity between the OLP and NP groups although previous studies found some positive effect for syntactic complexity (but not lexical complexity; Ahmadian & Tavakoli, 2014; Ellis & Yuan, 2005; Yuan & Ellis, 2003). The comparison of the PTP group vs. the OLP group vividly makes clear the positive effect of on-line planning for accuracy and the positive effect of pre-planning for increasing fluency and syntactic complexity.
The higher accuracy in on-line planning and higher fluency and syntactic complexity in pre-task planning could be explained by the way learners directed and activated their attentional resources and components of the composing processes. Firstly, the opportunity for pre-task planning decreases the load of ‘proposer’, ‘translator’ and ‘evaluator’ processes at the time of writing because they planned in advance the content and language of their narration that made some lemmas and structures more salient; this may contribute to fluency and syntactic complexity. In contrast, the provision of time for on-line planning will trigger the ‘proposer’, ‘translator’ and ‘evaluator’ processes, which may lead to an increase in accuracy but will arguably have an adverse effect on fluency. Therefore, it seems that the crucial factor for improving accuracy depends on the moment-by-moment opportunities in on-line planning that allow learners to reach their rule-based system to search in their linguistic repertoire to maximize accuracy while performing the task. Secondly, due to the limitation of attentional capacity, and lack of automaticity as proposed by Limited Attentional Capacity Model (Skehan, 1998) and also by Overload Hypothesis (Kellogg, 1990), there is possibly a trade-off between accuracy and complexity. In other words, enhancing accuracy came with the price of complexity and as such rejects the Robinson’s (2001) claims that there is no price for the processing-capacity limitation to have the joint enhancement of complexity and accuracy at the same time.
VII Conclusions
The purpose of this article is an attempt to demonstrate the actual processes that learners are using in different time planning conditions based on Hayes’s (2012) model of text production. In addition, different self-corrections in each planning condition were investigated to see the effect of planning on revising behavior of the learners. Moreover, we examined the relationship between ‘process’ and ‘product’ via quantitative CAF measures of the final products to corroborate the results. Further, we added a new experimental condition, PTOLP, in which the participants enjoyed the opportunity for both pre- and on-line planning. In sum, the findings reveal that both pre- and on-line planning improve performance: pre-planning reduces the pressure on ‘planner/proposer’, ‘translator’, and ‘evaluator/reviser’ processes at the time of writing that leads to more fluency and syntactic complexity; on-line planning pushes ‘translator’ and ‘evaluator/reviser’ processes to produce more accurate language. Time pressure, however, appears to work only with pre-planning and adding on-line planning to pre-planning is no better than pre-planning.
The findings of the present study have an important implication for language pedagogy. Given the fact that planning does significantly impact quantity and quality of L2 writing, we can convincingly argue for the beneficial effects of planning on different dimensions of L2 writing processes, and we are in a better position to manipulate the cognitive aspect of writing in different task conditions. However, despite the contributions of the present study, not all the interrelated variables can be investigated in a one shot study, and thus it opens up a range of possibilities for future research in L2 composition. First, it would be worthwhile to examine how these planning conditions along the continuum of proficiency and with different genres interact with writing processes and the quality of the final product. Second, the relationship of other elements in Hayes’s (2012) model of the text production process such as working memory, motivation, and task environment with subprocesses and the quality of the results have not been investigated so far. As astutely mentioned by Hayes and Nash (1996, p. 54), ‘research focusing on single writing process without sufficient attention to its context within the total writing activity may well yield misleading results’, and therefore deserves more attention.
Footnotes
Acknowledgements
We are sincerely grateful to the anonymous reviewers and editors of Language Teaching Research for their insightful comments and suggestions on the earlier drafts of this paper.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
