Save Your Strokes: Chinese Handwriting Practice Makes for Ineffective Use of Instructional Time in Second Language Classrooms

Abstract

Handwriting practice is the most time-consuming activity for learners of Chinese as a foreign language (CFL). CFL instructors report allocating at least one third of their course time to handwriting practice although it prevents students from engaging in meaningful communication, especially in the earliest stages of learning. Given the amount of time students spend in a college course is relatively fixed, the preregistered study presented herein examines the best use of students’ time when primary goals are word acquisition and communication. This work replicates a pilot study examining CFL word recognition in an online learning environment (ASSISTments) and the effects of supplemental handwriting practice. We examined word acquisition and recognition while manipulating condition (no-handwriting practice and with-handwriting practice), and posttest test point (1 [immediate], 2 [1 day delay], and 3 [1 week delay]). Two-way repeated measures analyses of variance revealed significant main effects for both condition and posttest test point in online and on-paper measures of word recognition and handwriting. Potential implications for CFL instruction and directions for future work are discussed.

Keywords

Chinese as a foreign language (CFL) instruction word acquisition word recognition handwriting randomized controlled trial

Background

As a logographic language, Chinese has numerous features that set it apart from Western languages. Chinese reading and writing systems use characters that are formed with radicals and strokes that are often only vaguely related to their meaning or pronunciation. As such, it can be difficult for learners to extract accurate meaning or pronunciation from characters alone. Characters are not equivalent to words in the Chinese language. Chinese words are formed by one or more characters, and most words are disyllabic in nature, or formed by two characters. Handwriting is the action of producing these characters or words by hand from memory.

Chinese is also a tonal language. Pinyin, the standard Romanized system for transliteration, is often used to help learners understand pronunciation and to connect a word’s sound to its meaning. In order to successfully read Chinese, students must learn to rapidly combine the three aspects of each word: visual (orthography), pronunciation (phonology), and meaning (semantics; Shen, 2004). In fact, research has shown that Chinese courses exhibit greater difficulty conforming to the American Council on the Teaching of Foreign Languages proficiency guidelines (ACTFL; 2012), and that learners taking Chinese as a foreign language (CFL) often find it difficult due to its orthographic nature, which increases the burden of retrieval and retention (Chinese Language Committee, 2009). However, it may be possible to address this added burden through pedagogical approaches that harness technology and minimize emphasis on handwriting, as we will show herein.

De Francis (1984) and Allen (2008) both isolated handwriting practice as the most time-consuming activity for CFL learners, noting that it significantly slows the learning process and prevents students from engaging in meaningful communication, especially in the earliest stages of learning. Hand-copying characters by following stroke order is one of the most commonly used practices for writing and word recognition. Many CFL teachers believe that this type of mechanical repetition helps students solidify word recognition skills. However, the need to write characters and words by hand has rapidly declined in nearly all Chinese social settings (Allen, 2008; Xie, 2014). Still, support for handwriting practice has been found in Chinese-language literacy contexts (as a native language) in both reading-development research (e.g., Chan, Ho, Tsang, Lee, & Chung, 2006; Huang & Hanley, 1994; Leck, Weekes, & Chen, 1995; Packard et al., 2006; Tan, Spinks, Eden, Perfetti, & Siok, 2005) and neuroimaging studies of normal adult subjects (e.g., Siok, Perfetti, Jin, & Tan, 2004). Tan et al. (2005) indicated that for Chinese children, the ability to read Chinese words is more strongly related to writing skills than phonological awareness, when compared with the results of children learning alphabetic native languages. Tan, Xu, Chang, and Siok (2013) also indicated that primary-school children’s reading development may be negatively affected by transitioning handwriting practice to Pinyin or typed practice. Such theories have enhanced CFL teachers’ beliefs that handwriting should be required to help reading development. But these hypotheses were based on native-language learners rather than second-language learners; should the effects of writing share the same mechanism?

Although different CFL programs maintain different requirements and pedagogical goals for learning (Everson, 2011), several researchers have suggested a close relationship between Chinese word recognition and handwriting (e.g., Cao et al., 2012; Cao et al., 2013; Guan, Liu, Chan, Ye, & Perfetti, 2011; Ke, 1998). For instance, Guan et al. (2011) compared the effects of three types of online writing tutors for Chinese: handwriting, reading only, and Pinyin typing. They suggested that both writing skill and phonological awareness, as proposed by Tan et al. (2005), may play roles in CFL reading, noting practical implications for the integration of handwriting and Pinyin typing in promoting reading Chinese in second-language contexts. Xu, Chang, Zhang, and Perfetti (2013) compared the effectiveness of different approaches on CFL learners’ orthographic knowledge of Chinese and found that writing and animation helped improve form recognition, that reading led to stronger recall of meaning and sound, and that writing promoted character reproduction from memory. However, their results were based on a sample of foreign-language learners with orthographic knowledge and general understanding of stroke-order rules—students who were regularly assigned the task of writing words from memory on homework and quizzes. Unfortunately, when foreign-language learners begin learning Chinese, they lack this knowledge and understanding. Hsiung, Chang, Chen, and Sung (2017) also found that writing exercises helped students memorize the orthography and output of Chinese characters. However, their experiment was conducted on CFL learners who were studying abroad in Taiwan; learners who had not only mastered orthographic knowledge but who had spent much of their study time focused on Chinese language learning. This suggests that their results were not particularly generalizable.

Guan et al.’s (2011) hypothesis that writing helps reading in Chinese might be true for native-language learners; there is no doubt that native language teachers and learners alike should continue to value the tradition and art of handwriting. However, as researchers and educators have come to realize the unbalanced input and output of handwriting at novice levels, the principle of “listening and speaking first” has gained traction, proposing nonsynchronized character/word recognition and production (Cui, 1999; Jiang, 2007). This principle makes sense for CFL teachers, who often share the primary concerns of (1) finding the most efficient ways to reach high-proficiency levels and (2) helping lower-level students communicate as quickly as possible in order to maintain interest. Real-world communication typically requires skill in speaking and reading. As such, it makes sense to reduce the amount of time that lower level students are expected to spend on handwriting practice.

To better understand the current state of novice- and intermediate-level CFL teaching, we surveyed 27 secondary- and college-level instructors in the United States. Responses were representative of 17 college instructors and 10 secondary-level instructors at 26 schools. Of those polled, 70% of instructors required their students to be able to write all learned Chinese words by hand. This statistic increased to 88% when only considering college level instructors. Many CFL instructors reported believing that handwriting is the most reliable approach for novel word acquisition, recognition, and retention, and feeling that it should be undertaken from the start. When asked, “When assigning homework to students, what percentage of time do you expect students to spend on writing by hand? (Handwriting-required practices include copying characters/words, essay writing, answering questions based on text, translating English to Chinese, etc.),” instructors at the college level who required students to be able to write all words by hand expected students to spend an average of 44% of their homework time on handwriting. Even instructors without strict handwriting requirements expected students to spend an average of 30% of their homework time on new word memorization techniques. Unfortunately, these instructors are allocating at least one third of their course time on a form of practice that may not actually be helping their students learn and retain novel words.

Nonnative beginners typically use few orthographic strategies in their approaches to character learning, while more advanced students tend to rely more heavily on orthographic knowledge (Shen, 2005). The time commitment required to build such a knowledge base adds significantly to a student’s workload (Shen, 2005). As such, we believe that the most valuable focus for CFL instructors is not whether handwriting should be included, but rather, how to isolate the most efficient strategies for teaching CFL and understanding how those strategies may differ for novices.

Before the influx of technology infused learning, “reading before writing” was a rather contrived concept. Warschauer and Healey (1998) stated that the development of information technology has provided foreign-language instructors and learners with new possibilities. Computerized Chinese instruction began more than a decade ago and instructors frequently turn to computer-based tools in the digital age (Xie, 2014). Language learning principles have been developed to suit the context of computerized language instruction and researchers believe that computerization has made proficiency-based Chinese instruction more efficient and more aligned to the guidelines of foreign-language learning in the 21st century (Xie, 2014). Zhu, Liu, Ding, and Peng (2009) noted that Pinyin typewriting can be beneficial for both the phonological and orthographic processing of Chinese characters. Zhu, Shum, Tse, and Liu (2016) even suggested that CFL beginners should rely on the Pinyin-input method found in word processors (e.g., Microsoft Word) rather than practicing conventional handwriting techniques, as the former medium led to better performance in essay-writing tasks. Whereas students would traditionally combine orthography, phonology, and semantics (Shen, 2004), Zhu et al.’s (2016) work suggested that reducing some of this burden through the affordances of technology led to more efficient and effective outcomes (Appendix A explains how the Pinyin input method can be used to type in Chinese). In the present preregistered study, online practices were used to teach novel target words by providing one or two formats of each word (orthography, phonology, or semantics) and asking students to supply the third. For instance, when prompted to supply the orthographic aspect of a word, students entered their response using Pinyin rather than hand-copying characters. Similarly, when prompted to supply the phonological aspect of a word, they entered Pinyin with tone marks.

The present study specifically considered recognition of disyllabic, or two-character, words. Word recognition is the ability of a reader to recognize written words correctly and effortlessly. Everson (1998) defined Chinese word recognition as “deriving both the phonetic codes (or pronunciation) as well as lexical meaning from printed Chinese characters.” In the present study, we measured “isolated word recognition,” or a reader’s ability to recognize words individually without contextual help. It should be noted that in Chinese, word recognition is different from character recognition. From a psychological perspective, two-character Chinese words require that readers assemble characters, increasing processing complexity above that necessary for single characters (Tan & Perfetti, 1999). Previous work has suggested that rapid word recognition is the main component of fluent reading. When considering communication, words (rather than characters) are the basic unit of a sentence; the vocabulary list of most CFL textbooks are built on the foundation of words.

A pilot version of this work (Lu, Ostrow, & Heffernan, 2019) revealed that handwriting practice was an ineffective use of instruction time for CFL learners; participants scored significantly lower on online portions of word recognition posttests after spending 30% of their practice time on character/word hand-copying exercises. These significant differences were apparent immediately following word acquisition practice sessions and on repeated testing 1 day and even 1 week later. Worse, results of on-paper posttests did not reveal significant gains for those who had spent time focused on hand copying. As such, pilot results lent credibility to the proposition that CFL instructors should not sacrifice novice learning time to handwriting instruction when word acquisition and communication are of primary concern.

The present study serves as a replication of our pilot work. This study was accepted as a preregistered publication in AERA Open prior to data collection and analysis; all results and claims made thereabout came subsequent to the paper’s initial conditional acceptance with further review and minor revisions. The study considered word recognition while manipulating two independent variables: condition and posttest test point. Condition included two levels: No-Handwriting (NH) practice and With-Handwriting (WH) practice. Test point indicated posttest time point and included three levels: 1 (immediate), 2 (1 day delay), and 3 (1 week delay). Given that the amount of time students spend in a college course is relatively fixed, we sought to understand the best use of students’ time when primary goals are word acquisition and communication. We sought to confirm that students perform better on word-recognition tasks when conventional amounts of handwriting practice are eliminated to make more time for online word-acquisition practice. Thus, we hypothesized that, as observed in our pilot work, students would score significantly higher on word recognition posttests when they were not subjected to hand-copying exercises.

Method

Participants

Participants included 60 students enrolled in an intermediate Chinese class in the fall 2018 semester at a university in the northeastern United States. The average age of participants was 19.45 years (SD = 1.25 years), with a distribution of 25 males and 35 females, 13 freshmen, 29 sophomores, 9 juniors, 8 seniors, and 1 graduate student. Students participated in the course for credit and were enrolled via a placement exam or following experience in a preliminary Chinese course after displaying “Novice-High” ACTFL Levels of language proficiency. All participants had prior knowledge of Pinyin and were regularly required to be able to read and type all new words, but were not regularly required to hand-copy words from memory on homework or quizzes.

On the first day of the experiment, 52 of the 60 sampled students participated in the pretest, word practice session, and Posttest 1, while eight students missed class. On the second day, 51 of the remaining 52 subjects participated in Posttest 2. On the eighth day, 49 of the remaining 51 participants subjects in Posttest 3. Analyses were conducted using treated rather than intent-to-treat methodologies, therefore taking these smaller samples into account.

Setting

This study was conducted using ASSISTments, an online learning platform that provides students with immediate feedback and teachers and researchers with robust student-level data (Heffernan & Heffernan, 2014; Ostrow & Heffernan, 2019). While the system is more commonly used for mathematics education, content from other domains including statistics, physics, chemistry, electronics, biology, history, English, and now Chinese has been created and researched using its research infrastructure, E-TRIALS. Previous research on this platform has explored the intricacies of CFL, including approaches to word recognition (Lu, Ostrow, & Heffernan, 2016; Lu, Ostrow & Heffernan, 2018) and the benefits of various feedback mediums (Lu, Xiong, & Heffernan, 2017). All participants took daily in-class quizzes using their laptops and had completed at least two in-class quizzes on ASSISTments before the study began, establishing class-wide familiarity with the system. The pretest, word practice session, and posttests were delivered using ASSISTments during class time. Supplemental handwriting practice was conducted on paper. The pilot version of this work followed the same structure.

Materials

Two sets of five Chinese words were curated by the first author of this work. These sets, “Word Set X” (with words 1–5) and “Word Set Y” (with words 6–10) are shown in Table 1. All 10 target words were two-character words selected from the Second-Class Vocabulary listed in the Outline of Graded Vocabulary for HSK (HSK Department, Chinese Government, 1992). To increase the likelihood that these words were novel to study participants, the first author verified that none of the words could be found in preliminary course textbooks.

Table 1

Target Words, Pinyin Pronunciations, and English Meanings

Target Word	Pinyin	English Meaning
Word Set X
稳定	wěndìng	stable
仔细	zǐxì	carefully; attentive
标准	biāozhǔn	standard; criterion
原因	yuányīn	reason; cause
其实	qíshí	actually; in fact; as a matter of fact
Word Set Y
距离	jùlí	distance; range
后悔	hòuhuǐ	regret
熟悉	shúxī	be familiar with; know well
主动	zhǔdòng	initiative
于是	yúshì	hence; as a result; and then

For the word practice session, the first author developed two 85-second introductory videos for Word Set X and Word Set Y, available for reference at Lu (2018). These videos introduced the new words by reading each word aloud twice in Chinese, reading its English meaning aloud twice, and then reading the word twice aloud again in Chinese. Chinese characters, along with their Pinyin and English meanings, were shown on screen while each word was read aloud. To create the practice session, the first author then developed seven question types for each word, as shown in Table 2. After watching the appropriate introductory video, participants cycled through these seven question types two to three times during a 13-minute word-practice session. If randomly assigned to practice handwriting for a particular word set, students had to hand-copy each word three times using proper stroke order on paper at the beginning of the practice round. A sample of the hand-copy practice sheet is provided in Figure 1.

Table 2

Question, Answer, and Hint Types and Correct Responses for the Word 稳定

Question Type	Answer Type	Hint Type	Correct Response
1. Choose the correct meaning of the word.	Multiple choice	wěndìng	稳定 → stable
2. Choose the correct Pinyin of the word.	Multiple choice	stable	稳定 → wen3ding4
3. Listen and choose the word that matches the sound.^a	Multiple choice	stable	wen3ding4 → 稳定
4. Type out the English meaning of the word.	Entry	wěndìng	稳定 → stable
5. Choose the word that matches the meaning.	Multiple choice	wěndìng	stable → 稳定
6. Type out Pinyin of the Chinese word. Please type out tones as well, and use 1 for first tone, 2, for second tone, 3 for third tone, 4 for fourth tone, and 0 for neutral tone. For example, if you see 你好, you should type out “ni3hao3.”	Entry	stable	稳定 → wen3ding4
7. Type out words based on meaning.	Entry	wěndìng	stable → 稳定

Audio was provided through a YouTube video showing the question, with the word read aloud twice. Students could replay the video if needed.

Figure 1.

Hand-copy practice prompt and worksheet example.

The pretest and the online portion of all posttests contained the same 10 problems (one for each word) presented randomly to each participant at each test point (see Appendix B). Each problem contained two subtasks focused on word recognition, requiring participants to enter the meaning or pronunciation of each word using Pinyin after viewing the Chinese characters for the word. All test points included online word recognition tasks and on-paper handwriting tasks. Participants were expected to answer online portions using ASSISTments (following the format of the pretest) and produce characters from memory on a provided paper worksheet, as prompted by the word’s Pinyin and English meaning. The order of the 10 target words on the handwriting worksheet was randomized for each test point. A posttest worksheet sample is included in Appendix C.

Design

The present study comprises two conditions that all participants experienced using a crossover design: NH and WH. The only difference between these conditions was that students in the WH condition began each round of practice with a handwriting exercise on paper that took approximately 30% of their practice time (see Figure 1), while those in the NH condition spent all of their practice time in ASSISTments cycling through word acquisition practices (see Table 2). In a counterbalanced fashion, participants received Word Set X or Word Set Y in their randomly assigned condition, followed by an immediate posttest, before moving to the remaining set/condition, as shown in Figure 2. This approach was used to control for both word and condition order. Minimal washout was considered based on word novelty.

Figure 2.

Experimental design.

Within the NH condition, participants practiced the 10 target words using ASSISTments. Thirteen-minute practice sessions for each word set were designed using two rounds: the first round contained question types one to four, while the second contained question types five to seven (see Table 2). Rounds were assigned randomly to balance the distribution of question types. If participants were able to finish the second round within the allotted time, they were offered another iteration of the first and second rounds. To help participants become familiar with the novel words at the start of the practice session, question types one and two were presented in a linear fashion for each word. Subsequent rounds featured fully randomized question types. While practicing within ASSISTments, participants were able to access hints (orthographic, phonological, or semantic, as shown in Table 2) and could ultimately access the correct answer in order to move on to the next question. Hints always provided the missing facet in the word acquisition task; for example, if participants were given a sound and asked to choose the orthography, the hint would provide the meaning of the word. All materials are available at Lu (2018) for further reference.

Within the WH condition, students began each 13-minute practice session by completing a hand-copying worksheet (see Figure 1). A prompt in ASSISTments (see Figure 1) directed students to complete and hand in this worksheet before moving on to the online content (as described for the NH condition). As such, approximately one third of learning time in the WH condition was devoted to handwriting practice in order to allow students to practice hand-copying characters, mimicking the course structure followed by the majority of CFL instructors.

Procedure

Blocking Participants and Randomization

To increase the chance that participants were randomly assigned to groups with equal variance, the first author blocked and randomized students based on their prior knowledge, producing groups that were sufficiently homogenous. The first author used students’ average grades from daily class quizzes prior to participation in the study as a measure of prior knowledge. To block and randomize participants, the first author rank ordered these scores, paired the two highest performing participants and all subsequent pairs, and randomly assigned each pair member to a condition progression (NH → WH or WH → NH). The same method was then used to divide condition progression groups into four subgroups in order to counterbalance the influence of word set order (as shown in Figure 2).

Experimental Process

Prior to the experiment, the first author explained the procedure to study participants and reminded them to take advantage of each question type and to use the hint function as necessary to keep moving forward. Participants then took the pretest to assess initial knowledge of the 10 target words. Participants then began the first round of their randomly assigned practice content. They were given 13 minutes for the first practice session, which began with an 85-second video introducing their assigned word set. After watching the video, participants were expected to work through the practice questions at their own pace. If assigned to the WH condition, participants were expected to first submit a handwriting practice worksheet before moving on to their online content. A posttest (1a) on the assigned word set was provided immediately following the 13-minute practice session. This process was then repeated for a second session of practice in which participants experienced the alternate word set and condition. They again had 13 minutes, including an introductory video, to proceed through handwriting (if applicable) and online practice as assigned. A posttest (1b) on the new word set was provided immediately following this second practice session. Posttest 2 covering both word sets was given on day three of the experiment during a regularly scheduled course meeting. Posttest 3 was then given during scheduled course time 8 days later. The full experimental design is depicted in Figure 2.

Training and Delivery

The first author was in charge of enacting the procedure and monitored the experiment together with two teaching assistants. To ensure that the experiment ran smoothly, the first author, who also conducted the pilot study, trained two teaching assistants on the procedure using the flowchart shown in Figure 2. All 52 study participants who attended class on day one took the pretest, participated in the word practice session, and took the immediate posttests (1a and 1b). A stopwatch was used to ensure that all participants received the same practice time (13 minutes) in each session, but posttests were self-paced and participants could take as much time as they needed.

Scoring Protocol

All online pretest and posttest scores were sourced from log data collected by ASSISTments. Anonymized data are available at Lu (2018) for further reference. Participants’ average posttest scores were calculated by adding the number of questions answered correctly and dividing that sum by the total number of questions on each test. Partial credit scores were generated for answers with otherwise accurate Pinyin using the wrong tones. The first author scored the handwriting portions of each posttest (blindly) by calculating the number of characters written accurately divided by the total number of characters on each test.

Results

Our null hypothesis was that there would be no difference between NH and WH conditions on either online or on-paper portions of word recognition and handwriting posttests. Furthermore, we predicted that scores would decrease with each test point, as observed in our pilot work, denoting forgetting. Tables 3 and 4 present mean scores and standard deviations for all test points by condition, with Table 4 providing practice session metrics and a comparison to our pilot work.

Table 3

Sample Sizes, Means (Standard Deviations), and Pairwise Comparisons for Pretest and Posttest Test Points by Condition for Online and On-Paper Scores

	Online						On-Paper
	NH		WH		t	p	NH		WH		t	p
Test Point	N	M (SD)	N	M (SD)	t	p	N	M (SD)	N	M (SD)	t	p
Pretest	52	0.05 (0.09)	52	0.06 (0.12)	−1.00	.322	52	0.04 (0.08)	52	0.05 (0.12)	−1.42	.162
Posttest 1	52	0.67 (0.25)	52	0.56 (0.28)	3.05	.004**	52	0.33 (0.26)	52	0.45 (0.34)	−2.92	.005**
Posttest 2	51	0.43 (0.29)	51	0.39 (0.29)	1.95	.058	44	0.29 (0.26)	44	0.35 (0.29)	−2.13	.039*
Posttest 3	49	0.32 (0.27)	49	0.29 (0.26)	0.87	.391	41	0.28 (0.24)	41	0.31 (0.28)	−0.68	.503

Note. NH = no handwriting; WH = with-handwriting.

p < .05. **p < .01.

Table 4

Mean (Standard Deviation) for Pretest, Posttest Test Points, and Practice Session Metrics Across Pilot and Present Studies by Condition for Online and On-Paper Scores

			Pilot study	Present study
Pretest	Online	NH	0.04 (0.07)	0.05 (0.09)
	Online	WH	0.05 (0.08)	0.06 (0.12)
	On paper	NH	N/A	0.04 (0.08)
	On paper	WH		0.05 (0.12)
Posttest 1	Online	NH	0.76 (0.22)	0.67 (0.25)
	Online	WH	0.64 (0.27)	0.56 (0.28)
	On paper	NH	0.09 (0.10)	0.33 (0.26)
	On paper	WH	0.13 (0.13)	0.45 (0.34)
Posttest 2	Online	NH	0.50 (0.28)	0.43 (0.29)
	Online	WH	0.41 (0.30)	0.39 (0.29)
	On paper	NH	0.08 (0.09)	0.29 (0.26)
	On paper	WH	0.10 (0.12)	0.35 (0.29)
Posttest 3	Online	NH	0.39 (0.27)	0.32 (0.27)
	Online	WH	0.32 (0.24)	0.29 (0.26)
	On paper	NH	0.11 (0.10)	0.28 (0.24)
	On paper	WH	0.11 (0.12)	0.31 (0.28)
Problems seen			37.46 (12.12)	31.86 (17.58)
Attempts			1.27 (0.22)	1.29 (0.28)
Hints			0.15 (0.11)	0.14 (0.11)
Time per online problem (sec)			16.92 (62.21)	27.84 (35.01)
Time per online problem (sec)			Mdn = 7.11	Mdn = 9.62
Time per hand-copy practice (sec)			236.98 (95.82)	270.54 (126.62)
Time per hand-copy practice (sec)			30.38% of overall practice time	34.69% of overall practice time

Note. NH = no handwriting; WH = with handwriting.

As anticipated, most participants received null scores on both online (M = 0.05, SD = 0.11) and on-paper (M = 0.04, SD = 0.10) portions of the pretest. As we counterbalanced practice orders and all students participated in both conditions, we performed two paired t-tests to determine whether there were any within group differences at pretest by condition. No significant differences were observed within groups in the online portions of the pretest, with students performing approximately the same in the NH condition (M = 0.05, SD = 0.09) and the WH Condition (M = 0.06, SD = 0.12), t(51) = −1.0, p = .322. Similarly, no significant differences were observed in the on-paper portion of the pretest, with students performing approximately the same in the NH condition (M = 0.04, SD = 0.08) and the WH Condition (M = 0.05, SD = 0.12), t(51) = −1.42, p = .162. Thus, we concluded that these groups were largely homogeneous and that we could proceed with planned posttest analyses.

We then conducted a two-way repeated measures analysis of variance to examine the main effects and interactions of condition (NH, WH) and posttest test point (1, 2, 3) on the online portions of posttest scores. There was a significant main effect of condition, F(1, 47) = 6.49, p = .014, with a moderate effect size ( $η_{p}^{2}$ = 0.12) indicating both a statistically and practically significant difference between online word acquisition practice (M = 0.47, SE = 0.04) and with-handwriting practice (M = 0.41, SE = 0.04). We explored this main effect further using post hoc paired t tests. Figure 3a graphs the scores of each condition by test point. As hypothesized, students exhibited a clear downward trend over time, representing forgetting. This graph also depicts relatively stable reliable differences between conditions. There was a significant difference observed between conditions on Posttest 1, with students in the NH condition (M = 0.67, SD = 0.25) outperforming those in the WH condition (M = 0.56, SD = 0.28), t(51) = 3.05, p = .004, 95% CI [0.04, 0.18], Cohen’s d = 0.41. There was also a marginally significant difference observed between conditions on Posttest 2, with students in the NH condition (M = 0.43, SD = 0.29) slightly outperforming those in the WH condition (M = 0.39, SD = 0.29), t(50) = 1.95, p = .058, 95% CI [−0.001, 0.09], Cohen’s d = 0.14. However, significant differences were not observed between conditions by Posttest 3, with students scoring approximately the same in both the NH condition (M = 0.32, SD = 0.27) and the WH condition (M = 0.29, SD = 0.26), t(48) = 0.87, p = .391. Still, these results suggest that the NH condition produced better word acquisition on average than the WH condition, especially in early measures of learning, reaffirming that time spent on handwriting instruction in CFL classes may be misplaced.

Figure 3.

Percent correct on pretest and each posttest test point by condition for (a) online word recognition tasks (left) and (b) on-paper handwriting tasks (right).

There was also a significant main effect of posttest test point, F(2, 94) = 80.30, p < .001, with an impressive effect size ( $η_{p}^{2}$ = 0.63), indicating we could reject the null hypothesis that there was no change across test points. Results revealed evidence of learning immediately following practice sessions (Posttest 1) and were then suggestive of forgetting, as anticipated. Pairwise comparisons revealed significant differences between Posttest 1 (M = 0.61, SE = 0.03) and Posttest 2 (M = 0.40, SE = 0.04), p < .001, Posttest 1 and 3 (M = 0.31, SE = 0.04), p < .001, and Posttest 2 and 3, p < .001.

A second two-way repeated-measures analysis of variance was conducted to examine the main effects and interactions of condition (NH, WH) and posttest test point (1, 2, 3) with regard to students’ scores on the on-paper portions of each posttest. There was a marginally significant main effect of condition, F(1, 36) = 3.91, p = .056, $η_{p}^{2}$ = 0.10, with students in the WH condition (M = 0.39, SE = 0.05) outperforming those in the NH condition (M = 0.31, SE = 0.04). We explored this main effect further using post hoc paired t-tests and observed a significant difference between conditions on Posttest 1 and Posttest 2. At Posttest 1, students in the WH condition (M = 0.45, SD = 0.34) outperformed those in the NH condition (M = 0.33, SD = 0.26), t(51) = −2.92, p = .005, 95% CI [−0.20, −0.04], Cohen’s d = 0.40. This difference remained at Posttest 2, with students in the WH condition (M = 0.35, SD = 0.29) outperforming those in the NH condition (M = 0.29, SD = 0.26), t(43) = −2.13, p = .039, 95% CI [−0.12, 0.003], Cohen’s d = 0.22. However, significant differences were no longer observed between conditions by Posttest 3, with students in the WH condition (M = 0.31, SD = 0.28) performing approximately the same as those in the NH condition (M = 0.28, SD = 0.24), t(40) = −0.68, p = .503. Figure 3b graphs scores by condition and test point. These results suggest that while handwriting led to better hand-copying skill initially, differences were nonexistent at 1 week, suggesting no lasting impact of limited handwriting practice for CFL learners.

On-paper portions of posttests also exhibited a significant main effect of test point, F(2, 72) = 15.33, p < .001, with a large effect size ( $η_{p}^{2}$ = 0.30), indicating we could reject the null hypothesis that there were no changes across test points. Pairwise comparisons revealed significant differences between Posttest 1 (M = 0.40, SE = 0.05) and Posttest 2 (M = 0.33, SE = 0.04), p < .001, and between Posttest 1 and Posttest 3 (M = 0.30, SE = 0.04), p < .001. There was no significant difference observed between Posttest 2 and Posttest 3, p > .05.

We also examined word practice session data from ASSISTments to help inform our analysis. Data revealed that each participant saw an average of 31.86 problems (SD = 17.58), made an average of 1.29 attempts per problem (SD = 0.28), and used an average of 0.14 hints per problem (SD = 0.11). Average time spent per problem was 27.84 seconds (SD = 35.01 seconds), while median time spent was 9.62 seconds (SD = 7.85 seconds). While assigned to the WH condition, all participants completed the first round of handwriting practice within the allotted time. Twenty-one participants were then able to start a second round of handwriting practice, and three participants were able to start a third round. When considering completed handwriting problems, participants spent an average of 4.51 minutes (270.54 seconds, SD = 126.62 seconds) on handwriting practice or 34.69% of overall practice time.

Discussion

We hypothesized that students would perform better on word recognition tasks if they were able to spend more of their practice time on word acquisition activities rather than allocating approximately 30% of their practice time to handwriting practice. Findings suggested a significant difference between practice with and without handwriting when considering immediate word recognition outcomes, favoring the removal of handwriting exercises (p = .004), with a marginally significant lasting impact 3 days later (p = .058). Although these results were not as robust as those observed in our pilot study, they trended in the same direction and reaffirmed that replacing handwriting with additional word acquisition training may lead to stronger word recognition in the short term. This gain was lost 1 week later, denoting a natural forgetting curve. These findings suggest that handwriting is an ineffective use of practice time for CFL learners who wish to focus primarily on word acquisition and communication; students scored significantly lower on word recognition tasks after spending 34.69% of their practice time on handwriting exercises, mimicking the structure of a traditional CFL course. This aligned with the findings of our pilot work in which students scored significantly lower on word recognition tasks after spending 30.38% of their practice time on handwriting exercises.

When considering students’ performance on handwriting tasks, findings suggested a significant difference between practice with and without handwriting when considering immediate handwriting outcomes, favoring the inclusion of handwriting practice exercises (p = .005), with a marginally significant lasting impact 3 days later (p = .039). This gain was lost 1 week later, again denoting a natural forgetting curve. These findings differed from those observed in our pilot work (interestingly, our pilot work did not reveal significant differences on handwriting outcomes), but they did not deviate from our expectations. The format of the handwriting portion of each posttest asked students to write a word’s characters as prompted by its Pinyin and meaning. The purpose of this exercise was to measure how well students could write the target words from memory, not to measure word recognition. It is logical that students who practiced handwriting were more likely to successfully write each word. However, it is worth reminding readers that proficiency in handwriting is not necessarily helpful for beginning CFL learners hoping to communicate and efficiently build their vocabulary. Thus, we prioritized gains in word recognition in the present study as suggesting greater promise for CFL learners. Plus, observed gains in both word recognition and handwriting were lost after 1 week, suggesting that spending 30% of course time on handwriting practice may be ineffective, but that simply filling that time with additional word acquisition practice may not be a viable solution to support long-term retention.

Still, the results of both our pilot work and the present study indicated that spending 30% to 35% of practice time on handwriting appears to hinder students’ immediate word recognition. While these results do not suggest that handwriting should be removed from CFL curricula all together, they speak to the efficient use of students’ learning time. As the amount of time students spend in a college course is relatively fixed, our results support that the best use of students’ time, if their primary goals are communication and vocabulary growth, is on word acquisition tasks rather than handwriting practice.

Our pilot work and the present study both took place at the same university in two class sessions of the same course over a 2-year period. Both studies consisted of the same 10 target words. The two classes requirements for handwriting were the same, and neither class required students to be able to write characters from memory in everyday homework or exams. Both classes frequently utilized computers, were familiar with typing characters and words using Pinyin, and had previously been exposed to the learning platform used for study implementation, ASSISTments. Pretests for both the pilot study and the present work confirmed that students were not familiar with the target words. Learning curves, forgetting, and observed differences between conditions and across test points were largely comparable. Learning habits within practice sessions were also largely comparable, as shown in Table 4, but participants in the present study spent considerably more time per problem (M = 27.84 seconds, SD = 35.01) than those in the pilot study (M = 16.92 seconds, SD = 62.21). This would have led to fewer repetitions of target words experienced within allotted practice time in the present study, which may explain deflated scores on Posttest 2 in comparison with our pilot work.

One major limitation of this work was our measure of “long-term” results. It is possible that word recognition may change over longer periods of time and future work should explore longer term outcomes by extending the duration of practice and by considering posttests with greater delay. It is also important to note that handwriting practice is not the only activity that may facilitate performance on handwriting posttests; word familiarity may also influence performance. Participants who did not practice handwriting may have been able to write out the target words based on familiarity from word acquisition practice. It is easy to imagine that characters with fewer strokes would be easier to remember and write out. Unfortunately, the present study did not control for stroke number across target words or consider confounding character difficulty.

Furthermore, findings from the present study suggested that students scored significantly higher on word recognition posttests when they were not subjected to handwriting practices. However, both our pilot work and the present study took place in a Chinese course that had adopted a computer-assisted learning approach, one that did not focus on strengthening handwriting practices in its everyday structure. It is possible that this approach may have weakened students’ performance on handwriting tasks and iterations of this work should be considered in more traditional CFL settings in which 30% of course time is regularly spent on handwriting practice. Future work could also extend beyond word recognition to examine the efficacy of handwriting on reading outcomes, in order to determine the optimal length of handwriting practice required to balance reading gains and resulting handwriting skill.

The present study improved on our pilot work by pretesting students’ handwriting ability, thereby enhancing the validity of the experiment. It also raised supplemental questions regarding the importance of “efficiency” as a criterion in CFL learning. While many studies have supported handwriting in Chinese instruction (e.g., Hsiung et al., 2017), most have failed to consider the efficient use of students’ time and some have even failed to rule out learning time as a major confounding factor. As such, the present work fills a critical gap in CFL literature while presenting results that challenge the standard of practice in CFL instruction. Essentially, CFL students should save their strokes: Handwriting practice is an ineffective use of instructional time in second language classrooms.

Footnotes

Appendix A

Appendix B

Appendix C

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors of this work have been funded in part by NSF grants (e.g., 1440753, 1252297, 1109483, 1316736, 1535428, 1724889, 1636782, 1759229, 1822830, 1031398, 1903304, 1931523, 1940236, 1917713), the US Department of Education Institute for Education Sciences (e.g., R305A170137, R305A170243, R305A180401, R305A120125, & R305C100024), the Graduate Assistance in Areas of National Need program (e.g., P200A180088 & P200A150306 ), EIR and the Office of Naval Research (N00014-18-1-2768), and Schmidt Futures. Views presented herein are those of the authors and are not necessarily endorsed by these funding agencies.

ORCID iD

Korinn S. Ostrow

Authors

XIWEN LU is a senior lecturer in Chinese at Brandeis University and a PhD candidate in the Learning Sciences & Technologies Program at Worcester Polytechnic Institute.

KORINN S. OSTROW is a research scientist at Worcester Polytechnic Institute. Her current work focuses on how technology can be used to scale educational experimentation.

NEIL T. HEFFERNAN is the William Smith Professor of Computer Science at Worcester Polytechnic Institute. He is known for developing ASSISTments, software that helps students learn mathematics as it assesses their knowledge. ASSISTments is used by more than 100,000 students in the U.S. each year.

References

Allen

J. R.

(2008). Why learning to write Chinese is a waste of time: A modest proposal. Foreign Language Annals, 41, 237–251. doi:10.1111/j.1944-9720.2008.tb03291.x

American Council on the Teaching of Foreign Languages. (2012). ACTFL proficiency guidelines (Electronic version). Retrieved from https://www.actfl.org/publications/guidelines-and-manuals/actfl-proficiency-guidelines-2012

Cao

Rickles

Zhu

Chan

Harris

. . . Perfetti

C. A.

(2013). Early-stage visual-orthographic processes predict long-term retention of word form and meaning: A visual encoding training study. Journal of Neurolinguistics, 26, 440–461. doi:10.1016/j.jneuroling.2013.01.003

Cao

Chan

D. H. L.

Lawrence

J. M.

Harris

L. N.

Guan

. . . Perfetti

C. A.

(2012). Writing affects the brain network of reading in Chinese: A functional magnetic resonance imaging study. Human Brain Mapping, 34, 1670–1684. doi:10.1002/hbm.22017

Chan

D. W.

C. S.-H.

Tsang

S.-M.

Lee

S.-H.

Chung

K. K. H.

(2006). Exploring the reading-writing connection in Chinese children with dyslexia in Hong Kong. Reading and Writing, 19, 543–561. doi:10.1007/s11145-006-9008-z

Chinese Language Committee. (2009). Modern Chinese common character list (Hanyu Tongyong Zibiao). Beijing, China: Commercial Press.

Cui

(1999). About the reforms in patterns of basic Chinese teaching. Chinese Teaching in the World, 1999(1), 3–8.

De Francis

. (1984). The Chinese language: Fact and fantasy. Honolulu: University of Hawaii Press.

Everson

M. E.

(1998). Word recognition among learners of Chinese as a foreign language: Investigating the relationship between naming and knowing. Modern Language Journal, 82, 194–204. doi:10.2307/329208

10.

Everson

M. E.

(2011). Best practices in teaching logographic and non-Roman writing systems to L2 learners. Annual Review of Applied Linguistics, 31, 249–274. doi:10.1017/S0267190511000171

11.

Guan

C. Q.

Liu

Chan

D. H. L.

Perfetti

C. A.

(2011). Writing strengthens orthography and alphabetic-coding strengthens phonology in learning to read Chinese. Journal of Educational Psychology, 103, 509–522. doi:10.1037/a0023730

12.

Heffernan

N. T.

Heffernan

C. L.

(2014). The ASSISTments ecosystem: Building a platform that brings scientists and teachers together for minimally invasive research on human learning and teaching. International Journal of Artificial Intelligence in Education, 24, 470–497. doi:10.1007/s40593-014-0024-x

13.

Hsiung

Chang

Chen

Sung

(2017). Effect of stroke-order learning and handwriting exercises on recognizing and writing Chinese characters by Chinese as a foreign language learners. Computers in Human Behavior, 74, 303–310. doi:10.1016/j.chb.2017.04.022

14.

HSK Department, Chinese Government. (1992). The Handbook outlining the graded vocabulary for HSK [《汉语水平（词汇）等级大纲》]. Beijing, China: Beijing Language Institute Press.

15.

Huang

Hanley

(1994). Phonological awareness and visual skills in learning to read Chinese and English. Cognition, 54, 73–98. doi:10.1016/0010-0277(94)00641-W

16.

Jiang

(2007). An experimental study on the effect of the method of “teaching the learner to recognize characters more than writing.” Chinese Teaching in the World, 2007(2), 91–97.

17.

(1998). Effects of strategies on the learning of Chinese characters among foreign language students. Journal of the Chinese Language Teachers Association, 33, 93–112. doi:10.1111/j.1944-9720.1998.tb01335.x

18.

Leck

K. J.

Weekes

B. S.

Chen

M.-J.

(1995). Visual and phonological pathways to the lexicon: Evidence from Chinese readers. Memory and Cognition, 23, 468–476.

19.

(2018). Data documentation for “Save Your Strokes: Chinese Handwriting Practice Makes for Ineffective Use of Instructional Time in Second Language Contexts.” Retrieved from http://tiny.cc/LuOstrowHeffernan2019Data

20.

Ostrow

K. S.

Heffernan

N. T.

(2016, November). Improving students’ character recognition ability using on-line practice. Paper presented at ACTFL Annual Convention and World Languages Expo, Boston, MA.

21.

Ostrow

K. S.

Heffernan

N. T.

(2018, May). Understanding the complexities of Chinese word acquisition within an online learning platform. Paper presented at the National Chinese Language Conference, Salt Lake City, UT.

22.

Ostrow

K. S.

Heffernan

N. T.

(2019). Comparing instructional policies for Chinese as a second language: Is handwriting worthwhile? Manuscript in preparation.

23.

Xiong

Heffernan

N. T.

(2017, April). Experimenting choices of video and text feedback in authentic foreign language assignments at scale. Paper presented at the Fourth Annual ACM Conference on Learning at Scale, Boston, MA.

24.

Ostrow

K. S.

Heffernan

N. T.

(2019). Advancing the state of online learning: Stay integrated, stay accessible, stay curious. In Feldman

R. S.

(Ed.), Learning science: Theory, research, & practice (pp. 201-228). New York, NY: McGraw-Hill Education.

25.

Packard

J. L.

Chen

Gaffney

J. S.

Anderson

R. C.

(2006). Explicit instruction in orthographic structure and word morphology helps Chinese children learn to write characters. Reading and Writing, 19, 457–487. doi:10.1007/s11145-006-9003-4

26.

Shen

H. H.

(2004). Level of cognitive processing: Effects on character learning among non-native learners of Chinese as a foreign language. Language and Education, 18, 167–182. doi:10.1080/09500780408666873

27.

Shen

H. H.

(2005). An investigation of Chinese-character learning strategies among non-native speakers of Chinese. System, 33, 49–68. doi:10.1016/j.system.2004.11.001

28.

Siok

W. T.

Perfetti

C. A.

Jin

Tan

L. H.

(2004). Biological abnormality of impaired reading is constrained by culture. Nature, 431, 71–76. doi:10.1038/nature02865

29.

Tan

L. H.

Perfetti

C. A

. (1999). Phonological activation in visual identification of Chinese two-character words. Journal of Experimental Psychology: Learning, Memory, and Cognition, 25, 382–393.

30.

Tan

L. H.

Spinks

J. A.

Eden

G. F.

Perfetti

C. A.

Siok

W. T.

(2005). Reading depends on writing, in Chinese. Proceedings of the National Academy of Sciences of the United States of America, 102, 8781–8785. doi:10.1073/pnas.0503523102

31.

Tan

L. H.

Chang

C. Q.

Siok

W. T.

(2013). China’s language input system in the digital age affects children’s reading development. Proceedings of the National Academy of Sciences of the United States of America, 110, 1119–1123. doi:10.1073/pnas.1213586110

32.

Warschauer

Healey

(1998). Computers and language learning: An overview. Language Teaching, 31, 57–71. doi:10.1017/S0261444800012970

33.

Xie

(2014). 中文教学与时俱进 [Chinese teaching in modern society]. In Zhou

(Ed.), Chinese language in the world (Vol. 3, pp. 69–74). Guangzhou, China: Zhongshan University Press. Retrieved from http://web.csulb.edu/~txie/papers/Chinese_teaching_in_modern_society.pdf

34.

Chang

Zhang

Perfetti

(2013). Reading, writing, and animation in character learning in Chinese as a foreign language. Foreign Language Annals, 46, 423–444. doi:10.1111/flan.12040

35.

Zhu

Shum

S. M.

Tse

S. B.

Liu

J. J.

(2016). Word-processor or pencil-and-paper? A comparison of students’ writing in Chinese as a foreign language. Computer Assisted Language Learning, 29, 596–617. doi:10.1080/09588221.2014.1000932

36.

Zhu

Liu

Ding

Peng

(2009). The influence of Pinyin typewriting experience on orthographic and phonological processing of Chinese characters. Acta Psychologica Sinica, 41, 785–792. doi:10.3724/SP.J.1041.2009.00785