The value of picture-book reading-based collaborative output activities for vocabulary retention

Abstract

This study investigated the effects of three instructional modes: picture-book reading-only (PRO), picture-book reading plus vocabulary instruction (PRVI), and picture-book reading plus reading-based collaborative output activity (PRCOA) on young adult EFL (English as a foreign language) learners’ vocabulary acquisition and retention. Eighty Taiwanese university students with low to intermediate level English proficiency from three English reading classes participated in each of the three modes once during three weekly 100-minute sessions. Vocabulary knowledge was tested through two post-tests using a modified Vocabulary Knowledge Scale: immediately after each instructional mode to measure students’ vocabulary acquisition, and one month later to measure their word retention. The results showed that the PRVI mode was the most helpful for immediate word learning. However, the PRCOA mode was the most effective for word retention, and most conducive to bettering students’ productive knowledge in both acquisition and retention. This study suggests that explicit learning from vocabulary instruction that directs students’ attention to the words to be learned did not guarantee greater vocabulary gains than incidental learning where new words can be learned as by-products of classroom collaborative output activities. Without trying to memorize words, students learned vocabulary through mental investment in group discussions and generative activities, leading to their mastery of productive word knowledge.

Keywords

EFL picture books reading-based collaborative output activity vocabulary instruction vocabulary retention young adult learners

I Introduction

The importance of developing word knowledge through reading has been well documented in first language (L1) studies. Reading research has also shown that extensive and sustained reading is fundamental for second language (L2) learners to build vocabulary. However, as Nation (2001) noted, many L2 learners ‘do not experience the conditions that are needed for this kind of learning to occur’ (p. 232). This point also applies to English as a foreign language (EFL) contexts, such as Taiwan, where young adult learners do little reading outside of class.¹ Studies showed that Taiwanese university students generally have a small vocabulary size, limited to 1,000 to 3,000 word families (Huang, 2004; Yang, 2002), which is far from the baseline vocabulary size of 8,000 to 9,000 word families for independent pleasure reading with 98% as the coverage (Nation, 2006). These EFL learners are trapped in a vicious circle. Because limited vocabulary makes it hard for them to do pleasure reading, they miss out on an opportunity for natural vocabulary learning. This being the case, questions arise such as whether EFL learners can pick up words simply through in-class reading for meaning, whether supplementary activities enhance the vocabulary learning that has occurred during reading and, if so, what kinds of supplementary activity are most effective.

II Literature review

A number of researchers have offered evidence of the helpfulness of reading-only for college-level EFL learners to pick up vocabulary incidentally (e.g. Brown, Waring, & Donkaewbua, 2008; Waring & Takaki, 2003). Waring and Takaki examined the rate at which English vocabulary was initially learned and then retained from reading a graded reader. Fifteen Japanese female university students were asked to recall 25 substitute English words on three test measurements (form recognition, meaning recognition, and meaning translation tests) immediately after the reading, one week later, and three months later to test different types of word knowledge. The results showed that the students could learn words from the graded reader incidentally (about 18% of the 25 items acquired), but that most were not retained three months after reading (nearly 4% of the 25 items retained). Waring and Takaki concluded that reading one graded reader was insufficient to help build new vocabulary, which is similar to Milton’s (2009) report that reading alone might not be sufficient for vocabulary acquisition to take place.

Different from research investigating vocabulary growth through reading-only, other studies have examined EFL vocabulary learning through reading supplemented with another activity. Sonbul and Schmitt (2010), for example, compared the effects of reading-only and reading plus explicit vocabulary instruction on the English vocabulary learning of 40 Arabic students. Three levels of students’ vocabulary knowledge were assessed (form recall, meaning recall, and meaning recognition) immediately after the experiment and one week later. They found that reading plus explicit instruction was more effective than reading-only for vocabulary acquisition and retention at all three levels. Similarly, Laufer and Rozovski-Roitblat (2015) studied the effectiveness of three ‘task type’ conditions (reading only, reading with a dictionary, and reading plus word-focused exercises) and three ‘number of encounters’ (with target words) conditions in mixed combinations on vocabulary retention among 185 young adult EFL learners. The results of active recall, passive recall, active recognition, and passive recognition vocabulary tests revealed that the reading plus word-focused exercises condition yielded the best results, regardless of the number of encounters with the target lexical items. Of the two other conditions, reading with dictionary was more effective than reading-only. Laufer and Rozovski-Roitblat concluded that learning new words depends on what learners do with the words. Other researchers have also found that reading plus a word-focused enhancement condition is superior to reading-only for vocabulary learning (e.g. Laufer, 2000; Paribakht & Wesche, 1997; Rassaei, 2016). In these studies, word-focused enhancement activities included consulting dictionaries, doing vocabulary exercises, having direct vocabulary instruction, and making or completing sentences with new words.

These intentional vocabulary-learning activities are common in EFL reading instruction (Day, 2003). For example, in Taiwan, to start a reading activity, the teacher typically introduces the new topic to be read and pre-teaches the related key words or phrases. Students then read the text, often word-by-word and relying mainly on dictionaries when they encounter unfamiliar words. As the next step, students usually answer comprehension questions and complete vocabulary exercises or write a short summary using the newly learned words. The teacher then clarifies misunderstandings if needed. Intentional learning under this condition, however, neither prepares these learners to become readers in English nor, as studies show, helps boost their vocabulary size. In a similar vein, Rott, Williams, and Cameron (2002) compared the reading-only approach with the reading plus multiple-choice glosses approach and found that, while readers with access to L1 glosses did initially learn more words than readers who read without any enhanced condition, five weeks later the new vocabulary knowledge of both groups had decayed to the same level. The gains from a reading plus vocabulary-enhancement condition are often not sustained; the initial learning may be in vain.

Taking a different approach, Lee (2007) examined the effects on vocabulary learning of picture-book reading with a think-aloud activity. Fifteen female Taiwanese college students first silently read two English picture books and then read them a second time using a think-aloud protocol, in which they not only verbalized their thoughts while reading each page, freely attending to words, passages, or illustrations, but also responded to the researcher’s questions designed to prompt comprehension and reflection. The Vocabulary Knowledge Scale was administrated to determine the immediate learning of 10 target words from each picture book, indicating 14% initial learning. However, long-term vocabulary retention was not addressed in her study. Also, Lee used the Flesch-Kincaid Grade Level Readability Formula to estimate the readability of the two picture books at 4.2 and 6.6 grade levels, which she deemed appropriate because her participants had 7.3 years of formal English instruction. This assumption of comparability of years of English instruction, however, ignores the vast difference between English instruction in school for native speakers and English instruction for learners of the language.

Lee’s (2007) study, however, brought two interesting new insights to light: (1) picture books may be appropriate as reading material for beginning to intermediate older EFL learners, and (2) a supplementary activity not focused on intentional, direct word learning may be effective in vocabulary development. As to the first point, Day and Bamford (1998) suggested that reading very easy books helps language learners quickly build both reading confidence and vocabulary knowledge. Given Aikman’s (1995) observation that topics in picture books may reflect sophisticated, mature issues and life experiences that will capture the interest of older learners, picture books may be suitable for young adult learners with limited vocabulary to read in a single class period with time left for discussion or other activities (Murphy, 2009). Moreover, the copious illustrations found in picture books add visual to printed information, thereby supporting comprehension through both textual and illustrated contexts and reinforcing the meanings of words (Aikman, 1995). As Polette (1989) observed, ‘primarily because the learning is based on the experience of bonding a word to a new perception of a visual image’ (p. 78), language learners may become enthusiastic about discovering new words and using them afterwards.

Sun’s (2015) study of 27 young adult EFL learners’ engagement in five picture-book lessons sheds light on the influence of non-vocabulary-enhancement activities on vocabulary growth. Each lesson in her study consisted of three stages: warm-up, individual picture-book reading, and a collaborative post-reading activity that encouraged learners to interact with text, illustrations, and peers to construct and reflect on story meaning. She found that the text interactions and group discussions facilitated students’ understanding of unfamiliar words presented in the text. Activities that involved predicting for meaning and reflective journal writing, which required good comprehension of the text, provided opportunities for students to collaboratively figure out the word meanings, which incidentally contributed to their vocabulary learning. In this way, vocabulary learning could take place naturally in activities where the learning focus is not to help learners acquire new words, but to facilitate their understandings of the message behind the story. Most importantly, according to Sun, this kind of activity created a space in which learning was enjoyable and active, which is essential for language learning but rarely seen in a typical EFL class in Taiwan. While Sun’s qualitative data revealed the value of these output activities for word learning, further, more robust verification is needed to show the degree to which words can be learned from this kind of supplementary activity.

To further investigate the helpfulness of word-focused and story-meaning-focused activities for vocabulary growth using picture books as reading material for readers with limited vocabulary size, this study aimed to compare the effects of three instructional modes on university EFL learners’ vocabulary acquisition and retention: picture-book reading-only (PRO), picture-book reading plus vocabulary instruction (PRVI), and picture-book reading plus reading-based collaborative output activity (PRCOA).

III Method

1 Participants

The participants in this study were 84 EFL students enrolled in three sophomore English reading classes at a university in Taiwan. The three classes were randomly assigned as Group A (27 students), B (31 students), and C (26 students). The students’ average age was 20 and, based on the college entrance examination, their English proficiency was at the low to intermediate level. Based on the pre-tests of the three picture books selected as reading material in this study, data from four students were excluded because they showed either receptive or productive knowledge of target words prior to the intervention. Therefore, the final analysis was based on 80 participants (Group A: 26 students; Group B: 29 students; Group C: 25 students). To ascertain that the three groups’ English proficiency was at the same level prior to the experiment, an analysis of variance (ANOVA) was conducted to compare the 80 participants’ scores on the final exam of the previous semester.² The results indicated that the score average for Group A was 63.62 (SD = 6.81), for Group B 63.10 (SD = 7.95), and for Group C 63.88 (SD = 7.73), with no significant differences among three groups (F (2,77) = .075, p = .928).

2 Study design

The intervention took place during three regular 100-minute English reading sessions at intervals of three weeks, starting from the third week of the semester. During each treatment session, the three groups read the same picture book, but each group was exposed to a different type of instructional mode (PRO, PRVI, or PRCOA). At the end of intervention, all three groups had read the three picture books and experienced all three modes but in different order. One week before each treatment session, a pre-test of the 13 target words from the selected picture book for the session and two control words that did not occur in the three picture books was administered to each group, rather than on the same day of the session in order to prevent learning from doing pre-tests. The purpose of having two non-target items in each test was also to check the pre-test-learning effect.³ After each treatment session, the three groups immediately took the same test as an immediate post-test to assess their initial word learning (acquisition). One month after each session, the same test was repeated as a delayed post-test to assess retention. Table 1 displays the design of the study.

Table 1.

Design of the study.

Week	Group A (n = 26)	Group B (n = 29)	Group C (n = 25)
1	Course introduction
2	Pre-test for Picture book 1
3	Picture book 1 session: The Three Little Wolves and the Big Bad Pig
	PRO	PRCOA	PRVI
	Immediate post-test for Picture book 1
5	Pre-test for Picture book 2
6	Picture book 2 session: The Three Robbers
	PRCOA	PRVI	PRO
	Immediate post-test for Picture book 2
7	Delayed post-test for Picture book 1
8	Pre-test for Picture book 3
9	Picture book 3 session: Those Darn Squirrels!
	PRVI	PRO	PRCOA
	Immediate post-test for Picture book 3
10	Delayed post-test for Picture book 2
13	Delayed post-test for Picture book 3

3 Target words

To identify target words, 20 vocabulary items assumed to be unfamiliar to the participants were chosen from each of the three picture books by the instructor, and a pilot test of those 60 words was administrated to 67 sophomores at the same university. Words unfamiliar to at least 99% of the students were selected as the target words. The final 39 target words (13 from each picture book) are shown in Appendix 1.

4 Reading material

As vocabulary learning through reading involves a great deal of word-guessing from context, the helpfulness of the context is a key factor influencing learning rate (Waring & Nation, 2004). Hulstijn (1992), however, found that contextual information does not always facilitate word learning. Picture books, with their combination of text and illustrations, can maximize vocabulary uptake by providing pictorial cues to offset the infrequent occurrence of some words in the text and insufficient linguistic contextual information for readers to correctly infer word meanings. What Zambo (2005) refers to as dual-coding, ‘using pictures and text simultaneously, or coding information visually and verbally’ (p. 504), suggests the potential for picture books to enhance vocabulary acquisition. Therefore, picture books were selected as reading material in the current study.

The criteria of students’ interest and linguistic difficulty were used to select the picture books for this study. According to the result of a picture-book-interest questionnaire, the top three themes the participants favored were fractured fairy tales, human heart, and friendship. With regard to optimal level of linguistic difficulty, there is some agreement in the literature that for fluent reading with comprehension, a text should have between 96% to 99% coverage of known words (Hu & Nation, 2000). Five picture books in which, based on the instructor’s judgment, fewer than 4% of the running words are unfamiliar to the students were chosen for each of the three themes. Then a copy of each book was provided for each of 34 sophomores from another class at the same university to read independently and list all unfamiliar words they encountered. On the basis of the results, books that had average of coverage rate of less than 96% were withdrawn. The final selections were The Three Little Wolves and the Big Bad Pig by Eugene Trivizas and Helen Oxenbury (1993) for fractured fairy tales, The Three Robbers by Tomi Ungerer (2008) for human heart, and Those Darn Squirrels! by Adam Rubin and Daniel Salmieri (2008) for friendship.

5 Procedures of three instructional modes

The following is a detailed description of each of the three instructional modes, the PRO mode, the PRVI mode, and the PRCOA mode.

a PRO mode

After a 20-minute warm-up talk related to the theme of a chosen picture book, a copy of the book was given to each student. Before starting to read, students were told to read for meaning and use context and pictorial clues to infer the meanings of unknown words. Students were then given 30 minutes to read their own copies alone and silently without looking up any words in a dictionary, after which they were given 10 minutes to answer general comprehension questions without referring to the text in order to make sure that they read for meaning. In order to control for time spent on the task across the three modes, a library of 40 thematically related supplemental picture books was provided in class pleasure reading for the next 40 minutes.

b PRVI mode

In this mode, upon the completion of the comprehension questions, the instructor provided a 40-minute explicit vocabulary lesson on all target words. Instead of writing the definition of each word on the blackboard, the instructor walked the students through the process of inferring word meaning from context and pictorial clues. For example, to lead the students to infer the meaning of outskirts from Those Darn Squirrels!, the instructor had them use the relevant illustration and context of the opening page, which began the narrative with these words: On the outskirts of town, at the edge of the forest, there was a little old house. The only thing older than the little house was the man who lived in it: Old Man Fookwire. The instructor first had students look at the illustration picturing five houses of different sizes and at various locations and asked them to identify the house that was little, looked old, and stood close to the forest. She then had students determine from the illustration whether that house was located in the center of the town or in the part of the town farthest from the center. Finally the instructor used suburbs, a synonym of outskirts that was familiar to the students, to reinforce the meaning of outskirts. Sometimes, when students showed signs of difficulty, the instructor would draw the word on the blackboard or use actions to explain it.

c PRCOA mode

In the PRCOA mode students were asked to form groups of four to participate in a collaborative output task, defined here as any activity that is open-ended and generative, stimulates creativity and imagination, promotes discussion about the input text with peers, and invites students to use both language and art to express their perspectives. From Lewison and Gossard’s (1994) Literacy activities handbook, the Predicting for Meaning and Reflective Journal activity was selected for The Three Little Wolves and the Big Bad Pig, the Open Mind activity for The Three Robbers, and the I Am Poem activity for Those Darn Squirrels!. Table 2 describes the three activities in detail. Because the Predicting Meaning and Reflective Journal activity involves the cycle of predicting, reading, confirming, integrating, and predicting again, for The Three Little Wolves and the Big Bad Pig the collaborative output task was combined with independent reading before students completed the comprehension questions, whereas students read independently and answered the comprehension questions before engaging in the collaborative output tasks for The Three Robbers and for Those Darn Squirrels!. During each activity, students shared their thoughts about the picture book, exchanged their ideas of how to present their thoughts through the activity, and discussed how to design the poster that would be their product of the activity. After that, students in each group shared their work with the whole class. While participating in the group discussion, students were encouraged to use English but allowed to speak in Chinese so that full sharing of perspectives and ideas would not be hindered by a language barrier. However, students were asked to use only English when sharing their poster with the whole class.

Table 2.

Descriptions of three reading-based collaborative output activities.

Activities	Descriptions
Predicting Meaning and Reflective Journal	This activity helps students learn how to interact with text and illustrations, and invites them to reflect on their new learning and understanding.
Predicting Meaning and Reflective Journal	Activity poster
	The Three Little Wolves and the Big Bad Pig
	Predicting based on title & cover	Predicting again	Predicting ending	Our thinking about the Story (words & graphic images)

Open Mind	This activity helps students speculate about the three robbers’ inner thoughts and feelings, in which they analyze the three robbers’ change of heart. In this activity, students put ‘The Three Robbers’ at the top of the Open Mind oval. Inside the oval-shaped outline, they write words or phrases and draw symbols or pictures to illustrate what the three robbers are thinking about, what they care about, and how they perceive others. The Three Robbers
I Am Poem	This provides the framework for a reflective poem to help deepen students’ understanding of the story of Those Darn Squirrels! and develop insights into Old Man Fookwire’s or the Squirrels’ wishes and struggles. Students could also use art to reflect on their thinking and to design their poem poster.I Am poem
	1st Stanza I am … I wonder … I hear … I see … I want … I am … (the 1st line of the poem repeated)	2nd Stanza I pretend … I feel … I touch … I worry … I cry … I am … (the 1st line of the poem repeated)	3rd Stanza I understand … I say … I dream … I try … I hope … I am … (the 1st line of the poem repeated)

6 Vocabulary instrument

To measure students’ vocabulary acquisition and retention of target words, Paribakht and Wesche’s (1997) Vocabulary Knowledge Scale (VKS) was modified for the present study. The VKS, which combines self-report and performance items to elicit learners’ self-perceived and demonstrated knowledge of a specific word, reflects changes in vocabulary knowledge during relatively limited instructional periods (Paribakht & Wesche). The researcher modified the VKS from a five-category to a four-category scale (see Table 3) to avoid the concern that variation in student interpretation of Categories III (I think it means) and IV (I know it means) of the original VKS might arise (Paribakht & Wesche, 1993).

Table 3.

Modified Vocabulary Knowledge Scale (VKS).

Self-report categories
I. I don’t remember having seen this word before.
II. I have seen this word before, but I don’t know what it means.
III. I know this word. It means _______________________________. (synonym or translation)
IV. I can use this word in a sentence: ________________________________________________________. (If you do this section, please also do section III.)

Following the basic unknown/known dichotomy (Horst, 2005; Paribakht & Wesche, 1997), Categories I (completely unfamiliar) and II (partially known knowledge) of the modified VKS were conflated into the ‘unknown’ word category, and Categories III (receptive knowledge) and IV (productive knowledge) into the ‘known’ word category. An item that was either reported as not known (Category I or II) or defined incorrectly (the synonym or translation of the item provided in Category III was wrong) was considered unknown and assigned a score of zero. An item that was either defined correctly or defined correctly and used appropriately in a sentence was considered known and assigned a score of one. Note that ‘appropriately’ here means that the use of an item in a sentence in Category IV was both semantically and grammatically correct, even if other parts of the sentence contained errors. The researcher and her colleague evaluated students’ responses, and the Cohen’s kappa agreement rates were .99, .99, and .99 for the three pre-tests (one for each picture book); .98, .97, and .97 for the three immediate post-tests; and .99, .98, and .97 for the three delayed post-tests.

IV Results

The descriptive statistics of the immediate and delayed post-test vocabulary knowledge scores for each instructional mode for each picture book are reported in Table 4. Because each participant had no receptive or productive knowledge about the 39 target words prior to the intervention, all pre-test scores were zero. Therefore, the immediate and delayed test scores were treated as vocabulary acquisition and retention scores respectively.

Table 4.

Descriptive statistics of acquisition and retention scores for the three picture books by the three instructional modes.

	Acquisition		Retention
	Mean* (Std)	Rate (percentage)	Mean* (Std)	Rate (percentage)
PRO mode:
The Three Little Wolves and the Big Bad Pig	7.15 (2.36)	55	5.35 (2.48)	41
The Three Robbers	7.20 (1.98)	55	5.32 (1.75)	41
Those Darn Squirrels!	6.86 (1.87)	53	4.97 (2.11)	38
PRVI mode:
The Three Little Wolves and the Big Bad Pig	12.31 (.76)	95	5.59 (1.84)	43
The Three Robbers	12.19 (1.23)	94	5.27 (2.09)	41
Those Darn Squirrels!	12.20 (1.15)	94	5.44 (2.04)	42
PRCOA mode:
The Three Little Wolves and the Big Bad Pig	10.08 (1.93)	78	7.96 (2.07)	61
The Three Robbers	10.28 (1.77)	79	8.34 (1.91)	64
Those Darn Squirrels!	9.46 (1.88)	73	7.65 (2.53)	59

Notes. * Maximum = 13. PRCOA = picture-book reading plus reading-based collaborative output activity; PRO = picture-book reading-only; PRVI = picture-book reading plus vocabulary instruction.

1 Effects of each instructional mode on vocabulary acquisition and retention

Paired-samples t-tests indicated that students experiencing the PRO mode made significant vocabulary gains immediately after the treatment session, and still retained significant word knowledge one month later for The Three Little Wolves and the Big Bad Pig (acquisition: t = 15.45, p < .001; retention: t = 10.99, p < .001); for The Three Robbers (acquisition: t = 18.19, p < .001; retention: t = 15.21, p < .001); and for Those Darn Squirrels! (acquisition: t = 19.81, p < .001; retention: t = 12.66, p < .001). As indicated in Table 4, on average students acquired over one-half of the target words from reading picture books alone (54%)⁴ and retained two-fifths of the words one month later (40%). Results of a one-way ANOVA found no significant differences in acquisition (F (2,77) = .22, p = .81), or in retention (F (2,77) = .27, p = .76) across the three picture books.

Students experiencing the PRVI mode also built up significant immediate vocabulary learning and retention for The Three Little Wolves and the Big Bad Pig (acquisition: t = 87.14, p < .001; retention: t = 16.33, p < .001), for The Three Robbers (acquisition: t = 50.40, p < .001; retention: t = 12.86, p < .001), and for Those Darn Squirrels! (acquisition: t = 52.83, p < .001; retention: t = 13.32, p < .001). Averagely, as shown in Table 4, students acquired nearly all of the target items from reading picture books together with direct vocabulary instruction (94%), but they only retained about two-fifths of the items one month later (42%). A one-way ANOVA found no significant differences in acquisition (F (2,77) = .11, p = .90) or in retention (F (2,77) = .17, p = .84) across the three picture books.

Similarly, students who participated in the PRCOA mode made significant vocabulary acquisition and retention gains for The Three Little Wolves and the Big Bad Pig (acquisition: t = 26.05, p < .001; retention: t = 19.22, p < .001), for The Three Robbers (acquisition: t = 31.25, p < .001; retention: t = 23.48, p < .001), and for Those Darn Squirrels! (acquisition: t = 25.65, p < .001; retention: t = 15.43, p < .001). As displayed in Table 4, across the three picture books, students acquired over three-fourths of the target words from reading picture books plus collaborative output activities (76%) and retained more than three-fifths of the words one month later (61%). Again, a one-way ANOVA found no significant differences in acquisition (F (2,77) = 1.40, p = .25) or in retention (F (2,77) = .70, p = .50) across the three picture books. Across the three modes, vocabulary acquisition and retention did not vary among the three picture books.

2 Comparison of effectiveness in vocabulary acquisition and retention among three instructional modes

As Table 4 indicates, differences were found among the three modes in the acquisition and retention scores for each picture book. One-way ANOVAs were conducted to assess the impact of the three different instructional modes (PRO, PRVI, and PRCOA) on students’ vocabulary acquisition and retention, as measured by the immediate and delayed post-tests. The results revealed that for each picture book there were statistically significant differences among the three modes both in the immediate post-test (The Three Little Wolves and the Big Bad Pig, F (2,77) = 57.30, p < .01, η² = .60; The Three Robbers, F (2,77) = 56.54, p < .01, η² = .59, and Those Darn Squirrels!, F (2,77) = 67.64, p < .01, η² = .64), and in the delayed post-test (The Three Little Wolves and the Big Bad Pig, F (2,77) = 11.72, p < .01, η² = .23; The Three Robbers, F (2,77) = 23.24, p < .01, η² = .38; and Those Darn Squirrels!, F (2,77) = 11.00, p < .01, η² = .22).

The results of post hoc comparisons shown in Table 5 revealed that, for each of the three picture books students in the PRVI mode made significantly more immediate vocabulary gains than those in the PRO mode. However, over a one-month period, the students in the two modes retained a similar number of target words. Like those in the PRVI mode, students in the PRCOA mode for each picture book also acquired significantly more knowledge about the target words than students in the PRO mode. Moreover, after one month students in the PRCOA mode still retained significantly more target words than those in the PRO mode. Moreover, students in the PRVI mode acquired significantly more target words for each picture book than those in the PRCOA mode. However, one month later students in the PRVI mode retained significantly fewer target words than those in the PRCOA mode. Overall, these results indicated that among the three instructional modes, the PRVI mode was the most effective for vocabulary acquisition, but the PRCOA mode was the most effective for vocabulary retention.

Table 5.

Comparisons of acquisition and retention scores among the three modes for the three picture books.

	Acquisition	Retention
The Three Little Wolves and the Big Bad Pig	PRO < PRVI (t = −11.14)PRO < PRCOA (t = −4.83)PRVI > PRCOA (t = 5.72**)	PRO = PRVI (t = −.41, p = .683)PRO < PRCOA (t = −4.08)PRVI < PRCOA (t = −4.46)
The Three Robbers	PRO < PRVI (t = −10.86)PRO < PRCOA (t = −6.03)PRVI > PRCOA (t = 4.61**)	PRO = PRVI (t = −.09, p = .926)PRO < PRCOA (t = −6.03)PRVI < PRCOA (t = −5.70)
Those Darn Squirrels!	PRO < PRVI (t = −12.40)PRO < PRCOA (t = −5.14)PRVI > PRCOA (t = 6.24**)	PRO = PRVI (t = −.84, p = .407)PRO < PRCOA (t = −4.29*)PRVI < PRCOA (t = −3.43)

Notes. The ‘greater than’ symbol (>) indicates that the test score of the former mode is significant greater than that of the latter mode. The ‘less than’ symbol (<) indicates that the test score of the former mode is significant less than that of the latter mode. The ‘equal to’ symbol (=) indicates that the test score of the former mode is not significantly different from that of the latter mode; **p < .001, *p < .01. PRCOA = picture-book reading plus reading-based collaborative output activity; PRO = picture-book reading-only; PRVI = picture-book reading plus vocabulary instruction.

3 Change in depth of vocabulary knowledge among three modes

Because the dichotomous scores of ‘unknown’ and ‘known’ each subsumed two levels of depth of word knowledge (completely unfamiliar and partially known for a score of 0 and receptive and productive knowledge for a score of 1), two similar scores might mask different levels of self-reported word knowledge. Further examination of changes in depth of word knowledge across the three modes on vocabulary acquisition and retention was explored to see practical differences among these modes. Table 6 shows the self-reported responses over types of vocabulary knowledge for the three modes for the three picture books combined.

Table 6.

Responses over types of vocabulary knowledge for three instructional modes.

Group	Test	Vocabulary Knowledge (percentages in parentheses)
Group	Test	Completely unfamiliar	Partially known	Receptive	Productive
PRO	Pre-test	742 (71)	298 (29)	0 (0)	0 (0)
	Immediate Post-test	259 (25)	216 (21)	417 (40)	148 (14)
	Delayed Post-test	276 (27)	348 (33)	355 (34)	61 (6)
PRVI	Pre-test	797 (77)	243 (23)	0 (0)	0 (0)
	Immediate Post-test	34 (3)	27 (3)	735 (71)	244 (23)
	Delayed Post-test	182 (17)	423 (41)	356 (34)	79 (8)
PRCOA	Pre-test	779 (75)	261 (25)	0 (0)	0 (0)
	Immediate Post-test	124 (12)	120 (11)	474 (46)	322 (31)
	Delayed Post-test	151 (14)	249 (24)	433 (42)	207 (20)

Note. Total number of responses = 1040. PRCOA = picture-book reading plus reading-based collaborative output activity; PRO = picture-book reading-only; PRVI = picture-book reading plus vocabulary instruction.

As had been shown, the PRVI mode was found most effective for vocabulary acquisition. However, as shown in Table 6, the percentage of productive knowledge acquisition for the PRCOA mode was the highest (31%), followed by the PRVI mode (23%) and the PRO mode (14%). The differences in productive knowledge among the three modes were all significant (PRO/PRVI: MD = −1.20, t = −5.55, p < .001; PRO/PRCOA: MD = −2.18, t = −11.21, p < .001; PRVI/PRCOA: MD = −.98, t = −4.55, p < .001). With regard to productive vocabulary knowledge retention, the differences between PRO and PRCOA (MD = −1.83, t = −11.85, p < .001) and between PRVI and PRCOA (MD = −1.60, t = −10.24, p < .001) were both significant, but the difference between PRO and PRVI was not significant (MD = −.23, t = −1.71, p = .089). This showed that the PRCOA mode (20%) was most effective, compared to the PRVI mode (8%) and the PRO mode (6%). These findings support the superiority of the PRCOA mode over the other two modes in facilitating productive vocabulary knowledge acquisition and retention.

V Discussion

The findings of this study demonstrated that university EFL learners could acquire vocabulary with each of the three instructional modes, PRO, PRVI, and PRCOA, and retain some knowledge one month later. Among the three modes, the PRVI mode was the most effective for vocabulary acquisition, followed consecutively by the PRCOA mode and PRO mode. However, for vocabulary retention the PRCOA mode was superior to the other two modes, between which no difference was found.

Overall, the present study found that reading plus (either PRVI mode or PRCOA mode) was more helpful than reading alone (PRO mode) for vocabulary acquisition. As Laufer and Shmueli (1997) claimed, when students read for meaning without any supplementary activities, new words in a text may go unnoticed, especially when they are not important for comprehension. This appeared to account for the predominant rate of completely unfamiliar word knowledge in the PRO mode on the immediate test as compared with the other two modes. In line with previous research (Lee, 2003; Sonbul & Schmitt, 2010), this study found that vocabulary was best acquired when students received direct vocabulary instruction combined with reading. A possible explanation for this finding is that a teacher’s instruction encourages students to pay close attention to each of the lexical items to be learned, leading to higher rates of acquisition.

One month after each treatment session ended, however, a different picture emerged. The PRCOA mode proved to be the most effective for vocabulary retention, followed by the PRVI and PRO modes, between which there was only a 2% difference in average retention rate. The vocabulary knowledge initially learned from the PRVI mode had decayed to the same level as the PRO mode, although the difference in average acquisition rate between the two was large (40%). That is, the advantage of teacher explanation disappeared after one month, indicating that the initial vocabulary learning from reading with vocabulary teaching was fragile. This finding contradicts Sonbul and Schmitt’s (2010) study showing that one week after the teaching session, students in the group of reading together with direct teaching of target words retained more words than the reading-only group. This discrepancy may be due to the vocabulary instruction given and/or the timing of the delayed tests. The vocabulary instruction in the present study was contextually relevant, so students might experience two aspects of learning at once, i.e. explanations of words and utilization of contextual and pictorial information. On the other hand, in Sonbul and Schmitt’s study, the meaning of each target item was given directly to the students. At that time students needed to focus only on form and meaning of the words being taught, which could result in a lower cognitive load. Also, the retention test in Sonbul and Schmitt’s study was assessed only one week after the session, which might not be long enough to realistically represent long-term retention.

The superiority of the PRCOA mode over the PRO and PRVI modes for long-term retention found in this study can be interpreted from an involvement load perspective as proposed by Laufer and Hulstijn (2001). As they claimed, the presence and degree of involvement of need, search, and evaluation are the determining factors in vocabulary retention (Laufer, 2001). In the PRCOA mode, negotiating the meaning of unknown words with peers entails the ‘need’ and ‘search’ components of involvement; students’ need to communicate with group members and complete the task propelled their collaborative search for the meanings of uncertain words. Compared with both the PRO and PRVI modes, in which the meanings of target words were left to guesswork or were explained by the instructor, the discussion of a word’s meaning with peers taking place in the PRCOA mode incidentally strengthened its retention. Moreover, during the three activities, students occasionally produced the words verbally while communicating with peers and, in order to express their feelings and ideas on the poster, they wrote phrases or sentences using some of the words. The use of words implies ‘evaluation’, the third component of involvement. As Rassaei (2016) claimed, activities that require learners to evaluate ‘the suitability of target words in an unfamiliar sequence of actions or situations imagined and created by the learner’ would promote word knowledge (p. 15). Students’ use of some target items to complete their poster during the PRCOA mode further enhanced their retention of those items.

As Bromley (2007) asserted, word learning requires giving students ‘a variety of opportunities to … use words actively in authentic ways’ (p. 536). The use of words in the text to express insights and ideas on a poster provided a second chance for students to learn vocabulary incidentally. Once a word is understood, whether through reading alone or through learning with peers, it might be retrieved and used generatively to complete an activity task, leading to enhanced productive lexical knowledge. For example, members of Team 5 of Group C used the word squirrel five times in their I Am poem to express the birds’ feelings toward those squirrels (for the black arrows showing their use of the word in the I Am poem, see Figure 1), each use semantically appropriate and grammatically correct. Although squirrel was new to all of them prior to the session, the receptive and productive knowledge of the word they acquired was still intact one month later.

Figure 1.

The poster of I Am Poem activity made by Team 5 of Group C.

Observing the poster made by Team 5 of Group C, some may argue that because the output activity of producing a poster required productive vocabulary use, it is not surprising that students in the PRCOA mode had higher acquisition and retention of productive word knowledge. However, although fewer than one fourth of the target words were used on the posters, and only a few students were trying to produce the target words verbally during group discussions, the higher achievement was a group phenomenon, suggesting it was not just talking and making a poster but the nature of the collaborative activities that can account for the superiority performance of students experiencing the PRCOA mode.

The output activities in the present study were open-ended, creative, generative, and collaborative, qualities that stimulate thinking and imagination, provoke discussions for story meaning, and promote learning enjoyment. These activities could intrinsically motivate students to be engaged and activate their willingness to learn as they became immersed in performing the task. As Schmitt (2008) asserted, any activity that ‘leads to more and better engagement should improve vocabulary learning’ (pp. 339–40). In the dialogue below, Team 1 of Group A’s discussion about the word loot shows how one student’s idea initiated the whole team’s efforts to discern the meaning of the word, which incidentally corrected another student’s misunderstanding and helped his teammates acquire and retain productive knowledge of it. Because students could communicate in either Chinese or English as they chose, most of the following conversation was translated from Chinese and the rest was corrected for language.

[They were discussing some bad things the three robbers had done to represent the evil side of the robbers, and Figure 2 presents the poster they made.]

AS1:

They got lots of money from those people. We can draw some money and jewels below these frightened people.

AS3:

How about drawing two sacks of money and gold?

AS4:

Why two?

AS3:

To represent the double O’s in the word loot.

AS2:

Loot? What is that?

AS1:

Here says, ‘There they carried their loot.’ [pointing at the page showing the text]

AS2:

I didn’t see it! Let me read this again.

AS4:

Doesn’t loot mean the chest they carried to their hide-out? Look! [pointing at the page illustrating one of the robbers carrying a chest and preparing to enter their hide-out, followed by another one holding a red axe]

AS3:

Here says [turning to the next page], ‘They had trunks and chests full of gold, jewels, money, watches, wedding rings, and precious stones.’ I think it means the valuable stuff in that chest.

AS2:

I think ‘the riches they got from people’ seems more logical than ‘the chest they carried’ for the word loot. Look here! [pointing at the page illustrating the three robbers looking at a chest full of gold] They were checking the gold and jewels in the chest they carried to their place. The chest itself, I think, is not important to these three robbers, but the gold inside the chest is.

AS4:

Generally speaking, most people would think gold and jewels are more valuable than a wooden chest.

AS1:

The double O’s could work! The gold, jewels, and money in this chest are also round shaped like the letter O.

AS3:

The double O’s also look like the eyes of the frightened passengers [pointing at the page-opening illustrating the three robbers threatening the passengers]. Their eyes are so big and round.

AS4:

Perhaps we could draw three round sacks. The middle one represents the O’s in the word loot, and the other two represent these people’s eyes. This arrangement could show that these three robbers derived great riches from the helpless and frightened people.

Figure 2.

The poster of Open Mind activity made by Team 1 of Group A.

This interaction exemplifies students’ unintentional vocabulary learning with teammates while working on a creative project. One group member’s idea seemed to help others, like AS2, notice words they might have overlooked while reading alone, which could explain the significantly smaller percentage of target words that were still unfamiliar immediately after students experienced the PRCOA mode in comparison to the PRO mode. AS2’s question and AS4’s inquiry about loot drew their teammates’ attention to the word itself and to the sentence with it. In order to decide how to represent the evil side of the three robbers, the whole team had to find out what loot meant, which prompted them to use context and pictorial clues, share their opinions on the meaning, and then work together to determine the more logical answer of two alternatives. As the team members negotiated meaning, they constantly reviewed the lexical context of the item and examined the relevant illustrations. Without deliberate practice of the item in a sentence, the attention focused on the item, the need to understand it, and the time spent intensely interacting with peers to discuss its meaning resulted in students’ repeated exposure to the item in its context, not only the sentence in which it appeared, but also its visual association with the illustrations showing the passengers’ frightened eyes and their own idea of drawing round sacks of money and gold. This would reinforce students’ recognition of the form of loot and facilitate their understanding of its meaning and grammatical use in a sentence, leading to enhanced productive word knowledge. On the immediate test, all members of this team were able to provide the correct meaning of loot and use it productively. A month later all retained it receptively, and indeed three retained productive knowledge of it, the only team in Group A to achieve this level of performance.

Overall, the need to interact and negotiate meaning with the story, illustrations, and peers in the PRCOA mode casually directed students’ attention to words in the text. Discussing interpretations with peers and working with one another to construct meaning provided students with opportunities to share, gain, and reshape word knowledge. Without trying to memorize words, students learned vocabulary through mental investment in group discussions and generative activities, during which they cognitively processed the new knowledge and then used it in productive ways to express their ideas and create their artifacts, which naturally expanded their vocabulary knowledge. As Johnson, Johnson, and Smith (1991) claimed, students need to actively construct their own knowledge because ‘learning is conceived of as something a learner does, not something that is done to the learner. Students do not passively accept knowledge from the teacher or curriculum’ (p. 1). From this perspective, the amount and variety of cognitive processing they undertake while dealing with unfamiliar lexical items influence the depth of knowledge of them that learners acquire (Paribakht & Wesche, 1997) and the strength of its retention (Newton, 2013).

VI Conclusions

This study compared the effectiveness of picture-book reading-only (PRO), picture-book reading plus vocabulary instruction (PRVI), and picture-book reading plus collaborative output activity (PRCOA) to enhance young adult EFL learners’ vocabulary acquisition and retention. The findings can be summarized as follows (> symbolizes ‘more effective than’; = symbolizes ‘equally effective’):

Vocabulary acquisition: PRVI > PRCOA > PRO

Vocabulary retention: PRCOA > PRVI = PRO

Overall, reading together with explicit vocabulary instruction highlighting attention to lexical items resulted in more effective initial word learning. However, with opportunities to negotiate meaning of unfamiliar vocabulary with peers and/or to use it in an interactive task, reading-based collaborative output activities promoted better word retention and better productive vocabulary knowledge acquisition and retention. This is not to suggest that direct vocabulary instruction does not play an important role in word learning. The findings of the present study, however, indicate the need for EFL reading programs to shift to a more constructive view of vocabulary learning because, as Milton (2008) stated, ‘success is not just a question of the rate of language or vocabulary learning but also of the time spent in meaningful foreign-language activity’ (p. 234).

In light of these findings, it is suggested that college instructors of EFL integrate reading-based collaborative output activities designed for text-meaning-searching/making in their teaching pedagogies in order to promote students’ vocabulary learning and to create opportunities for learning enjoyment, for which picture books are especially suited. Teachers’ own involvement is also important in these activities. They should actively circulate among working groups to intervene as needed or simply to listen to each group’s discussion and share their own insights into the story so that students can feel teachers’ enthusiasm and love of reading. Teachers can open up conversations by posing questions that reveal students’ understanding and encourage meaning constructions or by modeling how they interact with the text and illustrations and make sense of them. It is also important to keep in mind that students’ interest in an activity might flag if overly structured procedures and lack of flexibility make it seem like routine class work. Rather, to motivate interactions among peers and promote cognitive engagement with new words, learning activities that prompt original thinking and allow students to bring their imagination and creativity into full play should be provided.

As is the case with any study, this study has some limitations that should be acknowledged. First, the participants in this study experienced three different treatment sessions on three occasions (week 3, week 6, and week 9) and, therefore, the pre-test, immediate post-test, and delayed post-test for each set of 13 target words were administrated separately at different points in time. It is possible that students learned to anticipate that a vocabulary test would come at the end of each lesson, and that they directed their attention to vocabulary learning, which could have affected the test scores for latter two sessions. One suggestion for future research is to carry out the treatment sessions in the same week, and to have the immediate post-test at the end of the week to avoid task–test–task–test effect. Second, incidental vocabulary acquisition from reading is considered a gradual, cumulative learning process (Waring & Nation, 2004). As Waring and Nation (2004) asserted, ‘the ability to recognize which words occurred in the text and which words did not indicates that some familiarity with the form of a word has been achieved’ (pp. 16–17), which is an important step prior to working out what it means (Waring & Takaki, 2003). Therefore, using the basic unknown/known dichotomy to examine participants’ vocabulary learning may fail to recognize their attention to the forms of unfamiliar vocabulary words. Clearly, learners’ partial knowledge of a lexical item should be taken into account in future research.

Footnotes

Appendix

Appendix 1.

List of 39 selected target words.

The Three Little Wolves and the Big Bad Pig		The Three Robbers		Those Darn Squirrels!
Lexical item	Frequency	Lexical item	Frequency	Lexical item	Frequency
wheelbarrow (n)	2	fierce (a)	1	darn (a)	3
crumble (v)	1	cape (n)	4	outskirts (n)	1
frightened (a)	4	blunderbuss (n)	2	grump (n)	1
beaver (n)	2	axe (n)	2	crisp (a)	1
prowl (v)	4	faint (v)	1	birdfeeder (n)	6
rhinoceros (n)	3	fled (v)	1	squirrel (n)	25
barbed (a)	3	carriage (n)	3	fist (n)	2
padlock (n)	4	plunder (v)	1	devise (v)	2
hopscotch (n)	1	hide-out (n)	1	furious (a)	1
bolt (v)	1	loot (n)	1	launch (v)	1
dynamite (n)	1	trunk (n)	2	sail (v)	1
scorched (a)	1	bundle (v)	1	stack (v)	1
scent (n)	3	glittering (a)	1	disguise (n)	1

Note. Part-of-speech in parentheses: a = adjective; n = noun; v = verb.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research is supported by a grant from the Ministry of Science and Technology in Taiwan (MOST 102-2410-H-424-001-).

Notes

References

Aikman

C.C.

(1995). Ideas in practice: Picture books and developmental students. Journal of Developmental Education, 19, 28–32.

Bromley

(2007). Nine things every teacher should know about words and vocabulary instruction. Journal of Adolescent and Adult Literacy, 50, 528–537.

Brown

Waring

Donkaewbua

(2008). Incidental vocabulary acquisition from reading, reading-while-listening, and listening to stories. Reading in a Foreign Language, 20, 136–163.

Day

R.R.

(2003). Teaching reading: Comprehension questions. Teacher Talk, 18. Available at: http://www.cape.edu/docs/TTalk0018.pdf (accessed June 2016).

Day

R.R.

Bamford

(1998). Extensive reading in the second language classroom. New York: Cambridge University Press.

Horst

(2005). Learning L2 vocabulary through extensive reading: A measurement study. The Canadian Modern Language Review, 61, 355–382.

Nation

(2000). Unknown vocabulary density and reading comprehension. Reading in a Foreign Language, 13, 403–430.

Huang

C.C.

(2004). University students’ vocabulary knowledge, content knowledge and reading comprehension. Journal of National Tainan Teachers College: Humanities and Social Science, 38, 125–153.

Hulstijn

J.H.

(1992). Retention of inferred and given word meanings: Experiments in incidental vocabulary learning. In: Arnaud

Bejoint

(Eds.), Vocabulary and applied linguistics (pp. 113–125). London: Macmillan.

10.

Johnson

D.W.

Johnson

R.T.

Smith

K.A.

(1991). Active learning: Cooperation in the college classroom. Edina, MN: Interaction Book Co.

11.

Laufer

(2000). Electronic dictionaries and incidental vocabulary acquisition: Does technology make a difference? In: Evert

. (Eds.), Proceedings of the 9th EURALEX International Congress (pp. 849–854). Stuttgart: Stuttgart University.

12.

Laufer

(2001). Reading, word-focused activities and incidental vocabulary acquisition in a second language. Prospect, 16, 44–54.

13.

Laufer

Hulstijn

J.H.

(2001). Incidental vocabulary acquisition in a second language: The construct of task-induced involvement. Applied Linguistics, 22, 1–26.

14.

Laufer

Rozovski-Roitblat

(2015). Retention of new words: Quantity of encounters, quality of task, and degree of knowledge. Language Teaching Research, 19, 687–711.

15.

Laufer

Shmueli

(1997). Memorizing new words: Does teaching have anything to do with it? RELC Journal, 28, 89–108.

16.

Lee

H.N.

(2007). Investigating EFL adult learners’ vocabulary acquisition through reading picture books. Unpublished doctoral dissertation, Indiana University, Bloomington, IN, USA.

17.

Lee

S.H.

(2003). ESL learners’ vocabulary use in writing and the effects of explicit vocabulary instruction. System, 31, 537–561.

18.

Lewison

Gossard

(1994). Literacy activities handbook. El Segundo, CA: Galaxy Institute for Education.

19.

Milton

(2008). Vocabulary uptake from informal learning tasks. Language Learning Journal, 36, 227–237.

20.

Milton

(2009). Measuring second language vocabulary acquisition. Bristol: Multilingual Matters.

21.

Murphy

(2009). Using picture books to engage middle school students. Middle School Journal, 40, 20–24.

22.

Nation

(2001). Learning vocabulary in another language. Cambridge: Cambridge University Press.

23.

Nation

(2006). How large a vocabulary is needed for reading and listening? The Canadian Modern Language Review, 63, 59–82.

24.

Newton

(2013). Incidental vocabulary learning in classroom communication tasks. Language Teaching Research, 17, 164–187.

25.

Paribakht

T.S.

Wesche

(1993). Reading comprehension and second language development in a comprehension-based ESL program. TESL Canada Journal, 11, 9–29.

26.

Paribakht

T.S.

Wesche

(1997). Vocabulary enhancement activities and reading for meaning in second language vocabulary development. In: Coady

Huckin

(Eds.), Second language vocabulary acquisition: A rationale for pedagogy (pp. 174–200). New York: Cambridge University Press.

27.

Polette

(1989). Using ABC books for vocabulary development in the secondary school. English Journal, 78, 78–80.

28.

Rassaei

(2016). Effects of three forms of reading-based output activity on L2 vocabulary learning. Language Teaching Research. DOI: 10.1177/1362168815606160

29.

Rott

Williams

Cameron

(2002). The effect of multiple-choice L1 glosses and input-output cycles on lexical acquisition and retention. Language Teaching Research, 6, 183–222.

30.

Schmitt

(2008). Instructed second language vocabulary learning. Language Teaching Research, 12, 329–363.

31.

Sonbul

Schmitt

(2010). Direct teaching of vocabulary after reading: Is it worth the effort? ELT Journal, 64, 253–260.

32.

Sun

C.H.

(2015). The three little wolves go to college: A picture book lesson for young adult EFL learners. Journal of Adolescent and Adult Literacy, 59, 183–195.

33.

Waring

Nation

(2004). Second language reading and incidental vocabulary learning. In: Albrechtsen

Haastrup

Henriksen

(Eds.), Angles on the English-speaking world: Writing and vocabulary in foreign language acquisition (pp. 97–110). Copenhagen: Museum Tusculanum Press, University of Copenhagen.

34.

Waring

Takaki

(2003). At what rate do learners learn and retain new vocabulary from reading a graded reader? Reading in a Foreign Language, 15, 130–163.

35.

Yang

Y.F.

(2002). Developing vocabulary size and analysis test for identifying TVES college students’ reading levels. Proceedings of the 19th Conference on English Teaching and Learning (pp. 489–499). Taipei: Crane.

36.

Zambo

(2005). Using the picture book ‘Thank you, Mr. Falker’ to understand struggling readers. Journal of Adolescent and Adult Literacy, 48, 502–512.

37.

Rubin

Salmieri

(2008). Those Darn Squirrels! New York: Clarion.

38.

Trivizas

Oxenbury

(1993). The Three Little Wolves and the Big Bad Pig. New York: Aladdin.

39.

Ungerer

(2008). The Three Robbers. New York: Phaidon.

	Acquisition	Retention
The Three Little Wolves and the Big Bad Pig	PRO < PRVI (t = −11.14)PRO < PRCOA (t = −4.83)PRVI > PRCOA (t = 5.72**)	PRO = PRVI (t = −.41, p = .683)PRO < PRCOA (t = −4.08)PRVI < PRCOA (t = −4.46)
The Three Robbers	PRO < PRVI (t = −10.86)PRO < PRCOA (t = −6.03)PRVI > PRCOA (t = 4.61**)	PRO = PRVI (t = −.09, p = .926)PRO < PRCOA (t = −6.03)PRVI < PRCOA (t = −5.70)
Those Darn Squirrels!	PRO < PRVI (t = −12.40)PRO < PRCOA (t = −5.14)PRVI > PRCOA (t = 6.24**)	PRO = PRVI (t = −.84, p = .407)PRO < PRCOA (t = −4.29*)PRVI < PRCOA (t = −3.43)