Abstract
This study investigates the effectiveness of online versus paper-based reading strategy instruction on EFL learners’ reading comprehension. Fifty-seven university students from two intact reading classes with comparable proficiency levels were assigned to participate in either a paper-based or an online reading instructional module. Both groups received comparable reading strategy instruction in four areas: Global (keyword list, prediction, overview, and text structure), Problem-solving (reading rate, listening, summarizing, and concept mapping), Support (dictionary, grammar book, translator, highlight, and record book), and Socio-affective (music and peer interaction) strategies. The only difference was the medium: the instruction was paper-based in one class and computer-based in the other class. After each group’s four-hour training and reading period, students’ reading comprehension was measured using a written recall and further analyzed with a weighted propositional system. Results showed that the online reading group outperformed the paper-based group on overall reading comprehension. An interview session provided insights into students’ perceptions of the online module. Findings of this study can encourage teachers to incorporate web-based reading strategy training into their curriculum.
Keywords
Introduction
The ability to read fluently is a hallmark of successful language learning. An awareness of this crucial skill has sparked researchers’ interest in understanding complex reading processes. Researchers hope that findings on strategy use can facilitate and enhance learners’ reading comprehension (Sheorey and Mokhtari, 2008). Research has shown, for example, that reading becomes a more active process when readers use their prior knowledge (Carrell, 1987) and cultural background to anticipate the ideas in a text (Steffensen and Joag-Dev, 1984). Researchers have also examined the strategies that skilled readers use and the conditions under which they use these strategies (Block, 1992; Devine, 1993; Kern, 1989). This area of research is important for L2 readers because effective reading strategies will result in improved reading comprehension (Sheorey and Mokhtari, 2001).
Modern technology can support strategy-based instruction in several ways: computer-assisted concept mapping clarifies text structures (Liu et al., 2010); automatic differentiations of text difficulty facilitates skill development (Huang and Liou, 2007); reading rate programs train readers to improve their reading fluency (Freese, 1997); highlighting accelerates language acquisition (Ridder, 2002); and computer-mediated feedback enhances interactive reading (Murphy, 2010). Most studies focus on one or two features; relatively few integrate multiple online strategies. Because strategy instruction can improve reading comprehension and texts are moving from paper to digital formats, this question arises: Is online reading strategy instruction more effective than paper-based instruction for improving reading comprehension?
Research Background
Second Language Reading Strategies
Reading requires comprehension. Readers use a wide variety of strategies to acquire, understand, store, and retrieve the information they read. Pritchard (1990: 275) defines a reading strategy as ‘a deliberate action that readers take voluntarily to develop an understanding of what they read’. Grabe (1991: 379) identifies such reading strategies as ‘adjusting reading speed, skimming ahead, considering titles, headings, pictures and text information, anticipating information to come, and so on’. In L1 settings, reading comprehension instruction involves strategic reading development (Grabe, 2004). The general agreement among L1 researchers is that teaching a repertoire of reading strategies directly – through modeling and scaffolding – contributes to better reading comprehension (Pressley, 2002). L2 reading research has also focused on the development of strategic interactions with the text (Grabe, 2004). Strategies such as the Cognitive Academic Language Learning Approach, or CALLA, proposed by Chamot and O’Malley (1994), and Strategy-Based Instruction, or SBI, advocated by Cohen (1998), have the potential to enhance EFL academic reading curricula (Grabe, 2004).
Researchers who investigate reading of L1 and L2 learners have documented the important role of metacognitive awareness in reading comprehension (Kern, 1989; Mokhtari and Reichard, 2002; Pressley, 2002; Pressley and Afflerbach, 1995). Skillful readers constantly exercise their strategic awareness and monitor their comprehension during the reading process. Commonly used strategies include inferencing, identifying important information in the text, monitoring comprehension, summarizing, generating questions, and evaluating one’s understanding (Pressley and Afflerbach, 1995). Pressley (2002) breaks the text comprehension process down into before-reading strategies (setting a purpose, skimming the text to get an overview, and activating prior knowledge); during-reading strategies (taking notes, rereading selectively to make sure the text is understood, evaluating information, and guessing the meaning of words); and after-reading strategies (summarizing main ideas and discussing the text with others). O’Malley and Chamot (1990) suggest three categories of language learning strategies: cognitive (manipulating or transforming the material to be learned), metacognitive (thinking about the learning process), and socio-affective (working cooperatively or asking questions for interaction and affective control). Oxford (1990) separated learning strategies into two broad categories: direct and indirect strategies. Direct strategies involve mental processing of the target language, such as cognitive, memory and compensation strategies, while indirect strategies support and manage language learning, such as metacognitive, affective, and social strategies.
The concept of affective strategies in Oxford’s model (1990) was more specifically defined as strategies for lowering students’ affective filter, paying attention to students’ emotions, and promoting self-encouragement. Oxford (1990) suggested that listening to music was an effective way to release tensions during the reading process. Although these strategies are defined or categorized somewhat differently by each scholar, they are all designed to help students actively engage in reading.
As the awareness and use of reading strategies have increasingly been identified as indicators of good reading comprehension, more emphasis has been placed on helping students become strategic readers (Mokhtari et al., 2008). This in turn has prompted researchers to design instruments to measure students’ use of reading strategies while reading. The first such instrument, used widely around the world for assessing students’ reading strategy use, is the Metacognitive Awareness of Reading Strategies Inventory, known as MARSI (Mokhtari and Reichard, 2002; Mokhtari et al., 2008). This 30-item inventory is designed to measure students’ strategy use in reading academic materials. Strategies are classified into three broad categories: (1) global reading strategies used to set the stage for reading (setting the purpose for reading, previewing, and predicting the text); (2) problem-solving strategies used to deal with reading problems in understanding the text (checking one’s understanding when encountering conflicting information, re-reading for understanding); and (3) support reading strategies that provide tools to sustain reading (dictionaries and other support systems). These three categories of strategies help researchers identify what type of strategy students are using while reading.
The second instrument, the Survey of Reading Strategies, known as SORS (Mokhtari and Sheorey, 2002), is similar to MARSI but specifically designed to help ESL teachers increase their students’ metacognitive awareness. SORS measures the frequency and types of reading strategies used by L2 adolescent and adult students as they read academic texts. SORS also groups its reading strategies into three categories: global strategies are strategies that learners planned carefully to monitor their reading; problem-solving strategies are the actions that readers use while they are dealing directly with the text, including guessing the meaning of unknown words and visualizing information read; and support strategies are the strategies that readers adopt to facilitate comprehension, such as looking up new words in a dictionary, taking notes, and highlighting the text (Mokhtari and Sheorey, 2002; Mokhtari et al., 2008). This inventory helped students increase their awareness of strategies for better reading comprehension.
Technology and Second Language Reading Strategies
Grabe (2004: 46) developed 10 instructional strategies for second language reading: (1) ensure word recognition fluency, (2) emphasize vocabulary learning and create a vocabulary-rich environment, (3) activate background knowledge in appropriate ways, (4) ensure effective language knowledge and general comprehension skills, (5) teach text structures and discourse organization, (6) promote the strategic reader rather than teach individual strategies, (7) build reading fluency and rate, (8) promote extensive reading, (9) develop intrinsic motivation for reading, (10) plan a coherent curriculum for student learning.
Several features of modern technology can support these instructional principles: computer-assisted concept mapping (Liu et al., 2010) for teaching text structures; the automatic differentiation of the difficulty level of reading materials within an extensive reading program for enhancing vocabulary acquisition and developing strategic reading skills (Huang and Liou, 2007); reading rate training programs for training reading fluency (Culver, 1991; Freese, 1997); highlighting for developing linguistic knowledge (Ridder, 2002); and computer-mediated feedback for interactive reading (Murphy, 2010).
Liu et al. (2010) investigated the influence of a computer-assisted concept mapping learning strategy on the English reading comprehension of EFL college learners. They also investigated whether computer-assisted concept mapping strategies affected students’ strategy use. Drawn from four EFL classes, the 192 first-year university students who participated in this study were divided into an experimental group and a control group. Before beginning the study, the researchers had all students take a pre-reading test and a pre-strategy use questionnaire and then used an independent-samples t test to ensure that there were no significant differences between the two groups in terms of reading proficiencies and strategy use. The pre-reading test scores indicated that the two groups had similar populations of above-average and poor readers. The reading strategy instruction for the experimental group involved teaching computer-assisted concept mapping, whereas the instruction for the control group used traditional approaches, such as vocabulary and grammar. The results showed that the poor readers benefited more than the above-average readers did from computer-assisted concept mapping in that it improved their reading comprehension more substantially. Also, students who used computer-assisted concept mapping were more likely to use such reading strategies as listing, reinforcing, and reviewing.
Huang and Liou (2007) incorporated textual gloss, highlighting, bilingual concordances, and word lists into an online extensive reading program whose aim was to develop students’ vocabulary acquisition. The reading materials were from Sinorama, an electronic corpus of more than 5000 texts that introduce cultural features of Taiwan. The reading program integrated four different word lists into the program’s vocabulary design. Two lists were familiar: the General Service List, which covers the 2000 most basic English word families, and the High School Frequent Word List, which is used to write high school English textbooks in Taiwan. The other two word lists targeted academic vocabulary (the University Word List) and words that are frequently used in Sinorama texts but do not appear on the other three lists (the Sinorama High-Frequency Word List). The researchers then sequenced the reading texts by sorting the texts with the highest number of familiar words and the lowest number of target words as easiest and therefore to be read first. They then highlighted the target words in red and provided the translations of these Chinese equivalents within the text as a device for getting the readers’ attention. The words that were highlighted in red when they first appeared were subsequently highlighted in green without the Chinese translation. These typographical enhancements were aimed at focusing learners’ attention on these new words. Over a 12-week span, pre- and post-tests evaluated the vocabulary acquisition of 38 college EFL students and found that the learners’ vocabulary scores had improved significantly. Such gains confirm the benefits of an extensive reading program with vocabulary enhancement features that include a corpus-based syllabus and the sequencing of readings according to different word lists to accommodate learners’ individual needs.
Freese (1997) drew upon reading rate and comprehension research and suggested how computer-assisted software can be used to facilitate reading comprehension. She suggested that software should accommodate the various strategies or processes that students use during reading and should also include drill and practice to develop automatic recognition skills and flexibility for better comprehension. Reading rate training provides students with feedback on how fast they read and what strategies they use when they read articles of different difficulty levels. With this feedback, students can modify their reading rate and shift their reading strategies according to the reading materials and their purposes for reading.
Likewise, Culver (1991) implemented a Reading Mastery Program in an ESL class of 20 students and examined the effectiveness of this reading program on reading comprehension and speed. This program included the following five modules: (1) listing difficult vocabulary with definitions and examples before presenting the text; (2) presenting the reading with a skimming mode in which students could set different skimming speeds on the computer (slow, medium, fast or very fast) to practice reading speed training; (3) providing comprehension questions to develop critical reading skills, such as identifying problem/solution or cause/effect and making inferences; (4) providing cloze tests to assess students’ comprehension levels; and (5) offering interactive components for students to choose different possible outcomes of the story. The results showed that this program improved students’ reading comprehension and vocabulary and increased their reading rate. The findings confirmed the benefits of computers in improving students’ reading performance.
Ridder (2002) investigated whether highlighting and using hyperlinks on glossed vocabulary in electronic texts encouraged students’ willingness to consult dictionaries and led to better reading comprehension. Sixty second-year university students who were native speakers of Dutch participated in this study. They read two electronic versions of French economics texts that were of comparable length and difficulty and whose glosses provided French definitions and Dutch translations. The only difference between the two texts was that one text hyperlinked and underlined the glossed words in blue font, whereas the other text did not hyperlink or use any typographical enhancement to indicate the glossed words. Students’ reading comprehension skills were measured using multiple-choice questions, written recall, and a search-and-find task in which answers could be found in the text or by combining information from several paragraphs. The results showed that the blue font and underlining encouraged clicking behaviors. Highlighting and indicating glossed words in blue font increased students’ willingness to consult dictionary definitions. As for whether intense clicking of the highlighted vocabulary contributed to better text comprehension, the study did not find any significant effect on the multiple choice question results or the free recall, but the typographical enhancements clearly had a positive effect on the results of the search-and-find task.
Murphy (2010) investigated whether computer-mediated feedback for a web-based reading comprehension exercise improved reading comprehension and promoted quality interaction. A total of 425 first-year university English majors in EFL classes in Japan participated in this study. They received either (1) correct answers to comprehension questions or (2) computer-mediated elaborated feedback in which students were provided with hints, such as where to find the answers in the text, or rephrased questions to encourage students to retry and find answers on their own. If students still could not find the answers, they could interact and collaborate with their peers to read through the passages using computer-mediated tools. The results showed that students in the module with computer-mediated elaborated feedback achieved better reading comprehension.
The above studies demonstrate how computer technology can facilitate reading comprehension with the help of rate-building lessons, electronic glossing, computer-assisted concept mapping, and computer-mediated communication. This study seeks to investigate whether computer-based instruction is more effective than paper-based instruction in the college EFL teaching context.
Methodology
Participants
This quasi-experimental design involved 57 first-year college students (38 males and 19 females) from two intact classes of an EFL reading course at a national university in Taiwan. Students were randomly assigned to Class A, which learned online reading strategies (n=32), and Class B, which learned paper-reading strategies (n=25). The online group (M=517.34, SD=22.90) and the paper group (M=511.40, SD=29.33) had comparable proficiency levels, confirmed by a sample TOEIC test administered prior to the experiment (t = 0.16; p = 0.87).
Materials
To reduce instructional variables, both modules were designed and implemented by the researcher as instructor. Both groups read an article on Aquaculture (801 words; 12.1 Flesch-Kincaid grade level), adapted from http://www.northernaquafarms.com/aquaculture/index.html. The article was chosen because its length was suitable for a two-hour experiment, the intermediate difficulty level was appropriate for these students’ proficiency level, and the topic was relevant to the students’ majors.
Design
During the four-hour experiment, the two classes received the same instructions. The researcher divided the strategy activities into three stages: before-reading (30 minutes), during-reading (90 minutes), and after-reading (60 minutes). Each stage included different components of Global, Problem-solving, Support, and Socio-affective strategies. The remaining hour allowed students to navigate recursively among the three stages of reading strategy instruction. The only difference was the format of the paper versus online formats.
The Online Reading Strategy Module
To facilitate online instruction, activities were packaged in the researcher’s new web-based reading strategy program (http://eng.iem.sju.edu.tw). Its interfaces and design were significant improvements over a 2009 version. In the before-reading stage, students became familiar with online Global strategy activities: (1) the Important Word List (L1 definitions and L2 translations of the text’s key words); (2) Guess What (pictures and questions inviting predictions of the text’s content and relating it to their personal experiences); (3) Overview (a preview of the text to arouse interest); and (4) Text Structure (a hyperlinked outline of the text’s organization). These warm-up reading activities – making predictions about what the text would say and getting an overview of the text’s content and structure – helped students make a reading plan before they started reading.
In the during-reading stage, students became familiar with Problem-solving, Support, and Socio-affective strategies. The Problem-solving activities were (5) Reading Rate (selecting a reading speed and setting a clock) and (6) Read for Me (text-to-speech software). The Support activities were (7) Dictionary (links to online bilingual or monolingual dictionaries, such as Google or Merriam-Webster online dictionaries); (8) Grammar Book (links to online grammar websites); (9) Translator (online machine translation sites, such as Google Translation http://translate.google.com.tw); (10) Highlight (a digital annotation tool to mark unknown words and take notes); and (11) Record Book (archived Highlight notes). The Socio-affective activity was (12) Music (links to online radio and music programs, such as the BBC at http://www.bbc.co.uk/radio/). Students could listen to the music to lower their tension level and their affective filter during the process of reading. In this stage, students were actively engaged with the text using a combination of strategies to comprehend the text.
The after-reading stage included Problem-solving and Socio-affective strategies. The Problem-solving activities were (13) Summarizing (an online text-summarizer) and (14) Concept Map (online mind-mapping software, such as Inspiration at http://www.inspiration.com/ for drawing semantic maps to visualize the information in the text). The Socio-affective strategy activity was (15) Chatroom (opportunities to discuss the text with peers).
These before-reading, during-reading, and after-reading strategies provided rich support for reading texts. As we shall see, the Online and the Paper-based reading modules both followed the same three stages and used the same 15 strategies and in a parallel sequence. The only difference was the medium. The parallel processes for both modules are shown in Appendix 1.
The Paper-based Reading Strategy Module
The Paper-based Reading Strategy Module followed the same stages and processes as the Online Reading Strategy Module. In the before-reading stage, students became familiar with Global strategies through paper handouts that provided: (1) a list of important keywords, (2) pictures and several prediction questions that also related the topic of the text to their own experience, (3) an overview of the storyline, and (4) outlines of text structures that were identical to those provided to the web-based reading group.
In the during-reading stage, students’ Problem-solving strategy activities involved (5) reading rate training activities and (6) CDs of the text being read aloud. For the Support activities, students could (7) use paper dictionaries to look up unknown words, (8) consult grammar books for grammatical problems, (9) read translated texts, (10) highlight or underline the words they wanted to annotate, and (11) keep notes in a paper notebook. For the Socio-affective strategies, students were able to (12) listen to music prepared by the instructor. In the after-reading stage, students became familiar with Problem-solving strategies through (13) paper summaries of the text and (14) instructions on drawing semantic maps using pencil and paper. Socio-affective strategy activities enabled students to (15) ask their peers questions and to discuss the text with them in class. A sample handout is shown in Appendix 2.
Data Collection and Analysis Procedures
At the end of the four-hour instruction period, both groups completed a written recall in either L1 or L2 as a measure of reading comprehension. The researcher adopted Johnson’s (1970) weighted propositional system to score recalls according to each pausal unit’s relevance to the message of the text. Two raters marked the boundaries of the idea units in the reading and ranked them into main ideas (5 points), supporting ideas (3 points), or details (1 point). Both agreed that this article had 11 main ideas, 16 supporting ideas, and 25 details, with a total score of 128. 1 The inter-rater reliability was a Pearson coefficient (r) of .93. The instructor as researcher and another experienced EFL teacher scored the 57 written recalls independently and reached an inter-rater reliability of 0.89.
In addition to students’ written recall, a focus group interview was conducted with eight volunteers from the online reading strategy group to supplement the reading comprehension results. Although both groups’ written recalls are the focus of this study, the interview session was conducted to solicit students’ perceptions of this new medium of instruction. The focus group interview session lasted for 40 minutes and was conducted by a research assistant in L1 to avoid students’ frustration with not being able to express themselves adequately in L2. At the beginning of the interview, the interviewer informed students that the purpose of this interview was to gather feedback and assured them that their comments were confidential. Students were also asked to express the ideas freely without being worried that their responses might interfere with their semester grade. The interview comments were transcribed verbatim and coded according to students’ perceptions of online instruction.
Results and Discussion
The independent-samples t test results indicated significant differences between the total scores of Group A, the online reading strategy group, and Group B, the paper reading group (t (55)=2.56; p<0.05). Table 1 shows that the total written recall scores of the online reading group (M=18.04, SD=10.60) were better than those of the paper reading group (M=11.84, SD= 6.60).
T-test of the Total Written Recall Scores for the Two Groups.
Note: *p<.05.
Likewise, the online reading strategy group (M=2.56, SD=1.48) identified main ideas more proficiently than the paper reading group did (M=1.52, SD= 1.01) (see Table 2). Even in identifying details, the online group achieved higher scores (M=2.88, SD=2.44) than the paper-reading group did (M=1.28, SD= 1.20) (see Table 3). The two groups’ performance on written recalls indicate that the group that underwent the web-based reading strategy training module showed greater comprehension of the text than the paper-reading group did in terms of identifying main ideas (t (55)= 3.02; p<0.01) and details (t (55)= 2.98; p<0.01). The interactive features, easy retrieval, and flexibility of the online strategy functions appear to have facilitated reading development among this small sample of EFL learners (n=32). The value of the supportive online reading strategy functions in this study confirms the positive findings of previous studies on the use of multimedia support in online reading environments (Chang, 2005; Chen, 2011; Chun and Plass, 1996; Huang and Liou, 2007; Murphy, 2010).
T-test of the Main Idea Recall Scores for the Two Groups.
Note. **p<.01.
T-test of the Detail Recall Scores for the Two Groups.
Note. **p<.01.
The higher standard deviation of the online reading groups’ comprehension scores compared to that of the paper reading group deserves further investigation. Although the overall performance of the online group was better than the other group, it appears that online learning may not be effective for all students/learners in the same way. Future studies should look into more factors relating to individual differences, such as learning styles or gender.
The qualitative data from the interview session also revealed students’ positive assessment of the online reading support. The most positive feedback from students was that online dictionaries provided a rich resource for vocabulary acquisition. Seven out of eight students (87.55%) expressed satisfaction with the easily accessible online dictionary function and its bilingual and monolingual dictionaries, encyclopedias, and bilingual concordances, a finding consistent with previous research (Huang and Liou, 2007). Students liked being able to find unknown words from multiple online sources and to keep an electronic notebook for future use. This easy-to-retrieve dictionary feature can facilitate the reading process for EFL learners, whose lack of vocabulary poses challenges for fluent reading.
The second most positive feedback on this online reading module was that the online highlighting function facilitated incidental vocabulary learning. Six out of eight students (75%) also indicated that the color-coded highlight function helped them track the parts worthy of attention and to re-read for understanding. This response is consistent with Ridder’s (2002) finding that highlighted or visible hyperlinks increase students’ willingness to consult electronic dictionaries. When new words are clearly marked and more glosses are consulted, incidental vocabulary acquisition is more likely to take place. For example, one student commented: I like to annotate the text using the color-coded highlight function. The words I annotated were easily archived in my personal notebook in this online reading strategy program. These notes helped me to review the parts for better comprehension (Student #3).
In addition to Support strategies, six out of eight students (75%) also appreciated the online reading rate function, one of the Problem-solving strategies, which allowed them to use a flexible rate selection mechanism to choose and develop their reading speed. This response confirms the findings of previous studies that computer-assisted reading programs increase reading speed and comprehension by allowing students to choose the rate at which to skim passages (Freese, 1997).
As for socio-affective strategies, the interview data also showed that the music and discussion functions lowered affective filters and increased social learning. Five out of eight students (62.5%) liked the music and discussion board in the online Socio-affective module because these functions lowered their level of anxiety and increased quality collaboration and interaction in real time. They felt they could ask questions more freely and in ways that could not be achieved as easily or as successfully in the paper-based module. This highlights the importance of being able to cultivate students’ ability to ask strategic questions to improve reading comprehension, which ties in with results from previous studies (Murphy, 2010).
Half of the students (50%) felt that computer-based concept mapping helped organize the text hierarchically. Four out of eight students perceived the computer-based concept mapping as helpful in allowing them to display their interpretation of the text in a hierarchical order more easily than they could in a paper-based context. One student commented: The concept mapping tool in this online reading strategy program helped me collect my thoughts after reading the article. It somehow helped me remember the text’s content better and to write up a better recall because I had drawn the major ideas using this mapping tool (Student #7).
Although the overall satisfaction with online instruction was positive, the feedback was not complaint-free. Two out of the eight students (25%) expressed dissatisfaction with the length of time and degree of effort required to become familiar with the online reading modules. Providing students with extra support and plenty of time to adapt to new online strategies might help to make it less stressful for students unfamiliar with online features.
Conclusion
This study shows that it is possible to use technology to design useful strategy functions that aid reading comprehension and that providing second language learners with more strategic choices and opportunities for reading authentic materials will improve their reading comprehension. The online reading strategy module has been shown to be more effective than the paper reading module in helping this small sample of EFL learners read and understand an intermediate level text.
Promising as these results are, this study has some limitations. The online group’s better comprehension scores may be sufficient for making a preliminary claim about the effectiveness of this online reading strategy instructional module. However, more qualitative data, such as observations, interviews with all of the students, and sophisticated tracking of students’ actual strategy use can help us understand why the online medium has better outcomes and why some features of online reading strategies appear to be more helpful than others. The findings of this study are applicable only to this small sample of EFL learners. Future studies should include larger samples over a longer period and recruit students of different language backgrounds so that the results can be more transferable. In addition, researchers can also lengthen the sessions so that the patterns of students’ strategy use can be observed in greater detail.
In conclusion, given recent advances in technology, the prospects for using computer technology to improve reading comprehension both inside and outside classrooms are promising. The results of this study serve to encourage more EFL teachers to engage their students in web-based reading strategy training. In this way, they can improve their students’ reading proficiency and help them to achieve the literacy skills needed in the digital age.
Footnotes
Appendices
Acknowledgements
The author would also like to thank the reviewers for their insightful comments for improving this paper.
Funding
This study was supported by the National Science Council, Taiwan, ROC, Project No. NSC 96-2411-H-129-001 and Project No. NSC 101-2410-H-019-015.
1.
The first coding step was to identify the idea units of the texts. Both raters marked the boundaries of idea units and ranked them into main ideas (5 points), supporting ideas (3 points), and details (1 point). For example, the raters agreed that the first paragraph of this reading text had two main ideas: (1) Aquaculture is an industry and (2) Aquaculture includes the cultivation of aquatic species of plants or animals. It has one supporting idea, (1) Aquaculture includes the cultivation of aquatic species of plants or animals in controlled systems, and three details: (1) commercial, (2) recreational, and (3) resource management purposes. If one student’s recall reads as, ‘Aquaculture is an industry for commercial purposes’, the recall raters will mark the boundaries of this student’s recall against the original scheme. This student’s recall maps 1 main idea (5 points: Aquaculture is an industry) and 1 detail (1 point: for commercial purposes) and therefore receives a total of 6 points.