Abstract
Reading comprehension is a critical skill for school success. Struggling readers can benefit from computer-assisted instruction that utilizes components of effective instruction (e.g., frequent practice, immediate feedback). The purpose of this study was to examine the effects of Headsprout Comprehension, a computer-assisted reading program, on the reading comprehension of six elementary students with high-incidence disabilities (i.e., learning disabilities, emotional disturbance, and other health impairment–attention deficit hyperactivity disorder (OHI-ADHD). A multiple baseline across participants’ design demonstrated that Headsprout Comprehension was functionally related to substantial increases in reading comprehension for all six participants as measured by Ohio Achievement Assessment (OAA) passage comprehension questions and AIMSweb Maze assessments.
Keywords
Reading comprehension is a critical skill for success in school and independent learning throughout life. Many students with mild disabilities struggle with reading comprehension because they lack the skills to read strategically (Gajria, Jitendra, Sood, & Sacks, 2007). That is, these students may have difficulty identifying important information, using background knowledge to make inferences, monitoring their understanding of the text, and using repair strategies (e.g., text look backs) when they fail to comprehend (Gajria et al., 2007). The pervasive problem of reading failure for students with disabilities is evident in the National Center for Educational Statistics (NCES; 2011) reading assessment report. Specifically, 68% of fourth graders receiving special education services scored below basic compared with 30% of students without disabilities.
Students with disabilities generally need intensive intervention using evidence-based practices to become proficient readers (National Reading Panel, 2000). Evidence-based practices for improving reading comprehension include answering literal and inferential questions, monitoring comprehension through self-questioning, completing graphic organizers such as story maps, and summarizing important information from a reading passage (Jitendra & Gajria, 2011; National Reading Panel, 2000). In addition, struggling readers need explicit instruction, guided practice with continuous feedback, and frequent opportunities to read and respond to text independently (Berkeley, Scruggs, & Mastropieri, 2010; Gardill & Jitendra, 1999; Solis et al., 2012). Computer-assisted instruction (CAI), which often incorporate these instructional components, has been demonstrated to be an effective intervention for increasing reading skills including word recognition, vocabulary, and comprehension (National Reading Panel, 2000).
High-quality computer programs enable students to learn independently by providing frequent opportunities for active responding, immediate feedback after each response, and consistent instructional delivery (e.g., Hall, Hughes, & Filbert, 2000; Macaruso & Walker, 2008; Sorrell, Bell, & McCallum, 2007). In addition, many computer programs are not only motivating to students, but they can also be easily customized for individual learning needs (Chambers, Abrami, McWhaw, & Therrien, 2001; Irasquin, Drent, & Verhoeven, 2005). For example, teachers can adjust difficulty levels, pacing of instruction, and reading passage selections based on students’ skill levels and interests.
CAI has been demonstrated to be effective for increasing reading comprehension skills of high school students with learning disabilities (Twyman & Tindal, 2006), middle school students without disabilities (Johnson-Glenberg, 2005), and first graders at risk for reading failure (Chambers et al., 2008). In a randomized control group study conducted with 144 first graders with and without disabilities, Savage, Abrami, Hipps, and Geault (2009) found that students who participated in CAI demonstrated significant immediate and long-term gains in reading comprehension and decoding skills compared with students who received balanced literacy instruction delivered by their classroom teachers. Similarly, Johnson-Glenberg (2005) found that 20 middle school students who participated in supplementary CAI were able to answer more open-ended comprehension and vocabulary questions correctly at the end of computer sessions. Although the research on CAI for reading comprehension has been promising, more research is necessary as new instructional computer programs continue to become available to struggling readers.
The purpose of this study is to extend previous research on CAI with reading comprehension by examining the effects of Headsprout Comprehension for elementary students with mild disabilities. Headsprout Comprehension was developed by the creators of Headsprout Early Reading, a computer program designed for students in kindergarten through third grade that targets phonics, phonemic awareness, vocabulary, reading comprehension, and fluency (Huffstetter, King, Onwuegbuzie, Schneider, & Powell-Smith, 2010). Headsprout Early Reading was found to be an effective intervention for improving the overall reading skills of 31 at-risk preschoolers (Huffstetter et al., 2010). In addition, Clarfield and Stoner (2005) found similar positive effects of Headsprout Early Reading for three kindergarten and first-grade students with attention deficit hyperactivity disorder (ADHD). All students showed increases in reading fluency and decreases in challenging behavior. Clarfield and Stoner concluded that the positive effects were the result of individualized, highly engaging instruction, combined with frequent response opportunities, immediate feedback, and reinforcement built into the program.
Headsprout Comprehension was designed for third through fifth graders. The design and structure of Headsprout Comprehension is similar to Headsprout Early Reading but focuses more on reading comprehension than on decoding. In addition to frequent response opportunities and immediate feedback, Headsprout Comprehension incorporates evidence-based components for improving reading comprehension. These components include embedded questions, graphic organizers, and vocabulary instruction. Headsprout Comprehension consists of 50 self-paced, online lessons. The students are taught to ask and answer four types of questions: Find the Fact (literal comprehension), Mostly About (main idea), Vocabulary (meaning derived from context), and Clue Words (inferential comprehension). Although the instructional design and procedures of Headsprout Comprehension are grounded in evidence-based practices, we could find no published research examining the effects of Headsprout Comprehension on student outcomes.
This study was designed to examine the effects of Headsprout Comprehension on the reading skills of students with high-incidence disabilities in third through fifth grade. Specifically, the experimenters asked the following research questions:
Method
Participants
The participants were 5 fifth graders and 1 third grader identified with high-incidence disabilities (i.e., learning disabilities, emotional disturbance, and ADHD). The special education teacher requested that the third grader be included in the study due to her minimal progress in reading comprehension. Table 1 shows demographic- and school-related information for each participant. All participants had Individualized Education Program (IEP) goals related to reading comprehension, attended regular classrooms for most of the school day, and received reading instruction in a resource room for at least one period per day. Data collection began after receiving Institutional Review Board’s (IRB) approval, parent consent, and student assent.
Student Characteristics.
Note. IQ Scores were not available. GLE = grade-level equivalency; OHI = other health impairment; LD = learning disability; ED = emotional disturbance.
Based on Woodcock Reading Mastery Test.
Two pre-assessments were used to identify participants: The Headsprout Test (Headsprout, 2009), which assessed reading fluency, and the Woodcock Reading Mastery Test, 3rd Edition (WRMT-III; Woodcock, 2011), which assessed word identification, word attack, word comprehension, and passage comprehension. Students who read at least 70 correct words per minute on the Headsprout Test and scored at least 2.0 grade level on The WRMT-III (i.e., overall reading ability score) were selected to participate. These pre-assessments were administered individually 3 days prior to beginning baseline data collection. See Table 1 for WRMT-III grade-level equivalencies for each participant.
Setting
The study took place in a special education resource room in an urban elementary school in Ohio where 100% of the student population qualified for free or reduced lunch. Computer practice took place during reading and language arts period in the morning or during study hall in the afternoon. The participants used one of the four resource room computers or the computers in the school’s computer lab. Eight to 12 students were present during sessions conducted in the classroom and 4 to 12 students were present during sessions conducted in the computer lab. Students who were present but not participating in the study engaged in teacher-led instruction or worked independently on the other classroom computers.
The primary data collector was a PhD student with 10 years of teaching experience. She implemented the intervention and conducted the assessments for all of the participants. In addition to formal assessment training as part of her teacher preparation program, the primary data collector had been administering AIMSweb and OAAs for several years. A second data collector, who was also a PhD student and an experienced teacher (9 years), collected interobserver agreement (IOA) data and procedural reliability data.
Definition and Measurement of Dependent Variables
Two dependent variables were examined in this study: (a) scores on OAA reading passage comprehension assessments and (b) scores on AIMSweb Maze passages. These assessments were administered in an alternating sequence on each session (i.e., Session 1, Maze first, OAA second; Session 2, OAA first, Maze second, etc.).
OAA passage comprehension assessment
OAA reading passages were either narrative or expository. Each passage was accompanied by a series of questions worth different numbers of points for a total of 13. Five of the questions were multiple choice (worth 1 point), two of the questions required a two-part answer (worth 2 points), and one question required a four-part answer (worth 4 points). For example, “Who was the main character and what did he do?” is a two-part question, and “What four important events happened in the story” is a four-part question. The fifth graders were given fourth-grade reading passages because that is the level on which they would be assessed for high-stakes testing. Questions were first selected from the questions published with each OAA passage. Then additional questions were supplemented by the primary data collector so that each assessment was consistently formatted with five 1-point questions, two 2-point questions, and one 4-point question. Answers were scored using the OAA passage keys and experimenter-created keys.
The OAA is a high-stakes standardized test that is positively correlated with the Developmental Reading Assessment, and AIMSweb curriculum-based measures.The experimenters decided to use OAA passages because of their utility for preparing students for the annual statewide, high-stakes test. In addition, the OAA passages questions were parallel to the four question types taught in Headsprout Comprehension (i.e., finding facts, main idea, vocabulary, inferences). The OAA passages used in this study were accessed from the Ohio Department of Education Released Test Materials. Selected passages were two to three pages in length with Flesch Kincaid readability grade levels of 2.4 to 4.1 (M = 3.01). The mean number of words per passage was 493 (range = 340–699).
AIMSweb Maze
The second dependent variable was the number of correct answers on AIMSweb Maze probes. AIMSweb Maze is a standardized, norm-referenced curriculum-based measurement (CBM) of reading comprehension. AIMSweb Maze passages consist of both narrative and expository text. The participants were presented with a Maze passage in which every seventh word was deleted and replaced by parentheses containing three answer choices. The participants were timed for 3 min as they read the passage silently and circled each contextually correct word choice. At the end of the 3 min, the primary data collector counted and recorded the number of correct responses using the standardized administration and scoring procedures delineated in the AIMSweb Maze Manual (Shinn & Shinn, 2002). Performance on Maze passages has been demonstrated to be a sensitive measure and predictor of student growth over time in global reading proficiency (Shin, Deno, & Espin, 2000).
Independent Variable
The independent variable was completing lessons from the Headsprout Comprehension computer program. Headsprout Comprehension consists of 50 lessons containing both narrative and expository text for students in third through fifth grade. The program provides for individualized, sequential instruction and requires mastery of each lesson before progressing to the next lesson. Each lesson focuses on learning how to answer one of the four types of common questions in reading comprehension: (a) finding facts, (b) making inferences, (c) finding the main idea, and (d) understanding vocabulary. The program also includes a motivational component that allows participants to earn virtual coins to spend on brief videos and games.
Experimental Design
A multiple baseline across participants design was used to examine the effects of Headsprout Comprehension. In a multiple baseline design, each participant begins the intervention in a staggered format after baseline data are stable. A functional relation is demonstrated if behavior change occurs only when the intervention is implemented for each participant. The two experimental conditions were baseline and intervention. For each group of three participants, the intervention was implemented with one participant at a time. Once a participant achieved an increased OAA score of 25% over his or her baseline mean, the next participant with stable baseline data began the intervention.
Procedures
Baseline
During each baseline session, the participants engaged in their typical reading instruction and then the primary data collector administered the AIMSweb Maze and OAA assessments at the end of the period. Typical reading instruction consisted of introducing a reading passage, presenting vocabulary and spelling activities, and providing opportunities such as whole group reading, partner reading, and independent reading. The reading passages used for typical instruction came from Houghton Mifflin Reading Collections basal curriculum and Houghton Mifflin Leveled Readers for third and fourth grade. When students completed their assignments, they engaged in self-selected activities that included computer games, writing center, drawing, and completing assignments from other classes.
Intervention
During intervention, the participants completed one Headsprout lesson each session, followed by completing the OAA and AIMSweb Maze assessments. Intervention sessions with Headsprout Comprehension were conducted during independent work time in the resource room on Mondays and Wednesdays and in the computer lab during study hall on Tuesdays and Thursdays. In addition to completing Headsprout lessons, the participants continued to receive the same typical reading instruction that occurred during baseline.
The procedures for the Headsprout lessons were as follows. The participants began the session by logging into www.headsprout.com. Once they logged on, the screen showed a lesson map and the number of lessons completed. The program automatically selected the next lesson in the participant’s lesson sequence and opened the first activity. Participants used mouse clicks to make responses to prompts. On each session, the program provided instruction targeting one of the four types of questions. The participants practiced reading sentences and passages with the specific question type and completed assessment activities. For example, participants received instruction on the type of question being taught (e.g., Vocabulary), practiced selecting the icon for Vocabulary, and answered brief questions related to vocabulary. Then the participants played a game in which they moved an animated fish to end of a course by selecting the correct answers to vocabulary questions. Finally, the participants completed a final review consisting of multiple-choice questions with three to four options and one correct answer.
The Headsprout Comprehension program provides built-in corrective feedback. For example, if a student selects the wrong answer to a Find the Fact question, the program erases the student’s answer with a virtual eraser, highlights the correct answer in the passage, and provides an explanation for why the answer was correct (and why the student’s answer was incorrect). Then, the program prompts the student to try the question again. If incorrect responding continues, the program provides more instruction and practice for that skill. While progressing through each Headsprout Comprehension lesson, the participants earned virtual coins for correct answers. At the end of the lesson, each participant was able to spend the coins on games or short videos built into the program. Each lesson took 15−30 min. While participants independently completed lessons, the primary data collector monitored their behavior. If any participant was off-task, she prompted the participant to continue working. Participants received verbal praise on task behavior, but no praise was given for correct responses. The number of Headsprout Comprehension lessons completed by each participant during intervention was as follows: Kent, 12 lessons; Alison, 9 lessons; Dana, 8 lessons; Erin, 13 lessons; Tracy, 15 lessons; and Travis, 9 lessons. The limited number of lessons completed was due to the time constraints at the end of the school year.
IOA
Prior to beginning data collection, a second observer received IOA training with practice passages until at least 90% agreement was attained for each measure. This criterion was met in two sessions. On 37% of baseline sessions and 34% of intervention sessions, the second data collector scored the assessments independently. The scores of both data collectors were compared item-by-item, and IOA was calculated by dividing the number of agreements by the number of agreements plus disagreements and multiplying by 100. Mean IOA was 98% (range = 83%−100%) for OAA passages and 99.5% (range = 93%−100%) for AIMSweb Maze passages. Mean IOA in each phase for each participant is shown in Table 2.
Mean IOA by Phase for Each Participant for Each Measure.
Note. IOA = interobserver agreement; OAA = Ohio Achievement Assessment.
Procedural Reliability
Procedural reliability data were collected on the assessment procedures used during baseline and on the assessment and intervention procedures used during intervention. Procedural reliability was measured across 26% of baseline assessment sessions and 25% of intervention sessions. Prior to the beginning of data collection, the primary data collector trained the observer by modeling each step while the observed completed the checklist. The procedural reliability checklists included steps for adhering to the procedures for administering the assessments (AIMSweb, OAA passage questions) and for giving the participants instructions for engaging in Headsprout Comprehension sessions. All sessions were audio-recorded on a computer or iPod. The observer listened to the recordings and used a checklist to score whether or not each procedural step was performed correctly and in the correct sequence.
Procedural reliability was calculated by dividing the number of steps completed correctly by the total number of steps and multiplying by 100. Mean procedural reliability for all participants was 100% for baseline and 96% (range = 90%−100%) for intervention. During Headsprout intervention sessions, mean procedural reliability was 98% for the assessment steps and 99% for the intervention steps. Mean procedural reliability for participants was as follows: Allison, 97%; Erin, 99%; Dana, 100%; Tracy, 98%; Kent, 99%; and Travis, 95%. Procedural reliability was not collected for the typical reading instruction in place during the study.
Social Validity
At the end of the study, the primary data collector administered an opinion questionnaire in which the participants ranked aspects of the program from one to five as follows: (a) I did not like it at all/it was very hard, (b) It was a little hard/I did not like some of it, (c) I did not like it/I disliked it, (d) I liked it, and (e) I loved it. In addition, participants were asked the following open-ended questions about the intervention: (a) What is the best part of the program? (b) What is the worst part of the program? (c) What would you tell a friend about the program? At the end of the study, the primary data collector also interviewed the special education teacher, the general education teacher, and the literacy teacher at the end of the study.
Results
OAA Passages
Figure 1 shows the number of points scored on the OAA passage assessment, and Table 2 shows the mean scores in each condition. Participants completed 8 to 15 lessons. All participants produced a similar pattern of responding. Baseline OAA assessment data were relatively stable for each participant with mean scores of 1.4 (Tracy) to 4.2 (Travis). Allison’s baseline data showed a downward trend in baseline, but no upward trends occurred in baseline for any participant. During intervention, all participants demonstrated an immediate and continuous upward trend ending with 9 to 11 points (out of 13) by the end of the intervention phase. The slope of the upward trend in intervention ranged between 0.55 (Kent) and 0.73 (Alison). Kent showed the most variability during intervention. Gains in mean intervention scores over baseline ranged between 3.3 points (Travis) and 6.2 points (Alison and Tracy).
Ohio Achievement Assessment passage scores for each student in each session.
AIMSweb Maze
Figure 2 shows the number of correct responses on AIMSweb Maze, and Table 2 shows mean scores in each condition. Baseline responding demonstrated relatively stable data for most of the participants, with Erin and Kent showing the most variability. The mean number of correct baseline responses ranged between 9.2 (Allison) and 14.0 (Erin). During intervention, all participants showed an overall upward trend ending with 16 to 22 correct responses. The slope of the upward trend in intervention ranged between 0.40 (Alison) and 1.43 (Travis). Mean intervention responses ranged between 12.3 (Tracy) and 21.5 (Erin). Erin showed the most variable responding, but her intervention data showed no overlap with baseline. Allison showed the least amount of gain with 50% of intervention data points overlapping with baseline. She was absent on Sessions 12 and 13. When she returned, her first data points fell below baseline, but she achieved a second upward trend ending in 15 correct responses by the end of the study.
AIMSweb Maze scores for each student in each session.
Social Validity
On the questionnaire, all six participants indicated that they liked or loved the program, and five participants indicated that they loved earning the virtual coins. On the open-ended interviews, participants said the best things about Headsprout Comprehension were earning coins, becoming a better reader, passing a lesson, and working on the computer rather than doing worksheets. Among the worst features reported by the participants was having to get several questions in a row correct to pass a lesson, doing more work after the program (i.e., the assessments), and not earning coins fast enough. Participants also indicated they would tell friends that the program will help them become better readers and that they would have to read carefully to find the right answers. During the informal interview, the teachers indicated that Headsprout was an easy-to-implement intervention for reading comprehension that would help students, but they thought the OAA assessments were administered too frequently.
Discussion
This study demonstrates that CAI using Headsprout Comprehension was functionally related to substantial improvements in reading comprehension for all six participants as measured by their OAA and Maze scores. This study supports previous research documenting the positive effects of CAI on reading comprehension outcomes for first graders (Chambers et al., 2008; Savage et al., 2009), middle school students (Johnson-Glenberg, 2005), and high school students (Twyman & Tindal, 2006) and extends those results to third- through fifth-grade students with mild disabilities. In addition, this study provides evidence of the effectiveness of a CAI reading comprehension program not previously examined in the literature. Consistent with prior CAI research, aspects of the Headsprout Comprehension program that likely contributed to successful outcomes include the use of individualized, engaging, and systematic instruction that incorporated frequent response opportunities with immediate feedback for each response. These elements of instruction have been documented to be evidence-based practices for increasing achievement across a range of diverse learners acquiring and maintaining a variety of academic skills (Alber-Morgan, 2010).
The effectiveness of the program may be related, in part, to student motivation. Social validity data indicated that participants liked the program. One participant even asked if she would be able to keep doing the program at home during the summer. At the end of the study, all participants were given the login information to continue the program at home. Three participants continued using the program at home during the summer. Prior to this, access outside of experimental sessions was denied. Erin and Travis continued to access the program an average of 3 times per week for a month and passed several more lessons. Dana continued to access the program for 5 months with an average of five logins per month. This indicates that three of the six participants found the program motivating enough to access it outside of school on their own time and complete additional lessons during summer vacation. This behavior is consistent with their positive ratings on the social validity questionnaires.
The social validity of the types of assessments used is a notable strength of this study. The most important outcome measure was based on sample OAA reading passages. The OAA is a high-stakes achievement test used as an outcome measure for which Ohio public schools are held accountable for annual student progress. Considering that the participants showed substantial improvements on OAA passage comprehension, they may be more likely to perform better on the reading comprehension subtest of the OAA. Accordingly, Headsprout Comprehension may be useful for helping struggling readers become better prepared for annual high-stakes assessments.
Limitations and Future Research
The participants in this study were six elementary students with mild disabilities who attended an urban school district in Ohio. The results of this study demonstrate improved reading comprehension for all of the participants on two dependent measures, answering reading comprehension questions and completing Maze passages. Although the results of this research are promising, the findings have limited external validity due to the low number of participants who were fairly homogeneous. Future research should attempt to examine the effects of Headsprout Comprehension with different populations of students. For example, the effects of this program can be examined for students of differing age, grade, and ability levels, and for students who are culturally and linguistically diverse. In addition, future research may attempt to assess reading comprehension in other ways that may provide additional insight, such as having the student retell or paraphrase the story and counting the number of accurate details.
In this study, Headsprout Comprehension was implemented as a supplement to typical reading instruction, but procedural reliability data were only collected on the Headsprout Comprehension and assessment procedures. Procedural reliability data were not collected on typical classroom instruction for which the Headsprout intervention was a supplement. During the Headsprout intervention, however, the students continued to also participate in their regular reading instruction. So, the results of this study may represent the combined effects of regular instruction plus Headsprout. Despite the fact that Headsprout was used in addition to typical instruction (that was also present in baseline), the data show that each student’s performance changed when and only when the Headsprout intervention began. So, there is evidence that typical instruction alone in this particular classroom had minimal influence on the outcome measures. Future research should attempt to systematically examine the components of ongoing instruction and compare it with CAI. Future research may also be enhanced by examining the effects of the Headsprout intervention in combination with other evidence-based practices such as Direct Instruction reading programs or Classwide Peer Tutoring systems.
Another limitation of this study is that the experimenters were unable to control for each participant’s exposure to reading instruction outside of the resource room. Differing amounts of experience with various passage topics may explain some of the variability of reading comprehension outcomes. Students may score higher on comprehension assessments for which they have more background knowledge. In addition, some participants may have increased their performance more quickly due to the possible influences of instruction in other settings (i.e., regular classroom or home). Future research may attempt to control for exposure to materials outside of the experimental setting to determine if outcomes are truly the result of the intervention. For example, experimenters can examine the grade-level curriculum and attempt to exclude those topics from selected reading materials and assessments.
A limitation due to time constraints is that the participants were only able to complete 8 to 15 of the 50 program lessons due to the end of the school year. Continued data collection throughout the 50 lessons may have produced additional information about the program’s effects, such as the extent to which students generalize reading comprehension skills or implement comprehension strategies independently. Future research should examine the effects of completing all 50 lessons and assess the effects of completing Headsprout Comprehension on generalization and maintenance of reading skills.
Future research would also be enhanced by the follow-up data on student reading performance. In this study, three of the participants continued to access Headsprout Comprehension during the summer upon completion of data collection, but the experimenters were unable to continue administering the assessments. As the computer program automatically keeps track of when students complete the lessons and how much time they spend on them, future research should examine the effects of allowing students to access the program at home.
Implications for Practitioners
Headsprout Comprehension was an effective intervention for increasing the reading comprehension skills of students with high-incidence disabilities. An important benefit of CAI is that it can be customized to meet individual learning needs, enabling students of varying ability levels to experience academic success. Clarfield and Stoner (2005) found that computer-based reading instruction is effective because it provides opportunities for individualized, highly engaging instruction, combined with frequent response opportunities, and reinforcement built into the program. Customized differentiated instruction for individual students is a necessary component of effective education in inclusive classrooms.
CAI can also provide motivating ways to help students practice important skills independently. In addition, the positive effects on OAA passage scores indicate that Headsprout Comprehension has the potential to help students improve their high-stakes testing scores, a goal of many schools and districts. Headsprout Comprehension is not a complete reading program and should only be used as a supplement to structured evidence-based reading practices. Teachers can implement Headsprout Comprehension during independent practice sessions or small group instruction. Students can independently log in to the computer at school or home and receive additional reading comprehension practice. Supplementing evidence-based reading instruction with Headsprout Comprehension on a large scale has the potential to help remediate the pervasive problem of reading failure in U.S. education.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.