Effects of TWA-Supported Digitally on Comprehension of Students With Autism Spectrum Disorder,Level 1

Abstract

Presenting text in digital format with annotation supports may relieve some of the cognitive load that hinders inferential comprehension for students with comprehension deficits. Science texts are particularly difficult, as the content may not be within the knowledge repertoire of a reader. The purpose of this study was to investigate the effects of using a reading strategy, Thinking before, While and After (TWA) -digitally supported (TWA-SD), on the comprehension of science text by students with autism spectrum disorder (ASD, Level 1) participants. A concurrent multiple probe single subject research design was used to individually deliver intervention. Results indicated the intervention was effective in increasing participants’ accuracy and quality of oral retellings of main ideas and details as well as their performance on general comprehension questions. Implications regarding the use of digitally supported strategies to increase access to the curriculum for students with ASD, Level 1 are discussed.

Keywords

autism exceptionality tablets/iPad technology perspectives reading content/curriculum area middle school age/grade level universal deign for learning educational perspectives methodologies

An increasing number of students with autism spectrum disorder (ASD) are being included in content area classrooms with their peers, requiring that teachers become familiar with academic interventions to promote their success (Chiang & Lin, 2007). Individuals with ASD exhibit symptoms including social impairments (social interaction and social communication) and restricted and repetitive interests and behaviors (American Psychiatric Association, 2013); diagnostic frameworks recognize significant heterogeneity in functional levels (documented by the presence or absence of co-occurring intellectual and language impairment). One subgroup of children with ASD that has increased substantially in number is children with ASD with no accompanying intellectual impairment is ASD, Level 1. This subgroup is differentiated from others with ASD based on their strengths in cognitive and language abilities. The diagnostic criteria for ASD changed with the publication of the Diagnostic and Statistical Manual, version five (DSM-5) in 2013 (American Psychiatric Association, 2013). The DSM-5 describes three levels of increasing severity based on the levels of support required for daily functioning. The least supports are needed in ASD, Level 1, whereas the most supports are needed in ASD, Level 3. Current estimates indicate nearly half of children with ASD are those classified at Level 1 (Centers for Disease Control and Prevention, 2014). There is a growing research base focusing on evidence-based practices for individuals with ASD that may provide guidelines for general and special educators providing educational programming across a wide range of skills. Although the research is promising, results are often limited when it comes to generalization in diverse vocational and classroom settings (Sartini, Knight, Spriggs, & Allday, 2017).

Autism and Comprehension Deficits

The unique reading profiles of students with ASD, Level 1 provide a challenge to educators delivering instruction in text-rich environments such as middle school where students read to learn more than they need to learn to read. Text comprehension tends to be challenging for students with ASD, Level 1. Despite their strengths, their reading comprehension performance is often hindered by an impaired ability to draw inferences and make cause and effect connections (Estes, Rivera, Bryan, Cali, & Dawson, 2011). These reading difficulties have been documented in a number of studies. For example, Estes, Rivera, Bryan, Cali, and Dawson (2011) tested a sample of 9-year-olds with ASD and found 60% were underachieving on basic academic skills such as spelling, word reading, and number skills compared to what was predicted based on their cognitive ability. Brown, Oram-Cardy, and Johnson (2013) conducted a meta-analysis of 36 studies comparing the reading comprehension of children and adults with ASD to typically developing (TD) individuals. Their results indicate individuals with ASD performed significantly lower than average TD individuals (standardized mean difference of .70 SD). These findings did not differ based on functional level of the participants.

Making inferences while reading is often associated with perspective taking. It is often difficult for students with ASD to imagine the thoughts, feelings, and perspectives of other people. This often leads to difficulty primarily in social situations, peer relationships, and novel situations (Baron-Cohen, 2008; Lopata et al., 2013). He may also pose a problem when students with ASD are trying to determine the author’s purpose, especially implicit inferences in text (Ricketts, Jones, Happé, & Charman, 2013). Making higher order inferences from text is a skill emphasized by the Common Core State Standards for all students in language arts classrooms across the nation. Teachers often use oral retellings as a formative assessment to gauge their students’ level of understanding of science text (Moss, 2004). Students with ASD, however, may find oral retelling challenging. Explicit instruction in digital tools may scaffold and guide their retelling, thus providing students with ASD more success (Lange, McPhillips, Mulhern, & Wylie, 2006).

Despite documented comprehension deficits, there is limited research regarding reading comprehension interventions for children with ASD. A review by El Zein, Solis, Vaughn, and McCulley (2014) on reading comprehension for students with ASD found 8 of the 12 studies included at least one child with ASD, Level 1. These eight studies investigated a variety of intervention procedures. Successful intervention procedures included direct instruction, graphic organizers, and structured questioning and cueing procedures. One approach to teaching reading comprehension that incorporates direct instruction, cueing, and graphic organizers is self-regulated strategy development (SRSD). SRSD has been validated as an evidence-based practice for students with disabilities by more than 25 years of research (Graham & Harris, 2003; Mason, 2004). One specific SRSD reading strategy is the think before reading, think while reading, and think after reading strategy (TWA; Mason, 2013; Mason, Meadan, Hedin, & Corso, 2006).

TWA Strategy

This approach scaffolds students’ reading by prompting them to focus on elements of an expository text before, while, and after reading (Mason, 2013; Mason, Meadan, et al., 2006; Mason, Snyder, Sukhram, & Kedem, 2006). Mason (2004) conducted a randomized research study demonstrating the efficacy of TWA. Students who exhibited a significant gap between decoding and comprehension skills (decoding > comprehension) were randomly assigned to receive either TWA instruction or reciprocal questioning. Following procedural mastery of these strategies, the focus of instruction shifts to self-monitoring of the students’ own comprehension (Mason, 2013). Results indicated struggling readers who received TWA demonstrated significantly higher expository reading comprehension skills compared to the group who received reciprocal questioning.

While TWA yields significant gains in comprehension of expository text for struggling readers, its efficacy has only been researched using students with learning disabilities and emotional, behavioral disabilities. Its efficacy has not been validated for students with ASD. Teaching students with ASD, Level 1 to generalize self-regulated reading strategies such as self-questioning and summarizing strategies may help to reduce the cognitive load involved in the complex process of reading comprehension (Sartini et al., 2017). Given that students with ASD often exhibit difficulty with self-regulation of procedural strategies such as perspective taking, determining the main idea, and focusing on important versus irrelevant details (Baron-Cohen, 2008; Baron-Cohen, Leslie, & Frith, 1985), TWA may be an effective strategy for facilitating expository reading comprehension skills. Also, the intervention allows for the mastery of steps as part of a progression involving modeling, collaborative practice, and independent practice. Based on the characteristics of students with ASD, Level 1 and the applicability of the learning elements of TWA to this population, TWA appears to be a reading strategy with the potential to increase the expository text comprehension of this groups (Howorth, Lopata, Thomeer, & Rodgers, 2016).

Technological Tools as Support for Students

Pennington (2010) concluded computer-assisted instruction was effective for teaching limited academic skills to students with ASD. In addition, Pennington indicated students with ASD had fewer behavioral issues when they were allowed to use computer-assisted instruction. The use of computers seemed to mediate appropriate social communication behaviors by these students in the classroom setting. Several studies reviewed by Pennington involved pretraining of students with ASD on the use of the computer programs, teacher-mediated corrective feedback, and eventual independent use of the computer-assisted academic programs by students. This model of instruction follows the tenants of explicit instruction (i.e., “I do, We do, You do”) used to teach many self-regulated reading strategies such as the TWA strategy (Archer & Hughes, 2011; Mason, Snyder, et al., 2006). Research indicates the generalization of effects from computer-aided instruction (CAI) is specifically seen when CAI is used for literacy supports (Moore & Calvert, 2000; Pennington, 2010). Technological tools also have promise for students with ASD when used to teach social behavioral skills and self-monitoring skills (Mintz, 2013). Wallen, Plass, and Brünken (2005) indicate if one or two carefully chosen annotation tools were available, the cognitive load is less, and students are able to use annotation tools to increase their science text comprehension.

Presentation of text in a digital format offers visual and auditory supports to help students with ASD, Level 1 scaffold their understanding and to minimize their cognitive load (Anderson-Inman, 2009; Bernacki, Byrnes, & Cromley, 2012). Digital annotation, for example, provides students with the ability to use digital highlighting, audio and “sticky” note taking supports to scaffold their regulation of reading comprehension strategy use (Bernacki et al., 2012). Audio notes available in some digital annotation software allow students to summarize text at the sentence and paragraph level and play it back to themselves after they finish reading (Grimshaw, Dungworth, McKnight, & Morris, 2007). These high-tech tools have the potential to positively support student learning.

Bouck, Savage, Meyer, Taber-Doughty, and Hunley (2014) examined the effects of low-tech and high-tech metacognitive tools for students with ASD. They found the iPad was more effective, efficient, and preferred by participants than low-tech tools such as paper and pencil checklists. These findings suggest digital tools can promote both self-regulation and monitoring of academic tasks for students with ASD and indicate a need for further research in digital tools to promote self-regulated reading strategy use by students with ASD.

This study continues the recommendations from Bouck et al. (2014) by adding high-tech enhancements to an evidence-based strategy, TWA. The purpose of this study was to investigate the effectiveness of TWA-supported digitally (TWA-SD) as a package on the expository reading comprehension achievement of students with ASD, Level 1. Specifically, would the addition of digital audio and visual annotation supports to TWA provide sufficient supports to increase the comprehension skills of students with ASD, Level 1? This package included the use of digital annotation through highlighting, “sticky notes” and audio notes on the iPad while students read digital text.

Method

Participants

Participants were recruited by contacting the director of special education of a large urban school district in the northeastern United States. The director then recruited principals, who then recruited special education teachers whose students met the inclusion criteria. Inclusion criteria for participants were as follows: (a) diagnosed with ASD, (b) had reading comprehension goals listed in their individual education plans, and (c) had a full-scale IQ higher than 85 as measured by the Woodcock-Johnson III (WJ III; Woodcock, McGrew, & Mather, 2007). Six eligible children were identified at two different schools, this study describes the data for three who all received English language arts (ELA) instruction in the same resource room in the same school. The other three students were not included in the study.

All participants demonstrated an average level of word identification coupled with below average fluency levels according to the criteria set forth by Hasbrouck and Tindal (2006). Further examination of their records revealed a discrepancy between decoding and reading comprehension skills (decoding > comprehension). This pattern provided evidence of significant comprehension difficulty relative to decoding skills (Nation, Clarke, Wright, & Williams, 2006). Participants’ decoding skills fell in the average to significantly below-average range and comprehension skills in the average to significantly below range. Participants did not have additional intellectual disabilities, nor were classified as students with learning, emotional, or behavioral disabilities. All participants were male, 11 years of age, and in sixth grade: two were African American and one Caucasian. One participant had a speech and language disorder in addition to ASD, Level 1.

Table 1 represents the characteristics of the participants involved in this study. The classroom teacher initially contacted parents of participants and written consent was obtained before working with the children. In addition to parental consent, participant assent was obtained via researcher-led interviews and having the participants circle a smiling face to indicate assent with a “yes” response and a frowning face icon to indicate a “no” response to each question.

Table 1.

Participant Demographic Information.

Participant	Ethnicity	DIBELS-Fluency (Winter)	WJ Subtest: Comprehension	WJ Subtest: Decoding
Michael Age 11	African American	45 wpm (<10th percentile)	55 Significantly below average	78 Low average
David Age 11	African American	91 wpm (<25th percentile)	86 Average	97 Average
Jaime Age 11	Caucasian	77 wpm (<10th percentile)	56 Significantly below average	67 Significantly below average

Note. Reading comprehension and decoding scores as indicated by Woodcock-Johnson III (WJ) subtests (Woodcock et al., 2007). Fluency scores as indicated by DIBELS and benchmarks according to Hasbrouck and Tindal (2006).

Setting

After institutional review board approval, this investigation was conducted in a middle school resource room during ELA instruction. The school was located in an urban, low socioeconomic city in the northeast United States. Although teachers were present while the researcher conducted the study, they were asked not to use the intervention materials with their students until the generalization phase to ensure fidelity of strategy instruction. Baseline, intervention, and maintenance sessions took place during 1:1 sessions in the participants’ resource room with the researcher serving as instructor. Maintenance data were collected 10 days after the completion of the intervention phase in the same setting, but with novel science and social studies text passages within the participants’ instructional Lexile level.

Outcome Measures

Instructional-level reading expository science passages were selected based on the participants’ performance on the Qualitative Reading Inventory-5 (QRI-5; Leslie & Caldwell, 2011). Accuracy, sequence, discussion of main ideas, and supporting details have been previously associated with high levels of expository text comprehension (Magliano, Millis, Levinstein, & Boonthum, 2011; Mason, 2004, 2013; Mason, Meadan, et al., 2006). Therefore, an oral retell summary statement was used to measure participants’ comprehension of text passages at baseline, intervention, maintenance, and generalization phases (Carnahan & Williamson, 2013; Mason, 2004; Mason, Snyder, et al., 2006). The retell was scored according to a rubric (Table 2) including measures of accuracy, main ideas with supporting details, and prompting. Two upper elementary and one middle school writing teacher reviewed the rubric for content validity. In addition, it was shown to three doctoral-level reading experts who all agreed the description of content on the rubric was valid.

Table 2.

Oral Retell Rubric.

	4	3	2	1	Circle 1
Main Fries
Accurate in ideas and sequence	Precise gist of the passage in same exact sequence as the story Extremely accurate and discusses all the main points at length	Precise gist of the passage In roughly the same sequence as the story Mostly accurate and discusses most of the main points	Unclear explanation of main idea Unclear sequence of the story Many inaccuracies and Unclear discussion of main points	No retell of the main ideas	4 3 2 1
Details
Accurate in major and minor details	Four or more text-based details Two or more details not explicitly slated, but inferred Embellished with reader’s prior knowledge Finer points that enrich the context of the passage provided (compares to existing knowledge)	Two or more text-based details One inferred detail Some reference to reader’s prior knowledge	Minimal text-based details No inferences No reference to the text	No details given to support the main idea	4 3 2 1
Prompting
	Retell is independent—no prompting	1–2 Verbal prompts (i.e., What else?)	3–4 Verbal prompts	4+ Verbal prompts	4 3 2 1
Total					/3=

Note. Adapted from Tindal and Marston (1990); and Haworth, Lopata, Thomeer, and Rodgers (2016)

After participants completed each passage, they were asked to independently read and answer 10 reading comprehension questions during baseline, intervention, and maintenance sessions. These questions were within their instructional Lexile level and generated by a free, web-based source (https://www.readworks.org/) for expository text passages according to grade and Lexile level. For all phases of the study, topics of the texts included natural resources, rocks and minerals, volcanoes, glaciers, and natural energy sources. These topics aligned with what was being taught by the general education science teacher in the participants’ science class. These questions were used, not only because they pertained to the passages being read but because they also assessed key information about the participants’ comprehension. Specifically, 10 questions elicited information on participant understanding of (a) text structure, (b) explicit information presented in text, (c) inferential understanding, (d) main idea, (e) vocabulary (two questions), (f) sentence-level syntax questions, (g) author’s purpose, (h) text implicit information, and (i) supporting details (Andreassen & Bråten, 2011; Baker, Gersten, & Scanlon, 2002; Berninger, Vermeulen, & Abbott, 2003).

Reliability

Interobserver agreement (IOA) was 100% on 30% of baseline retells and 100% on 30% of intervention retells. IOA was obtained from both the classroom teacher and an independent doctoral-level graduate student who was not otherwise involved in the study. Training consisted of listening to an audio recording of a participant giving a retell and the researcher, classroom teacher, and graduate student independently scoring the audio retell. Training continued to 100% agreement, using point-by-point agreement.

Fidelity

Fidelity of instruction was measured according to a checklist for each lesson. Lessons were audio recorded, and doctoral-level graduate students, who were not otherwise involved in the study, listened to 33% of lessons to ensure they were accurately delivered. Two scorers were shown a video of the researcher instructing a participant in the TWA strategy and trained to collect fidelity data until all researchers reached agreement of 100%. These criteria for interobserver reliability are based on the quality indicators set forth for single subject research design (Horner et al., 2005; Kratochwill et al., 2010). Fidelity of instruction was found to be 100% on 33% of all lessons.

Design and Procedures

Design

The effect of instruction in the TWA-SD strategy was investigated using a concurrent multiple probe single subject research design (Horner & Baer, 1978; Kazdin, 2011; Kennedy, 2005; Richards, Taylor, Ramasamy, & Richards, 1999). In a concurrent multiple probe design, baseline begins at the same time for all participants; intervention and maintenance phases occur in a staggered fashion for each group of participants according to the mastery of the intervention by the previous group (Horner & Baer, 1978). Once a stable trend in responding was evident during baseline, the researcher began the TWA-SD strategy intervention. This consisted of 12 lessons delivered by the researcher until participants met mastery criteria for use of the TWA-SD strategy. Five sessions of maintenance data were collected on all participants 10 days after intervention sessions concluded (Richards et al., 1999). Generalization data were also collected on all participants by the classroom teacher on participants’ independent use of the TWA-SD strategy on novel social studies texts 10 days after maintenance sessions concluded.

Intervention

All participants received explicit instruction in the three steps to thinking before reading (T) phase: (a) think about the author’s purpose, (b) think about what you know, and (c) think about what you want to know. They received explicit instruction in the three steps to thinking while reading (W) phase: (a) think about your reading speed, (b) look for connections to what you know, and (c) reread to understand. Finally, participants were instructed in the three steps to thinking after you read (A): (a) think about the main idea, (b) summarize information, and (c) think about what you learned (Mason, 2013). Explicit instruction included modeling by the researcher, supported practice, and independent practice. Fidelity checklists and lesson plans are available from the authors upon request.

Digital supports

Participants were taught, using the steps of explicit instruction, how to digitally highlight the main ideas in pink using the mnemonic prompts, “I pink this is the main idea,” details in green using the mnemonic “Do you a-green this is an important detail?,” and rhetorical information in orange using the mnemonic prompt, “Orange you glad this isn’t important?.” In addition, participants were taught, using the steps of explicit instruction, how to add digital “sticky notes” as reminders for each of the three steps of the TWA strategy and to record their audio retell summaries using the audio note feature. For this study, all of these digital supports were found in the iAnnotate application for iPad (see Figure 1). Criterion for participants to reach mastery of the TWA-SD strategy was independent use of eight out of nine steps of the TWA-SD strategy (Mason, 2004).

Figure 1.

Example comprehension questions (https://www.readworks.org/).

Baseline

After participant assent was obtained, each participant was provided a science text passage chosen jointly by the researcher and his special education teacher to read in digital format on an iPad mini. Instructional-level reading passages were selected based on the participants’ performance on the QRI-5 (Leslie & Caldwell, 2011). Passages were selected according to the participant’s instructional reading level as further determined by his WJ and Dynamic Indicators of Basic Early Literacy Skills (DIBELS) scores (shown in Table 1) and physical science curriculum topics (e.g., erosion, rocks and minerals, or volcanoes).

In baseline, the participants received the same instruction they would normally receive. Typical reading comprehension instruction in the resource room involved the teacher showing participants a text passage on an interactive white board, while reading aloud each paragraph and modeling self-questioning strategies. Then, the teacher asked participants to silently read a different text while she walked around and provided guidance and independent feedback. During baseline, participants were asked to orally summarize the passages they read independently. Baseline retells were scored by the researcher according to the same rubric used during the TWA-SD strategy intervention.

Training

As described by the Mason, Meadan, Hedin, and Corso (2006), the TWA protocol consisted of nine sessions. For fidelity purposes, these protocols were followed with the addition of digital annotation supports in lieu of paper-based supports. The protocol developed for this study involved the possibility of three additional sessions to reach mastery, if needed.

The TWA-SD intervention in this study consisted of 45-min sessions that included TWA instruction and behavior-specific feedback. Each session was highly structured and began with a 5-min review of rules and introduction to new vocabulary for the lesson passage. The next 35 min of each session included explicit instruction/training in the nine steps of the TWA-SD strategy. These steps were organized into three subcomponents: thinking before reading phase (think about the author’s purpose, think about what you know, and think about what you want to know), thinking while reading phase (think about your reading speed, look for connections to what you know, and reread to understand), and thinking after reading phase (think about the main idea, think about summarizing information, and think about what you learned; Mason, 2013). The final 5 min of each session were used to review the lesson content and performance with the participant. As proposed by Mason, Meadan, et al. (2006), lessons progressed from researcher modeling, to supported collaborative practice, to participant-independent practice.

Participants became figurative copilots in their reading working alongside the teacher to complete the steps (Mason, 2004). The researcher modeled the nine components of the TWA-SD strategy to gradually guide participants to complete the steps independently. The researcher only intervened when she observed errors in the strategy implementation or when participants asked for assistance with the digital tools. In this phase, the researcher facilitated the “What I Want To Learn” component by explicitly modeling for participants how to annotate the digital text by making audio notes and sticky note reflections of what they wanted to learn and similar reflections during and after reading. The researcher supported the participants’ practice to digitally highlight text and paraphrase for main ideas in audio notes, as well as to guide participants along a checklist of the TWA steps (Hedin, Mason, & Gaffney, 2011; Mason, 2013).

Thus, the total TWA strategy was broken into smaller parts to ensure participants had ample opportunity to observe modeling and participate in supported and independent practice to demonstrate mastery of the strategy steps (Deshler & Schumaker, 1993; Lenz, Ehren, & Deshler, 2005). Criteria for mastery of the TWA-SD strategy were participants’ independent use of eight out of the nine steps from the TWA in correct order (Mason, 2004) while also using the digital annotation tools (e.g., highlighting, audio notes).

Data Collection After Training

Once all participants had met mastery criteria, oral retell data were collected for five to six sessions without researcher instruction, and only verbal prompting (maximum of two) of “What do you do next?” until a stable trend was evident in the data (Howorth et al., 2016; Mason, 2013). Oral retells were collected via the audio note tool, and scored by both the researcher, and the classroom teacher. Again, no time limit was given for the participants to give a retell. The length of retells ranged from 1 to 6 min during the intervention phase. Performance for each participant was graphed. According to the criteria for exemplary single-case research (Horner et al., 2005), a small percentage (30% or seven participant retells) was also scored independently by a doctoral-level graduate student to ensure reliability. Agreement was 100% using point-by-point comparison (Horner et al., 2005).

Maintenance and generalization

Five additional maintenance probes were administered to each group to measure oral retells, and performance on comprehension questions 10 days after each participant had completed the intervention phase. Instruction in the strategy was not given, only oral prompts (maximum of two): “What do you do next?” Furthermore, as a measure of generalization, the classroom teacher gave participants expository digital text passages as PDF files to read on iPads during their normal social studies instruction. Participants were reminded they could use the TWA strategy, but no further prompts to use the strategy were provided. The classroom teacher chose text passages according to the independent reading levels appropriate for each participant. Participant-annotated PDF files were then given to the researcher in a password-protected file. The oral retells embedded in these files were evaluated for quality and accuracy of main ideas and details. Comprehension question accuracy was also recorded (See Figure 2). These measures were taken to evaluate whether participant performance was reliant on the presence of the researcher or whether participants’ TWA-SD strategy use could generalize to other content area expository reading tasks.

Figure 2.

TWA-digitally supported (SD) available digital annotation supports example.

Results

Visual analyses of differences in percentage of nonoverlapping data (PND) as well as the TAU-U index of overall effect for single-case design were used to evaluate the results of this intervention (Parker, Vannest, Davis, & Sauber, 2010; Scruggs & Mastropieri, 1998). Effect sizes of performance differences between baseline phase and intervention phase were calculated for participant performance on both quality of oral retell and comprehension question accuracy. TAU-U follows the same “S” sampling distribution as Mann–Whitney U and Kendall’s rank correlation, so p values and confidence intervals can be provided (Hollander & Wolfe, 1999).

The authors investigated the effectiveness of the TWA-SD intervention package, on the comprehension of participants with ASD, Level 1. The results (in Figures 3 and 4) indicate there was a functional relationship between use of TWA-SD (Mason, 2004) and the comprehension of text by sixth-grade male participants with a diagnosis of ASD, Level 1. As discussed below, all three participants increased their accuracy and quality of oral retellings of main ideas and details as well as their performance on general comprehension questions after learning to use the TWA-SD strategy.

Figure 3.

Graph of oral retell rubric scores.

Figure 4.

Graph of percentage of comprehension questions answered correctly.

Michael

Michael’s baseline performance on the retell rubric was represented by a mean of 1.44, median of 1.33 (range = 1.33–1.67) across three sessions. During the intervention phase, Michael’s performance on the retell rubric increased to a mean of 2.44, median of 2.66 (range = 1.33–3) across six sessions. His retell performance stabilized during the maintenance phase (range = 3–3.17) across four sessions and also across the generalization phase (range = 3–3.33) across five sessions.

Michael’s baseline performance on the reading comprehension questions was represented by a mean score of 34% across three sessions (range = 13–50%); median of 40% (range = 13–50%). His comprehension question scores increased markedly and immediately during the intervention phase to a mean of 71% (range = 50–100%). Interestingly, the accuracy of Michael’s comprehension questions continued to increase during maintenance (mean = 80%, range = 40–100%) but dropped sharply during the generalization phase (mean = 40%, range = 0–60%).

Jaime

Jaime’s baseline performance on the retell rubric was represented by a mean of 2.33, median of 2.33 (range = 2.24–2.33) across three sessions. His performances on the oral-retell measure increased markedly and immediately during the intervention phase to a mean retell rubric score of 3.5 (range = 3.33–3.67) across five intervention sessions. His retell accuracy scores remained stable during the maintenance phase with a mean score of 3.5 (range = 2.67–4).

Jaime’s accuracy on the baseline reading comprehension questions was a mean of 67%, a median of 65% (range = 60–75%) across three baseline sessions. His mean score on comprehension questions increased immediately and markedly during the intervention phase to a mean score of 78% (range = 50–100%) across six sessions. His performance stayed stable during the maintenance phase at a mean score of 78% (range = 60–100%) during the maintenance phase. No retell or comprehension question generalization data were gathered for Jaime during the generalization sessions, as he was absent during these sessions.

David

David’s baseline performance on the retell rubric was represented by a mean of 2.25, median of 2.0 (range = 2.33–2.67) across four sessions. His retell rubric scores increased immediately and markedly to a mean of 3.28 (range = 3–3.67) during the intervention phase. During the maintenance phase, David’s scores remained above baseline at a mean of 3.11 (range = 2.67–3.67) across three sessions. During the generalization phase, however, David’s scores trended back toward baseline with a mean of 2.67 across three sessions.

During the baseline phase, David’s accuracy on the reading comprehension questions was a mean of 40%, median of 35% (range = 10–55%) across four sessions. His comprehension question scores also increased immediately and markedly during the intervention phase to a mean of 71% (range = 60–100%) across six sessions. David’s scores also dropped sharply during the maintenance phase to a mean comprehension accuracy score of 47% (range = 30–70%) across three maintenance sessions. His scores on comprehensions question assessments continued to decline during the generalization phase to a mean score of 30% across three sessions (range = 0–80%).

Measures of Effect

Table 3 shows that all single-case design measures of effect (i.e., Percent Exceeding Mean [PEM], PND, and TAU-U) indicate positive effects of use of the TWA with digital tools across all participant groups for oral retell and variable effects on comprehension question answer accuracy. According to PND, 83% of the retell quality data for Michael did not overlap with baseline data. This indicates that the intervention was effective according quality indicators established by the Institute of Educational Sciences (Kratochwill et al., 2010). According to TAU-U, 88% of Michael’s retell quality intervention data indicate an improvement over baseline, which indicates an effective and significant outcome (p = .039, α = .05). Finally, 100% of Michael’s retell intervention data exceed the mean of baseline data (PEM; very effective). Similarly, 100% of Michael’s comprehension question intervention data exceed the mean of baseline data (PEM; very effective). According to PND, 67% of the comprehension question accuracy intervention data for Michael did not overlap with baseline data (questionable effect). Interestingly, the intervention was shown to be effective for comprehension question accuracy. According to TAU-U, 89% of Michael’s comprehension question accuracy data within the intervention phase indicate improvement over his baseline performance, which is also significant (p = .014, α = .05; Hollander & Wolfe, 1999; Parker et al., 2010; Scruggs & Mastropieri, 1998). Finally, 100% of Michael’s comprehension question accuracy data exceed the mean of baseline data (PEM; very effective).

Table 3.

Single-Case Design Measures of Effect.

Participant	Retell PEM	Retell PND	Retell TAU-U
Michael	100%Very effective	83%Effective	88%Effective
Jaime	100%Very effective	100%Very effective	100%Very effective
David	100%Very effective	100%Very effective	100%Very effective
Participant	Comp. PEM	Comp. PND	Comp. TAU-U
Michael	100%Very effective	67%Questionable	61%Questionable
Jaime	67%Questionable	67%Questionable	37%Ineffective
David	100%Very effective	50%Ineffective	58%Ineffective

Note. Effectiveness indicators from Hollander and Wolfe (1999); Parker, Vannest, Davis, and Sauber (2010); and Scruggs and Mastropieri (1998). PND = percent of nonoverlapping data.

Jaime also showed overall improvement in comprehension during the intervention phase when compared to baseline phase for both retell quality and comprehension accuracy, although only the retell quality improvement was significant. According to PEM and PND, 100% of Jaime’s intervention retell data did not overlap with baseline data (very effective). According to TAU-U, 100% of Jaime’s retell data indicate an improvement between phases (very effective). Interestingly, comprehension accuracy data for Jaime, according to both PEM and PND indicate 67% of the data between baseline and intervention phases do not overlap (questionable effect). TAU-U measures also indicate the intervention was ineffective for Jaime’s ability to answer comprehension questions accurately with only 37% of the comprehension accuracy data showing improvement between phases (Hollander & Wolfe, 1999; Parker et al., 2010; Scruggs & Mastropieri, 1998).

David also showed general improvement in retell quality and questionable improvement in comprehension question accuracy between baseline and intervention phases. According to PEM, PND, and TAU-U, 100% of David’s retell data did not overlap between phases during intervention. Although 100% of David’s comprehension question accuracy did not overlap according to PEM, only 50% of his comprehension accuracy data did not overlap according to PND, and only 58% of the data in the intervention phase indicated improvement in comprehension accuracy according to TAU-U (Hollander & Wolfe, 1999; Parker et al., 2010; Scruggs & Mastropieri, 1998).

Social Validity

All participants indicated the TWA-SD strategy helped them to become better readers. Regarding what he had learned, one participant responded he had learned to take his time, add audio summaries, and slow down to make sure he understood what he was reading. Another participant responded he had learned the goal of reading was to understand, not to “get done.” Regarding how the strategy could help other students, responses included, “They need to think before, while, and after they read” and “It will help them on their English Language Arts (ELA) exams.” None of the participants offered information to add to the strategy. Finally, the participants listed the following as what helped them the most from the strategy: (a) the thinking while reading stage (i.e., linking knowledge, reading speed, and rereading), (b) “I didn’t know I had to do any of that” (c) “Adding audio notes on what the reading reminded me of,” and (d) “Highlighting and looking first at the main idea and then the details.”

In addition, after the completion of the maintenance phase, all participants were interviewed by the first author who read questions from a technology use survey. This survey used Likert-type style questions with a high score of four containing the following questions: (a) Indicate how much you enjoyed using the iPads during your reading activities, (b) How easy was it for you to use the iPads? (c) How easy for you was it to highlight using iAnnotate? (d) How easy was it for you to use the TWA strategy? (e) How easy for you was it to use the TWA checklist on the iPad? (f) How likely are you to recommend the iAnnotate app to another student? and (g) How likely are you to recommend reading on the iPad to your teacher? The participants all responded that they found it easy to use the technology. The mean response for Likert-type style questions was 3.57 (range = 3.14–4). Participants found it particularly difficult to copy and paste text and to save their work using the iAnnotate application. This is important to know because it may make classroom management of student data a challenge for a teacher who may have to manually save each student’s annotated text.

Discussion

Most TD adolescents with accurate decoding skills are able to develop accurate reading comprehension skills (Chiang & Lin, 2007; Nation et al., 2006). Despite strong decoding skills and a good memory for details and facts, students with ASD, Level 1 often have difficulty with reading comprehension (Åsberg & Sandberg, 2010; Carnahan & Williamson, 2013). The purpose of this study was to examine the effects of the TWA-SD strategy on the expository comprehension of participants who have an ASD, Level 1 diagnoses. This strategy was chosen because it is effective for other populations of students (i.e., students with learning disabilities or emotional and behavior disabilities) who struggle with reading comprehension. This study adds to existing research support for instructional techniques that involve self-questioning and self-monitoring as effective for improving the comprehension skills and academic performance of students with ASD, Level 1. Digital supports were added to the TWA strategy as part of an intervention package (TWA-SD) to allow flexibility according to universal design for student engagement in reading (digital highlighting and audio note tools), and expression of paragraph-level summaries (via stylus written, typed, or audio-recorded self-notation).

Implications

The results indicate that the use of TWA-SD by participants with ASD, Level 1 who struggle with reading comprehension increased their comprehension as measured by retells. Results for comprehension questions answered correctly are more variable. Contrary to what theory and current research suggests, participants were not more likely to answer inferential questions inaccurately any more than explicit questions. More research with a larger participant sample is needed before any generalizations can be made from this result.

In addition, all participants indicated they enjoyed using the TWA-SD strategy, and it had changed their attitudes in a positive way toward reading. Bastug (2014) indicated positive attitudes toward reading were highly correlated to higher reading comprehension. This study therefore adds much needed information to the list of research-based effective content area interventions for students with an ASD diagnosis. Despite the general low accuracy in answering individual comprehension questions, it is important to note the significant overall increase in retell measures across intervention, maintenance, and generalization phases compared to baseline data, indicating a generalized effect on retell.

The TWA strategy has been long validated as an evidence-based instructional strategy for students with learning disabilities. The results of this study indicate this strategy can be used with digital supports to provide for visual and auditory self-regulation of participants’ own reading comprehension. In lieu of rewording the steps of the strategy in their own words, as is common in any self-regulated strategy instruction, each of these participants drew what looked like a “symbol” or “symbol sentence” to represent the steps of the strategy. The use of such graphical (rather than alphabetical or word based) mnemonics may be unique to students with autism who have often been described as visual thinkers. Furthermore, the use of TWA-SD was investigated as an intervention package. Further research is warranted on what specific digital annotation features yielded the most impact on students’ learning of the strategy and the effects of those tools on the reading comprehension outcomes. Component analysis of each of the steps of the TWA strategy and how the digital tools support each component could contribute to a greater understanding of maintaining and generalizing the effects of this intervention.

Limitations

Although this study was successful on multiple measures, there are many limitations that warrant discussion. Namely, due to the public school setting of this study and variability of middle-school schedules, a stable baseline was not reached for the comprehension question accuracy measure. This may contribute to the variability of the results and limits the conclusions that can be made to the functional relationship between use of the TWA-SD intervention package and the ability to answer prewritten comprehension questions. For instance, the quality and accuracy of participants’ retells improved with their use of the TWA-SD strategy; however, their performance and accuracy in answering multiple-choice and short-answer comprehension questions were more variable. Another limitation is the small male sample, which restricts generalization of findings, and the administration protocol. Since many students receive their instruction in a large group setting, it is unknown whether similar outcomes would be achieved if the TWA-SD was delivered in a group format. Finally, this study examined the effectiveness of this intervention for expository reading comprehension, its efficacy for narrative reading comprehension cannot be inferred from the findings.

Need for Future Research

Based on this study, future studies should seek to evaluate this protocol with a larger, more diverse group of students with ASD. It might also be useful to study the intervention in a randomized group study, so the relative efficacy of TWA-SD can be determined compared to other reading interventions. Future researchers might investigate the types of short answer and comprehensions questions missed by students with ASD as they may show patterns that provide more information to guide instruction. For example, what types of comprehension questions are most frequently answered incorrectly? Furthermore, in order to increase the ecological validity of this intervention, future research should examine use of the TWA-SD in more inclusive classrooms, with classroom texts, and taught by teachers rather than researchers. As such, instructional, behavioral, and technological logistics of use of this strategy on a larger scale should be investigated. Also, in order to further generalize these findings, participants of both genders should be included in future studies.

Conclusion

The TWA-SD intervention package was investigated as a tool to develop expository reading comprehension of adolescents diagnosed with ASD, Level 1. This strategy was chosen because it has been shown to be effective for other populations of students who struggle with expository reading comprehension. The digital supports were added because they allow flexibility according to universal design for student engagement in reading (highlighting tools), and expression of paragraph-level summaries (via written, typed, or audio-recorded notes). All participants indicated they enjoyed using the TWA strategy, and it had changed their attitudes in a positive way toward reading. Although the intervention was delivered individually to participants, a functional relationship was shown for increasing their oral retell skills after using the TWA-SD intervention package. This adds much needed information to the list of research-based effective content area instructional interventions for students with an ASD, Level 1 diagnosis.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported in part by a grant from the United States Department of Education, Office of Special Education Programs, Project Title: Preparing Leaders for the Digital Age, Grant Number: H325D100026.

References

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Arlington, VA: Author.

Anderson-Inman

(2009). Supported eText: Literacy scaffolding for students with disabilities. Journal of Special Education Technology, 24, 1–8. Retrieved from http://www.tamcec.org/jset-index/supported-etext-literacy-scaffolding-for-students-with-disabilities/

Andreassen

Bråten

(2011). Implementation and effects of explicit reading comprehension instruction in fifth-grade classrooms. Learning & Instruction, 21, 520–537.

Archer

Hughes

C. A.

(2011). Explicit instruction: Efficient and effective teaching. New York, NY: Guilford.

Åsberg

Sandberg

A. D.

(2010). Discourse comprehension intervention for high-functioning students with autism spectrum disorders: preliminary findings from a school-based study. Journal of Research in Special Educational Needs, 10, 91–98.

Baker

Gersten

Scanlon

(2002). Procedural facilitators and cognitive strategies: Tools for unraveling the mysteries of comprehension and the writing process, and for providing meaningful access to the general curriculum. Learning Disabilities Research & Practice, 17, 65. Retrieved from http://onlinelibrary.wiley.com.gate.lib.buffalo.edu/journal/10.1111/%28ISSN%291540-5826

Baron-Cohen

(2008). Autism, hypersystemizing, and truth. Quarterly Journal of Experimental Psychology, 61, 64–75.

Baron-Cohen

Leslie

A. M.

Frith

(1985). Does the autistic child have a “theory of mind”? Cognition, 21, 37–46.

Bastug

(2014). The structural relationship of reading attitude, reading comprehension and academic achievement. International Journal of Social Sciences and Education, 4, 931–946.

10.

Bernacki

M. L.

Byrnes

J. P.

Cromley

J. G.

(2012). The effects of achievement goals and self-regulated learning behaviors on reading comprehension in technology-enhanced learning environments. Contemporary Educational Psychology, 37, 148–161. doi:10.1016/j.cedpsych.2011.12.001

11.

Berninger

V. W.

Vermeulen

Abbott

R. D.

(2003). Comparison of three approaches to supplementary reading instruction for low-achieving second-grade readers. Language, Speech & Hearing Services in Schools, 34, 101–116. doi:10.1044/0161-1461(2003/009)

12.

Bouck

E. C.

Savage

Meyer

N. K.

Taber-Doughty

Hunley

(2014). High-tech or low-tech? Comparing self-monitoring systems to increase task independence for students with autism. Focus on Autism & Other Developmental Disabilities, 29, 156–167. doi:10.1177/1088357614528797

13.

Brown

H. M.

Oram-Cardy

Johnson

(2013). A meta-analysis of the reading comprehension skills of individuals on the autism spectrum. Journal of Autism and Developmental Disorders, 43, 932–955.

14.

Carnahan

C. R.

Williamson

P. S.

(2013). Does compare-contrast text structure help students with autism spectrum disorder comprehend science text? Exceptional Children, 79, 347–363. Retrieved from www.cec.sped.org

15.

Centers for Disease Control and Prevention. (2014). Prevalence of autism spectrum disorders among children aged 8 year old: Autism and developmental disabilities monitoring network, 11 sites, United States, 2010. Morbidity and Mortality Weekly Report, 63, 1–12.

16.

Chiang

H. M.

Lin

Y. H.

(2007). Reading comprehension instruction for students with autism spectrum disorders: A review of the literature. Focus on Autism and Other Developmental Disabilities, 22, 259–267.

17.

Deshler

D. D.

Schumaker

J. B.

(1993). Strategy mastery by at-risk students: Not a simple matter. The Elementary School Journal, 94, 153–167.

18.

El Zein

Solis

Vaughn

McCulley

(2014). Reading comprehension interventions for students with autism spectrum disorders: A synthesis of research. Journal of Autism and Developmental Disorders, 44, 1303–1322.

19.

Estes

Rivera

Bryan

Cali

Dawson

(2011). Discrepancies between academic achievement and intellectual ability in higher-functioning school-aged children with autism spectrum disorder. Journal of Autism and Developmental Disorders, 41, 1044–1052.

20.

Fuchs

L. S.

Fuchs

Hosp

M. K.

Jenkins

J. R.

(2001). Oral reading fluency as an indicator of reading competence: A theoretical, empirical, and historical analysis. Scientific Studies of Reading, 5, 239–256.

21.

Graham

Harris

K. R.

(2003). Students with learning disabilities and the process of writing: A meta-analysis of SRSD studies. New York, NY: Guilford Press.

22.

Grimshaw

Dungworth

McKnight

Morris

(2007). Electronic books: Children’s reading and comprehension. British Journal of Educational Technology, 38, 583–599. doi:10.1111/j.1467-8535.2006.00640.x

23.

Hasbrouck

Tindal

G. A.

(2006). Oral reading fluency norms: A valuable assessment tool for reading teachers. Reading Teacher, 59, 636–644. doi:10.1598/RT.59.7.3

24.

Hedin

L. R.

Mason

L. H.

Gaffney

J. S.

(2011). Comprehension strategy instruction for two students with attention-related disabilities. Preventing School Failure, 55, 148–157.

25.

Hollander

Wolfe

D. A.

(1999). Non-parametric statistical methods (2nd ed.). New York, NY: Wiley.

26.

Horner

R. D.

Baer

D. M.

(1978). Multiple-probe technique: A variation of the multiple baseline. Journal of Applied Behavior Analysis, 11, 189–196.

27.

Horner

R. H.

Carr

E. G.

Halle

McGee

Odom

Wolery

(2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71, 165–179.

28.

Howorth

Lopata

Thomeer

Rodgers

(2016). Effects of the TWA strategy on expository reading comprehension of students with autism. British Journal of Special Education, 43, 39–59.

29.

Kazdin

A. E.

(2011). Single-case research designs. New York, NY: Oxford University Press.

30.

Kennedy

C. H.

(2005). Single-case designs for educational research. Upper Saddle River, NJ: Prentice Hall.

31.

Kratochwill

T. R.

Hitchcock

Horner

R. H.

Levin

J. R.

Odom

S. L.

Rindskopf

D. M.

Shadish

W. R.

(2010). Single-case designs technical documentation. What Works Clearinghouse.

32.

Lange

A. A.

McPhillips

Mulhern

Wylie

(2006). Assistive software tools for secondary-level students with literacy difficulties. Journal of Special Education Technology, 21, 13–22.

33.

Lenz

B. K.

Ehren

B. J.

Deshler

D. D.

(2005). The content literacy continuum: A school reform framework for improving adolescent literacy for all student. Teaching Exceptional Children, 37, 60–63.

34.

Leslie

Caldwell

J. S.

(2011). Qualitative Reading Inventory–5. Boston, MA: Pearson Education.

35.

Lopata

Thomeer

M. L.

Volker

M. A.

Lee

G. K.

Smith

T. H.

Rodgers

J. D.

… Toomey

J. A.

(2013). Open-trial pilot study of a comprehensive school-based intervention for high-functioning autism spectrum disorders. Remedial and Special Education, 34, 269–281.

36.

Magliano

Millis

Levinstein

Boonthum

(2011). Assessing comprehension during reading with the Reading Strategy Assessment Tool (RSAT). Metacognition & Learning, 6, 131–154. doi:10.1007/s11409-010-9064-2

37.

Mason

L. R.

(2004). Explicit self-regulated strategy development versus reciprocal questioning: Effects on expository reading comprehension among struggling readers. Journal of Educational Psychology, 96, 283–296. doi:10.1037/0022-0663.96.2.283

38.

Mason

L. H.

(2013). Teaching students who struggle with learning to think before, while, and after reading: Effects of self-regulated strategy development instruction. Reading & Writing Quarterly: Overcoming Learning Difficulties, 29, 124–144. doi:10.1080/10573569.2013.758561

39.

Mason

L. H.

Meadan

Hedin

Corso

(2006). Self-regulated strategy development instruction for expository text comprehension. Teaching Exceptional Children, 38, 47–52. Retrieved from http://www.cec.sped.org

40.

Mason

L. H.

Snyder

K. H.

Sukhram

D. P.

Kedem

(2006). TWA+ PLANS strategies for expository reading and writing: Effects for nine fourth-grade students. Exceptional Children, 73, 69–89.

41.

Mintz

(2013). Additional key factors mediating the use of a mobile technology tool designed to develop social and life skills in children with autism spectrum disorders: Evaluation of the 2nd HANDS prototype. Computers in Education, 63, 17–27. doi:10.1016/j.compedu.2012.11.006

42.

Moore

Calvert

(2000). Brief report: Vocabulary acquisition for children with autism: Teacher or computer instruction. Journal of Autism and Developmental Disorders, 30, 359–362.

43.

Moss

(2004). Teaching expository text structures through information trade book retellings. The Reading Teacher, 57, 710–718.

44.

Nation

Clarke

Wright

Williams

(2006). Patterns of reading ability in children with autism spectrum disorder. Journal of Autism and Developmental Disorders, 36, 911.

45.

National Institute of Child Health and Human Development. (2000). Report of the national reading panel. Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction (NIH Publication No. 00-4769). U.S. Government Printing Office, Washington, DC: U.S. Government Printing Office. Retrieved from https://www.nichd.nih.gov/publications/pubs/nrp/Documents/report.pdf

46.

Parker

R. I.

Vannest

K. J.

Davis

J. L.

Sauber

S. B.

(2010). Combining non-overlap and trend for single case research: Tau-U. Behavior Therapy, 42, 284–299. doi:10.1016/j.beth.2010.08.006.

47.

Pennington

R. C.

(2010). Computer-assisted instruction for teaching academic skills to students with autism spectrum disorders: A review of literature. Focus on Autism and Other Developmental Disabilities, 25, 239–248.

48.

Richards

S. B.

Taylor

R. L.

Ramasamy

Richards

R. Y.

(1999). Single subject research. San Diego, CA: Singular.

49.

Ricketts

Jones

C. R.

Happé

Charman

(2013). Reading comprehension in autism spectrum disorders: The role of oral language and social functioning. Journal of Autism and Developmental Disorders, 43, 807–816.

50.

Sartini

Knight

V. F.

Spriggs

A. D.

Allday

R. A.

(2017). Generalization strategies to promote text comprehension skills by students with ASD in core content areas. Focus on Autism and Other Developmental Disabilities, 33, 150–159. doi:10.1177/1088357617735815.

51.

Scruggs

T. E.

Mastropieri

M. A.

(1998). Summarizing single-subject research: Issues and applications. Behavior Modification, 22, 221–242.

52.

Tindal

G. A.

Marston

D. B.

(1990). Classroom-based assessment: Evaluating instructional outcomes. Merrill Publishing Co.

53.

Wallen

Plass

J. L.

Brünken

(2005). The function of annotations in the comprehension of scientific texts: Cognitive load effects and the impact of verbal ability. Educational Technology Research and Development, 53, 59–71.

54.

Woodcock

R. W.

McGrew

K. S.

Mather

(2007). Woodcock-Johnson III. Rolling Meadows, IL: Riverside.