Augmented Reality Supporting Reading Skills of Students with Autism Spectrum Disorder

Abstract

Teachers’ use of video modeling has been established as an evidence-based practice for teaching students with autism spectrum disorder (ASD). Augmented reality (AR) applications can be used as tools to provide trigger-based, video-modeled instructional supports to students with ASD. The use of AR in this way may help teachers implement evidence-based reading skills practice such as video modeling and provide more independent practice opportunities. It also provides more options for student engagement and concept representation. This article describes ideas for how to use a particular AR application to (a) teach phonics and word identification, (b) support reading fluency, (c) embed videos into texts as cues for reading comprehension, (d) teach content area vocabulary words, and (e) use video models during transition planning.

Keywords

video modeling augmented reality reading autism spectrum disorder

Devon is an 6-year-old third grade student diagnosed with autism spectrum disorder (ASD). Samantha is a 12-year-old seventh grade student diagnosed with ASD and dyslexia. Marcus is a 16-year-old high school student with ASD, cerebral palsy, and a speech and language impairment, who uses an iPad to communicate. Students like Samantha, Devon, and Marcus who have been diagnosed with ASD, exhibit traits including (a) social impairments (e.g., social-interaction and social-communication) and (b) restricted and repetitive interests and behaviors (American Psychiatric Association, 2013). These students are logical and procedural thinkers, and enjoy when their daily routines are consistent. However, they have difficulties with reading, which is necessary for learning in content areas. Devon struggles with phonics, word identification, and fluency. Samantha and Marcus have difficulties with reading comprehension and understanding difficult vocabulary. Their struggles with reading hinder their learning across various content areas such as English language arts, social studies, and science. Thus, their teachers are exploring effective ways to enhance their students’ reading skills, so the students can learn successfully in their classes (see Note 1).

Students with Autism Spectrum Disorder

Reading is an essential skill for every student to learn successfully in school as well as in their lives beyond school (Hallahan, Kauffman, & Lloyd, 1999). However, learning how to read is not easy for many students. According to the National Assessment of Educational Progress report (U.S. Department of Education, 2017), only 37% of fourth-grade students and 36% of eighth-grade students read at the proficient level. The report also indicated students with disabilities have more significant difficulties with reading compared to their typical peers. Legislation such as the Every Student Succeeds Act (2015) and Individuals with Disabilities Education Improvement Act (2004) mandates teaching all students, including students with disabilities, essential reading skills (e.g., phonics, fluency, vocabulary, comprehension) using research-based reading instruction (Whalon, Al Otaiba, & Delano, 2009).

Early in reading development, students have to develop phonics and word identification skills required to read texts accurately and fluently, but the ultimate goal of reading is comprehending the text (Ricketts, 2011). According to previous research, students with ASD have a considerably wide range of reading skills. For example, some students can do well with word identification but have difficulties with reading comprehension, while some students are unable to decode but have reasonable word identification skills (Nation, Clarke, Wright, & Williams, 2006). However, students with ASD tend to experience more difficulties with reading comprehension than decoding and word identification; in addition, vocabulary and comprehension skills are highly related (Nation et al., 2006; Ricketts, Jones, Happé, & Charman, 2013).

Text comprehension tends to be challenging for students with ASD because of an impaired ability to draw inferences, understand idioms and rhetorical language, and make cause-and-effect connections (Estes, Rivera, Bryan, Cali, & Dawson, 2011). Making inferences while reading is often associated with perspective taking. It is often difficult for students with ASD to imagine the thoughts, feelings, and perspectives of other people and characters while reading. This may pose a problem when they are trying to determine the author’s purpose and implicit inferences in text (Ricketts et al., 2013). Making higher-order inferences from text is a skill emphasized by the Common Core State Standards for all students in language arts classrooms across the nation (National Governors Association Center for Best Practices & Council of Chief State School Officers, 2010). Teachers often use oral retellings as a formative assessment to gauge their students’ level of understanding of text (Moss, 2004). However, students with ASD may find oral retelling challenging.

Video Modeling and Augmented Reality

Instructional and assistive technology has been shown to play an important role in providing learning support to students with ASD and is a legally required consideration for students with disabilities (Morgan, Yeager, Murphy, & Spies, 2018). In particular, video modeling (VM) has been shown to be an evidence-based practice for teaching tasks to students like Samantha, Devon, and Marcus who have been diagnosed with ASD and prefer visual stimuli (Bellini & Akullian, 2007). VM was developed based on the social learning theory, which emphasizes that people can learn from modeling (Bandura, 1969). To learn target skills (e.g., social, behavioral, communication, academic skills), students watch a VM in which others (i.e., VM) or themselves (i.e., video self-modeling) demonstrate targeted skills (Buggey & Ogle, 2012). Literature suggests that VM is an evidence-based practice for teaching social, behavioral, communication, functional, and behavioral skills to students with ASD (Bellini & Akullian, 2007).

Video modeling is an effective intervention to support students who struggle with literacy-based academic tasks (Marcus & Wilder, 2009; Mechling & Hunnicutt, 2011). For example, students can watch VM on word reading, letter naming, or vocabulary usage to learn the skills (e.g., Marcus & Wilder, 2009; Mechling & Hunnicutt, 2011; Morlock, Reynolds, Fisher, & Comer, 2015). In addition, VM of self-monitoring questions are ways to support students with ASD (Bellini & Akullian, 2007; Cihak et al., 2016). Explicit modeling using digital annotation tools and self-questioning may scaffold and guide retelling, thus potentially improving reading performance (Lange, McPhillips, Mulhern & Wylie, 2006). In addition, VM can help teachers provide added distributive practice of reading skills in an independent way for students. Students can watch the VM using laptops, tablets, or smartphones for independent learning. However, a problem that teachers often confront when implementing VM is that students with ASD may have difficulty maintaining attentional focus on the video model (Cihak et al., 2016). The use of AR applications may alleviate this problem.

Augmented Reality to Support Reading

Many AR applications exist that allow teachers to control the video content being modeled during reading instruction. Teachers can help students maintain their attentional focus on VM through the use of AR tools. However, the use of AR apps for teaching students with ASD is a new topic in the field, and more research is needed to determine its effects. Research studies examining the use of AR for teaching academic vocabulary to students with ASD report promising effects (Cihak et al., 2016; McMahon, Cihak, Wright, & Bell, 2016).

Table 1 lists AR applications for mobile devices (e.g., tablets, smartphones) that can be used to support reading for students with ASD. Several AR application are either free or low cost, including AR Flashcards Space Lite, Blippar, HP Reveal, Just a Line, and Our Discovery Island: Phonics Tricksters. This article focuses specifically on the use of the HP Reveal application. HP Reveal allows teachers to create video models, an evidence-based practice for students with ASD. Teachers can use the application for providing effective video-modeled reading interventions to this population.

Table 1.

Examples of Augmented Reality Applications to Support Reading.

Name	URL	Features
AR Flashcards Space Lite	http://arflashcards.com/ar-flashcards-space-free/	Scan pictures or objects to trigger video demonstration^a Flashcards^b
Blippar	https://www.blippar.com	Attach video models to 2D or 3D objects^a Scan words to trigger video demonstration^a Scan pictures or objects to trigger video demonstration^a
Just a Line	https://justaline.withgoogle.com	Make simple drawings with device and create video to share
HP Reveal	https://www.hpreveal.com/	Attach video models to 2D or 3D objects^a Scan words to trigger video demonstration^a Scan pictures or objects to trigger video demonstration^a
Our Discovery Island: Phonics Tricksters	https://appadvice.com/app/our-discovery-island-phonic-tricksters/673987542	Game-based app to practice letter-sounds

Make your own feature.

Ready-made feature.

HP Reveal is a customizable platform that allows users to attach user-made video models to 2D or 3D objects. First, users need to create a “trigger” image, such as a vocabulary flashcard or a portion of text. This is accomplished simply by clicking on the “+” button on the right top of the app screen. Next, the video models or pictures are associated with the trigger image. Then, when users scan the trigger image using the app, the video or picture associated with it will appear. Augmented reality is interactive through its ability to combine both real and virtual objects. It provides an enhanced view of the real-world environment by adding virtual computer-generated information (Carmigniani & Furht, 2011).

Augmented reality applications like HP Reveal may help to maintain attentional focus as they require the student to scan a picture based “trigger” to play the video or see an image. For example, using an iPad and the HP Reveal application, a teacher could make a video of himself or herself pronouncing a content-area vocabulary word such as “vapor” while showing a pot of steaming water. When a student scans a teacher-created flashcard of the word “vapor” using a mobile device with the HP Reveal application installed, they can view the video as many times as needed. Using AR apps, teachers can create and provide VM easily to students with ASD.

The following sections describe how HP Reveal can help with various reading tasks including phonics, fluency, comprehension, and vocabulary.

English Language Arts Class

Devon is a first-grade student with ASD who struggles with phonics and word recognition. Repeated reading, where a teacher reads aloud first then students read aloud to the teacher, is an effective reading intervention for students with disabilities who struggle with reading (Lee & Yoon, 2017). Using the HP Reveal application, his teacher created a deck of flashcards for phonics (e.g., cat, sat) and sight words (e.g., you, about) that Devon has difficulties with. These flashcards serve as trigger images (see Figure 1). The teacher created a short video on how to pronounce the word “sat,” showing how to segment (s-, a-, t-) and blend (sat) the sounds. The teacher took a picture of the flashcard (sat) as a trigger image, then attached the video to the image. Using the HP Reveal application, Devon scans the cards to watch the teacher-created videos and practice the target phonics and sight word skills.

Figure 1.

Phonic flashcard trigger (left) and flashcard triggering the video to play for the student (right).

Devon also struggles with reading fluency. His teacher decides to create a read-aloud book using the HP Reveal application. First, the teacher takes a picture of each page as a trigger image. Then, she records a video that reads aloud each sentence on that page. The video shows the teacher using her finger to follow the words while reading. Each video has its own trigger image, which is each page of the text. When reading, Devon is able to scan a trigger image to practice reading the book independently (see Figure 2). To support his reading comprehension, the teacher also creates a video asking reading comprehension questions (e.g., Who are main the characters? What is the problem? How is problem solved?) as he reads. The teacher associates the video to the appropriate book page such that it aligns with the specific reading comprehension question (see Figure 3).

Figure 2.

Reading fluency: Reading text trigger (left) and teacher-created video screenshot of the teacher reading the text aloud (right).

Figure 3.

Reading comprehension: Reading text trigger (left) and teacher-created video screenshot of a comprehension question (right).

Science Class

Samantha, a 12-year-old, has strong decoding skills but weak reading comprehension of grade-level texts. She often finishes her reading assignments before her typical peers but does poorly on measures of reading comprehension. Poor reading comprehension is causing difficulties in many of her academic classes, including determining which information is relevant and irrelevant in texts.

Samantha may benefit from the use of VM and VM self-questioning strategies. Such strategies are effective for students with ASD who struggle with reading comprehension, or the ability to understand a text (Bellini & Akullian, 2007). Using the HP Reveal application, triggers for VM self-questioning strategies can be embedded in a student’s version of a text. For example, a teacher could create an AR based video self-model of the student asking the following at various points in the text. “What do I know about this already?” “What questions do I have?” “How does this connect to something I already knew?” “How can I put that in my own words?” This would ultimately help the student to practice self-questioning techniques and generalize across various types of text passages. The triggers could also be put onto notecards that can be used as self-questioning prompts while the student is studying or reading a self-selected text (see Figure 4). Explicit modeling in digital annotation tools and self-questioning scaffold and support their comprehension, thus providing students with ASD more success in reading (Lange, McPhillips, Mulhern, & Wylie, 2006).

Figure 4.

Reading comprehension: Reading text trigger (left) and reading text triggering the video to play for the student (right). The text is an author-written example. The picture is used under a creative commons agreement from https://pixabay.com.

In addition, Samantha often struggles with understanding the meaning of content area vocabulary. She is able to read the actual word, but her lack of understanding of the meaning of the word hinders her learning in content area subjects because vocabulary and reading comprehension are correlated (Nation et al., 2006). To provide support, her teacher prepared a deck of flashcards for difficult vocabulary words. For example, for photosynthesis, the teacher took a picture of the word. He found a video explaining photosynthesis on YouTube. He then uploaded the video using the HP reveal application. After that, he attached the video to the trigger image of a photosynthesis vocabulary card. Samantha is now able to review all the vocabulary cards to understand the meaning first before reading her textbook. Later, while reading the textbook, she can watch the photosynthesis video again if she needs to review the definition.

Vocational Education

Marcus is in enrolled in vocational classes at his high school. He is unsure of his career path, however. His reading fluency and comprehension skills are limited. He benefits from having multiple representations of the text presented to him. His teacher is exploring ways to help him read about future careers choices and improve his understanding of job descriptions and application requirements. She has given him a pamphlet from a local vocational training center that explains the process, but he and his parents struggle to read it due to low adult literacy levels. One way his vocational skills teacher could help is to create a video reading of the pamphlet for him to watch and listen to with his family.

Repeated reading is also an evidence-based practice that can help promote fluent reading skills, the ability to read quickly and accurately. Creating video models of a fluent reading of a text that students can watch before reading the text themselves has been shown to increase oral reading fluency (Therrien, 2004). A VM can be linked to an AR trigger of the book’s cover using HP Reveal. This is an easy way of allowing the student to have access to the text and to practice oral reading fluency independently. Similarly, Marcus’s teacher could create a how-to video for the completion of a specific job application. With repeated watching and practice of this skill, Marcus should feel confident when the time comes to actually apply for a job on his own.

Conclusion

When deciding which type of technology to use as an instructional tool, it is important to consider the learning goals and students’ learning needs. The International Society for Technology Education has included this sentiment in its teacher standards. In addition, it is apparent that using technology itself does not make changes in students’ learning; it depends on the use of evidence-based instructional variables embedded in the technology-based program or how teachers integrate technology into their lesson (Vrasidas & Glass, 2010). Innovative use of technology, such as AR applications, to integrate evidence-based teaching strategies can transform learning for students with ASD and promote their inclusion in the community.

The use of AR applications is not without challenges. High-tech tools in classrooms comes with the associated costs of tablets and high-speed Internet availability. It is relatively easy to use and create the videos, but teacher training is required. Also, creating the video models and trigger links can be time intensive for busy teachers. Setting up the environment to be conducive to using the AR applications also takes time (e.g., making sure the triggers are placed strategically for learning). In addition, students can be distracted while using the tablet to use the application; for example, they can get out of the application to play games. Teachers need to set up clear behavior expectations while using the app. Also, it might be helpful to use the “Guided Access” accessibility feature, locking the app with a passcode, available on some devices.

However, AR applications to support reading activities for students with ASD have many potential benefits. These types of interactive applications promote student motivation, communication, and interaction and make learning fun. Research shows that when students are provided multiple means of content representation, engagement, and expression, in-depth learning occurs and students remember more (Rose, Gravel, & Domings, 2012). Ultimately, the use of AR applications is a strategy for implementing evidence-based reading instructional practices such as video modeling, repeated reading, segmenting and blending, and self-questioning strategies for comprehension.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Notes

References

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: Author.

Archer

A. L.

Hughes

(2011). Explicit instruction: Effective and efficient teaching. New York, NY: Guilford.

Bandura

(1969). Principles of behavior modification. New York, NY: Holt.

Bellini

Akullian

(2007). A meta-analysis of video modeling and video self-modeling interventions for children and adolescents with autism spectrum disorders. Exceptional Children, 73, 264–287. doi:10.1177/001440290707300301

Buggey

Ogle

(2012). Video self-modeling. Psychology in the Schools, 49(1), 52–70.

Furht

(Ed.). (2011). Handbook of augmented reality. Springer: Science & Business Media.

Cihak

D. F.

Moore

Wright

McMahon

D. D.

Gibbons

M. M.

Smith

(2016). Evaluating augmented reality to complete a chain task for elementary students with autism. Journal of Special Education Technology, 31, 99–108. doi:10.1177/0162643416651724

Estes

Rivera

Bryan

Cali

Dawson

(2011). Discrepancies between academic achievement and intellectual ability in higher-functioning school-aged children with autism spectrum disorder. Journal of Autism and Developmental Disorders, 41, 1044–1052. doi:10.1007/s10803-010-1127-3

Every Student Succeeds Act, Pub. L. No. 114-95 (2015).

10.

Hallahan

D. P.

Kauffman

J. M.

Lloyd

J. W

. (1999). Introduction to learning disabilities. Boston, MA: Allyn & Bacon.

11.

Individuals with Disabilities Education Improvement Act, 20 U.S.C. § 1400 (2004).

12.

Lange

A. A.

McPhillips

Mulhern

Wylie

(2006). Assistive software tools for secondary-level students with literacy difficulties. Journal of Special Education Technology, 21(3), 13–22. doi:10.1177/016264340602100302

13.

Lee

Yoon

S. Y.

(2017). The effects of repeated reading on reading fluency for students with disabilities: A meta-analysis. Journal of Learning Disabilities, 50, 213–224.

14.

Marcus

Wilder

D. A.

(2009). A comparison of peer video modeling and self video modeling to teach textual responses in children with autism. Journal of Applied Behavior Analysis, 42, 335–341. doi:10.1901/jaba.2009.42-335

15.

McMahon

Cihak

D. F.

Wright

R. E.

Bell

S. M.

(2016). Augmented reality as an instructional tool for teaching science vocabulary to postsecondary education students with intellectual disabilities and autism. Journal of Research on Technology in Education, 48, 38–56. doi:10.1080/15391523.2015.1103149

16.

Mechling

L. C.

Hunnicutt

J. R.

(2011). Computer-based video self-modeling to teach receptive understanding of preposition by students with intellectual disabilities. Education and Training in Autism and Developmental Disabilities, 46, 369–385.

17.

Morgan

J. J.

Yeager

K. H.

Murphy

S. J.

Spies

T. G.

(2018). Legal and ethical considerations regarding the integration of assistive and educational technology for students with disabilities: Perspectives from the United States of America. In Trimmer

Dixon

Findlay

Y.S.

(Eds.) The Palgrave handbook of education law for schools (pp. 397–411). Cham, Switzerland: Palgrave Macmillan.

18.

Morlock

Reynolds

J. L.

Fisher

Comer

R. J.

(2015). Video modeling and word identification in adolescents with autism spectrum disorder. Child Language Teaching and Theory, 31, 101–111 doi:10.1177/0265659013517573

19.

Moss

(2004). Teaching expository text structures through information trade book retellings. Reading Teacher, 57, 710–718.

20.

Nation

Clarke

Wright

Williams

(2006). Patterns of reading ability in children with autism spectrum disorder. Journal of Autism and Developmental Disorder, 36(7), 911–999.

21.

National Governors Association Center for Best Practices & Council of Chief State School Officers. (2010). Common Core State Standards for Language Arts. Retrieved from http://www.corestandards.org/ELA-Literacy/

22.

Ricketts

(2011). Research review: Reading comprehension in developmental disorders of language and communication. Journal of Child Psychology and Psychiatry, 52(11), 1111–1123.

23.

Ricketts

Jones

C. G.

Happé

Charman

(2013). Reading comprehension in autism spectrum disorders: The role of oral language and social functioning. Journal of Autism and Developmental Disorders, 43, 807–816. doi:10.1007/s10803-012-1619-4

24.

Rose

D. H.

Gravel

J. W.

Domings

(2012). Universal design for learning “unplugged”: Applications in low-tech settings. In Hall

T. E.

Meyer

Rose

D.H.

(Eds.) Universal design for learning in the classroom (pp. 120–134). New York, NY: Guilford.

25.

Therrien

W. J.

(2004). Fluency and comprehension gains as a result of repeated reading: A meta-analysis. Remedial and Special Education, 25, 252–261. doi:10.1177/07419325040250040801

26.

U.S. Department of Education. Institute of Education Sciences, National Center for Education Statistics. (2017). National Assessment of Educational Progress Reading Report Card. Retrieved from https://www.nationsreportcard.gov/reading_2017/#/nation/scores?grade=4

27.

Vrasidas

Glass

C. V.

(Eds.). (2010). Preparing teachers to teach with technology: Current perspectives on applied information technologies. Greenwich, CT: Information Age.

28.

Whalon

K. J.

Al Otaiba

Delano

M. E.

(2009). Evidence-based reading instruction for individuals with autism spectrum disorders. Focus on Autism and Other Developmental Disabilities, 24, 2–15.