Social Engagement of Elementary-Aged Children With Autism Live Animation Avatar Versus Human Interaction

Abstract

Social engagement is critical to the personal and social development of children. Individuals with Autism Spectrum Disorder (ASD) often exhibit low levels of social engagement with peers and adults, with clearly fewer instances of eye contact, initiation, joint attention, and empathy. Given the pivotal role of social engagement, procedures that naturally evoke higher rates of social engagement may be critical. Prior research suggests that cartoons and other animated characters are more appealing to children with ASD than traditional face-to-face interactions with adults. However, this assertion lacks consistent empirical support. In this study, five elementary age students with ASD participated in a series of interactions with adults and live animation avatars. An alternating treatments design was used to evaluate participants’ social engagement comparing their interactions with avatars to those with humans. Data show all five participants were consistently more attentive to the avatar than to human interventionists. In addition, all participants and their parents indicated the procedures were socially valid (i.e., preferred, feasible, and effective).

Keywords

autism < exceptionalityceptionality small N/single subject design < methodologies technology integration < technology perspectives live animation avatars engagement

Autism spectrum disorder (ASD) has become one of the most widely discussed and researched disabilities. According to the Autism and Developmental Disabilities Monitoring Network under the US Centers on Disease Control (2018), one out of 59 children has ASD. Children with ASD commonly experience impairments in functional communication, emotion recognition, and social interaction (Hooper, Poon, Marcus, & Fine, 2006). Common difficulties in social interaction are often associated with inability to prioritize social interaction over sensory stimulation and high rates of repetitive actions like hand-flapping or stereotypies (Goldman et al., 2008). Social and communication deficits along with behavioral impairments are pervasive in young children with ASD and without intervention can persist through adulthood (American Psychological Association, 2013; Hotton & Coles, 2016).

Communication deficits combined with social impairments may cause individuals with ASD to experience related academic problems, social frustration, and limited access to preferred professional opportunities. Individuals with ASD, regardless of age, have reported lower quality of life than their peers without disabilities, a discrepancy not explained by intelligence or symptom severity alone (van Heijst & Geurts, 2015). Quality of life differences may be attributed to pervasive academic deficits, especially in reading comprehension and problem solving (Keen, Webster, & Ridley, 2016). Many individuals with ASD will likely struggle to establish meaningful relationships and/or live independently (Howlin & Goode, 1998). The common theme with many of these persistent challenges is the inability to sustain social and professional relationships.

Social Engagement

One of the earliest and most important challenges for individuals with ASD is sustaining social engagement (Mundy, Sigman, & Kasari, 1990). Social engagement is not always clearly defined in the research literature, but it appears to be best characterized as active participation in a social situation including reciprocal play, communication, and attention (Bellini, Akullian, & Hopf, 2007). An essential aspect of social engagement begins with the ability to focus on a shared social experience with another person, often referred to as joint attention (Bakeman & Adamson, 1984). Individuals with ASD may exhibit limited joint attention when they fail to make eye contact, orient toward relevant stimuli, or follow gestures, signals, or prompts from an adult or peer in a social situation. Many have long-term deficits in pro-social interaction due to failure to acquire these behaviors in infancy (Clifford & Dissanayake, 2009). The ability to maintain appropriate eye contract is critical to joint attention and integral to social engagement (Cook et al., 2017).

Technology-aided Instruction and Interventions to Improve Social Engagement

In 2015, two large-scale systematic reviews of the ASD research literature were published (National Autism Center, 2015; Wong et al., 2015). Both review teams stated their purpose as to identify empirically-supported practices beneficial to individuals with ASD. Of the many practices identified in both reviews, strong support was found for behavioral interventions including modeling, prompting, providing antecedent interventions, and including reinforcement schedules to improve social engagement in individuals with ASD. Wong and colleagues (2015) titled one of their identified categories of empirically supported practices “technology-aided instruction and interventions” (TAII). They designated TAII as an intervention type “in which technology is the central feature supporting the acquisition of a goal for the learner” (p. 1960). In terms of ASD, technology is often broadly defined as any electronic platform used to facilitate therapeutic outcomes through the systematic application of hardware (e.g., computers, web resources, mobile devices, and virtual reality) and software subject to human, contextual, and other idiosyncratic influences (Odom et al., 2015). Studies investigating the impact of TAII found beneficial effects across a variety of dependent variables including social function, communication, social behavior, joint attention, school readiness, and academic skills.

Interest in TAII has increased as evidence suggesting that some individuals with ASD lack interest in traditional instructional models and may not benefit from the identified practices due to inadequate social engagement become more frequent. For example, McLaughlin and Carr (2005) indicated that rapport, which directly impacts teaching students with ASD, may be negative if a student associates a particular teacher with aversive activity such as chores or difficult tasks the student wants to avoid. To escape the demands of these activities, students may engage in behavior including physical aggression, threats, and other dangerous actions (Geiger, Carr, & LeBlanc, 2010; McLaughlin & Carr, 2005). Teachers and other interventionists may need to find alternative methods and strategies that make the learning experience more enjoyable and valuable to the student (Ohtake, Takahashi, & Watanabe, 2015).

Some authors hypothesize that TAII might appeal to children’s and adolescents’ common interest in video, animation, and other digital media (Goldsmith & LeBlanc, 2004). Technology allows the flexibility to incorporate restricted interests in ways potentially more focused and less abstract than traditional methods. Evidence suggests that indirect instructional methods that rely on pre-recorded videos can be effective and preferred by individuals with ASD (Kellems & Morningstar, 2012; Pond, 2017). Current advances in technology including streaming video and live animation enable new and sophisticated instructional methods with potential to capitalize on learning strengths of individuals with ASD (Wainer & Ingersoll, 2011) and are thus preferred over low-tech methods (Goldsmith & LeBlanc, 2004; Pond, 2017).

Live Animation

A new and promising TAII delivers instruction through live animation in which an animated character (avatar) is controlled by a human (avatar pilot) using a combination of hardware and software tools to allow real-time interactions with a preferred avatars (Klin, Lin, Gorrindo, Ramsay, & Jones, 2009). Piloted by trained interventionists, these avatars can interact with a child with ASD naturally, adapting instruction dynamically based on the individual’s responses without requiring complex programming or expensive revisions. As technologies like live animation become more accessible, its potential use for individuals with ASD is increasingly feasible (Ploog, Scharf, Nelson, & Brooks, 2013).

Unfortunately, research on the effectiveness of live animation in teaching individuals with ASD is limited. In one successful study, Hopkins et al. (2011) evaluated the effects of a computer-based instructional program featuring an avatar on the social skills of individuals with ASD. The researchers utilized FaceSay, a software program designed to teach individuals with ASD to accurately identify emotions exhibited by an avatar. Participants in the study were more likely to correctly identify emotions modeled by the avatar than by humans, and these skills generalized to human models in typical settings. One of the authors’ most important anecdotal findings was that the participants were highly engaged in the intervention procedures. They wanted to work on the computer and repeatedly asked for opportunities to use the technology.

Animation can be especially helpful in teaching necessary skills that are difficult to teach using other methods (Knight, McKissick, & Saunders, 2013). For example, teaching students how to properly cover themselves during urination. Ohtake et al. (2015) used animated elements in video hero modeling (VHM) to teach bathroom skills to Shinnosuke, a 12-year-old male student with ASD. Similar to traditional video modeling, VHM uses video to model the correct completion of a target skill, but the model is a preferred animated character. The student watches the video immediately before engaging in the target behavior. Consistent with the assertions from Bellini and Akullian (2007), this strategy is based on the hypothesis that the student will be more attentive while watching video modeling by a loved or admired character. Before participating in VHM, Shinnosuke was unsuccessful in learning the toileting skills from traditional in-vivo modeling and prompting (e.g., verbal, visual, and gestural). After receiving VHM daily for a month, Shinnosuke improved his performance in all the targeted toileting skills.

Similarly, animation provides a safe way to expose individuals with ASD to high-risk situations that might require individual emergency and lockdown plans (Clarke et al., 2014). Self, Scudder, Weheba & Crumrine (2007) created compared static video models of safety procedures during a natural disaster with virtual reality models created to provide immersive experiences when training safety skills during emergencies. They found that the immersive, dynamic environments contributed to faster acquisition of the target skills while avoiding exposure to dangerous or disruptive stimuli commonly used in emergency preparedness training. Animation provides interventionists the flexibility to incorporate idiosyncratic preferences into interventions and to design learning experiences associated with a diverse set of stimuli and experiences.

Despite these promising applications, Carter, Williams, Hodgins, & Lehman (2014) reported contrasting results. These researchers compared the level of social engagement and accuracy of responding in children with autism interacting with humans and with human-controlled avatars. The avatar pilot in this study was a trained actor with experience interacting with large groups and individuals in an interactive theme park setting. The actor spoke in Australian accent while he interacted with audience members and the other pre-recorded animated characters that were part of the scene. They observed 12 participants with autism during four conditions: interaction with a human therapist, interaction with a human-controlled interactive avatar, interaction with a human actor speaking like the avatar, and interaction involving a cartoon character. The researchers reported no statistically significant differences in the level of attention the participants exhibited in the four different conditions. The findings of Carter et al. (2014) appear to contradict previous research on the potential engagement of live animation and cast doubt on the viability, utility, and potential value of live animation as an intervention platform.

Purpose of the Present Study

The purpose of the present study is to evaluate the effects of live animation compared with traditional in vivo interactions on the social engagement of individuals with ASD. We also aimed to evaluate the feasibility of traditional interventionists (e.g., teachers and paraeducators) piloting the avatar. Much of the aforementioned literature on live animation involved the use of highly trained actors or individuals with significant training on the technology platform of interest. There is very little published research on the use of live animation with traditional interventionists with limited training or experience with the technology. This is unfortunate because most schools and clinics will not have the resources to engage expert pilots as interventionists. If live animation avatars are going to be widely used, then the technology will need to be accessible and user-friendly enough to be used by teachers. This study examined the effects of TAII using live animation on social engagement of individuals with ASD compared to in vivo interactions with humans.

Questions for Research

The research team conducted this study to answer two questions:

1. To what extent do individuals with ASD exhibit different levels of social engagement during interactions with an avatar compared to interactions with a human?

2. Is an avatar a socially valid method of interacting with individuals who have ASD?

Method

The institutional review board at the sponsoring institution approved all research procedures, recruitment materials, data management protocols, and consent procedures. Prospective participants were contacted and screened by a member of the research team to determine if the study was of interest and if the participant met all the inclusion criteria. Next, the first author and a second member of the research team met with the caregivers of the prospective participants and explained the study. If caregiver offered their consent, then the research team and the child’s caregiver met with the prospective participant to explain the study and obtain their assent to proceed.

Participants

Five males with ASD, between the ages of 8 and 10, were recruited from a specialized school primarily serving students with ASD. Inclusion criteria included (a) parental consent, (b) ASD diagnosis, (c) ability to communicate verbally, (d) successful completion of a screening document, and (e) deficits in communication as measured by Social Skills Improvement System (SSIS). Individuals between ages 8 and 10 were targeted for this study because we the research team had experience working with individuals with disabilities in these age ranges and we designed the procedures to be age-appropriate for individuals within this target age range.

A member of the research team screened all prospective participants using a brief phone questionnaire, asking parents first to specify the child’s age and autism diagnosis. If the child was within the appropriate age range and had a formal diagnosis, we continued the screening by asking parents about the prospective participant’s verbal and communication ability and his social skills, then discussed sharing testing information with the research team to confirm what was learned during the interview. Concerning verbal communication ability, the researchers asked first if the child could communicate verbally and if he could describe his preferences, as our methodology required participants to respond vocally to a series of simple questions about preference. Thus, parents were given the opportunity to indicate whether or not their child had the target skills and abilities. The next questions asked about social skills, including whether the child could recognize and express emotion in himself and others, and how the child would respond talking to an animated character. None of the prospective participants had prior experience speaking to an avatar using live animation, but all had extensive experience viewing multimedia on computer or tablet screens similar to those reported in previous research (Mazurek, Shattuck, Wagner, & Cooper, 2012). We then asked for information related to an autism diagnosis and/or information concerning special education qualification. Interviewees representing children who did not meet all criteria were notified at this point.

All selected participants had an educational or medical diagnosis of ASD and low social functioning levels (see Table 1). Results from the GARS-3 and ADOS-2 Module 1 tests indicated that all of them either met criteria for ASD or were very likely to have ASD. On their first visit, the parent or guardian of each participant completed the parent questionnaire of the SSIS, which confirmed the results of the participant’s comprehensive evaluation indicating high likelihood of having autism. All participants had scores under the 15th percentile in the Social Skills section of the SSIS prior to beginning the study. Similarly, participants' test scores from the Vineland-II indicate that communication and socialization (social skills) abilities were lower than average. Specific tests on language acquisition or ability level were not reviewed, since the study focused on behaviors associated with social engagement and not verbal communication ability.

Table 1.

Participant Characteristics and Test Scores.

Participant		Cognitive Ability		Autism			Adaptive Behavior			SSIS
Participant		Test	FSIQ	Score	Probability	SSIS Autism Raw Score	Composite (Percentile)	Communication (Percentile)	Socialization (Percentile)	Social Skills (Percentile)
Morty	10	SB	116	114^a	Very likely	29^c	61 (1)	65 (99)	51 (79)	70 (4)
Jerry	10	WISC-IV	128	108^a	Very likely	30^c	89 (23)	93 (32)	81 (10)	66 (2)
Larry	8	WCJ-IV	95	17^b	Met criteria	27^c	77 (6)	91 (27)	85 (16)	81 (11)
George	8	WPPSI-IV	64	112^a	Very likely	33^c	75 (5)	79 (8)	68 (2)	60 (1)
Kramer	9	WISC-V	95	Medical diagnosis		25^c	42 (21)	46 (34)	52 (55)	72 (5)

Note. Unless noted, all scores are standard scores. SB = Stanford Binet.

^aScore from the GARS-3 test.

^bScore from ADOS-2 Module 1 test. Adaptive behavior scores for Morty, Jerry, Larry, and George were from the Vineland II, and Kramer’s score is from the Behavior Assessment System for Children (second ed. BASC-2).

^cAbove average raw scores.

Settings and Equipment

The study was conducted in a controlled educational lab on a university campus, which included a large intervention room with an attached observation room separated by one-way windows. The lab was also equipped with a video recording system having cameras at three different angles. All sessions took place in the same room. For sessions using the avatar, the equipment was set up before the participant entered the room. To ensure consistency across both conditions, the equipment in the room was always located in the same place, and the participants were not permitted to touch or directly manipulate the equipment. If a participant approached or touched the equipment they were politely redirected back to their seats.

Intervention Personnel

During sessions, an interventionist was present in the room or the avatar was visible on the screen. Three individuals were trained as interventionists. The avatar pilot was one of two trained research assistants—both female undergraduate students studying special education. The person controlling the avatar and the interventionist were randomized each session to control for carry-over effects. The participants did not meet or see the individual controlling the avatar at any point during the study.

The Avatar

The avatar used in the study was Marla, a tropical fish similar to some of the characters in the movie Finding Nemo. This avatar was selected because it was highly rated by children of similar ages in our pilot sessions and its colorful scales and dynamic background were engaging. We also considered the technical aspects of controlling Marla and believed that her animations and functions were a good match for this particular study. Marla’s facial expression and the direction of her eyes could be changed. For this study Marla’s emotions were set at neutral, and her eyes were adjusted so they were always directed at the participant. Finally, an animated fish was selected over the human animated characters because the animations of the human characters were often distracting and unpleasant to the children in the pilot sessions.

The pilot (a research assistant) controlled the movements of the avatar using a Dell Inspiron 7559 laptop with a Waycom Intuos touch tablet, with Invirtua 3d Digital Puppeteer software (Invirtua, 2016). Marla was displayed on a 32-inch Dell monitor with a webcam that projected the participant on the screen so the avatar pilot could see him. The laptop used to control the avatar was located adjacent to the room housing the monitor and webcam, connected via HDMI cable. The avatar pilot had full control and was able to communicate in real time with the participant.

Avatar Training

To pilot the avatar, the two members of the research team were trained to use the computer, Waycom Intuos tablet, and software. Pilots attended at least five sessions to practice using the technology and were asked to help develop step-by-step instructions for each part of setup and piloting process. This activity was intended to support the training of future pilots as well as encourage acquisition and development of the skills necessary to operate the technology. Finally, three pilot sessions were conducted with children without disabilities in the target age ranges to identify any potential problems or issues prior to active data collection.

Measures

Four behaviors associated with social engagement were selected for data collection: making eye contact, maintaining correct body position, appropriately attending, and responding appropriately. These variables were selected as critical components of social engagement derived from the research literature that could be operationally defined for direct observation. Eye contact was defined as head and shoulders oriented toward the speaker. Correct body position required sitting upright in the chair with the entire back touching the chair and feet touching the floor in front of the chair or hanging off the front of the chair. Appropriate attending was defined as the subject was not to be touching or manipulating clothing, picking at fingernails or skin, touching or rubbing hair on the head or arms, scratching excessively, flapping hands, rocking excessively, or making repetitive vocalizations (e.g., repeating questions multiple times and repeatedly greeting people in the room). Appropriate responding was recorded when the child did not talk when the avatar/human was speaking and responded within five seconds after a question was asked with a contextually appropriate vocalization.

Social engagement for this study was defined as the percent of intervals during which the target behaviors were observed. To calculate social engagement for each session, a member of the research team divided each session into 10-second intervals. The presence of each target behavior was coded using whole-interval recording (Cooper et al., Heron, & Heward, 2007). After coding for the presence of each of the four target behaviors during a given 10-second interval, a member of the research team calculated the social engagement for that interval by summing the total number of target behaviors observed and dividing by the total number of target behaviors. For example, if a participant made eye contact, remained seated, and responded appropriately for the full 10 seconds then three of the four target behaviors were observed and the percent socially engaged for that interval would 75% (i.e., three behaviors observed divided by four possible behaviors for each interval). With the percent socially engaged calculated for each interval, the member of the research team coding that session would calculate the percent socially engaged for the session by averaging the percent engaged for each interval across the total number of intervals for that session.

Data Collection

All sessions were audio and video recorded. The average session length was 12 minutes with the shortest session lasting approximately 10 minutes and the longest session lasting 24 minutes. A set number of questions was asked every session, so the length of sessions was determined by participant responses. The intervention setting included three different cameras to provide multiple angles of every interaction throughout the session. One camera was set up to provide a direct shot of the participant’s face throughout the session; the other two cameras were mounted on the ceiling to provide angled views of the interaction. After recording, videos were uploaded to Box and shared with two trained coders. The videos were available to be watched and re-watched on a browser with an interface allowing coders to stop, start, record time stamps, and alter the speed of playback if necessary. Members of the research team entered data on an Excel data sheet while watching the recordings from each session.

Interobserver Agreement

Two independent observers watched each recorded session and independently coded instances of the target behaviors. These coders were undergraduate research assistants who were not directly involved in running sessions for this project but were interested in gaining research experience by working on the project as data collectors. The second author met with each student and provided training on how to code videos. Training was ended when the data collector coded a mock session with 80% agreement with the scores from the second author. On average, 70% of the sessions were dual coded and interval-by-interval agreement was calculated (Cooper et al., 2007). The percentage of agreement was calculated by dividing the total number of agreements by the total number of opportunities to agree and multiplying by 100. Agreement exceeded 80% for nearly every variable (see Table 2). Exceptions included two variables for Morty (listening/responding and body positioning) and one for George (eye contact). All exceptions were reviewed by all members of the team before being entered into the final data set. Disputes were resolved by consensus after examining the operational definitions and reviewing the recorded clip in question.

Table 2.

Interobserver Agreement for Each Condition and All Dependent Variables.

Participant	Sessions Coded (%)	Human				Avatar
Participant	Sessions Coded (%)	Eye Contact (%)	Body Position (%)	Attending (%)	Responding (%)	Eye Contact (%)	Body Position (%)	Attending (%)	Responding (%)
Morty	75	94	100	90	75	81	99	77	83
Jerry	88	95	90	91	93	83	92	92	92
Larry	63	100	99	97	91	70	96	81	100
George	63	94	93	100	95	82	100	87	100
Kramer	25	97	100	100	97	100	94	94	94

Note. The percentage of sessions coded represents the percentage of all sessions. All other percentages represent the within-session percentage agreement using the interval-by-interval method.

Research Design

An alternating treatments design (Cooper et al., 2007) was used to explore the effects of a live animation avatar on the social engagement of five individuals with ASD. An alternating treatments design provides a means by which researchers can compare two intervention conditions by alternating between these conditions and examining the effects on a repeated measures outcome of interest. In this case, the avatar and human conditions were alternated and a measure of engagement was collected throughout each session. The alternating treatments design is recognized as appropriate for experimental single-case research by the What Works Clearinghouse (WWC, 2017) and our use of the design meets the WWC design standards with reservations and the quality indicators for single-subject research set forth by Horner et al. (2005).

Procedures

Participants were randomly assigned to begin with either the avatar or the human condition; they alternated between the avatar and the human to counterbalance conditions within and across participants. Only one human and one avatar session were completed per day. Regardless of condition, the interventionist met the participant at the door of the observation room to begin each session and remained present at all times. A parent was present in the room for all sessions but did not interact with the participant, who sat with his back to his parent to reduce the potential for distractions. The research team was comprised of researchers and pre-service special educators with experience observing instruction in the specialized school from which participants were recruited. In addition, all but one of the team members had direct experience delivering instruction in schools to individuals with ASD. The appropriate stimulus (human or avatar) was present in the room and asked the prescribed questions on 100% of sessions.

Questions

We created 180 questions categorized into 12 domains with 15 related questions per domain. Domains included family, school, favorites, emotions, and interests, along with “would you rather” and “what if” questions. For example, the animal domain included “What is your favorite animal?” A question in the family domain was “What does your family do for fun?” The question set was created to reduce the likelihood that participants would see the same question twice. We worked to create a diverse set of questions that would be fun to answer with comparable difficulty across sessions. The questions were reviewed by educators who had experience working with children with disabilities in this age range. Question sets were randomly selected prior to the session. We did not evaluate the accuracy of the participants response only whether or not their response was contextually appropriate. For example, if the participant indicated that his favorite animal was a bird, we would mark that as an appropriate response regardless of whether or not his caregivers, friends, or teachers would agree that this is his favorite animal. If participants did not answer a question within 10 seconds, the interventionist or avatar moved onto the next question on the list.

Avatar Condition

For the avatar condition, the equipment needed was set up and tested before the participant entered the room. A table with a 20-inch monitor was placed 4 feet in front of and facing away from the one-way window. In front of the table, two chairs were placed facing the monitor 4 feet from the table and 2 feet from each other. The equipment was always placed in the same position. Before each participant’s first session with the avatar, the interventionist followed a script to introduce the participant to the avatar before she appeared. After the introduction, the interventionist invited the avatar onscreen and engaged in brief small talk with her to demonstrate how to properly communicate with an avatar: The participant was to face the screen and speak clearly in a loud enough voice for her to understand. The avatar then read the questions selected for the session verbatim. Responses to participants' comments were limited to a single affirmation after each statement. Follow-up questions were permitted only if something was unclear to the avatar pilot.

Human Condition

For the human condition, two chairs were placed facing each other 5 feet apart: one for the interventionist and one for the participant. The monitor displaying the avatar was not in the room during the human condition. Before the first session with a participant, the interventionist introduced herself to the participant and explained that she was going to ask some questions and the participant should do his best to answer them. The interventionist then read the questions selected for that session to the participant verbatim. Affirmations and follow-up questions were limited identically across conditions. The interventionists were instructed to interact with the children as they would any child in a school setting.

Data Analysis

The data were analyzed using visual analysis (Wolfe, Barton, & Meadan, 2019). Visual analysis is a procedure for the visual inspection of graphical summaries of data. The level, trend, and variability are evaluated to determine if there are differences between the data paths across research conditions. In our use of an alternating treatments design, we were particularly sensitive to level changes across conditions but also calculated averages and slope coefficients to provide additional indicators for comparison.

Social Validity

Following each participant’s final session, the parent and the participant each had a conversation with a researcher and answered open-ended questions from a social validity survey regarding their experience with and opinions of the social skills intervention. The content of the questions was informed by the Intervention Rating Profile (Briesch, Chafouleas, Neugebauer, & Riley-Tillman 2013) and the methodology by recommendations from other measures of social validity (Leko, 2014). The purpose was to determine the social importance of the goals, procedures, and outcomes. All parents were asked the same questions, as were all participants. All of these social validity sessions were videotaped so the responses could later be transcribed and coded using a qualitative data process, as described by Merriam (2002), so themes could be identified. Parents were asked the following questions:

1. How did your child feel about talking with the avatar?

2. Did your child talk about the avatar with you or with friends in your home? If so, what kinds of things did the child (ren) say?

3. What changes, if any, did you notice in your child’s social skills or communication ability after this experience?

4. Was this experience successful in improving your child’s communication ability in different locations (at home, school, etc.)?

5. Do you think your child’s behavior is severe enough to justify using this technology?

6. Would you be willing to participate in a follow-up study using an avatar?

Participants were asked the following questions

1. How did you feel about talking to the avatar?

2. Would you rather talk with the avatar or with someone else?

Results

Social Engagement During Human Sessions

All data are summarized in Figure 1. On average, participants exhibited all target behaviors during 24% of the intervals during the human sessions. With all five participants very little variability was observed in social engagement across all sessions (range = 14%–31%). Within-subject variability was highest with Morty, ranging from 14% to 27% engaged. There was limited evidence of an increasing or decreasing trend in the data. Morty and Kramer had near zero slope coefficients. Jerry’s data had a slight positive coefficient (b = 0.013), and George and Larry had slight negative coefficients (−0.054 and −0.014, respectively). Overall, the social engagement data were low and stable during the human condition for all participants.

Figure 1.

Percentage of engagement during the human and avatar conditions for all participants.

Social Engagement During Avatar Sessions

Participants exhibited higher levels of social engagement during the avatar sessions than during the human condition, with full engagement (i.e., all four engagement behaviors observed during the interval) noted during 43% of intervals, on average for all participants. These data were more variable and, in some cases, exhibited stronger trends than data from the human sessions. Variability was low in the Avatar sessions for Jerry (range = 31%–48%), Kramer (range = 39%–53%), and Larry (range = 37%–46%). George’s data were highly variable, ranging from 46% engagement in the second avatar session to 92% in the final avatar session. Morty was highly engaged during the first two avatar sessions, with percentage of engaged intervals at 86% and 72%, respectively. However, Morty’s social engagement was lower on the final two sessions, at 21% and 25%, respectively.

Visual analysis for these data revealed a consistent pattern of higher social engagement for George, Kramer, and Larry during the avatar condition. The data for Jerry and Morty were less compelling. Jerry’s social engagement during the avatar sessions was higher in all adjacent comparisons; however, his social engagement in the second human session was equivalent to the level observed during the final two Avatar sessions. Morty’s social engagement data during the Avatar sessions had the strongest trend observed across all sessions and participants, with a negative slope coefficient of −0.325.

Social Validity

During the open-ended social validity interviews, parents uniformly felt their child enjoyed coming and talking to the avatar and believed this experience had been successful in improving their child’s communication ability outside the clinic. Parents also agreed that their child’s needs were significant enough to justify the use of live animation avatars in an intervention. When asked whether the participant talked about the avatar outside of the study, most of the parents noted that their child shared their experiences with the avatar at home with their families or with friends. One parent qualified her response: “I don’t think so, just because it’s not something they would [likely] volunteer that information.” Parents also reported that they observed positive changes in their child’s communication skills including “making a new friend at swim team,” “just trying to talk more with people, because usually he would just stick by himself and not really interact with anyone,” and “talking with a new therapist he caught himself interrupting and. . . Stopped himself.” One parent commented that the child was “just becoming more aware.” All of the parents responded that they would be willing to support their child’s participation in a follow-up study using an avatar.

Participants' comments when asked generally about the avatar ranged from “loved it” to “not quite sure at first, but [eventually] liked it” and “not quite sure what to think, but [was] curious.” Participants also said that their experiences in the study were “great” and “fun but weird.” Overall, all but one of the participants indicated that they enjoyed talking to the avatar and preferred speaking with an avatar over a human. The one participant who did not indicate that he enjoyed speaking with the avatar also indicated that he did not like speaking to people in general.

Discussion

The purpose of this study was to compare the social engagement of individuals with ASD interacting with a live animation avatar to their social engagement interacting with humans. On average the participants exhibited higher social engagement during the avatar sessions compared to lower, stable interaction levels with humans. The questions and procedures were identical; only the conversation partner—human or avatar—differed between conditions. For three of the five participants, a clear functional effect was evident, suggesting that conversation with the avatar evoked higher levels of social engagement for individuals with ASD than interaction with human interventionists. However, the data from two of the participants, Morty and Jerry, were not conclusive. In addition to demonstrating higher levels of social engagement, participants reported that they enjoyed talking with the avatar. All but one of participants indicated they would rather speak to the avatar than a human.

These findings support the common assertion that individuals with ASD prefer opportunities to interact with avatars or other TAII representations (Baron-Cohen et al., 2009; Goldsmith, & LeBlanc, 2004; Hopkins et al., 2011; Pond, 2017). The current study provided self-reported as well as direct observation data supporting the conclusion that individuals with ASD exhibited higher quality interactions with an avatar, but it did not seek evidence of why talking with an avatar might be preferred. Other researchers have suggested that TAII methods might be more engaging for individuals with ASD because fewer social demands are associated with the interactions (Moore & Calvert, 2000). This study involved sets of comparable questions asked in a similar context with a parent and intervention staff member present during both human and avatar conditions. Thus, it seems likely that reduced social demands that might have affected findings were due to aspects of the avatar’s visual display: perhaps the more colorful presentation of the avatar and its exaggerated facial expressions.

The current study contradicts the findings of Carter and colleagues (2014) that opportunities to watch animation or interact with an avatar did not produce higher levels of social engagement that might lead to more accurate responding. Our results indicated that the avatar was associated with higher social engagement. We controlled for possible confounding variables associated with the setting, interventionist training, and interaction types across the human and avatar conditions. Although we are uncertain why our results were different than some published previously, we consider our setup practical for clinical or school-based interventions. Experience of our interventionists in interacting with individuals with ASD and in using avatars was limited. We employed less expensive and more readily available technology than used in other studies.

Several participants experienced an apparent “novelty” effect of the Avatar on social engagement. For example, the data from the first Avatar sessions for Morty and Jerry show higher levels of social engagement than expected, given engagement during the human sessions; however, these rates steadily dropped off after the first few sessions. The percentages of social engagement in the final two Avatar sessions for these participants appeared indistinguishable from the human sessions. This pattern might also be interpreted in terms of the idiosyncratic preferences of these participants, having not been fully captured by the avatar. For example, many individuals with ASD exhibit strong sensitivity to auditory or tactile stimuli, which were not intentionally manipulated in the presentation of the avatar in this study. These preferences are a variable that could be controlled in future studies.

Limitations

The data for all participants was lower than expected across all phases. Thus, the separation between data points for the avatar and human conditions was smaller than we had anticipated, although the overall pattern for all participants was consistent. In addition, for Morty, Jerry, and Larry there is a negative trend evident in the data suggesting that the initial interactions with the avatar were more engaging than later interactions. This might be explained by satiating on the novel aspects of the avatar over time or it might be related to doing the same task repeatedly. The latter assertion may be supported by the slight downward trend evident in the human condition data for Jerry and Larry. This low social engagement may be due to the nature of the activity (answering a long list of questions) or to the novelty of the clinical setting. Another possible explanation involves our operational definitions of the target behaviors observed for social engagement. Our standard for social engagement included clear orientation toward the speaker and minimal peripheral responses (e.g., fidgeting, touching hair, etc.). This definition of social engagement was sensitive enough to capture differences between these conditions but may not reflect the diversity of social expectations across contexts and cultures. Overall, the experimental procedures and the data collection practices may have contributed to difficult to interpret patterns in the data.

Several methodological limitations should be noted. First, participants were randomly assigned to either the avatar-first or human-first condition, but subsequent sessions alternated between these conditions rather than being randomized. Randomization is ideal, but both counterbalancing and randomization are recognized as appropriate methodologies in the research literature (What Works Clearinghouse, 2017). Furthermore, we found no evidence of a sequence effect explaining the patterns observed in the data. Randomization was not possible due to the constraints of taking down and setting up the technology for the avatar.

Second, the recommended number of data points per condition is five (What Works Clearinghouse, 2017). None of the participants met this standard, but all but one participant (George) met the design standards with reservations by having at least four data points in each phase. George had only three data points per phase, but his data were more consistent and the difference between human and avatar was more compelling than the patterns observed for other participants. Also, the What Works Clearinghouse standards (2017) require that studies demonstrate an effect at three different points in time; the study met this requirement. Ideally, we would have collected more data due to the variability across participants, but this was not possible due to time and resource constraints for both the research team and the participating families. Third, the questions used in the socially validity interview could have been worded in a way to reduce socially desirable responding and to be more open ended for participants and caregivers.

Finally, this study focused exclusively on engagement while interacting with a live animation avatar. Several issues relative to the utility of an avatar as an interventionist were not included in the data collection or recommendations for practice. For example, we do not collect regarding the fading of the avatar in an intervention setting. It is possible that individuals prefer working with avatars and continue to avoid direct interactions with humans and fail to develop the necessary relationship skills to meet their needs. Although this was beyond the scope of this investigation it is worth noting as a limitation. Also, the cost of the equipment and the training required to pilot the avatars may be prohibitive for many special education professionals.

Suggestions for Future Research

Future studies might explore the individual and contextual factors that lead to higher social engagement with live animation over traditional intervention practices. The variability in our findings suggests that additional uncontrolled variables, including idiosyncratic preferences, may need to be considered in future studies. In addition, this study did not include instruction on a specific skill, thus not demonstrating use of an avatar as an effective intervention strategy. Although there is some evidence that avatars with live animation can be effective intervention agents (Hopkins et al., 2011), future research on TAII using avatars could help to identify the extent to which avatars can be used to teach students new skills.

It is possible that access to preferred live animation avatars further exacerbates social avoidance in individuals with ASD. Specifically, if individuals with ASD rely on animated instructional experiences will they then miss out on opportunities to learn important skills from interacting with teachers and peers in person? Researchers might consider exploring opportunities to incorporate interactions with avatars and humans during the course of the intervention. This blended approach to intervention may help us understand more about the necessary conditions to facilitate in person social interaction through a history of interacting with avatars. In addition, researchers and practitioners would benefit from experimental investigation of generalization strategies to reduce reliance on avatars moving forward.

Implications for Practitioners

Practitioners might benefit from exploring the use of technology in their classrooms or clinics. The findings of this study suggest that individuals with ASD are more likely to attend to an avatar than to a human interventionist. Given the previously documented limits on social engagement in individuals with ASD, practitioners would benefit from employing technology that can be dynamically adjusted to incorporate existing motivation to learn and other idiosyncratic preferences. Also of note is that the avatar in this study was piloted by inexperienced undergraduate students majoring in special education. Thus, practitioners with extensive experience working with individuals with ASD and relatively little experience working with live animation avatars can effectively operate this technology. As extensive training is not necessary, this technology is feasible in applied settings with limited time and resources to support staff development. Strategies that can encourage social engagement in instructional settings are worth investment of our time, resources, and research.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by a grant from the McKay School of Education at Brigham Young University.

ORCID iDs

Ryan O. Kellems

Cade T. Charlton

Author Biographies

Ryan O. Kellems is an associate professor and Fulbright Scholar in the the Counseling Psychology and Special Education department at Brigham Young University. Prior to coming to BYU Dr. Kellems was on the faculty at the University of Oregon where he worked with the National Post-School Outcomes Center. His primary research interests are video modeling, transition for students with disabilities, post-school outcomes of students with disabilities, assistive technology, and working with young adults with autism in community-based vocational settings. One of Dr. Kellems areas of expertise is using the latest technology to assist students with disabilities as they transition to life after school. Prior to completing his doctoral degree Dr. Kellems was a high school teacher where he worked in both rural and urban settings as a special education and Social Studies teacher.

Cade T. Charlton grew up in St. Anthony, ID and is the son of two career educators. He was introduced to Applied Behavior Analysis (ABA) and Organizational Behavior Management as a freshman at Utah State University. With just enough knowledge to be dangerous, he worked with several private companies as an ABA service provider. Prior to completing the MBA program, Cade started working as a consultant with the Center for the School of the Future where he fell in love with helping teachers adopt and implement better services for at risk students. These consulting opportunities in priority schools coupled with federally- and locally funded research projects provided Cade with a wealth of opportunities to explore the application of the principles of ABA to improving organizational supports for individuals with disabilities at school.

Brooke Black majored in Experience Design and Management at Brigham Young University. She worked as a member of an undergraduate research team and continues to actively serve her professional and local community.

Heidi Bussey completed her doctorate in the Educational Inquiry, Measurement, and Evaluation program at Brigham Young University. She was immediately hired as an assistant professor in the Counseling Psychology and Special Education Program. After many contributions in special education, Dr. Bussey left the university to work as an entrepreneur in education.

Rachel Ferguson majored in Special Education at Brigham Young University. While studying special education, she diligently worked as a member of an undergraduate research team until her graduation. She is currently working as a special educator for Alpine Transition and Education Center (ATEC) in American Fork, Utah and is pursuing a graduate degree from the University of Kansas.

Bruna F. Gonçalves majored in Special Education at Brigham Young University. As a key member of her undergraduate research team, Bruna’s work was essential to multiple professional publications and other dissemination efforts. She is currently working for the Disability Services at the Church of Jesus Christ of Latter-Day Saints in Salt Lake City, Utah.

Mikaela Jensen majored in Special Education at Brigham Young University. Her work as a research assistant was remarkably productive during her time at BYU culminating in five publications in special education.

Sara Vallejo majored in Special Education at Brigham Young University. While studying special education, she diligently worked as a member of an undergraduate research team until her graduation.

References

American Psychiatric Association . (2013). Diagnostic and statistical manual of mental disorder (5th ed.). American Psychiatric Publishing.

Bakeman

Adamson

L. B.

(1984). Coordinating attention to people and objects in mother-infant and peer-infant interaction. Child Development, 55(4), 1278–1289. https://doi.org/10.2307/1129997

Baron-Cohen

Golan

Ashwin

(2009). Can emotion recognition be taught to children with autism spectrum conditions? Philosophical Transactions of the Royal Society of London B: Biological Sciences, 364(1535), 3567–3574. https://doi.org/ 10.1098/rstb.2009.0191

Bellini

Akullian

(2007). A meta-analysis of video modeling and video self-modeling interventions for children and adolescents with autism spectrum disorders. Exceptional Children, 73(3), 264–287. https://doi.org/10.1177/001440290707300301

Bellini

Akullian

Hopf

(2007) Increasing social engagement in young children with autism spectrum disorders using video self-modeling. School Psychology Review, 36(1), 80–90. https://doi.org/10.1080/02796015.2007.12087953

Briesch

A. M.

Chafouleas

S. M.

Neugebauer

S. R.

Riley-Tillman

T. C.

(2013). Assessing influences on intervention implementation: Revision of the Usage Rating Profile-Intervention. Journal of School Psychology, 51(1), 81–96. https://doi.org/10.1016/j.jsp.2012.08.006

Carter

E. J.

Williams

D. L.

Hodgins

J. K.

Lehman

J. F.

(2014). Are children with autism more responsive to animated characters? A study of interactions with humans and human-controlled avatars. Journal of Autism and Developmental Disorders, 44(10), 2475–2485. https://doi.org/10.1007/s10803-014-2116-8

Clarke

L. S.

Embury

D. C.

Jones

Yssel

(2014). Supporting students with disabilities during school crises: A teacher’s guide. Teaching Exceptional Children, 46(6), 169-178. https://doi.org/10.1177/0014402914534616

Clifford

Dissanayake

(2009). Dyadic and triadic behaviors in infancy as precursors to later social responsiveness in young children with autistic disorder. Journal of Autism and Developmental Disorders, 39, 1369–1380. https://doi.org/ 10.1007/s10803-009-0748-x

10.

Cook

J. L.

Rapp

J. T.

Mann

K. R.

McHugh

Burji

Nuta

(2017). A practitioner model for increasing eye contact in children with Autism. Behavior Modification, 41(3), 382–404. https://doi.org/ 10.1177/0145445516689323

11.

Cooper

J. A.

Heron

T. E.

Heward

W. L.

(2007). Applied behavior analysis (2nd ed.). Pearson.

12.

Geiger

K. B.

Carr

J. E.

LeBlanc

L. A.

(2010). Function-based treatments for escape-maintained problem behavior: A treatment-selection. https://doi.org/10.1007/BF03391755

13.

Goldman

Wang

Salgado

M. W.

Greene

P. E.

Kim

Rapin

(2008). Motor stereotypies in children with autism and other developmental disorders. Developmental Medicine and Child Neurology, 51(1), 30–38. https://doi.org/10.1111/j.1469-8749.2008.03178.x

14.

Goldsmith

T. R.

LeBlanc

L. A.

(2004). Use of technology in interventions for children with autism. Journal of Early and Intensive Behavior Intervention, 1(2), 166–178. https://doi.org/10.1111/j.1469-8749.2008.03178.x

15.

Hooper

S. R.

Poon

K. K.

Marcus

Fine

(2006). Neuropsychological characteristics of school-age children with high-functioning autism: Performance on the NEPSY. Child Neuropsychology, 12(4–5), 299–305. https://doi.org/10.1080/09297040600737984

16.

Hopkins

I. M.

Gower

M. W.

Perez

T. A.

Smith

D. S.

Amthor

F. R.

Wimsatt

F. C.

Biasini

F. J.

(2011). Avatar assistant: Improving social skills in students with an ASD through a computer-based intervention. Journal of Autism and Developmental Disorders, 41(11), 1543–1555. https://doi.org/10.1007/s10803-011-1179-z

17.

Horner

R. H.

Carr

E. G.

Halle

McGee

Odom

Wolery

(2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71(2), 165–179. https://doi.org/10.1177/001440290507100203

18.

Hotton

Coles

(2016). The effectiveness of social skills training groups for individuals with autism. https://doi.org/10.1007/s40489-015-0066-5

19.

Howlin

Goode

(1998). People with autism and Asperger's syndrome. In Volkmar

F. R.

(Ed.), Autism and pervasive developmental disorders (pp. 209–241). Cambridge University Press.

20.

Keen

Webster

Ridley

(2016). How well are children with autism spectrum disorder doing academically at school? An overview of the literature. Autism, 20(3), 276–294. https://doi.org/10.1177/1362361315580962

21.

Klin

Lin

D. J.

Gorrindo

Ramsay

Jones

(2009). Two-year-old with autism orient to non-social contingencies rather than biological motion. Nature, 459(7244), 257. https://doi.org/10.1038/nature07868

22.

Knight

McKissick

B. R.

Saunders

(2013). A review of technology-based interventions to teach academic skills to students with autism spectrum disorder. Journal of Autism and Developmental Disorders, 43(11), 2628–2648. https://doi.org/10.1007/s10803-013-1814-y

23.

Leko

M. M.

(2014). The value of qualitative methods in social validity research. Remedial and Special Education, 35(5), 275-286. https://doi.org/10.1177/0741932514524002

24.

Mazurek

M. O.

Shattuck

P. T.

Wagner

Cooper

B. P.

(2012). Prevalence and correlates of screen-based media use among youths with autism spectrum disorders. Journal of Autism and Developmental Disorders, 42(8), 1757–1767. https://doi.org/10.1007/s10803-011-1413-8

25.

McLaughlin

Carr

E. G.

(2005). Quality of rapport as a setting event for problem behavior: Assessment and intervention. Journal of Positive Behavior Interventions, 7(2), 68–91. https//doi.org/10.1177/10983007050070020401

26.

Merriam

S. B.

(2002). Qualitative research in practice: Examples for discussion and analysis. Jossey-Bass.

27.

Moore

Calvert

(2000). Brief report: Vocabulary acquisition for children with autism: Teacher or computer instruction. Journal of Autism and Developmental Disorders, 30(4), 359–362. https://doi.org/10.1023/A:1005535602064

28.

Mundy

Sigman

Kasari

(1990). A longitudinal study of joint attention and language development in autistic children. Journal of Autism and Developmental Disorders, 20(1), 115–128. https//doi.org/10.1007/BF02206861

29.

National Autism Center . (2015). Findings and conclusions: National standards project, phase 2. Author. http://www.nationalautismcenter.org/national-standards-project/results-reports/

30.

Odom

S. L.

Thompson

J. L.

Hedges

Boyd

B. A.

Dykstra

J. R.

Duda

M. A.

Bord

(2015). Technology-aided interventions and instruction for adolescents with autism spectrum disorder. Journal of Autism and Developmental Disorders, 45(12), 3805–3819. https://doi.org/10.1007/s10803-014-2320-6

31.

Ohtake

Takahashi

Watanabe

(2015). Using an animated cartoon hero in video instruction to improve bathroom-related skills of a student with autism spectrum disorder. Education and Training in Autism and Developmental Disabilities, 50(3), 343–355.

32.

Ploog

B. O.

Scharf

Nelson

Brooks

P. J.

(2013). Use of computer-assisted technologies (CAT) to enhance social, communicative, and language development in children with autism spectrum disorders. Journal of Autism and Developmental Disorders, 43(2), 301–322. https://doi.org/10.1007/s10803-012-1571-3

33.

Pond

M.H.

(2017). Challenging assumptions about adolescents with autism spectrum disorders and technology [Master's thesis, Institution Awarding Degree]. http://rave.ohiolink.edu/etdc/view?acc_num=ucin1504878501061312

34.

Self

Scudder

R. R.

Weheba

Crumrine

(2007). A virtual approach to teaching safety skills to children with autism spectrum disorder. Topics in Language Disorders, 27(3), 242–253. https://doi.org/ 10.1097/01.TLD.0000285358.33545.79

35.

van Heijst

B. F.

Geurts

H. M.

(2015). Quality of life in autism across the lifespan: A meta-analysis. Autism, 19(2), 158–167. https://doi.org/10.1177/1362361313517053

36.

Wainer

A. L.

Ingersoll

B. R.

(2011). The use of innovative computer technology for teaching social communication to individuals with autism spectrum disorders. Research in Autism Spectrum Disorders, 5(1), 96–107. https://doi.org/10.1016/j.rasd.2010.08.002

37.

What Works Clearinghouse

. (2017). Standards handbook (Version 4.0). US Department of Education. https://ies.ed.gov/ncee/wwc/Docs/referenceresources/wwc_standards_handbook_v4.pdf

38.

Wolfe

Barton

E. E.

Meadan

(2019). Systematic protocols for the visual analysis of single-case research data. Behavior Analysis in Practice, 12(2), 492–502. https://doi.org/10.1007/s40617-019-00336-7

39.

Wong

Odom

S. L.

Hume

K. A.

Cox

A. W.

Fettig

Kucharczyk

Schultz

T. R.

(2015). Evidence-based practices for children, youth, and young adults with autism spectrum disorder: A comprehensive review. Journal of Autism and Developmental Disorders, 45(7), 1951–1966. https://doi.org/10.1007/s10803-014-2351-z