Using a 3D Immersive Virtual Environment System to Enhance Social Understanding and Social Skills for Children With Autism Spectrum Disorders

Abstract

Children with autism spectrum disorders (ASDs) have a severe deficiency in social understanding and skills. The present study utilized single-subject design (N = 3) with multiple probes to investigate the effectiveness of a three-dimensional social understanding system with a head-mounted display to improve social understanding and skills in three children with ASD. We tested the proposed system based on immersive virtual environments. The target behaviors of non-verbal communication, social initiations, and social cognition for each participant and the impact of using immersive digital equipment with this population were examined simultaneously. A preliminary empirical study for three participants diagnosed with ASD was conducted over a 6-week period. The findings indicate that participants’ targeted behaviors improved from baseline to intervention through maintenance following their use of the system. These suggest that the system may present an effective learning environment for the promotion of social understanding and skills in children with ASD.

Keywords

single-subject experimental research virtual environment social understanding and social skills children with autism spectrum disorders

Introduction

Autism spectrum disorders (ASDs) represent a collection of problems related to the development of basic functions caused by a disorder related to brain development. Characteristic symptoms include difficulties in social understanding and language development, unusual patterns of repetitive behavior, and lack of imagination (American Psychiatric Association, 2000). Due to these problems, individuals with ASDs have difficulty with social understanding, use of appropriate gestures and social skills, maintenance of friendships, and involvement in social play (Machintosh & Dissanayake, 2006). In addition, these difficulties are direct or indirect consequences of social interaction deficits (Chung, Mosconi, Drewry, Matthews, & Tasse, 2007). Several researchers have indicated that promotion and reinforcement of social skills is important for the maintenance of those social skills in persons with ASDs (Fatima, Nazneen, Gregory, & Rosa, 2012; Odom & Watts, 1991). Similarly, Cheng and Ye (2010) suggested that individuals with ASDs could learn social skills through practical experiences of real situations. For example, Odom and Strain (1986) trained three young children with autism to initiate and share activities with peers. Teachers used prompts to increase their initiation and play. Their study demonstrated that the teacher’s prompts increased initiations in students with autism and enhanced their social responses. Specifically, effective social interaction skills (e.g., initiating greetings, play, imitating, sharing, taking turns, asking for help, and requesting things) were demonstrated and improved in the children with autism through guidance from the teachers. In addition, three of the four participants exhibited increased social initiations (SIs) and decreased problem behaviors at the end of the intervention (Gonzalez-Lopez & Kamps, 1997). The combination of modeling, promotion, reinforcement, and teacher guidance may be the most efficient and effective means of providing social skills training in individuals with ASDs (Devender, Stephanie, & Lan, 2010; Rebecca & Candice, 2010).

Technological advances can lead to novel and potentially more effective treatment strategies and enhance the quality of life of people with ASDs and their families (D. J. Brown, Neale, & Cobb, 1999). Virtual environments (VEs) are a form of computer-based learning centered on visual representation, which offers unique advantages for teaching abstract concepts, for example, social interaction (Herreraa, Jordanb, & Veraa, 2006). A similar theory, proposed by Suchman (2007), indicates that knowledge must be learned through practical eruditions and experiences via real situations. VEs offer the realism that makes generalization of treatment situations easier in people with ASDs. In addition, it is the most effective method of rehabilitation with respect to daily life skills (Lanyi & Tilinger, 2004; Strickland, McAllister, Coles, & Osborne, 2007). VEs provide users considerable flexibility in designing learning methods as they allow for customization of learning tasks and simulate three-dimensional (3D) versions of various social and instruction-based situations in a safe environment (Parsons, Mitchell, & Leonard, 2004; Putnam & Chong, 2008). Users can operate the system independently in a realistic simulated 3D world, which is a stress-free social environment (Josman, Milika Ben-Chaim, Friedrich, & Weiss, 2008). Interestingly, individuals with ASDs exhibit superior visual-spatial learning (Meadan, Ostrosky, Triplett, Michna, & Fettig, 2011). A number of researchers suggest that the features of VEs can be effectively applied to children with ASDs (Millen, Edlin-White, & Cobb, 2010; Parsons, Mitchell, & Leonard, 2005). The characteristics of virtual simulation systems may be effective to guide people with ASD in the acquisition of social understanding and skills, and to reduce inappropriate social behaviors (Kathy & Howard, 2001).

Some studies have used desktop VE to enhance social skills training. For instance, Andersson, Josefsson, and Pareto (2006) designed a game-based VE for social skills training. In this study, they offered the users opportunities to explore different social locations and try a variety of behavioral responses. The authors reported that the game-strategy VE improved social skills in children with autism. Similarly, Leonard, Mitchell, and Parsons (2002) and D. J. Brown, Neale, and Cobb (1999) developed “virtual café” and “virtual city” programs, respectively. Their studies intended to train the social skills (e.g., finding a place to sit down or catching a bus) of individuals with ASD. They reported that adolescents with ASD demonstrated that they can use the desktop VE to improve specific skills, and that some participants transferred their learning from the virtual world to the real world. Similarly, Charitos et al. (2000) used a scenario titled “Returning Home” in a VE application. It assisted children with autism to organize their behaviors with respect to daily activities. The study showed that it is essential to provide challenges at appropriate levels for children with ASDs, and this could enhance positive behaviors.

Situational learning that relies on the use of VE applications has been shown to be successful in teaching children with autism to initiate the principles of social skills by practicing repetition-function without limitation via technology-based systems (J. S. Brown, Collins, & Duguid, 1988; Neale, Leonard, & Kerr, 2007). For example, Cheng and Ye (2010) and Cheng, Chiang, Ye, and Cheng (2010) used a collaborative VE with a text-communication intervention based on a technology system to improve social interaction in children with ASDs. In addition, participants interacted with human users by engaging in various social events and used real-time communication (e.g., text, verbal communication, or facial expressions) to practice their target behaviors. These studies used a multiple-probe design to observe the performance of each targeted skill. Findings revealed that the collaborative VE enhanced target behaviors such as eye contact, appropriate social manners, listening to others, perspective-taking skills, and empathy. However, while designing social events in VE, we need to consider the nature of the presentation of virtual people, objects, and animated events for this population.

In recent years, the focus of display technology has progressed from the use of projection display systems to that of a head-mounted display (HMD). Immersive VE refers to VEs with highly visual, 3D, and completely immersive visual worlds, making them reliant on HMDs. Immersive VEs are thus superior to conventional VEs, which typically use projection display systems (Bayer, 2002; Takashi, 2002). In immersive VEs, users wear HMDs that consist of two small monitors attached to a high-speed computer with integrated head-position sensor controls, which control the direction from which the VE is viewed (Osterlund & Lawrence, 2012).

The advantage of HMDs is that they expand the field of view, are lightweight and small, and can present interactive spaces (e.g., using virtual theaters). As such, HMD technology could help overcome impairment in the ability to sustain attention, which is common in most people with ASDs.

A number of researchers have noted that children with ASDs have short attention spans (Clark, Feehan, Tinline, & Vostani, 1999; Rommelse, Franke, Geurts, Hartman, & Buitelaar, 2010), making it difficult for them to focus on sustained learning (Allen & Courchesne, 2001; Doyle & Arnedillo Sánchez, 2011; Golan & Baron-Cohen, 2006). Using a virtual HMD display mitigates this limitation and reduces distractions from the outside world. However, only a few studies have used HMDs for special education. Strickland, Marcus, Mesibov, and Hogan (1996) used immersive virtual reality with a virtual HMD as a learning tool for two autistic children, aged 7.5 and 9 years, to teach them to recognize the colors of cars and cross the street safely. Their study recommended using adjustable 3D VE and HMD tools with autistic children, to help participants understand virtual scenarios and the virtual world. They indicated that some users felt dizzy while wearing the HMD. Strickland et al. (2007) developed desktop VEs to teach fire safety skills to young (3–6 years old) children with ASDs. These included recognizing the danger of fire and responding appropriately (i.e., leaving the house swiftly and waiting outside in a predetermined place). They reported that 11 of the 14 children who took part completed the fire safety VE without error. In recent years, HMD technology has been improved, and more comfortable wearing equipment is now available.

A three-dimensional social understanding (3D-SU) system was developed in the present study and integrated with an HMD to enable individuals with ASDs to learn social understanding and skills. An HMD was used as it was expected to help participants with ASDs to concentrate on 3D social scenarios and teaching content. The purpose of this study was to evaluate the effectiveness of the 3D-SU system in enabling participants with ASDs to learn non-verbal communication (NC), SIs, and social cognition (SC), by examining the performance of each participant on each of the target behaviors, and assessing the impact of using immersive equipment.

3D-SU and Skills Systems

System Design

The 3D-SU system creates an immersive simulated environment that promotes social understanding in people with ASDs. The system enables users to isolate themselves from the physical environment and navigate in an immersive environment. It involves control factors such as degree of immediacy of control, anticipation of events, and mode of control (Witmer & Singer, 1998). Users can interact with virtual objects and humans when they look around the environments. For instance, users can use a mouse to “drag” coins into the charge-box when they board the bus. Similarly, in the classroom situation, they can “drag” a pen to draw on the drawing board. The system presents two scenes: a virtual bus stop and a classroom. Each environment presents animated social events, enabling users to experience these events more fully. Both were presented on a virtual HMD (Model: I-Glasses PC 3D Pro) and a workstation laptop (a Pentium 3650MHZ processor and 64 MB RAM running Windows XP). The system was developed using 3DMax and Poser software for creating 3D models. The interaction function was developed using the kernel system, the Virtools Scripting Language (VSL), and an object-oriented language (C++). The display version was designed as a first-person view (FPV), in which the participants used the virtual HMD to view virtual characters and environments. Participants wore a virtual HMD to survey 3D scenarios that relied on arrow keys or a joystick, to immerse themselves in the VEs without distractions. In addition, each social question in the system included both a text description of the two 3D social scenarios and an aural description for those with low reading competency (see Figure 1). The instructional features used in the 3D-SU system are presented in Figure 2.

Figure 1.

Architecture of the 3D-SU system.

Figure 2.

Instructional features used in the 3D-SU system.

System Content

The 3D virtual bus stop and classroom environments were selected because subjects frequently encounter these social settings. These social environments (specifically, the bus stop and classroom) were suggested by ASD experts and teachers and parents of children with ASDs (see Figures 3 and 4), as these two environments could provide frequent opportunities to practice social interactions. The teaching content involved 3D social modeling, promotion, and awards of reinforcement in the system design (Rebecca & Candice, 2010). Twenty-four problem-based social questions were developed for use with the system. Some examples are as follows: (a) Would you like to choose a seat on the bus? (b) Can I talk loudly on the bus? (c) Can I hide my classmate’s pen? (d) Why can’t I do this? The questions were read aloud automatically by the system, which enhanced the knowledge of the questions for the users. Questions were true or false or multiple-choice formats with one correct answer, and they assessed NC, SI, and SC. The contexts of the questions and scenarios were examined by teachers and parents of children with ASDs and an ASD expert. The system provided a reward message to the participants (e.g., “Great! You have done well”) when an appropriate response was clicked. The message was accompanied by animated applause. A wrong answer elicited a “Try again!” message. At the same time, the teacher provided the participant with hints to produce the appropriate response.

Figure 3.

Scenario 1 of the 3D-SU system.

Figure 4.

Scenario 2 of the 3D-SU system.

Method

A single-subject experiment was combined with a multiple-probe across-subjects A-B-C design, to determine the efficacy of the 3D-SU system in promoting the comprehension of social protocols in individuals with ASD. This design was for a baseline-(A), intervention-(B), and maintenance-(C) periods to observe behavioral differences after treatment. The behavioral changes observed in each participant were also investigated. The experimental analysis of target behaviors was conducted using systematic observation. In addition, each participant was observed separately, to ensure stringent control and better understanding of his or her behavior. In addition, each participant had different deficits, requiring provision of individual assistance to operate the system.

Participants

Three boys who had been diagnosed with ASDs (aged = 10–13 years) were recruited from a local special education school. Their school records demonstrated that all three had social impairments characteristic of children with ASDs. These participants had basic cognitive and reading abilities. The Wechsler Abbreviated Scale of Intelligence III (WASI-IV; Wechsler, 2003) was used to evaluate verbal IQ (VIQ), performance IQ (PIQ), and full-scale IQ (FSIQ) in the three participants. The details of each participant’s performance on this test are shown in Table 1. Note that, for ethical considerations, pseudonyms have been used for all participants. Permission to include the participants in the study was secured from their parents; furthermore, all participants were glad to cooperate.

Table 1.

Details for Each Participant.

Participant	Adam	Ricky	Noah
Gender	M	M	M
Age (years)	12.3	11.5	10.6
FSIQ	84	82	80
PIQ	89	85	82
VIQ	81	82	75

Note. FSIQ = Full-Scale IQ; PIQ = Performance IQ; VIQ = Verbal IQ.

Adam was 12.3 years old and attended a mainstream school. Adam has a young sister without an ASD. Twice a week, he attended a training course at a special school to help him manage his deficits. His parent reported that Adam enjoyed mathematical calculations and playing with jigsaw puzzles. Furthermore, Adam’s basic social communication ability was sufficient for his daily life, but he had difficulty with social interaction (e.g., symptoms of echolalia) and demonstrated disruptive behavior.

Ricky was 11.5 years old and attended a mainstream school, supplemented by bi-weekly courses in a special school. He has an older brother without an ASD. Ricky’s IQ was higher than that of the other two study participants. He particularly enjoyed mathematics, creating graphical figures on a computer, and building complex castles, but he had a poor attention span. He exhibited repetitive behavior that involved breaking pencil tips continuously. Ricky also enjoyed using computers, especially watching animations. Regarding his disorder in social understanding and skills, Ricky had difficulties in basic communication and social interaction and exhibited emotional instability.

Noah was 10.6 years old, an only child in his family, and attended a special school. His special interest was consulting calendars and memorizing the dates of given days. His mother mentioned that he had a poor attention span and was easily distracted by others. He had basic social understanding and communication abilities but experienced difficulty in getting along with others. For example, he showed repetitive behaviors and often asked questions such as “What are you doing in the afternoon?” “What are you doing tomorrow?” and “Do you have any brothers or sisters?”

Setting and Design

This single-subject experiment was combined with a multiple-probe design for all participants, to observe changes in their target behaviors. Each participant was involved in baseline, intervention, and maintenance phases of three sessions for each phase delivered over 6 weeks during which, performance of the target behaviors was recorded. Levels of target behaviors were examined in the baseline and maintenance phases. For the intervention phase of the experiment, a standard laptop with the 3D-SU system was set up in a quiet room of the special school. Each participant manipulated the 3D-SU intervention system using the virtual HMD. A scene viewed by the participant was also projected on a large display screen, so that the researchers and teachers could view what the participant was viewing. In addition to the participants, one teacher and two researchers were involved in the experimental situation, to ensure the social integrity of the experiment (that participants were treated fairly and with respect) and observe behavioral changes related to social understanding and skills following the participants’ use of the 3D-SU. The teachers responded to a questionnaire assessing the effectiveness of the 3D-SU intervention.

Measurement

We developed two scales to assess changes in the participants’ social behaviors following implementation of the intervention: a social events card (SEC) and a social behaviors scale (SBS). Three target behaviors were assessed with this measurement: NC, SI, and SC. Both measurement tools were discussed in detail with professionals and ASDs experts to ensure their validity and suitability. The SEC involved 12 events that could occur in classroom and school environments, as suggested by the participant’s parents and ASDs experts. To evaluate behavioral changes in the participants, the SBS adopted criteria from studies by Tse, Strulovitch, Tagalakis, Meng, and Fombone (2007); Xu (2005); and Sparrow, Cicchetti, and Balla (1984). The SBS addressed behaviors related to NC, SI, and SC and consisted of 32 questions (16 true or false and 16 multiple-choice). Each participant randomly selected 3 event topics from the SEC and responded to 12 questions on the SBS measure. The measurements assessed the participants’ ability to perform these three target behaviors according to the following criteria:

The participant can understand the meaning of postures and gestures in other people (e.g., waving, shaking hands, or raising a hand to answer a question in class).

The participant can perform some gestures and express the meaning of a gesture (e.g., “What should I do when I have a question during class?”).

The participant can verbally initiate communication (e.g., he or she can answer the following questions correctly: “What should I say to someone who helps me to do something?” and “What should I say to the teacher when he or she says ‘Good morning’ to me?”).

The participant can interact with other people using initiate interaction (e.g., responding with “Good morning,” “Thank you,” or “Good-bye” when someone speaks in the same way).

3.1. The participant can understand SC (e.g., he or she can answer the following questions correctly: “Should I wait in a line or break in line when people are waiting to buy a drink?” “Is it okay to hide a classmate’s textbook?” and “Can I run in the classroom when a lesson is in session?”).

3.2. The participant can build interpersonal interactions (e.g., is able to play with other classmates or answer the question, “What can I do if my classmate speaks to me loudly during a class?”).

3.3. The participant can behave politely (e.g., he or she can answer the following questions appropriately: “What should I say if my classmate wants to play with me?” and “What should I do when I see that my classmate has fallen down the stairs?”).

The total SBS score possible for the six true or false questions in each session was 18; 3 points were awarded for each correct answer and no points for an incorrect answer. The total SBS score possible for the 6 multiple-choice questions in each session was 30; each answer was assigned a score on a 5-point scale ranging from 0 to 5. Specifically, a clear response was awarded a score of 5, a nearly appropriate response was awarded a score of 4, a moderately appropriate response was awarded a score of 3, a less appropriate response was awarded a score of 3, a very inappropriate response was awarded a score of 2, a completely inappropriate response was awarded a score of 1, and no response was assigned a score of 0. The researcher and two trained observers coded the responses of each participant. All the coders were aware of the proposed system and understood the intervention process. The study was videotaped during the baseline, intervention, and maintenance phases. The SBS was used to explore the effects of using the 3D-SU system on social understanding and skills in this special population.

Procedure

A single-subject design was combined with multiple probes across subjects, which involved repeated observations of the study participants. An initial probe was taken to determine the three subjects’ performance levels on each behavior in the sequence. Adam received probes in the sequence, but Ricky and Noah received severance probes. After Adam finished three probes and showed stable responses, Ricky began the assessment with the second probe. A similar procedure was adopted in Noah’s case. Furthermore, participants participated in the experimental study once a week over the course of a 6-week period. Each week, on the day designated for research, three to five sessions were conducted during the experimental study periods. Reinforcement candy was provided when a participant exhibited emotional stability. Each phase involved a teacher and two trained researchers who understood the principles of the study. Each researcher used the SEC and SBS to code each participant’s responses and performance. The entire period consisted of three baseline sessions, five intervention sessions, and three maintenance sessions, as described below.

Baseline

Each participant’s social understanding and skills were assessed prior to the implementation of the intervention system. The length of the baseline period differed for each participant, to ensure that the baseline phase was terminated at a stable performance level. Each participant chose three events from the SEC and received 12 questions from the SBS, which were asked by the teacher. The participants were required to provide verbal responses to these questions. The teacher also offered a prompt when a participant needed it. The baseline phase lasted for 3, 6, and 9 days for Adam, Ricky, and Noah, respectively. All data were collected during this phase by asking 12 questions, randomly selected from the SBS, in each session. The participants’ responses were assessed based on their correct and incorrect answers.

Intervention

The intervention was designed to improve the social understanding and social skills of the participants. Each participant completed five intervention sessions (see Figure 5). The participants were led individually to a quiet room by the teacher. Each intervention session lasted for approximately 30 to 40 min. Each participant was provided instructions on the use of the system and given the opportunity to practice before the actual intervention procedure began. The virtual HMD was used to display the 3D scenarios, in which the participants were directed to a bus stop, boarded the bus, viewed scenes on the streets, and went into their classroom. The virtual view was controlled by a keyboard and a head-position sensor, such that participants could move their heads and view the VE from their own viewpoints at all times.

Figure 5.

Participant using the 3D-SU system.

All the questions were presented sequentially to each participant. They answered either verbally or using the keyboard. For this, the “o” and “x” keys were used to answer true and false questions, respectively, and the number keys 1 to 4 were used to answer the multiple-choice questions. A correct response elicited the message, “Correct, well done! You are doing well,” accompanied by applause. Inappropriate responses received a message, “Incorrect, it is not like that! You can try it again.” The instruction was given promptly, and the participants were permitted to attempt each question 3 times before the next question appeared. If the participants exhibited signs of emotional instability, they were given a 5-min break or the session was paused and resumed only when they wished to continue operating the system. In addition, to obtain valid interpretations of the system operations, the researcher, an observer, and a teacher recorded the participants’ responses and assessed their performance.

Maintenance

Each participant went through a maintenance period of 20 days to assess effects on learning following the intervention phase. Again, the participants chose three events from the SEC and received 12 questions from the SBC, which were presented by the teacher. The participants provided verbal responses to these questions. This phase determined whether the participants were able to maintain these learned skills over a long period. For each session, each participant was presented with a randomly selected SEC scenario and 12 randomly selected questions from the SBS.

Data Collection

Each session was audiotaped and videotaped during the baseline, intervention, and maintenance phases. The three participants’ behaviors were assessed using the SBS with SEC. The observers and researchers then reviewed the videotapes and coded the participants’ responses. For each participant, data were collected from the last three baseline sessions, the last five intervention sessions, and the three maintenance sessions.

We determined inter-observer agreement (IOA) by assessing the agreement between independent observers (one teacher and two researchers) who had marked scores on the SBS for each participant. The IOA was calculated as the total number of agreements, divided by the sum of agreements and disagreements, multiplied by 100 (Tekin-Iftar, Kırcaali-Iftar, Birkan, Uysal, Yıldırım, & Kurt, 2001). IOA data were recorded for probe sessions in the baseline, intervention, and maintenance periods with a criterion level of 85% IOA. Thus, Adam’s, Ricky’s, and Noah’s evaluations had average IOA reliability of 97.4%, 97%, and 92.7%, respectively (see Table 2).

Table 2.

The results of IOA in Each Phase for Three Participants.

Participant	Baseline (%)	Intervention (%)	Follow-up (%)
Adam	95.83	97.5	99
Ricky	96	98.42	96.78
Noah	94.86	95.30	88

Results

All the data were visually inspected for evidence of behavioral change related to social understanding and skills.

Figure 6 shows that the non-overlapping data point was 100%, which suggests that the intervention using the 3D-SU and skills system was highly effective.

Figure 6.

Scores for answers provided during baseline, intervention, and maintenance, for each participant.

As evident in Figure 6, each participant performed differently at the baseline phase (i.e., Adam had a score range of 8 to 12, Ricky 10 to 14, and Noah 3 to 12, where the total possible score was 32). Ricky and Noah generally improved during the intervention phase, with occasional declines. Adam showed an improvement as early as the second session of the intervention phase. Although his scores declined slightly at the start of this phase, they had reached 25 by the end of the intervention. Similarly, Ricky achieved a score of 26 and Noah had scored 24 by the end of this phase. These scores suggest that the 3D-SU intervention system was effective in improving the target behaviors of NC, SI, and SC.

Adam

Adam’s mean score for appropriate answers was 10 during the baseline phase (see Table 3). His score ranged from 8 to 12 (change +4) for the baseline phase. His mean score for appropriate answers was 22 for the intervention phase, with a range of 18 to 25 points. This variation indicated that the data remained stable over time. Adam demonstrated significant progress from the baseline to the intervention phase. Furthermore, compared with the baseline phase (see Table 4), Adam’s SI behavior made a notable improvement (M = 6.2) after using the system. During the intervention period, Adam displayed a strong interest in the 3D-SU system, and was able to make clear explanations for NC behavior to the researcher. His performance continued to show stability throughout the maintenance phase, suggesting that his SI behavior was maintained over this period.

Table 3.

Performance in Each Phase for Each Participant.

Sessions	Adam			Ricky			Noah
Sessions	A	B	C	A	B	C	A	B	C
Range	8–12	18–25	26–28	10–14	20–26	25–27	3–12	15–24	24–26
Change	+4	+7	+2	+4	+6	+2	+9	+9	+2
M	10	22	27.3	11.8	23.5	26	8.7	19.8	25

Note. A = baseline; B = intervention; C = maintenance.

Table 4.

Scores for NC, SI, and SC Performance.

Experimental session	Adam			Ricky			Noah
Experimental session	A	B	C	A	B	C	A	B	C
NC (full score is 8)	2.3	5.6	6.7	2.75	5	6	2.5	4.8	6
SI (full score is 8)	3	6.2	7.3	3.25	5.17	6.3	2.17	4.4	6
SC (full score is 8)	2.7	5.2	6.7	3.25	7.17	7.3	2.3	5.6	6.7

Note. NC = non-verbal communication; SI = social initiations; SC = social cognition; A = baseline; B = intervention; C = maintenance.

Ricky

Ricky showed standard levels of the target behaviors. His mean score at baseline was 11.8 (see Table 3). Although Ricky’s scores dropped slightly during the baseline phase (scores of 12, 10, and 14 points; see Figure 6), his last baseline score was 14. His scores generally increased during the intervention but dropped slightly before stabilizing. This suggests that the 3D-SU intervention system helped him improve his target behaviors. Ricky’s SBS score increased from 20 to 26 during the intervention phase. A comparison of the mean scores for the intervention and baseline phases (M = 23.5 vs. 11.8, respectively) indicated that the intervention was effective. During the maintenance phase, the overlapping data point was 100% (see Figure 6). In terms of Ricky’s SC behavior, a great improvement was achieved in the intervention phase, and Ricky maintained this improvement during the experiment (see Table 4). Ricky got the highest score (7.3) in SC compared with the other participants. This result indicated that Ricky’s social understanding and skills improvement were maintained.

Noah

Noah did not perform as well as the other two participants did. Noah’s performance was unstable: His mean SBS score was 8.7, with a range of 3 to 12. He did not exhibit a stable score until Session 6 of the baseline phase. He was familiar with the process of manipulating the system; however, he failed to perform in a stable manner during Sessions 7 and 8 of the intervention phase (see Figure 6). His mean score for the target behaviors was 19.8 for this phase (see Table 3), with scores increasing from 15 to 24 during the intervention phase, as compared with the mean score during the baseline phase (M = 8.7, change +9). Analysis of the observation data showed that Noah’s scores increased at the beginning of the intervention phase (see Figure 6), and he exhibited a better concentration level while using the system. The non-overlapping percentage was 100% between the baseline and intervention phases, and the overlapping percentage was 33.3% between the intervention and maintenance phases. This indicated that the 3D-SU system was effective in helping Noah to improve his target behaviors. In addition, during the maintenance phase, Noah’s score increased from 24 to 26. His mean score for target behaviors during this phase was 25, as compared with 19.8 during the intervention, suggesting that his scores had improved over time and his performance remained consistent during the maintenance phase. Furthermore, during the baseline phase, his SI behavior was low (M = 2.17), and he was unable to interact with the researcher or exhibit appropriate behaviors. For instance, he often responded with unrelated questions instead of appropriate answers in this phase (see Table 4). After using the system, Noah exhibited fewer inappropriate behaviors. His mean scores for NC behavior rose from 2.1 in the baseline phase to 4.8 in the intervention phase and 6.4 in the maintenance phase.

Discussion

This study examined the effects of the 3D-SU system with a virtual HMD in terms of social understanding and skills (i.e., NC, SIs, and SC capability) in individuals with ASDs. The results indicate that the participants’ social understanding and skills improved following their use of the system. This demonstrates that the characteristics of realism and the context of the novel 3D-SU system are useful for this population. These findings are in agreement with those obtained by Standen and Brown (2005) and Bruno and Muzzupappa (2010), who stated that VEs facilitate concept development through practical activities. The situational learning environment, as suggested by J. S. Brown, Collins, and Duguid (1988), emphasizes that learning should apply to real-world situations. The system used in the present study demonstrated that exploration of an immersive VE provides exploration and involvement with social events, a finding that was consistent with similar studies conducted by Kawa and Pisula (2010) and Pierce and Courchesne (2001).

Moreover, the system adopted the approaches of modeling, promoting, reinforcing, and guiding, which helped the participants develop these target behaviors and maintain them over a period of time. This is congruent with the findings of Alberto and Fredick (2000). They suggested that instructional steps that involved identifying people in a picture, identifying objects and actions, and event sequencing could help students read visual information. The present study offers evidence for an intervention approach based on the use of immersive VEs to help individuals with ASDs with respect to their social skills and understanding. It may be concluded that the immersive VEs positively influence learning ability in people with learning disabilities, and that visual assistance with reciprocal interactions can assist them in learning social skills through repeated practice (Cheng et al., 2010; Cheng & Ye, 2010; Golan & Baron-Cohen, 2006; LaCava, Golan, Baron-Cohen, & Smith, 2007; Meadan et al., 2011).

With reference to the performance of each participant, Adam’s SI behavior was very quiet in the baseline phase, and he often ignored the questions he was asked. As a result of the intervention, he exhibited fewer inappropriate behaviors and better concentration when the researcher asked questions, and he responded with more plausible answers. Adam’s SC behavior was improved, and he started providing descriptive explanations to questions rather than simple “yes” or “no” responses. For instance, when asked, “Can I walk around during a lesson?” he answered, “I need go to back to my seat immediately and sit down when the teacher is teaching.” Interestingly, with reference to understanding NC, Adam raised his hand automatically when the virtual teacher asked, “Who can answer the question?” in the classroom scenario. This indicated that Adam understood NC behaviors and used appropriate gestures with the 3D-SU system. Similar experimental results have been demonstrated by Chuang (2003), indicating that a stress-free social environment significantly affects learning in people with ASDs and stimulates learning motivation. Furthermore, Adam seemed to understand social consequences through these animated 3D social scenarios. He repeatedly immersed himself in the virtual scenes and felt safe enough in the VE to exhibit his behaviors.

Regarding Ricky’s SI behavior, in the baseline phase, it was observed that he could not concentrate or listen attentively when researchers talked to him, and he exhibited obsessive behaviors. He did not invest appropriate thought before providing his responses. During the intervention phase, Ricky’s SC and NC behaviors improved, and he could provide clear explanations when he replied to the questions posed by the system. For instance, in the intervention phase, to the question “Can I speak loudly in a bus?” he replied, “No, speaking loudly will distract others!” This demonstrates that the 3D animated scenarios may have helped him understand social situations. However, the behavioral reactions from Adam and Ricky may be explained by the fact that the HMD isolated them from the outside world, which may have helped them concentrate on viewing 3D social scenarios. As their attention was focused on viewing and learning, they felt free to respond in this immersive world.

Noah often ignored questions and was unable to concentrate when researchers communicated with him. He liked playing with his toys in the baseline phase. However, Noah’s SI and SC behaviors improved gradually during the intervention phase, though he sometimes needed hints from the researcher. Following the intervention, Noah was found to behave more politely. He enjoyed watching the 3D bus scenario and concentrated on it, often becoming immersed in the situation when it appeared. Noah repeatedly asked the researcher, “When will the bus come?” and “Who will get on this bus?” In this sense, the system provided for anticipation of recurring events and the ability to control them, which may have helped Noah experience an excellent opportunity to learn social skills and behaviors in different situations (Strickland et al., 1996). His improvement was impressive in terms of SI behavior; for instance, he tried to establish a social relationship with the researchers. Noah also said “Thank you” when he finished using the system or received chocolates as a reward, and he waved goodbye when he left, especially notable as the scores that Noah had obtained during the baseline phase were relatively low in NC, SI, and SC.

The three teachers reported that the 3D-SU system provided visual stimuli and helpful assistance to their students in developing their social understanding and skills. All the teachers stated that the system motivated the children to learn these target behaviors through the 3D animation and scenarios. They encouraged their students to think about appropriate behaviors in real-life settings, applying what they had learned from the 3D-SU intervention system. Furthermore, the teachers suggested that the system could be used with children with other disabilities (e.g., mental retardation and emotional problems). They added that the virtual HMD may help children who have difficulty in concentrating on a task.

The VE system with HMD may have helped our participants to concentrate on learning and ignore distractions. They consistently tracked moving virtual objects in a scene with their eyes, head, and body reactions. The system offered environmental richness, scene realism, and meaningful operating experiences for children with ASDs via this immersive world (Witmer & Singer, 1998). It may increase learning effects and familiarity with new technology. This finding corroborates those revealed by Eyon (1997) and Strickland et al. (1996), who found that using an HMD encouraged children to focus and immerse themselves in a 3D virtual world. In addition, we can speculate that the designed virtual humans have more patience and provoked less stress in their interactions with individuals with ASDs in contrast to relative to actual human interactions (Tartaro & Cassell, 2008).

Some limitations of the present study should also be taken into consideration. We did not assess whether the participants went on to demonstrate these target behaviors in daily life. All the participants performed their tasks in the system with improved self-efficacy after the intervention. Although they had stereotyped characteristics (e.g., stubbornness), they worked hard to learn and memorize the basic principles of these target behaviors during the experiment. Several possible explanations can be provided for the positive changes in their target behaviors demonstrated in this experiment. First, one possible explanation is that all of the participants lived in a remote district and did not have access to appropriate educational resources to help them overcome difficulties resulting from their ASDs. The HMD with visual simulation was a novel technology in their learning experience, which may have improved their motivation to learn the target behaviors and stimulated their interest in exploring the VE (Irish, 2013; Parsons & Cobb, 2011). Another reason could be that these children with ASDs had above average IQs and selective attention, which may have helped them learn quickly despite their short attention spans (Witmer & Singer, 1998). Similar results were found by Doyle and Arnedillo Sánchez (2011); Bledsoe, Smith Myles, and Simpson (2003); and Tekin-Iftar et al. (2001).

Second, the relatively small sample size may have limited the generalizability of the data. However, the skills that were learned from this system and could be applied to the real world were the primary interests in this study. In our future studies, a larger number of participants will be sampled to collect enough data to reach the requirements for generalization. Third, the participants in the present study scored highly on the WASI. Therefore, further studies should involve participants with different degrees of disability. Finally, to use this system, participants with ASDs require standard criteria (e.g., basic reading ability and ability to operate computer technology), which can help children with ASDs to read and understand social contexts.

This study contributes to the literature on social skills intervention by demonstrating that the VE system presents direct instruction and visual concentration for children with ASDs learning academic tasks. The immersive approach provides whole engagement with participants who can have practical experiences with realistic social interactions. This technology can be used in general education settings to immerse these children in social situations with people. However, results of this study should be considered preliminary until further evidence is gathered through replication.

Conclusion

This study investigated performance in terms of social understanding and skills in NC, SIs, and SC after using the 3D-SU immersive system for individuals with ASDs. The data illustrates that the 3D-SU system can improve the utilization of reciprocal interactions for students with ASD. The system was less stressful for the participants than real-life training might be, and they felt free to interact with the virtual characters to practice their social understanding and skills. This study also provides a new line of research that might be used to create social opportunities for children with ASDs that include social practices of an academic nature. The major difficulty faced by these children is their inability to generalize their behavior from one environment to another. More virtual social scenarios can be developed for people with varying degrees of ASD severity, to help them cope with a greater number of potentially difficult social situations. Furthermore, future studies should assess social skills in individuals with ASDs who have below average IQ and devise ways of improving the maintenance effects of the 3D-SU system.

Footnotes

Acknowledgements

The authors thank the children with autism spectrum disorders (ASDs) and their parents who participated in this research by generously volunteering their time. They specially thank Dr. Terri Lewis for her help in English writing assistance.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors thank the National Science Council of the Republic of China, Taiwan for financially supporting this research under Contract No. NSC_99-2221-E-018-002.

References

Alberto

P. A.

Fredick

L. D.

(2000). Teaching picture reading as an enabling skill. Teaching Exceptional Children, 33, 60–64.

Allen

Courchesne

(2001). Attention function and dysfunction autism. Frontiers in Bioscience, 6, 105–119.

American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text rev.). Washington, DC: Author.

Andersson

Josefsson

Pareto

(2006). Challenges in designing virtual environments: Training social skills for children with autism. International Journal on Disability and Human Development, 5(2), 105–111.

Bayer

M. M.

(2002). Retinal scanning display: A novel HMD approach for army aviation. Proc. SPIE, 4711 (Helmet- and Head-Mounted Displays VII), 202–213.

Bledsoe

Smith Myles

Simpson

R. L.

(2003). Use of a social story intervention to improve mealtime skills of an adolescent with Asperger syndrome. Autism, 7, 289–295.

Brown

D. J.

Neale

Cobb

S. V.

(1999). The development and evaluation of the virtual city. International Journal of Virtual Reality, 4, 28–41.

Brown

J. S.

Collins

Duguid

(1989). Situated cognition and the culture of learning. Educational Researcher, 18, 32–42.

Bruno

Muzzupappa

(2010). Product interface design: A participatory approach based on virtual reality. International Journal of Human-Computer Studies, 68, 254–269.

10.

Charitos

Karadanos

Sereti

Triantafillou

Koukouvinou

Martakos

(2000). Employing virtual reality for aiding the organization of autistic children’s behavior in everyday tasks. In Proceedings of the 3rd International Conference on Disability, Virtual Reality and Associated Technologies, (ICDVRAT), (pp.147–170). Reading, United Kingdom: The University of Reading.

11.

Cheng

Chiang

Cheng

(2010). Enhancing empathy instruction using a collaborative virtual learning environment for children with autistic spectrum conditions. Computer & Education, 55, 1449–1458.

12.

Cheng

(2010). Exploring the social competence of students with Asperger syndrome in a collaborative virtual learning environment. Computer & Education, 54, 1068–1077.

13.

Chuang

(2003). Online virtual training environments with intelligent agents to promote social inclusion (Master’s thesis). Nottingham Trent University, Nottingham, UK.

14.

Chung

K. M.

Mosconi

S. R. M.

Drewry

Matthews

Tasse

M. J.

(2007). Peer-mediated social skills training program for young children with high-functioning autism. Research in Developmental Disabilities, 28, 423–436.

15.

Clark

Feehan

Tinline

Vostani

(1999). Autistic symptoms in children with attention deficit-hyperactivity disorder. European Child & Adolescent Psychiatry, 8, 50–55.

16.

Devender

R. B.

Stephanie

L. H.

Lan

L. G.

(2010). Impact of training peers and children with autism on social skills during center time activities in inclusive classrooms. Research in Autism Spectrum Disorders, 4, 619–625.

17.

Doyle

Arnedillo Sánchez

(2011). Using multimedia to reveal the hidden code of everyday behaviour to children with autistic spectrum disorders (ASDs). Computers & Education, 56, 357–369.

18.

Eyon

(1997). Computer interaction: An update on the avatar program. Communication, Summer, 22.

19.

Fatima

A. B.

Nazneen

H.V.

Gregory

D. A.

Rosa

I. A.

(2012, May). Towards a tool to support the authoring of social skills instructional modules. Presented at the International Meeting for Autism Research, Toronto, Canada.

20.

Golan

Baron-Cohen

(2006). Systemizing empathy: Teaching adults with Asperger syndrome and high functioning autism to recognise complex emotions using interactive media. Development and Psychopathology, 18, 589–615.

21.

Gonzalez-Lopez

Kamps

(1997). Social skills training to increase social interactions between children with autism and their typical peers. Focus on Autism and Other Developmental Disabilities, 12, 2–14.

22.

Herreraa

Jordanb

Veraa

(2006). Abstract concept and imagination teaching through virtual reality in people with autism spectrum disorders. Technology and Disability, 18, 173–180.

23.

Irish

J. E. N.

(2013). Can I sit here? A review of the literature supporting the use of single-user virtual environments to help autistic adolescents learn appropriate social communication skills. Computers in Human Behavior, 29(5), 17–24.

24.

Josman

Milika Ben-Chaim

Friedrich

Weiss

P. L.

(2008). Effectiveness of virtual reality for teaching street-crossing skills to children and adolescents with autism. International Journal on Disability and Human Development, 7, 49–56.

25.

Kathy

Howard

(2001). Social stories, written text cues, and video feedback: Effects on social communication of children with autism. Journal of Applied Behavior Analysis, 34, 425–446.

26.

Kawa

Pisula

(2010). Locomotor activity, object exploration, and space preference in children with autism and Down syndrome. Acta Eurobiologiae Experimentalis, 70, 131–140.

27.

LaCava

P. G.

Golan

Baron-Cohen

Smith

M. B.

(2007). Using assistive technology to teach emotion recognition to students with Asperger syndrome: A pilot study. Remedial and Special Education, 28, 174–181.

28.

Lanyi

C. S.

Tilinger

(2004). Multimedia and virtual reality in the rehabilitation of autistic children. In: Lecture Notes in Computer Science LNCS 3118, 22-28.

29.

Leonard

Mitchell

Parsons

(2002). Finding a place to sit: A preliminary investigation into the effectiveness of virtual environments for social skills training for people with autistic spectrum disorders. In Sharkey

P. M.

Sik Lányi

Standen

P. J.

(Eds.), Proceedings of the 4th International Conference VR in the Rehabilitation of People with Intellectual Disabilities: Virtual Reality and Associated Technologies pp. 249–257). Reading, United Kingdom: The University of Reading.

30.

Machintosh

Dissanayake

(2006). A comparative study of the spontaneous social interactions of children with high-functioning autism and children with Asperger’s disorder. Autism, 10, 199–220.

31.

Meadan

Ostrosky

M. M.

Triplett

Michna

Fettig

(2011). Using visual supports with young children with autism spectrum disorder. Teaching Exceptional Children, 43(6), 28–35.

32.

Millen

Edlin-White

Cobb

(2010). The development of educational collaborative virtual environments for children with autism. The 5th Cambridge Workshop on Universal Access and Assistive Technology, Cambridge, UK. March, 2010.

33.

Neale

Leonard

Kerr

(2002). Exploring the role of virtual environments in the special needs classroom. In Sharkey

Sik Lanyi

Standen

(Eds.), Proceedings of the 4th ICDVRAT (pp. 259–266). Reading, United Kingdom: The University of Reading.

34.

Odom

S. L.

Strain

P. S.

(1986). A comparison of peer-initiation and teacher-antecedent interventions for promoting reciprocal social interaction of autistic preschoolers. Journal of Applied Behavior Analysis, 19, 59–71.

35.

Odom

S. L.

Watts

(1991). Reducing teacher prompts in peer-mediated interventions for young children with autism. The Journal of Special Education, 25, 26–42.

36.

Osterlund

Lawrence

(2012). Virtual reality: Avatars in human space flight training. Acta Astronautica, 71, 139–150.

37.

Parsons

Cobb

(2011). State-of-the-art of virtual reality technologies for children on the autism spectrum. European Journal of Special Needs Education, 26, 355–366.

38.

Parsons

Mitchell

Leonard

(2004). The use and understanding of virtual environments by adolescents with autistic spectrum disorders. Journal of Autism and Developmental Disorders, 34, 449–466.

39.

Parsons

Mitchell

Leonard

(2005). Do adolescents with autistic spectrum disorders adhere to social conventions in virtual environment? Autism, 9, 95–117.

40.

Pierce

Courchesne

(2001). Evidence for a cerebellar role in reduced exploration and stereotyped behavior in autism. Biological Psychiatry, 49, 655–664.

41.

Putnam

Chong

(2008). Software and technologies designed for people with autism: What do users want? In Proceedings from Assets ‘08: The 10th International ACM SIGACCESS Conference on Computers and Accessibility (pp. 3–10). New York, NY: ACM.

42.

Rebecca

M. C.

Candice

(2010). Using paraprofessionals to teach social skills to children with autism spectrum disorders in the general education classroom. Journal of Instructional Psychology, 37, 161–169.

43.

Rommelse

N. N.

Franke

Geurts

H. M.

Hartman

C. A.

Buitelaar

J. K.

(2010). Shared heritability of attention-deficit/hyperactivity disorder and autism spectrum disorder. European Child & Adolescent Psychiatry, 19, 281–295.

44.

Sparrow

S. S.

Cicchetti

D. V.

Balla

D. A.

(1984). Vineland Adaptive Behavior Scale (2nd ed.). Circle Pines, MN: American Guidance Service.

45.

Standen

P. J.

Brown

D. J.

(2005). Virtual reality in the rehabilitation of people with intellectual disabilities: Review. Cyber Psychology & Behavior, 8, 272–282.

46.

Strickland

Marcus

Mesibov

Hogan

(1996). Brief report: Two case studies using virtual reality as a learning tool for autistic children. Journal of Autism and Developmental Disorders, 26, 651–659.

47.

Strickland

McAllister

Coles

C. D.

Osborne

(2007). An evolution of virtual reality training designs for children with autism and fetal alcohol spectrum disorders. Topics in Language Disorders, 27, 226–241.

48.

Suchman

(2007). Human machine configurations: Plans and situated actions (2nd ed.). New York, NY: Cambridge University Press.

49.

Takashi

(2002, April). Head mounted display. Displays, 23(1–2), 57–64.

50.

Tartaro

Cassell

(2008). Playing with virtual peers: Bootstrapping contingent discourse in children with autism. In ICLS’08 Proceedings of the 8th International Conference for the Learning Sciences (Vol. 2, pp. 382–389). International Society of the Learning Sciences.

51.

Tekin-Iftar

Kırcaali-Iftar

Birkan

Uysal

Yıldırım

Kurt

(2001). Using a constant time delay to leach leisure skills to children with developmental disabilities. Mexican Journal of Behavior Analysis, 27, 337–362.

52.

Tse

Strulovitch

Tagalakis

Meng

Fombone

(2007). Social skills training for adolescents with Asperger syndrome and high-function autism. Journal of Autism and Developmental Disorder, 37, 1960–1968.

53.

Wechsler

(2003). Wechsler abbreviated scale of intelligence-Fourth Edition (WISC®-IV). San Antonio, TX: Harcourt Assessment.

54.

Witmer

B. G.

Singer

M. J.

(1998). Measuring presence in virtual environments: A presence questionnaire. Presence, 7, 225–240.

55.

X. L.

(2005). Chinese Adaptive Behavior Scale. Taipei: The Center of Special Education, National Taiwan Normal University.