Abstract
In this study, the researchers explored the usage of a virtual reality (VR)–based social skills learning environment for children with autism spectrum disorder (ASD). Using OpenSimulator, the researchers constructed a desktop VR-based learning environment that supports social-oriented role-play, gaming, and design by children with ASD. Seven 10–14 years old children with ASD participated in this VR-based social skills program for 20+ hr on average. Data were collected via screen recording and observation of play- and design-oriented social skills enactment and pre- and postintervention Social Communication and Skills Questionnaires. Participants demonstrated an increased level of successful social skills performance from the baseline to the intervention phase. The findings provided preliminary evidence for the usage of a VR-based social skills learning environment for children with ASD.
Approximately 1 in 54 children in the United States is diagnosed with an autism spectrum disorder (ASD), and ASD affects all racial, ethnic, and socioeconomic groups (Centers for Disease Control and Prevention [CDC], 2020). Approximately 44% of children with ASD have IQ scores in the average and above range and may be the fastest growing segment (CDC, 2020; Rao et al., 2008). They can speak, read, and write but lack motivation and critical social skills that are vital to social engagement, such as participating in social roles and relationships, initiating and maintaining social interactions, engaging in play, sharing an affective experience or understanding the perspectives of others, collaboration and negotiation (Macintosh & Dissanayake, 2006). Hence, they can encounter problematic social experiences with their peer groups and have difficulty in navigating a social world full of complex situations in community life, school, and work (Church et al., 2000).
Social Skills Interventions for Children With ASD
In spite of increasing empirical studies and reviews of treatment or educational methods for children with ASD, no specific method has emerged as the established standard of practice for this heterogeneous learner group (Masi et al., 2017). However, among diverse evidence-based social skills interventions such as naturalistic interventions, video modeling, peer-mediated interventions, and group-based social skills training, learner interest and naturalistic environmental support emerge as two salient facets contributing to social skills improvement and voluntary social engagement. Social skills interventions have seen an increasing trend toward situated, naturalistic techniques from direct instruction, while the control for prompting social interactions is shifting from teacher verbal antecedents to self-monitoring, peer mediation, or naturally occurring stimuli (e.g., Ke et al., 2018; Rao et al., 2008; Schreibman et al., 2015; White et al., 2007). A situated, naturalistic intervention is typically conducted in loosely controlled contexts, uses naturally occurring antecedents and consequences, and incorporates the target child’s interests and preferences into the intervention activity or environmental arrangement (Wong et al., 2015). Increasing evidence indicates that generalization and performance improve in a naturalistic social skills learning environment (Schreibman et al., 2015; Spelke et al., 2013).
Moreover, there is a shift in autism research from a sole focus of cognitive theoretical accounts, such as theory of mind, executive functioning, or weak central coherence theory, to a social motivation explanation (Chevallier et al., 2012). Chevallier et al. argued that social motivation deficits play an important role in social dysfunction in ASD. As such, it is especially important for a social skills program to leverage and enhance learner’s attention and motivation toward social information and interaction, through increasing the relevance of the social stimulus to solve a task or the participant’s intrinsic interest for the stimulus and the task. Given the above discussion, an appropriately authentic and personally meaningful learning environment for children with ASD is a worthy candidate for development and research.
Using innovative technologies for social skills training emerges as a prominent and resource-efficient method (Grynszpan et al., 2014). Research shows that children with ASD tend to enjoy computerized interventions and have made significant learning gains through various technology-based training packages such as cartoons, video modeling, computer-assisted instruction, computer games, and virtual worlds (Grynszpan et al., 2014; Knight et al., 2013; Parsons, 2016). Technology-based educational programs can (a) provide students with instant and consistent feedback, (b) prioritize student agency in an active learning situation rather than making students passive participants, (c) dynamically control social demands that are overly challenging and confusing, (d) make abundant use of visually cued instruction and multimodal interaction, and (e) be time and cost-effective in providing individualized instruction (Ke et al., 2018). There is increasing interest and evidence in the benefit of using computer-based interactive technologies to transform education and increase autonomy and quality of life for individuals with ASD (Grynszpan et al., 2014; Knight et al., 2013; Walsh et al., 2017). A recent meta-analysis of interactive technology interventions used for the training of skills of individuals with ASD synthesized the results of 21 efficacy studies and indicated a positive, medium effect size (Cohen’s d = 0.47, CI [0.08, 0.86]) of the technology-based intervention compared with the control condition, and a large effect size (Cohen’s d = 0.79, CI [0.50, 1.09]) compared with the baseline condition (Grynszpan et al., 2014). The meta-analysis also indicated technology-based interventions as equally effective for individuals with different levels of autism. On the other hand, the most prevalent technology used for social skills education is still video technology (Qi et al., 2017; Reed et al., 2011). Research on the design of innovative technology-based educational tools for complex social and academic skills training for individuals with ASD is still limited (Ke et al., 2018; Knight et al., 2013).
Virtual Reality (VR) for Social Skills Training
Desktop VR emerges as a promising and cost-effective platform for situated and highly interactive social skills training. Desktop VR is a computer-generated three-dimensional simulation of real-life environments that is explored and interacted with by real-time users. VR could take variant forms, ranging from desktop computer rendering of a highly interactive virtual world to a fully immersive multisensory environment in laboratories (Mikropoulos & Natsis, 2011). VR has some intrinsic appeal as an instructional tool for children with ASD who tend to learn using visual support (Mitchell et al., 2007; Schreibman & Ingersoll, 2005). A VR-based learning environment allows participants to practice social skills in a nonthreatening, synthetic but realistic, motivating, controllable, and diversifiable environment during learning (Bellani et al., 2011). There is evidence that VR technologies can, and do, represent authentic and plausible scenarios and social encounters “such that people will behave and respond in a similar way in the virtual worlds as they do in the real world,” thereby reflecting and supporting real-world conventions, understanding, and behaviors for a range of different user groups (Blascovich et al., 2002; Georgescu et al., 2014; Parsons, 2016, p. 142; Schmidt & Schmidt, 2008).
The embedded construction kit of an open-source VR platform (e.g., OpenSimulator) affords active learning through design (Barry et al., 2003). Involving individuals with ASD as a participatory and collaborative designer (more than user) of the virtual world as a learning and social space will help to ground learner perspectives, needs, strengths, or preferences in the educational intervention design, thus fostering learner agency and sense of presence (Ke & Lee, 2016; Parsons, 2016). Supporting multisensory interactions for multiple users over distance is another salient feature of VR that makes social skills training accessible in diverse learning settings.
In spite of its promise, the VR-based learning environment is still understudied and warrants further development and testing. Early efforts on studying virtual-reality-based learning for individuals with ASD proceeded from embedding multimedia direct instruction in a virtual world (e.g., Stichter et al., 2014) to emphasizing multiplayer simulation and scenario-based social role-play (e.g., Ke & Lee, 2016; Didehbani et al., 2016; Mitchell et al., 2007). In a recent pretest–posttest study with 30 children with autism, Didehbani et al. (2016) examined the effects of a desktop VR-based social cognition training that involved children in social role-play with two to three peer- and clinician-controlled, confederate avatars in daily life social scenarios. The study showed participants’ improvements in measures of emotion recognition, social attribution, and executive function of analogical reasoning.
Therefore, the purpose of this study is to investigate the implementation of a VR-based learning environment in supporting social skills training via a packet of naturalistic, personally meaningful learning activities. These VR-based learning activities include scenario-based social role-play, group gameplay, and collaborative artifact design. Using the single-case research design, this study aimed to examine the effect of a VR-based, naturalistic social skills training program for children with ASD. The specific research question was: What effect does participating in VR-based social skills training have on the social skills performance of children with ASD?
Method
Participants and Setting
Seven 10–14 years old children with autism were recruited from a local autism center at a suburban area and participated in the study voluntarily with written consent received. They had a medical diagnosis of autism; the ability to speak, read, and write; and received grade-level academic instruction. The participants included one girl and six boys, of whom three were minorities and all were native English speakers (see Table 1). All participants were capable of using the computer and internet and had prior experience of playing digital games.
Participant Information.
Intervention: VR-Based Social Skills Training
Using OpenSimulator, the researchers constructed a 3D, desktop VR-based learning environment for social skills training. It encompassed a variety of play- and design-oriented social interaction tasks situated in VR-simulated everyday social scenes such as a neighborhood, a school, amusement parks, other public facilities (e.g., restaurants and shops), or story scenes (e.g., an underwater lab and a Western town). These social interaction tasks were organized into four types: virtual schooling (i.e., social skills instruction and practice), social role-playing in familiar and novel tasks, artifact design (to fulfill individual or group needs), and social gaming that allows or requires social interactions (see Figure 1 for details).
Virtual reality–based social interaction activities.
Apart from participant-controlled avatars, nonplayer characters were used as the background characters and elements of a triggering event (e.g., loud and chatty students at a school café). Three trained facilitators puppeteered multiple virtual social characters via an online, free voice morphing software. The facilitators included researchers and doctoral students majoring in education. The puppeteered social characters embodied and portrayed scene- and task-related social interaction stakeholders such as partners, clients, supervisors, competitors, and/or consultants. The facilitator prompting, following the principle of naturalistic intervention (Rao et al., 2008), acted as a supplement to the environmental stimuli or cues presented by the virtual social interaction tasks, simulated social scenes, and peers. Prompting was both verbal and nonverbal (e.g., via animated body postures and movements). The form of prompting was adaptive to suit participants’ needs and engage them in a social interaction task, ranging from passive proximity presence (e.g., following or standing before the participant), responding, initiation or inquiry (e.g., repeating a question or request), to modeling or instructive cuing (e.g., “Let me show you,” “I would suggest…”).
The VR-based social interaction scenes and tasks aimed to promote the practice and performance of the following social skills: initiating social interactions, interpersonal negotiation, self-identity expression, and flexible thinking (or cognitive flexibility). The operational definitions of these targeted social skills are provided below and in Table 2:
Enactments of the Targeted Social Skills.
Initiation: Frequency of verbalizations that are not in direct response to a preceding question or that occur at least 5 s after a preceding verbalization and nonverbal initiation of an interaction (e.g., wave to greet a peer’s avatar).
Interpersonal negotiation: An indication of the recognition of a conflict between one’s and another’s perspective, a reciprocal exchange (including opinion exchanges and reciprocal communication with a balance of perspective), verbal statements of the intent to collaborate, and the development of shared goals (Selman et al., 1986).
Positive self-identity expression: Demonstration of confidence or feelings of worth (Stainback et al., 1994) by explaining one’s own perspectives and preferences, describing individual differences, and identifying commonness with others.
Cognitive flexibility: Switching between solutions, tasks, or perspectives based on the changing contexts or emergent plan or rule changes (Geurts et al., 2009).
Procedure
Research design
A multiple baseline across participants design was used to examine the effects of the novel VR-based learning program. To reduce the time for completing the multiple baseline procedure and hence reduce the risk of attrition (Kratochwill et al., 2010), participants were randomly assigned to two subgroups. Given the fact that not all participants were available when the study started and the need to get participants to begin treatment relatively quickly after the baseline assessment, the researchers applied a partially nonconcurrent multiple baseline design (Watson & Workman, 1981) in each subgroup. Successive participants were randomly assigned to a priori (predetermined) baseline duration (ranging from three to five data points), and consecutively began the intervention after the priori baseline phase. For two participants in each subgroup, the baseline phase occurred in proximity. To bolster the rigor of the study (Carr, 2005; Harvey et al., 2004), follow-up data were also collected with four participants.
Intervention implementation procedure
Every participant received the VR-based intervention at home via an internet-connected computer installed with the VR viewer and equipped with a mouse and a headset. They logged into the VR learning environment using individual accounts. All participants received a 1-hr hands-on training on navigating and using the VR environment before the study. They were all able to perform basic object fetching and editing tasks and alternative modes of navigation at the end of the training.
The duration and frequency of the intervention sessions (lasting 0.75 to 1.25 hr per session, 1–2 sessions per week) allowed for an adjustment based on the participant’s progress and availability. Each child had to complete each social skill training activity to advance to next, based on the mastery model recommended by prior research of social skills training (Foster & Bussman, 2007). The participants went through the intervention program in an average of 20.29 hr (SD = 1.70), over 16–31 sessions. To evaluate the implementation fidelity of the VR-based intervention, the researchers randomly selected 30% of the intervention sessions of every participant and coded the fidelity by checking whether the preprogrammed content (e.g., scenes, object-embedded prompts) was presented, activities took place as planned, facilitators adhered to the general principle of naturalistic prompting, and learners were engaged in the planned activities. Each fidelity component was coded on a 0–2 scale, with 0 indicating nonoccurrence, 1 indicating an occurrence that needs improvement, and 2 indicating a fully expected occurrence. The congregated implementation fidelity score was 1.72, indicating a satisfactory (>.85) adherence.
Data Collection and Analysis
Baseline, intervention, and follow-up data
Baseline data on the targeted social skills were collected by observing and screen-recording every participant’s social interaction behaviors in person and in the virtual world. Specifically, the leading researcher observed and documented participants’ social interactions with their family members and other researchers either at home or in a public library based on participants’ preferences for approximately 20 min per session or observation point. The participants’ conversations and behaviors in VR social scenario simulations (e.g., social interactions at a virtual school and a virtual public facility), without any scaffolding or cuing, were also observed and screen recorded.
Participants’ social skills performance during the VR-based intervention phase were screen recorded and observed. The researchers conducted behavioral coding with the participants’ social interaction performance during VR-simulated social scenarios at each intervention session, using time sampling (per 30 s) as the primary unit of coding. The coding focused on inspecting the occurrence of both positive and negative behavioral performance of the targeted social skills by the participants without scaffolding or cuing. The coding was assisted by a self-developed event logging application that allows a quick logging and frequency count of the targeted social skill enactments during video analysis or live observation. The coding followed a structured protocol listing operational definitions and examples of the targeted social skills. Three trained coders independently coded a randomly selected 20% of the recordings, with the interrater reliability (κ) being .86. Through peer debriefing, the coders reached 100% agreement on the frequency and occurrence contexts for every core facet of the social skills performance and their exemplified events. They then coded the remaining recordings. Based on the social interaction behavioral coding results, the researchers then calculated the average frequency of successful enactments of each targeted social skill (i.e., average counts of successful enactments of each skill in a 3-min interval) in each baseline and intervention session or observation point.
Follow-up data were then collected from four (of the seven) voluntary participants 1 month post the intervention, over three to five observation points. To collect the social validity data, the researchers interviewed every participant for 5 min in the virtual world at the end of each interaction task on their perceptions and experience with open-ended questions (e.g., “How do you feel about past experiences?” “Why?”). The researchers also interviewed the parents by email on their satisfaction with the intervention activities after the program.
Pre- and postintervention social skills measures
The Social Communication Questionnaire (SCQ; Rutter et al., 2003) and Social Skills Questionnaire (SSQ; Spence, 1995) were used as supplementary measures of the participants’ social and communication competence before and after the training program. Both questionnaires are validated instruments used for children with ASD in prior empirical research and have satisfactory internal consistency. The SCQ current form is a 40-item, parent report measure of the individual’s social communication behavior during the most recent 3-month period. The items are administered in a yes/no response format (0–1) and completed by the parent (or other primary caregiver). The scale’s αindex of internal consistency was .69 (pretest) and .83 (posttest) in this current study. The SSQ is a 30-item questionnaire on social skill behaviors over the past 4 weeks in children, with each item rated by both parents and children on a 3-point Likert-type scale (0–2). The reported coefficient αs of the parent and pupil forms in the pre- and posttests were .86, .80, .96, and .92, respectively.
Data analyses
In this study, the researchers used visual analysis aided by nonparametric metric (Tau-U; Vannest et al., 2016) and paired t test statistical methods. The researchers conducted a visual analysis of multiple baseline across-participants graphs (i.e., graphing the data collected and visually inspecting the differences between phases and across participants) to determine whether there is a functional relation between the intervention and the outcome variables (Alberto & Troutman, 2006; Kratochwill et al., 2010). Specifically, the researchers focused on examining within- and between-phase data patterns in terms of consistency of changes in (a) level, or the mean score for the data within a phase; (b) trend, or overall direction of the data within a phase; (c) variability, or the range and standard deviation of the data; (d) overlap, or the proportion of data from one phase that overlaps with data from the previous phase (the smaller the overlap is, the more compelling the demonstration of effect is), and (e) consistency of data patterns in similar phases across participants. Overall, the visual analysis aimed to assess whether the data demonstrate at least three aforementioned indicators of the intervention effect for the participants.
To supplement and corroborate the visual analysis, the pre- and postintervention comparison analyses (paired t tests) were conducted with the pre- and postintervention questionnaire results and with the observed behavior measures in the baseline and intervention phases. The researchers also conducted a qualitative cross-case analysis with the videos and observation notes to further explore the actions and reactions of participants (as multiple cases) related to different social scenarios and interaction tasks during the intervention. The processes of individual-case coding and constant cross-case comparative analysis (Yin, 2017) were performed with the goal of identifying common and different characteristics and behavioral patterns found in individuals who were successful and those who were challenged by the interaction tasks. The qualitative case analysis helped the researchers better interpret meaningful trends governing participants’ social skills performance in the context of the activities and scenario features of the VR-based intervention.
Results
General Effect on Social Skills Performance
All participants completed the designated VR-based social interaction tasks. There is a general increase in the averaged frequencies of successful performance of the targeted social interaction skills by the participants from the baseline to the intervention phase, as the descriptive statistics (Table 3) portrays. The four retained participants maintained a better level of performance in negotiation, positive self-identity expression, and cognitive flexibility at the follow-up than the baseline. The paired t tests with all participants’ baseline and intervention data indicated an overall significant social skill performance improvement (Table 4).
Means and Standard Deviations of the Average Frequencies Per 3 Minutes in Social Interaction Skills Performance.
Note. The average frequency refers to the number of successful enactments per 3 min in each baseline or intervention session; the means and standard deviations of average frequencies were then calculated across all baseline or intervention sessions for each participant.
Paired t Test for Seven Participants on the Average Frequencies Per 3 Min in Social Interaction Skills Performance.
Social validity
Participants constantly expressed satisfaction with the VR-based social interaction activities during interviewing. Their parents reported increased voluntary engagement by the participants in daily social interactions post the intervention. Four participants requested and continued the VR-based social interaction activities even after the study was completed.
Performance of Specific Social Skills
The graphical data series of and nonoverlap metric (Tau-U) calculated with the baseline and intervention data showed an improvement of social skills performance by the participants. However, the intervention effect on specific social skills could fluctuate across sessions related to the VR intervention activities, as discussed in the next section.
Initiation of social interactions
Figure 2 illustrated the level and trend of successful social initiation during the baseline and intervention phases performed by the participants in two subgroups. The numerical values on the Y coordinate were the average frequency of successful enactments of interaction initiation (per 3 min), and the X coordinate showed baseline and intervention sessions or observation points. The average Tau-U values of the participants in two subgroups were 0.64, p < .001, 90% CI [0.39, 0.90], and 0.52, p = .01, 90% CI [0.19, 0.85], respectively. That is, 64% or 52% of the intervention phase data showed improvement, and the improvement trend was significant, after controlling for the baseline trend.
Initiation performance (frequency per 3 min) in the baseline and intervention sessions.
A visual analysis of individual participants’ initiation performance between the baseline and intervention phases also showed a trend of improvement, except for Participant 5. Participant 5 showed reduced initiation performance from the baseline to the intervention phase, Tau-U = −0.5, p = .17 (or nonsignificant), 90% CI [−1, 0.11]. Qualitative observation indicated that he tended to be occupied by exploring virtual artifacts or simulated social scenes and was relatively passive in social interactions in the intervention phase. On the other hand, he portrayed active and appropriate social initiation performance in the sessions when the intervention activities (e.g., artifact design for clients) made him learn to identify and cater to others’ needs or perspectives (e.g., in Sessions 7–8, and 17). Such a pattern on the potential moderation of the intervention task feature on the initiation performance was also found with another participant (e.g., Participant 3) whose social-initiation performance trend fluctuated across sessions.
Overall, the Tau-U of each participant in Subgroup 1 was positive, ranging from 0.41 to 0.80, which indicated a moderate to large improvement in the interaction-initiation performance. In Subgroup 2, Participants 6 and 7 showed a consistent and enhanced interaction-initiation performance from the baseline to the intervention, with the Tau-U values ranging from 0.93 to 1 that suggested a large improvement.
Interpersonal negotiation
Figure 3 illustrated an increasing trend in the successful enactments of interpersonal negotiation from the baseline to the intervention phase for all participants. The average Tau-U values of the participants in two subgroups are 0.75, p < .001, 90% CI [0.49, 1.00], and 0.64, p = .002, 90% CI [0.30, 0.97], respectively. That is, 75% or 64% of the intervention phase data showed significant improvement, after controlling for the baseline trend. For Participants 1 and 5–7, the intervention lacked an immediate effect. Not until after three to five intervention sessions, did these participants show increased and successful enactments of negotiation. From being originally noncommunicative, unaware, or withdrawn with a conflict, the participants in the intervention phase gradually portrayed more acts of acknowledging perspective differences, voluntary opinion seeking or explanation, and verbal or nonverbal expressions that integrated self and others’ perspectives.
Interpersonal negotiation (frequency per 3 min) in the baseline and intervention sessions.
Certain intervention activities, however, came short of motives or provisions for the practice and training of negotiation. A self-referential design task (e.g., building a dream house for oneself) or a game that prioritizes environmental exploration rather than interpersonal interactions (e.g., treasure hunting) was associated with less attempts of negotiation than other intervention activities. Participant 3, for example, reduced her negotiation performance in the last session after getting involved in building a virtual ski vacation home.
Positive self-identity expression
Figure 4 indicated an increased level in the self-identity expressions of most participants from the baseline to the intervention phase, but the data variability among the participants was evident. The average Tau-U values of the participants in two subgroups were 0.54, p < .001, 90% CI [0.28, 0.79], and 0.24, p > .05, 90% CI [−0.09, 0.57], respectively. The interphase improvement was medium and significant in the first subgroup but small and nonsignificant in the other, after controlling for the baseline trend.
Positive self-identity expression (frequency per 3 min) in the baseline and intervention sessions.
It was observed that participants’ attempts to explore and express self-identity was linked to their experience of pressure during intervention activities. Participant 1, for example, lacked positive identity expressions in the sessions (e.g., of 14, 18–19, 21, 23, 27–28) that featured the intervention activities of peer competition, such as virtual car racing, scavenge hunting, or design competition. Such a pattern was also observed with other participants when the activities involved a high-pressure social scenario (e.g., job interview in Session 27 of Participant 2 and in Session 13 of Participant 3). Reduced performance of self-identity expression was also linked to the intervention activities that prioritized environment exploration and the fulfillment of external demands (e.g., customer service role-playing or client-requested artifact design in Sessions 30, 32–33 of Participant 1, in Sessions 17, 30 of Participant 2, in Session 27 of Participant 3).
Participants 4, 5, and 7 enacted positive self-identity expression in less than one third of the intervention sessions. These sessions were linked to the activities of storytelling, creative artifact design, and strategy gaming (e.g., a Five in a Row chess game). It appeared that self-identity expression was cultivated by the acts of creativity or spontaneity.
Cognitive flexibility
The graphical data series in Figure 5 showed an increased level in the cognitive flexibility performance of the participants from the baseline to the intervention phase. The average Tau-U values of the participants in two subgroups are 0.52, p < .001, 90% CI [0.26, 0.78], and 0.66, p = .001, 90% CI [0.32, 0.99], respectively. That is, 52% or 66% of the intervention phase data showed significant improvement, after controlling for the baseline trend.
Cognitive flexibility (frequency per 3 min) in the baseline and intervention sessions.
The increasing trend of cognitive flexibility performance showed variability. Participant 1, for example, manifested improved performance of cognitive flexibility in the sessions when the intervention activities featured collaborative representation or design, such as a collaborative storytelling game, a team design quest, or a math puzzle game that prompted one to interact with other characters to gather task-relevant cues. On the other hand, his flexibility performance dropped when the intervention activities involved peer competition or lacked an intriguing theme (e.g., Sessions 19–23, 32, 36). Similarly, Participant 3 demonstrated an improved flexibility performance in the sessions when the intervention activities made her practice interactive storytelling (e.g., Sessions 12, 17), imaginary role-playing (e.g., Session 22), or creative design (e.g., constructing an underwater laboratory in Session 27). She then showed least enactments of flexibility in the sessions when the intervention activities omitted a narrative (e.g., math gaming in Sessions 13–14 or artifact design without a backdrop story in Sessions 18, 25, 31).
Participants 2 and 4 showed a gradual increase in the cognitive flexibility performance during the intervention. Participant 2’s flexibility enactments increased with collaborative gameplay or design activities but dropped with competitive gameplay (e.g., playing a Gomoku game in Session 21) or protocolled social role-play (e.g., job interviewing in Session 26). Participant 4’s flexibility performance improved slowly but steadily, but it dropped when he initially played the Five in a Row chess game (in Session 23). As observed, he failed to check alternative positions of a future move, predict the future move of the opponent, or identify a move that would help in the future position. Instead, he repeated a routine strategy and constantly got his move predicted and blocked by the opponent. After multiple rounds of chess gameplay with different opponents, he portrayed an obvious increase in flexibility performance in the next session. In Session 29, he experienced another flexibility-performance drop when the intervention activity made him practice solving a challenging social problem (e.g., verbal bullying). But he managed to resume the flexibility performance in the subsequent session with the same activity. In Session 31, he was found involved in exploring the virtual space (a snow resort) rather than the designated design activity, thus missing flexibility enactments.
Participants 5–7 all manifested an increased level in the cognitive-flexibility performance during the intervention. After the initial two to five intervention sessions, they showed more successful attempts to identify, switch, and use alternative rules or solutions with the simulated interaction tasks. Their flexibility performance dropped when the activities featured protocoled social role-play (e.g., job interviewing in Session 15 of Participant 5, and customer service role-playing in Sessions 26–27 of Participant 7) or environment exploration (e.g., scavenge hunting in Sessions 17–18 by Participant 5 and construction site inspection in Sessions 22 and 24 by Participant 6).
Pre- and Postintervention Social and Communication Competence
A comparative analysis with the pre- and postintervention SCQ and SSQ responses indicated a numerical reduction of the ASD symptom (or SCQ scores) and an improvement of social and communication competence of the participants (in SSQ) after the intervention. But the differences were not statistically significant, potentially due to a small sample size. Tables 5–6 summarize the descriptive statistics and paired-samples t test results as well as the SCQ and SSQ scores of individual participants.
Paired t Test Results for SCQ and SSQ Measures.
Note. SSQ = Social Skills Questionnaire; SCQ = Social Communication Questionnaire; ASD = autism spectrum disorder.
a One participant failed to complete postintervention measures and was not included in the analysis.
SCQ and SSQ Results of Participants.
Note. SSQ = Social Skills Questionnaire; SCQ = Social Communication Questionnaire.
Discussion and Conclusions
The current study findings support the usage of the VR-based learning environment to promote the social skills performance of participants with ASD. The participants demonstrate an improved level of social skills performance from the baseline to the intervention phase. The improved performance of negotiation, self-identity expression, and cognitive flexibility is maintained at follow-up. The average Tau-U values of the participants’ social skills performance between the baseline and the intervention phases are generally positive and significant. The corresponding graphical data series generally portray an increased level. The comparative analyses with the pre- and postintervention social communication and skills questionnaire responses do not indicate a significant result, though there is a numerical reduction of the ASD symptom and improvement of reported social and communication competence post the intervention.
On the other hand, the effect of the VR-based, play- and design-oriented training varies across intervention activities among participants on the targeted social skills, especially self-identity expression. The finding on the data variability among participants supports the argument of prior research that it is difficult for a single social skill training task or method to be effective for all participants with ASD (Ke et al., 2018; Parsons, 2016). The study indicated that the participants’ social skills performance can be impacted by how they react to a specific training task and the associated social stimuli (e.g., a backdrop story, a game rule that encourages collaboration vs. competition). Specifically, a self-referential design task may fail to accommodate interpersonal negotiation but motivate self-identity expression. A task that features collaborative play or design tends to facilitate the practice of negotiation and cognitive flexibility, whereas a play or design quest that prioritizes environmental exploration may reduce the practice of these skills. The finding echoes the social motivation explanation of autism (Chevallier et al., 2012) on the importance of learners’ motivation or agency in social engagement and interaction performance. An implication is that VR-based training activities differ in their social affordance and should be selected and designed in alignment with the learner and learning needs to promote a comprehensive set of social skills.
The VR-based training activities lack an immediate effect on the participants’ performance of complex social skills such as negotiation and cognitive flexibility. Such a pattern may be due to the lack of a direct experience of demanding social scenarios (e.g., customer service, job interview, and competitive gaming) for the participants with ASD in everyday social interactions. Hence, these activities are novel and can be initially overwhelming for the participants, though the participants do manage to improve and manifest the targeted social skills after continuous learning and practice with those complex social interaction tasks. Another potential reason is that participants may experience a learning curve for performing multifaceted social interactions in the virtual world. Additional orientation on the VR technology can be provided before the initial training sessions.
In conclusion, this exploratory study provides preliminary evidence on the positive effect of the desktop VR-based, play- and design-oriented social skills training for children with ASD. The study also illustrates the challenges in designing and implementing a comprehensive social skill intervention for a heterogeneous group of learners with ASD. Based on the study findings, a future VR-based social skills learning environment should (a) dynamically select and align the type, sequence, and social stimuli of the play- and design-oriented training activities based on individual learners’ engagement and performance states and (b) allow a longer duration of practice and learning with the novel or demanding social tasks.
Implications, Limitations, and Future Research
The study findings suggest the feasibility of using an open-source, desktop VR platform to support situated, play- and design-oriented training of multifaceted social skills with children with ASD. VR-based social skills practice can occur in an informal learning setting (e.g., at home) to cater to individuals’ schedules and preferences, thus supplementing school education and acting as a naturalistic extension of their social and learning engagement. As the findings imply, the accessibility and participatory nature of the desktop VR platform should enable both educators and learners to adaptively design or select appropriate scenes, tasks, and presentation features as required while sustaining a longitudinal social learning program.
This exploratory study is a partially nonconcurrent multiple baseline design that cannot control history effects that might be coincidental with the intervention application, even though in this study historical events are relatively unlikely to affect all participants since the participants are not in the same school district or neighborhood. It should also be noted that due to a small sample that is typical for applied and clinical research on a phenomenon with low prevalence, the paired t test analyses with the pre- and postintervention SSQ and SCQ measures lack the statistical power. The paired t test results should be interpreted with caution. It is warranted for the future applied and clinical research to study the VR-based social learning environment with a larger sample and an alternative or a more rigorous research design, such as a concurrent multiple baseline with additional baseline data observation points, a changing criterion single-case design that evaluates the intervention effects on fostering a social skill over time, or an experimental control group study that includes the measure of generalization.
Besides, in this study desktop, participatory design–integrated VR rather than cave automatic virtual environment is examined. The usage of the desktop platform with the open-source VR software helps to promote the accessibility of a virtual learning environment in both informal and formal learning settings. Future research could replicate the current study with a cave VR system to further examine the design and effectiveness of an immersive learning environment for individuals with ASD. In this study, a set of play- and design-based social interaction tasks with variant social scenes or stimuli are examined in situ as a holistic packet of VR-based intervention. To better study the relative effectiveness of each type of intervention activities in relation to the learners’ performance, future researchers should consider conducting systematic and iterative design-based research to examine whether and how individual participants will perform differently in response to the different design and implementation features of a VR-based social skills training program.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Spencer Foundation, Chicago, IL (Grant Number 201400178).
