The Preliminary Efficacy of Cognitive Behavioral Therapy Virtual Reality Job Interview and Workplace Communication Training Program for Autistic Adults

Abstract

Background:

The majority of autistic adults experience social communication differences that may lead to frustrations during the job application process, even among those with strong work-related skills or histories. Compared with traditional lecture or role-play training, virtual reality (VR) technology generates more realistic and immersive simulated environments, which is crucial for autistic adults who often have difficulties envisioning real-world situations. The study developed a Cognitive Behavioral Therapy (CBT) VR Job Interview and Workplace Communication Training (CBT-VR-JIWCT) program, considering the features of autistic adults, and examined its feasibility and preliminary efficacy.

Methods:

The eight-session program contained various VR scenarios delivered via head-mounted displays as a highly immersive training environment for practicing job interviewing and communication skills. We examined the feasibility and efficacy of the program through the interviewer-rated Mandarin Chinese Version of the Adapted Mock Interview Rating Scale (adapted MIRS-Chinese), speech analysis, and content analysis. Eighty autistic adults were randomized into the intervention group (n = 40) and the control group (n = 40). Among them, 30 from the intervention group eventually completed the training program, whereas 36 from the control group completed pre- and post-tests without intervention.

Results:

Overall, autistic adults reported minimal simulator sickness symptoms. The attendance rate was high. Compared with the controls, the intervention group had significantly better interview performance, as rated by significant group-by-time interactions on the total adapted MIRS-Chinese scores (p < .001), speech volume (p = .006), as well as a nonsignificant larger group-by-time interaction observed on the frequency of achievement words (p = .056).

Conclusion:

CBT-VR-JIWCT is feasible and effective in enhancing job interview skills among autistic adults. Speech and content analyses demonstrated value in objectively assessing improvements in interview performance, such as word choice and presentation styles, within this population. Overall, this study underscores the potential of VR technology as a promising tool for enhancing employment opportunities among autistic adults.

Community Brief

Why is this an important issue?

Autistic adults frequently face challenges in the context of job applications that primarily stem from inadequate interviewing skills and social communication difficulties. The recurrently experienced frustration can foster conditioned anxiety, diminish self-confidence, and curtail motivation to seek employment opportunities, leading to high unemployment rates within the autistic population.

What was the purpose of this study?

This study developed a unique virtual reality (VR) training program. It integrated a highly immersive VR environment, generated through a head-mounted display, with cognitive behavioral therapy (CBT)-based coaching lessons tailored for autistic adults. This study examined both the feasibility of a using VR technology with autistic adults and the preliminary efficacy of this innovative program.

What did we do?

We recruited 80 autistic adults. We then randomly assigned them to the VR group (n = 40) who underwent our VR training program or the control group (n = 40) who did not receive the training. The VR training program included coaching on job interviewing and communication skills for various workplace situations based on VR simulated environment. We collected the adverse effects when they used the VR platform. To evaluate the preliminary efficacy of the VR training program, we assessed their interview performance on the rating scale, as well as content and prosodic features by speech analysis methods, to compare the performance before and after the training.

What were the results of the study?

Overall, autistic adults reported minimal discomfort when viewing VR materials. The attendance rate for the VR training program was good. Compared with the controls, the VR group demonstrated significantly improved interview performance. Moreover, they exhibited a greater increase in speech volume, total word count, total phrase count, and the use of achievement-related words compared with the control group.

What do these findings add to what was already known?

Previous studies have demonstrated the effectiveness of using computer screens for job interview training; however, empirical evidence on using highly immersive VR equipment in job training remains limited. Our randomized controlled trial further confirmed the efficacy of using head-mounted displays (HMDs) in enhancing job interview skills. Also, in addition to traditional assessment on rating scales, speech and content analyses may help decompose objective improvement in interview performance, including word choosing and ways of presentation.

What are the potential weaknesses in the study?

The relatively small sample size may potentially limit the study's ability to detect small-effect differences after training. Additionally, the study assessed outcomes in a laboratory environment, which may not capture the complexities and nuances of real-life job interviews and employment scenarios.

How will these findings help autistic adults now or in the future?

The results suggest that our VR training program is both feasible and effective in enhancing job interview skills among autistic adults, demonstrating its potential as a valuable tool for addressing the unique challenges faced by this population in the employment context. With the incorporation of innovative technology, our program successfully engaged and motivated unemployed autistic adults who had previously given up searching for jobs. This study paves the way for continued advancements in technology-driven interventions and support services for autistic adults in the job market.

Keywords

autism spectrum disorder virtual reality job interview training

Background

Autistic individuals often demonstrate distinct variations in verbal and nonverbal communication, social perception, and social styles compared with their non-autistic peers. These differences can impede effective communication with coworkers, hinder integration into workplace culture,^1,2 and lead to bias and misunderstandings from non-autistic colleagues.^3,4 Moreover, the challenges in social interaction may pose a significant obstacle during traditional job interviews, which prioritize communicative and social skills over the specific skills required for a given position.⁴ Furthermore, anxiety during interviewing may pose further difficulties,⁵ possibly further hindering interview performance of autistic adults. The above mentioned is reflected in higher rates of unemployment among autistic adults compared with their non-autistic peers.⁶ Therefore, it is crucial to identify effective strategies for enhancing occupational communication and social skills in this population.

However, providing job training for autistic adults poses significant challenges. First, past workplace experiences of being treated unkindly or misunderstood may lead to reluctance to attempt real-world exposure. Second, many workplace scenarios, such as noisy and crowded environments or public speaking, are difficult to simulate effectively through traditional one-on-one role-play. Third, the preliminary efficacy of interventions in a simulated environment may not readily transfer to real-world settings. In response to these challenges, recent evidence strongly supports the integration of innovative virtual reality (VR) technology as a means to improve learning and living skills in autistic adults.^7–9 VR technology creates artificial environments that closely resemble real-world scenarios, providing realistic visual and auditory sensations to the users. This allows autistic adults to easily envision themselves in various situations and repeatedly practice relevant skills without the fear of negative consequences in the real world.

Several studies have examined the feasibility and efficacy of VR systems in improving skills in autistic individuals. While most of these studies have focused on children and adolescents, they have addressed various areas, including emotion recognition,^10–12 social skills,^13,14 and communication skills.^13,15,16 Some studies have also specifically examined the use of VR technology for job interview training (JIT).^17–19 One noteworthy study by Smith et al.²⁰ examined a unique computerized job interview simulator system called the Virtual Reality Job Interview Training (VR-JIT). This system allows trainees to watch video clips of the interviewer, answer questions, and receive immediate feedback within a virtual platform. The study found that the VR-JIT improved job interview skills and subsequent employment outcomes in autistic adults.^20,21 To enhance the applicability to autistic youths and assess the effectiveness in community settings, Smith et al. developed a modified version called Virtual Interview Training for Transition Age Youth (VIT-TAY). The VIT-TAY includes three difficulty levels and focuses on 10 essential job interviewing skills based on existing literature.²² In a randomized controlled trial (RCT), 48 transition-age autistic youth received school-based pre-employment services along with additional VIT-TAY training, whereas 23 transition-age autistic youth received services as usual. The results demonstrated significant improvement in job interview skills and increased job access among those who received VIT-TAY, highlighting the potential benefit of VR technology combined with JIT.²³ A recent study (Adiani et al. 2022) demonstrated a closed-loop adaptive VR-based JIT platform named Career Interview Readiness in VR.²⁴ This program combined a real-time physiology-based stress detection module and gaze detection module to provide individualized adaptation and showed initial feasibility among autistic users. Overall, these studies support the potential of VR technology as a valuable tool for improving various skills, including job interview skills, in autistic adults.

It is important to acknowledge that the studies mentioned earlier, which explored the use of VR systems for JIT, primarily used technologies such as computer screens or tablets. In contrast, head-mounted displays (HMDs) offer users a more immersive and realistic simulated environment.^25,26 HMDs provide a heightened sense of immersion by excluding external visual stimuli and directing the user’s attention solely toward the simulated scenarios. This characteristic of HMDs minimizes the disparity between the artificial training environment and the real-world settings, thereby enhancing the transfer of learned skills from the simulated environment to the actual job interview settings. Pioneering studies have demonstrated the feasibility and efficacy of HMD-assisted VR equipment for social skill training in autism.^25,27 On this basis, we specifically focused on the application of HMD technology in JIT.

The present study aimed to develop a training program, Cognitive Behavioral Therapy VR Job Interview and Workplace Communication Training (CBT-VR-JIWCT) program, that incorporates both CBT coaching and a virtual training environment displayed through HMDs. We conducted a RCT to evaluate the feasibility and preliminary efficacy of this program in improving job interview skills among autistic adults. We hypothesized that participants who received CBT-VR-JIWCT program would demonstrate better interview performance and verbal expression than those in the control group. Moreover, previous studies evaluated training effects predominantly by interviewer-rated job interview performance. We take a step further, aiming to evaluate the preliminary efficacy of both interviewer-rated job interview performance and objective assessment, such as speech analysis, content analysis, and coherence analysis, in order to provide a more comprehensive evaluation of the intervention effect.

Methods

Participants

We recruited autistic adults from the Adult Autism clinics in the Department of Psychiatry, National Taiwan University Hospital. All participants met the diagnostic criteria of Autism Spectrum Disorder (ASD) of the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5). The inclusion criteria were: (1) Young adult aged between 18 and 45 who had a diagnosis of ASD by a licensed psychiatrist; (2) having a full-scale Intelligence Quotient (IQ) >70 on the Wechsler Adult Intelligence Scale, Fourth Edition (WAIS-IV); (3) and scored ≧26 on the Autism Spectrum Quotient (AQ), indicating a significant level of autistic traits. Participants with the following conditions were excluded: (1) having a history of major psychiatric disorders (e.g., bipolar disorder, schizophrenia, or other psychotic disorders) or major neurological diseases and (2) having sensory sensitivities that would preclude participation in CBT-VR-JIWCT program.

We stratified a total of 80 participants by sex and randomized them into two groups: the “CBT-VR-JIWCT” group (VR group, n = 40) or the control group (n = 40) (Fig. 1). Six participants in the intervention group refused VR practice due to uncomfortable side effects (an overview of the side effects is given in “Simulator Sickness Questionnaire” later). Three of them agreed to reallocate into the control group. This relocation may lead to underestimating the severity of the side effect in the VR group, but was expected to have limited influence on the main outcome. The VR group completed the eight-session training; the control group did not receive any specific coaching on job-related skills. Both groups underwent a pre-test and a post-test. Throughout the training period, both groups attended the Adult Autism Clinics regularly every 1–3 months. During these visits, the psychiatrist spent approximately 10–20 minutes discussing coping strategies for stressful life events. All participants reported not receiving professional training outside the study during the study period. All participants voluntarily engaged in the training without payment. In the VR group, we reminded the participants weekly to attend training sessions and offered flexibility on the training schedule concerning their availability. In the VR group (n = 34), 30 completed the intervention and post-test, whereas in the control group (n = 43), 36 completed the post-test, resulting in a final sample of 30 (males n = 23, 77%) vs. 36 (males n = 30, 83%) for analysis. The mean ages of each group were 26.43 (standard deviation [SD] 5.79) and 28.75 (SD 5.84), respectively (Table 1). The two groups did not statistically differ in age, sex, and the levels of autistic characteristics.

FIG. 1.

Flow diagram of study design. IQ, Intelligence Quotient; AQ, Autism Spectrum Quotient; SRS, Social Responsiveness Scale; EQ, Empathy Quotient; VR, virtual reality.

Table 1.

The Demographic Data of the Study Sample

	The VR group (n = 30)		The control group (n = 36)
	Mean (or n)	SD (or %)	Mean (or n)	SD (or %)	t	p
Age	26.43	5.79	28.75	5.84	1.61	0.112
Males	n = 23	79.3%	n = 30	83.3%	x² = 0.46	0.498
Females	n = 7	20.7%	n = 6	16.7%	x² = 0.46	0.498
Full-scale IQ	101.70	19.09	99.23	16.18	−0.74	0.463
Autism Spectrum Quotient (AQ)
Socialness	38.12	6.70	39.55	6.34	0.96	0.339
Mindreading	22.22	3.83	23.03	3.87	0.66	0.515
Patterns	12.46	3.17	13.52	2.90	1.08	0.285
Attention to details	12.06	2.22	12.10	1.97	0.15	0.883
Attention switching difficulty	17.64	3.26	18.39	2.50	1.09	0.283
AQ total scores (TW)	102.54	12.85	106.58	11.61	1.26	0.213
AQ (UK)	31.89	7.18	34.81	5.82	1.86	0.068
Empathy Quotient (EQ)	24.80	7.78	21.17	9.25	−1.73	0.090
Social Responsiveness Scale (SRS)
Social communication	45.93	16.54	43.06	13.34	−0.45	0.653
Stereotyped behaviors	22.64	7.82	22.90	6.99	0.26	0.795
Social awareness	19.75	4.67	21.23	4.29	1.51	0.136
Social emotion	14.68	4.77	15.42	4.80	0.65	0.519
SRS total scores	103.00	28.59	102.6	25.8	0.18	0.856

VR, virtual reality; SD, standard deviation; IQ, Intelligence Quotient.

CBT-VR-JIWCT

The training program consists of eight 60-minute sessions, using an HTC VIVE HMD to provide immersive VR scenarios for practicing job interviewing and communication skills in various workplace scenarios, such as briefings and handling disagreements. Each theme was practiced at basic and advanced difficulty levels, with the basic scenario more supportive and the advanced scenario containing harsh comments and doubts to train participants to respond calmly and appropriately.

Session 1 focused on teaching participants about anxiety, providing relaxation techniques to cope with it, and using the Anxiety Rating Scale to quantify anxiety levels (0–10).²⁸ In the subsequent sessions (sessions 2–7), participants underwent relaxation training before VR exposure, and they practiced the same VR scenario twice, with specific coaching in between. The training program addressed various key aspects necessary for a successful job interview.²² These included portraying oneself as a diligent, dependable, and cooperative team player who demonstrates professionalism and negotiating skills, maintains a positive and honest demeanor, expresses genuine interest in the position, and establishes overall rapport with the interviewer. During the briefing scenarios (sessions 4–5), participants acquired skills for delivering concise presentations to an audience of 50 people in a conference room. They received coaching on speech organization and presentation skills aimed at capturing the audience’s attention. This involved avoiding overly technical or detailed language, providing specific examples, summarizing key presentation points, using eye contact to engage the audience, and using body language to emphasize key ideas. In the disagreement scenarios (sessions 6–7), participants learned to handle a conflict situation with a colleague where they were challenged by a colleague. They were coached to proficiently adopt the six steps of the Program for the Education and Enrichment of Relational Skills (PEERS®) program²⁹ in these scenarios, including “keep cool, listen, repeat, explain, say sorry, and solve the problem.” Session 8 served as a review and conclusion of the main points covered in each session. Supplementary Table S1 provides an overview of all the VR scenarios in the eight sessions.

Throughout the eight sessions, we incorporated several elements from the PEERS Social Skills Training program,²⁹ which is shown as effective in improving overall social skills in autistic young adults in Taiwan.³⁰ These elements included coaching with specific steps, role-play demonstrations and rehearsals, enhancing perspective-taking ability, and emphasizing appropriate body language. After exposure to the VR training scenario in each session, we provided individualized coaching, such as “avoiding criticizing former employers” or “providing one example to support your advantage,” based on the participants’ performance. We pointed out inappropriate responses, demonstrated appropriate examples, and encouraged immediate rehearsal practices. We prompted participants to consider the perspective of the interviewer. For example, the question “How would you react if you were not selected for the position?” may assess the interviewee’s frustration tolerance toward challenging questions. Therefore, it is important to show determination to continuous self-transcendence and applying the position again after self-enhancement rather than simply saying, “I would just move on to the next job.” Finally, we consistently emphasized appropriate body language, including maintaining eye contact, offering social smiles and courtesies, as well as adjusting speech volume, speed, and pitch. In addition, in each session, we used cognitive restructuring based on CBT principles to address unhelpful or dysfunctional cognitive patterns, such as all-or-nothing thinking and catastrophizing, which could lead to poor performance. Our goal was to identify alternative thinking to replace these maladaptive beliefs. Personalized coaching was an integral part of our approach, with participants receiving immediate feedback on their practice performance. To provide immediate feedback to participants, we outlined their strengths and areas to be improved and offered specific coaching accordingly. We demonstrated appropriate responses on the spot and encouraged them to practice again. If the performance was improved, we promptly gave verbal encouragement. Repetitive exposure to VR scenarios was a deliberate strategy to enhance the efficacy of the training and desensitize anxiety responses. We summarize the specific details of the training protocol in Supplementary Table S2. The VR scenarios experienced by the participants are shown in Supplementary Figure S1. The training team consisted of one psychiatrist and three research assistants with backgrounds in psychology. To ensure fidelity to the program, they underwent extensive training that included demonstration, observation, and close supervision before they began delivering the training independently. Before the implementation of this clinical trial, we invited three autistic adults to try this VR training program. Their valuable feedback on the training format and the VR platform helps optimize the training program.

Procedures

The Research Ethics Committee of National Taiwan University Hospital approved this study before implementation [No.201712172RINC]. After informed consent procedures in the clinics, all participants completed baseline assessments, including IQ, AQ,³¹ Social Responsiveness Scale (SRS),³² and Empathy Quotient (EQ)³³ before randomization. The enrolled participants were later randomized into the VR group or the control group. The participants in both groups underwent “Mock job interview” at the 1st week and the 8th week of the training period, as the assessments of pre-test and post-test performance. We recorded the participants’ responses in the pre-test and post-test to evaluate the interview performance on the Mandarin Chinese Version of the Adapted Mock Interview Rating Scale (adapted MIRS-Chinese) and speech characteristics. The evaluators were blind to the grouping status of the participants.

Measures

AQ

The AQ³¹ is a self-report questionnaire developed to quantify autistic traits in adults with normal IQ on WAIS. It consists of 50 theoretically derived statements depicting personal views, habits, and preferences pertinent to the unique profile of autistic people. Each statement is rated on a four-point scale, with answer categories “definitely agree,” “slightly agree,” “slightly disagree,” and “definitely disagree.” The former two are scored “1” and the latter two are “0,” leading to the total score ranging from 0 to 50, where a higher score depicts the autistic end of the continuum. The AQ has satisfactory internal consistency (0.82) and test–retest reliability (0.70) and good discriminative validity and screening properties for the diagnosis of ASD at a threshold score of 26.³⁴ The Chinese AQ had good psychometric properties with a 5-factor structure, that is, socialness, mindreading, patterns, attention to details, and attention switching.³⁵

EQ

The EQ³³ is a measure of empathy. Around 81% of autistic adolescents and adults score less than 30 on the EQ, compared with 12% of non-autistic controls; the groups together report excellent internal consistency (0.92) and test–retest reliability (0.97). The Chinese version of EQ has satisfactory reliability and validity (Huang HY & Gau SS, unpublished).

SRS

The SRS³² is a 65-item rating scale on the level of autistic characteristics in natural settings over the past 6 months. It includes domains of social awareness, social information processing, capacity for reciprocal social communication, social avoidance, and autistic mannerisms. Items were rated on a 4-point Likert scale from “0” (not true) to “3” (almost always true). Higher scores reflect greater difficulties in social communication and more prominent autistic features. The Chinese-language SRS has demonstrated a satisfactory four-factor structure with high internal consistency (Cronbach’s alpha, 0.94–0.95), that is, social communication, stereotyped behaviors/interest, social awareness, and social emotion,³⁶ and has been widely used to assess social difficulties in Taiwan.

Simulator sickness questionnaire

The simulator sickness questionnaire (SSQ) is a 16-item questionnaire for the assessment of simulator sickness.³⁷ Each SSQ item receives a rating between 0 and 3. The SSQ comprises three categories as follows: nausea, oculomotor, and disorientation. A total simulator sickness score is defined as the accumulation of the sum scores of all three categories multiplied by a constant of 3.74.³⁸ The higher scores indicate more severe sickness symptoms. The severity of sickness symptoms was “negligible” if the total score <5, “minimal” if 5–9, “significant” if 10–14, “concerning” if 15–19, and “bad” if ≧20.³⁹ The SSQ is recently used to provide an optimal virtual environment for HMDs. This study used the SSQ to assess the adverse effects of the VR platform. A total of 107 autistic adults were invited to test on the VR platform (see Supplementary Table S3). Some experienced mild sickness on the VR testing platform. The most common symptoms included mild eyestrain (21%), blurred vision (16%), difficulty focusing (15%), general discomfort (15%), fatigue (13%), and vertigo (13%). The SSQ total score was 7.83, indicating a “minimal” level of sickness symptom severity.³⁹

Adapted MIRS-Chinese

We referred to the Mock Interview Rating Scale (MIRS),⁴⁰ a validated tool for evaluating job interview performance in adults with schizophrenia, which has recently been applied in autistic adults.⁴¹ We translated the MIRS into Mandarin Chinese, maintaining the original items and 5-point rating anchors, while incorporating additional items assessing speech amount, speech volume, appropriate body language, and confidence specific for autistic individuals. This adapted version, the “adapted MIRS-Chinese,” consisted of 14 items: (1) comfort level, (2) negotiation skills, (3) conveying oneself as a hard worker (dependable), (4) sounding easy to work with (teamwork), (5) sharing things in a positive way, (6) sounding honest, (7) sounding interested in the position, (8) sounding professional, (9) establishing overall rapport with the interviewer, (10) overall impression, (11) speech amount, (12) speech volume, (13) appropriate body language, and (14) confident attitude. Each item was rated on a scale from 1 to 5. We summed the scores of the 14 domains of each individual to derive a total score. The higher score indicated better performance.

The mock job interview entailed standard questions like: “Please give a one-minute brief introduction of your past work experience and reasons for leaving,” “Why do you want to apply for this job?,” and “What are your strengths that make you more suitable for this job than others?” We recorded the trainees’ performance into audio files for scoring purposes and randomly assigned these audio files to three raters who were blind to the experimental condition. Before this study began, the three raters independently rated the same five audio files and attained a high degree of reliability (intraclass correlation coefficient = 0.91).

Data Analysis

Speech analysis

We used the openSMILE V2.0 software⁴² to analyze the audio files of the pre- and post-test performance. The analyzed performance dimensions included voice volume, pitch, and quality (i.e., the trembling tendency of a voice). The volume was the mean intensity of the voice. The pitch was defined as the fundamental frequency (F0) of a speech on a logarithmic semitone scale. The voice quality was quantified by several acoustic features, such as jitter (indexing aperiodicity of the F0 signal), shimmer (the difference of the peak amplitudes of consecutive F0 periods), and Harmonics-to-Noise Ratio (indexing the relative amount of additive noise in the voice).

All these performance-indexing speech characteristics indicate a speaker’s emotional state. Specifically, emotions that relate to a higher arousal state or activation, such as anger and happiness, are associated with higher volume, pitch, and pitch variability (the standard deviation of the pitch). In contrast, sadness and boredom usually relate to lower volume, pitch, and pitch variability.^43–46 Anxiety, often found in the interviewees during job interviews, was associated with an increased pitch, probably due to increased tension of laryngeal and vocal fold muscles under an anxious state.⁴⁷ In contrast, anxiety was related to lower pitch variability,^46,48 increased proportion of pauses,⁴⁶ and change in voice quality (increased jitter).⁴⁹

Content analysis

Linguistic Inquiry and Word Count (LIWC)⁵⁰ is a text analysis program that counts the words relevant to psychologically meaningful categories in written texts or speeches. LIWC transferred qualitative description into quantitative data (i.e., the relative proportions of categories in texts) and can avoid subjectivity and inconsistency in human coding. Chinese LIWC⁵¹ was developed from English LIWC dictionary and revised according to Chinese language characteristics. It contains a total of around 6800 words across 30 linguistic categories and 42 psychological categories. The reliability and validity of Chinese LIWC and its equivalence to English LIWC have been proven satisfactory.⁵¹ In this study, we first selected occupation-related word categories from Chinese LIWC. We then compared the percentages of each word category spoken by the VR and control groups between the pre-test and post-test.

Coherence analysis

To investigate text coherence within and between paragraphs, we applied the Sentence-Bidirectional Encoder Representations from Transformers (SBERT),⁵² which integrates the Siamese network with a pre-trained Bidirectional Encoder Representations from Transformers (BERT) model. BERT⁵³ is a pre-trained language model developed by Google for solving Natural Language Processing (NLP) problems. Compared with traditional NLP approaches, the BERT model takes the position and context of a word in a sentence into account. The SBERT extends from BERT and can output a semantically meaningful vector representation for an inputted text, be it a word, a sentence, or a paragraph.

These vector representations outputted from SBERT allowed us to quantify the content coherences of a trainee’s pre- or post-test speeches. Specifically, we operationalized the coherence between two consecutive pieces of text by the semantic similarity score calculated from the cosine similarity between the two vectors (−1: highly opposite meanings; 1: highly similar meanings).⁵⁴ For each paragraph, a sliding window of two consecutive sentences yielded several coherence scores, which were then averaged to produce a mean coherence for each paragraph. Similarly, but at a larger textual scale, we could represent each paragraph by a vector, and we could calculate a coherence score from two consecutive paragraphs. We could then average these between-paragraph coherence scores to derive a mean between-paragraph coherence.

Feasibility study

Feasibility was operationalized as the level of adverse effect while using VR platform, adherence to the intervention, and user experience. Specifically, we evaluated the adverse effects of the VR platform by the SSQ total scores in all participants. As for adherence, we calculated the attendance rate for the VR group, which was presented by the percentage of intervention sessions attended by the participants. User experience was assessed through verbal inquiries regarding the ease of using VR platforms and the level of engagement.

Statistical analyses

We conducted all statistical analyses using SAS 9.4 software. We compared the demographics, degree of autistic traits, and other characteristics between the VR and control groups using Student t-tests and chi-square analyses. For the primary outcome, we conducted generalized estimating equations (GEEs)⁵⁵ to evaluate whether CBT-VR-JIWCT was associated with changes between the pre-test and the post-test in the adapted MIRS-Chinese scores. Specifically, we examined whether there was a significant group-by-time interaction to evaluate the intervention effect after 8-week training.

For speech, content, and coherence analyses, considering that more than half of the variables did not follow a normal distribution, we used nonparametric Mann–Whitney U tests to examine the differences between the pre-test and post-test within the two groups. We also conducted GEE to evaluate group-by-time interaction in voice volume, pitch, quality, selected LIWC categories, and the two coherence scores. GEEs are suitable for analyzing data with a repeated measures structure,⁵⁶ whereas Mann–Whitney U tests may be more efficient or robust for highly skewed or non-normal data,⁵⁷ as in our case. In addition, GEEs primarily focus on modeling mean differences,⁵⁸ whereas Mann–Whitney U tests provide insights into the overall distribution of the data, not just the central tendency.⁵⁹ Therefore, as both sets of information are important, we presented them together. The effect size r of each Mann–Whitney U test was also computed.⁶⁰ In general, an absolute value for r below 0.3 was considered a small effect, between 0.3 and 0.5 as a medium effect, and above 0.5 as a large effect.

Results

Job interview skills

We compared job interview role-play performance measured on the adapted MIRS-Chinese between the pre-test and post-test. We observed a significant group-by-time interaction (z = 6.33, p < 0.001) on the adapted MIRS-Chinese total scores, indicating that the VR group had more improvement on performance scores after 8-week training compared with the control group. (Fig. 2 and Supplementary Table S4)

FIG. 2.

The comparison of the adapted MIRS-Chinese total scores between the pretest and post-test in the VR and the control groups. (A) The VR group (B) The control group (C) The VR group vs. the control group. *GEE analysis revealed a significant group-by-time interaction. Adapted MIRS-Chinese, Mandarin Chinese Version of the Adapted Mock Interview Rating Scale; VR, virtual reality; GEE, generalized estimating equation.

Speech analysis

In the post-test interview, both VR and control groups showed a significantly increased volume and decreased pitch than in the pre-test interview (Supplementary Table S5). When comparing the difference in speech volume, pitch, and quality between the pre-test and post-test performance, we found that after the 8-week training, the improvement in volume was significantly greater in the VR group (mean 0.0034, SD 0.0047) compared with the control group (mean 0.007, SD 0.0030) (U = 655, p = 0.010), whereas improvements in pitch and speech quality were not significant between the two groups (Table 2). These results aligned with the findings from the GEE analyses, which also showed a significant group-by-time interaction (z = 2.76, p = 0.006) on speech volume. (Table 2 and Supplementary Table S4)

Table 2.

The Comparison of the Improvement After Training in Speech Analysis, Content Analysis, and Coherence Analysis Between the Virtual Reality and the Control Groups

	The VR group (n = 30)		The control group (n = 36)
	Mean	SD	Mean	SD	U statistic	p value	Effect size, r
Speech analysis
Volume	0.0034	0.0047	0.0007	0.0030	655	0.010^a	0.319
Pitch	−21.2	37.8	−8.4	12.6	385	0.814	0.029
Quality	0.0007	0.0125	−0.0019	0.0107	532	0.527	0.078
Content analysis
Total word count	153.67	255.96	63.42	172.69	624.5	0.028	0.270
Total phrase count	96.22	159.57	42.03	113.20	609.5	0.044	0.248
“I” words	1.98	2.30	1.88	2.73	539	0.233	0.147
“Social process” words	1.72	4.54	1.74	1.97	587	0.081	0.215
“Affect process” words	1.71	3.70	1.41	2.59	492	0.470	0.089
“Positive emotion” words	1.62	2.24	1.19	1.80	556	0.167	0.170
“Cognitive process” words	6.21	4.05	4.52	4.52	600	0.057	0.234
“Work-related” words	3.31	2.06	2.53	2.33	564	0.141	0.181
“Achievement” words	2.82	2.54	1.74	1.95	607	0.047	0.244
Non-fluencies	0.45	2.62	1.36	2.69	395	0.102	0.201
Fillers	−0.02	2.95	0.84	2.23	372	0.056	0.235
“Negative emotion” words	−0.25	2.10	0.28	1.24	380.5	0.070	0.223
Coherence analysis
Between paragraph coherence	0.07	0.19	0.01	0.14	567	0.132	0.185
Within paragraph coherence	0.01	0.08	0.00	0.08	486	0.401	0.103

There’s a significant group-by-time interaction shown in GEE analysis.

VR, virtual reality; SD, standard deviation; GEE, generalized estimating equation.

Content analysis

Both VR and control groups showed significant gains in total word count, total phrase count, “I,” “social process,” “affect process,” “positive emotion,” “cognitive process,” and “work related” words at the post-test compared with the pre-test. However, in the control group, not only useful words but also redundant words such as “non-fluencies” and “fillers” increased significantly at post-test than pre-test (Supplementary Table S6).

Using the nonparametric Mann–Whitney U test to compare between-group differences on the change in total word counts, total phrase counts, and the category words after intervention (Table 2), we found that the VR group had significantly more improvement in total word count, total phrase count, and the “achievement” words compared with the control group. In GEE analysis, the significance of the group-by-time interaction disappeared, but a nonsignificant larger group-by-time interaction was observed in “achievement” words (z = 1.91, p = 0.056) (Supplementary Table S4).

Coherence analysis

Using the SBERT to analyze coherence within or between paragraphs (Table 2), we found a non-significant improvement of between-paragraph coherence in the VR group than the control group by the nonparametric Mann–Whitney U test, which aligned with the findings from the GEE analyses (Supplementary Table S4).

Feasibility study

The attendance rate during the 8-week training was 88.2% for the VR group. The mean SSQ scores were 7.83, indicating a “minimal” level of sickness symptom severity during the use of the VR platform. Almost all participants verbally reported having an enjoyable experience using VR training platform.

Discussion

As one of the studies applying VR technology in training autistic adults, this study demonstrated the high feasibility of using VR technology in this population due to its well-tolerated nature. The adverse effects were generally rare and mild, and the attendance rate during the 8-week training was acceptable. The preliminary efficacy of this VR training program was validated by a significant group-by-time interaction when evaluating the adapted MIRS-Chinese scores and speech analysis. Overall, autistic adults in the VR training group had significantly increased adapted MIRS-Chinese total scores, speech volume, and trend-level increased “achievement” words after training, compared with the control group. Besides, they exhibited an attitude of being more accountable, expressed genuine interest in the position, portrayed themselves as a cooperative team player, and maintained a more positive demeanor after training. Therefore, this study provides evidence to support that CBT-VR-JIWCT may be a feasible and efficacious program to enhance job interview skills for autistic adults.

The findings of improved job interview performance in the VR group align with previous studies using VR technology for training autistic adults. For instance, Strickland et al.¹⁹ demonstrated that practicing interview skills through role-play with human-controlled avatars in computerized environments led to enhanced content and delivery of responses during job interviews. Moreover, Smith et al.^20,23 presented the effectiveness of a computerized job interview stimulator system in improving job interview skills and subsequent employment outcomes. These findings, combined with ours, suggest that VR technology is promising in training autistic adults for job interview scenarios.

Apart from previous studies using computer screens or tablets as training platforms,^{17–20,23,61} our program represents a novel approach as one of the VR job training programs designed for autistic adults, utilizing innovative VR equipment, HMDs, which offer a more realistic simulation environment. This high degree of realism theoretically increases the potential to apply the acquired skills in real-world situations. Kourtesis et al.²⁷ recently developed an HMD-assisted VR system for training social skills in autistic adults (n = 25). Although their system had high acceptability and feasibility, there was no control group, and the improvement of the performance was not quantified after training. Other existing RCTs focusing on HMDs for life skills in autistic individuals recruited relatively smaller sample sizes (n < 15).²⁵ Our study stands out with its RCT design and larger sample, providing novel evidence for the efficacy of implementing HMDs in job training for autistic adults.

Furthermore, we assessed the preliminary efficacy of the VR training program through multiple means, including interviewer-rated job interview performance and objective quantitative assessments involving speech analysis, content analysis, and coherence analysis. Both VR and control groups exhibited a significant increase in volume and a decrease in pitch during the post-test compared with the pre-test, with the VR group showing a notably greater improvement in volume and greater group-by-time interaction compared with the control group. Previous research has highlighted how anxiety can influence various prosodic features, leading to a higher pitch,⁴⁷ reduced pitch variability,^46,48 increased proportion of pauses,⁴⁶ and alterations in voice quality (increased jitter).⁴⁹ Moreover, successful interviews have been associated with higher voice intensity^62,63 and greater pitch variability.^64,65 In our study, the increase in volume and decrease in pitch observed in both groups during the post-test suggest that repeated practice with mock job interviews contributed to anxiety reduction and improved overall interview performance. In addition, the VR group demonstrated a more significant improvement in volume after the intervention, further supporting the preliminary efficacy of the VR program in enhancing interview-related speech characteristics. These findings were consistent with the results obtained from interview role-play scoring, where both groups showed a significant increase in total adapted MIRS-Chinese scores at the post-test. Notably, the VR group demonstrated a significantly greater group-by-time interaction in total adapted MIRS-Chinese score in comparison with the control group, indicating the importance of individual coaching and VR scenario practice during the 8-week intervention. Our subsidiary analysis revealed a significant correlation between the adapted MIRS-Chinese total score and the speech volume at the post-test (Spearman’s ρ = 0.412, p < 0.001), indicating that speech volume was correlated to overall interview performance (Supplementary Fig. S2, Supplementary Table S7). Further investigation is warranted to explore the impact of the intervention on pitch variability and its association with interview performance.

Regarding the content analysis, both VR and control groups showed significant gains in total word count, total phrase count, “I,” “social process,” “affect process,” “positive emotion,” “cognitive process,” and “work related” words at the post-test compared with the pre-test, suggesting that repeated practice itself could lead to more abundant speech content in autistic adults. However, it became apparent that the increased speech content in the control group was not solely attributed to an augmentation of useful words but was also influenced by an increase in redundant words such as “non-fluencies” and “fillers,” which was not observed in the VR group. These results indicate that while repeated practice alone can enrich the speech content, the VR training program might have more effectively increased the use of pertinent and valuable words during interviews compared with the control group. Previous study also indicated that interview performance was associated with not only more fluent speech and an increase in positive words but also related to fewer filler words and minimizing negative words.⁶⁶

Furthermore, the VR job training group exhibited a notable increase in total word count, total phrase count, and the use of “achievement” words after the training compared with the control group using Mann–Whitney U test, whereas only a nonsignificant larger group-by-time interaction still preserved in “achievement” words in GEE analysis. Findings in total word count, total phrase count, and the use of “achievement” words were not replicated by the GEE method. Future studies may consider a larger sample size to achieve adequate statistical power for conducting the GEE method and may explore the longitudinal changes in participants’ performance during and after training. In addition, Naim et al.⁶⁶ demonstrated an association between better interview performance and the use of more unique words and more “we” (versus “I”), which was not observed in our study. Incorporating additional word categories to thoroughly examine the program’s efficacy should be considered in the future.

Limitations

There were several limitations in this study. First, the study was underpowered to definitively establish the intervention’s efficacy; however, it did provide preliminary insights that can guide the design of future trials. Future studies may consider larger and more diverse samples. As for the feasibility, we only collected adverse effect while using VR platform, adherence to the intervention, and user experience to assess feasibility. Future studies may consider using standard questionnaires to evaluate satisfaction or in-depth interviews to assess user experience. Second, the study measured all outcomes in a laboratory environment with artificial test scenarios, which may not fully replicate real-life situations. Therefore, there was a possibility that the intervention did not correlate with actual increases in obtaining or retaining employment in real-world settings. The transferability of the intervention effect from the experimental environment to the real world should be examined. It would be valuable to incorporate real-life outcomes, such as employment rates, in the follow-up studies to examine the connection between our training program and ecologically valid outcomes. Third, we chose highly immersive VR equipment, HMD, to enhance training realism. However, its limited availability in communities restricts the scalability of our program. Despite this, most autistic participants are interested in this novel technology⁶⁷ that might increase their motivation toward the training. Another advantage is that our replayable simulated scenarios provide an opportunity for repeated exposure practice and may potentially reduce the labor costs of traditional intervention. In the future, investigating partnerships with community institutions will be valuable to enhance scalability and reach more potential beneficiaries. Implementation preparation costs could also be a significant area for further research. In addition, the use of scripted VR scenarios may lead to unexpected responses from autistic adults if the scenarios do not align with their personal situations. Future VR-JIT programs may address these issues specifically. Moreover, although all participants reported not receiving professional training during the study period, there were no objective measurements for this claim. As one of the novel VR job training programs for autistic adults and the first study to examine training effects for autistic adults through speech and content analyses, our findings need to be validated in larger independent samples. Regarding job interview performance, we successfully applied speech and content analyses to evaluate job interview performance, but a larger sample may be required to enhance statistical power for detecting small-effect differences in voice characteristics and content. Furthermore, future studies may consider analyzing additional prosodic information (e.g., pitch variability, portion of speech pauses, and so on) and expanding the range of word categories. Exploring correlations between these prosodic and lexical features and job interview performance would provide a more comprehensive understanding of the intervention’s impact and refine its implementation for autistic adults. Finally, due to the small size of the female subsample, this study lacked sufficient statistical power to conduct a subsample analysis. Verifying whether sex differences affect interview performance and identifying the possible reasons for these differences would be valuable in the future.

Conclusion

In conclusion, CBT-VR-JIWCT has demonstrated both preliminary efficacy and feasibility in improving job interview performance among autistic adults. By leveraging innovative technology, it has successfully engaged many unemployed autistic adults who had previously abandoned their job-seeking efforts due to unsuccessful experiences. The scripted VR scenarios are valuable in helping autistic adults manage anxiety during exposure to job interview situations. This study underscores the promising potential of VR technology as a motivating tool for work preparation among autistic adults. Furthermore, the incorporation of speech and content analyses can be valuable in assessing job interview performance in this population. These analyses offer valuable insights that can inform future interventions and research in this area, ultimately contributing to improved support and opportunities for autistic adults in the job market.

Authors’ contributions

Y.-L.C. conceptualized the study, designed the curriculum, recruited participants, and conducted clinical assessment. M.O. and R.-H.L. designed virtual reality training materials. T.-R.H. performed speech, content, and coherence analyses. P.-R.C. performed statistical analyses and drafted the article. All authors edited and approved the final article. The article has been submitted solely to this journal and is not published.

Footnotes

Acknowledgments

The authors thank research assistants Ms. Su-Chin Pan, Mr. Zu-Chin Ting, Ms. Fang-Yu Hsiao, and Ms. Yi-Jing Lee for their contributions in this training program. The authors also thank Mr. Yi-Hao Wei for assisting the analysis. The authors also thank the staff of National Taiwan University Hospital-Statistical Consulting Unit (NTUH-SCU) for statistical consultation and analyses.

Author Disclosure Statement

All authors declare no conflicts of interest.

Funding Information

The work presented in this article is supported by grants from Minister of Science and Technology (MOST 108-2628-H-002-009-MY3, 111-2410-H-002-156-MY2), National Taiwan University Hospital (UN109-007, UN112-0037), and National Health Research Institutes (NHRI-EX110-11008PC, NHRI-EX111-11008PC, NHRI-EX112-11008PC), Taiwan. The funding bodies had no roles in the study design, collection, analysis, interpretation of data, and article writing.

Supplemental Material

References

Scott

, Milbourn

, Falkmer

, et al. Factors impacting employment for people with autism spectrum disorder: A scoping review. Autism. 2019; 23(4):869–901; doi: 10.1177/1362361318787789

Baldwin

, Costley

, Warren

. Employment activities and experiences of adults with high-functioning autism and Asperger’s disorder. J Autism Dev Disord. 2014; 44(10):2440–2449; doi: 10.1007/s10803-014-2112-z

Sasson

, Faso

, Nugent

, Lovell

, Kennedy

, Grossman

. Neurotypical peers are less willing to interact with those with autism based on thin slice judgments. Sci Rep. 2017; 7(1):40700; doi: 10.1038/srep40700

Whelpley

, May

. Seeing is disliking: Evidence of bias against individuals with autism spectrum disorder in traditional job interviews. J Autism Dev Disord. 2023; 53(4):1363–1374; doi: 10.1007/s10803-022-05432-2

Sarrett

. Interviews, disclosures, and misperceptions: Autistic adults’ perspectives on employment related challenges. DSQ., 2017; 37(2); doi: 10.18061/dsq.v37i2.5524

Roux

, Shattuck

, Rast

, Rava

, Anderson

. National autism indicators report: Transition into young adulthood. Philadelphia, PA: Life Course Outcomes Research Program, AJ Drexel Autism Institute, Drexel University. 2015.

Stock

, Davies

, Wehmeyer

, Lachapelle

. Emerging new practices in technology to support independent community access for people with intellectual and cognitive disabilities. NeuroRehabilitation. 2011; 28(3):261–269; doi: 10.3233/NRE-2011-0654

Gal

, Lamash

, Bauminger-Zviely

, Zancanaro

, Weiss

. Using multitouch collaboration technology to enhance social interaction of children with high-functioning autism. Phys Occup Ther Pediatr. 2016; 36(1):46–58; doi: 10.3109/01942638.2015.1040572

Howard

, Gutworth

. A meta-analysis of virtual reality training programs for social skill development. Computers Educ. 2020; 144:103707; doi: 10.1016/j.compedu.2019.103707

10.

Chen

C-H

, Lee

, Lin

L-Y

. Augmented reality-based video-modeling storybook of nonverbal facial cues for children with autism spectrum disorder to improve their perceptions and judgments of facial expressions and emotions. Comput Hum Behav. 2016; 55:477–485; doi: 10.1016/j.chb.2015.09.033

11.

Didehbani

, Allen

, Kandalaft

, Krawczyk

, Chapman

. Virtual reality social cognition training for children with high functioning autism. Comput Hum Behav. 2016; 62:703–711; doi: 10.1016/j.chb.2016.04.033

12.

Serret

, Hun

, Iakimova

, et al. Facing the challenge of teaching emotions to individuals with low- and high-functioning autism using a new Serious game: A pilot study. Mol Autism. 2014; 5(1):37; doi: 10.1186/2040-2392-5-37

13.

, Moon

, Sokolikj

. Virtual reality–based social skills training for children with autism spectrum disorder. J Spec Educ Technol. 2022; 37(1):49–62; doi: 10.1177/0162643420945603

14.

Lorenzo

, Gómez-Puerta

, Arráez-Vera

, Lorenzo-Lledó

. Preliminary study of augmented reality as an instrument for improvement of social skills in children with autism spectrum disorder. Educ Inf Technol. 2019; 24(1):181–204; doi: 10.1007/s10639-018-9768-5

15.

Bernardini

, Porayska-Pomsta

, Smith

. ECHOES: An intelligent serious game for fostering social communication in children with autism. Inform Sci. 2014; 264:41–60; doi: 10.1016/j.ins.2013.10.027

16.

Trepagnier

, Olsen

, Boteler

, Bell

. Virtual conversation partner for adults with autism. Cyberpsychol Behav Soc Netw. 2011; 14(1–2):21–27; doi: 10.1089/cyber.2009.0255

17.

Burke

, Bresnahan

, Li

, et al. Using Virtual Interactive Training Agents (ViTA) with adults with autism and other developmental disabilities. J Autism Dev Disord. 2018; 48(3):905–912; doi: 10.1007/s10803-017-3374-z

18.

Burke

, Li

, Grudzien

, Garcia

. Brief report: Improving employment interview self-efficacy among adults with autism and other developmental disabilities using Virtual Interactive Training Agents (ViTA). J Autism Dev Disord. 2021; 51(2):741–748; doi: 10.1007/s10803-020-04571-8

19.

Strickland

, Coles

, Southern

. JobTIPS: A transition to employment program for individuals with autism spectrum disorders. J Autism Dev Disord. 2013; 43(10):2472–2483; doi: 10.1007/s10803-013-1800-4

20.

Smith

, Ginger

, Wright

, et al. Virtual reality job interview training in adults with autism spectrum disorder. J Autism Dev Disord. 2014; 44(10):2450–2463; doi: 10.1007/s10803-014-2113-y

21.

Smith

, Fleming

, Wright

, et al. Brief report: Vocational outcomes for young adults with autism spectrum disorders at six months after virtual reality job interview training. J Autism Dev Disord. 2015; 45(10):3364–3369; doi: 10.1007/s10803-015-2470-1

22.

Huffcutt

. An Empirical Review of the Employment Interview Construct Literature. Int J Select Assess. 2011; 19(1):62–81; doi: 10.1111/j.1468-2389.2010.00535.x

23.

Smith

, Sherwood

, Ross

, et al. Virtual interview training for autistic transition age youth: A randomized controlled feasibility and effectiveness trial. Autism. 2021; 25(6):1536–1552; doi: 10.1177/1362361321989928

24.

Adiani

, Itzkovitz

, Bian

, et al. Career Interview Readiness in Virtual Reality (CIRVR): A platform for simulated interview training for autistic individuals and their employers. ACM Trans Access Comput. 2022; 15(1):Article 2–A28; doi: 10.1145/3505560

25.

Skjoldborg

, Bender

, Jensen de Lopez

. The efficacy of head-mounted-display virtual reality intervention to improve life skills of individuals with autism spectrum disorders: A systematic review. Neuropsychiatr Dis Treat. 2022; 18:2295–2310; doi: 10.2147/NDT.S331990

26.

Sherman

, Craig

. Understanding Virtual Reality: Interface, Application, and Design. Morgan Kaufmann Publisher; 2018.

27.

Kourtesis

, Kouklari

, Roussos

, et al. Virtual reality training of social skills in adults with autism spectrum disorder: An examination of acceptability, usability, user experience, social skills, and executive functions. Behav Sci (Basel). 2023; 13(4):336; doi: 10.3390/bs13040336

28.

Bloch

. Healing from depression: 12 weeks to a better mood. Nicolas-Hays, Inc.; 2009.

29.

Laugeson

, Gantman

, Kapp

, Orenski

, Ellingsen

. A randomized controlled trial to improve social skills in young adults with autism spectrum disorder: The UCLA PEERS((R)) Program. J Autism Dev Disord. 2015; 45(12):3978–3989; doi: 10.1007/s10803-015-2504-8

30.

Chien

, Tsai

, Chen

, et al. Effectiveness, durability, and clinical correlates of the PEERS social skills intervention in young adults with autism spectrum disorder: The first evidence outside North America. Psychol Med. 2023; 53(3):966–976; doi: 10.1017/s0033291721002385

31.

Baron-Cohen

, Wheelwright

, Skinner

, Martin

, Clubley

. The autism-spectrum quotient (AQ): evidence from Asperger syndrome/high-functioning autism, males and females, scientists and mathematicians. J Autism Dev Disord. 2001; 31(1):5–17; doi: 10.1023/a:1005653411471

32.

Constantino

, Gruber

. Social Responsiveness Scale (SRS) manual. Western Psychological Services: Los Angeles, CA; 2005.

33.

Baron-Cohen

, Wheelwright

. The empathy quotient: An investigation of adults with Asperger syndrome or high functioning autism, and normal sex differences. J Autism Dev Disord. 2004; 34(2):163–175; doi: 10.1023/b:jadd.0000022607.19833.00

34.

Woodbury-Smith

, Robinson

, Wheelwright

, Baron-Cohen

. Screening adults for Asperger Syndrome using the AQ: A preliminary study of its diagnostic validity in clinical practice. J Autism Dev Disord. 2005; 35(3):331–335; doi: 10.1007/s10803-005-3300-7

35.

Lau

, Gau

, Chiu

, et al. Psychometric properties of the Chinese version of the Autism Spectrum Quotient (AQ). Res Dev Disabil. 2013; 34(1):294–305; doi: 10.1016/j.ridd.2012.08.005

36.

Gau

SS-F

, Liu

L-T

, Wu

Y-Y

, Chiu

Y-N

, Tsai

W-C

. Psychometric properties of the Chinese version of the Social Responsiveness Scale. Res Autism Spectrum Disord. 2013; 7(2):349–360; doi: 10.1016/j.rasd.2012.10.004

37.

Kennedy

, Lane

, Berbaum

, Lilienthal

. Simulator Sickness Questionnaire: An enhanced method for quantifying simulator sickness. Int J Aviation Psychol. 1993; 3(3):203–220; doi: 10.1207/s15327108ijap0303_3

38.

Bimberg

, Weissker

, Kulik

. IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) On the Usage of the Simulator Sickness Questionnaire for Virtual Reality Research. Institute of Electrical and Electronics Engineers Publisher: New York, NY; 2020, pp. 464–467.

39.

Stanney

, Kennedy

, Drexler

. Cybersickness is not simulator sickness. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 41(2), pp. 1138–1142, 1997 doi: 10.1177/107118139704100292

40.

Smith

, Burke-Miller

, Bornheimer

, et al. Psychometric properties of the mock interview rating scale for schizophrenia and other serious mental illnesses. Front Psychiatry. 2023; 14:1150307; doi: 10.3389/fpsyt.2023.1150307

41.

Smith

, Sherwood

, Genova

, et al. Psychometric properties of the mock interview rating scale for autistic transition-age youth. Front Psychiatry. 2023; 14:1235056; doi: 10.3389/fpsyt.2023.1235056

42.

Eyben

, Wöllmer

, Schuller

. Opensmile: The munich versatile and fast open-source audio feature extractor. In: Proceedings of the 18th ACM international conference on Multimedia. October 2010, Firenze, Italy. Association for Computing Machinery: New York, NY; 2010, pp.1459–1462.

43.

Ramakrishnan

, El Emary

IMM

. Speech emotion recognition approaches in human computer interaction. Telecommun Syst. 2013; 52(3):1467–1478; doi: 10.1007/s11235-011-9624-z

44.

Memon

. Acoustic correlates of the voice qualifiers: A survey. arXiv. 2020; doi: 10.48550/arXiv.2010.15869

45.

Laukka

, Juslin

, Bresin

. A dimensional approach to vocal expression of emotion. Cognition Emotion. 2005; 19(5):633–653; doi: 10.1080/02699930441000445

46.

Laukka

, Linnman

, Åhs

, et al. In a nervous voice: Acoustic analysis and perception of anxiety in social Phobics’ speech. J Nonverbal Behav. 2008; 32(4):195–214; doi: 10.1007/s10919-008-0055-9

47.

Johnstone

, van Reekum

, Bänziger

, Hird

, Kirsner

, Scherer

. The effects of difficulty and gain versus loss on vocal physiology and acoustics. Psychophysiology. 2007; 44(5):827–837; doi: 10.1111/j.1469-8986.2007.00552.x

48.

Hagenaars

, van Minnen

. The effect of fear on paralinguistic aspects of speech in patients with panic disorder with agoraphobia. J Anxiety Disord. 2005; 19(5):521–537; doi: 10.1016/j.janxdis.2004.04.008

49.

Fuller

, Horii

, Conner

. Validity and reliability of nonverbal voice measures as indicators of stressor-provoked anxiety. Res Nurs Health. 1992; 15(5):379–389; doi: 10.1002/nur.4770150507

50.

Tausczik

, Pennebaker

. The psychological meaning of words: LIWC and computerized text analysis methods. J Lang Soc Psychol. 2010; 29(1):24–54; doi: 10.1177/0261927X09351676

51.

Huang

, Chung

, Hui

, et al. The development of the chinese linguistic inquiry and word count dictionary. Chinese J Psychol. 2012; 54(2):185–201.

52.

Reimers

, Gurevych

. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. arXiv. 2019; doi: 10.48550/arXiv.1908.10084

53.

Devlin

, Chang

M-WC

, Lee

, Toutanova

. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv. 2018; doi: 10.48550/arXiv.1810.04805

54.

Xia

, Zhang

, Li

. Learning similarity with cosine similarity ensemble. Inform Sci. 2015; 307:39–52; doi: 10.1016/j.ins.2015.02.024

55.

Ballinger

. Using generalized estimating equations for longitudinal data analysis. Org Res Methods. 2004; 7(2):127–150; doi: 10.1177/1094428104263672

56.

Liang

K-Y

, Zeger

. Longitudinal data analysis using generalized linear models. Biometrika. 1986; 73(1):13–22.

57.

Conover

. Practical nonparametric statistics. vol 350. John Wiley & Sons; 1999.

58.

McCulloch

, Searle

. Generalized, Linear, and Mixed Models. Wiley Publisher. Wiley; 2001.

59.

Mann

, Whitney

. On a test of whether one of two random variables is stochastically larger than the other. Ann Math Statist. 1947; 18(1):50–60; doi: 10.1214/aoms/1177730491

60.

Rosenthal

. Parametric measures of effect size. In: The handbook of research synthesis. Russell Sage Foundation; 1994, pp. 231–244.

61.

Kumazaki

, Yoshikawa

, Muramatsu

, et al. Group-based online job interview training program using virtual robot for individuals with autism spectrum disorders. Front Psychiatry. 2021; 12:704564; doi: 10.3389/fpsyt.2021.704564

62.

Macan

. The employment interview: A review of current studies and directions for future research. Hum Resource Manag Rev. 2009; 19(3):203–218; doi: 10.1016/j.hrmr.2009.03.006

63.

Huffcutt

, Conway

, Roth

, Stone

. Identification and meta-analytic assessment of psychological constructs measured in employment interviews. J Appl Psychol. 2001; 86(5):897–913; doi: 10.1037/0021-9010.86.5.897

64.

Scherer

. Personality inference from voice quality: The loud voice of extroversion. Euro J Social Psych. 1978; 8(4):467–487; doi: 10.1002/ejsp.2420080405

65.

Aronovitch

. The voice of personality: Stereotyped judgments and their relation to voice quality and sex of speaker. J Soc Psychol. 1976; 99(2):207–220; doi: 10.1080/00224545.1976.9924774

66.

Naim

, Tanveer

, Gildea

, Hoque

. Automated analysis and prediction of job interview performance. IEEE Trans Affective Comput. 2018; 9(2):191–204; doi: 10.1109/taffc.2016.2614299

67.

Newbutt

, Sung

, Kuo

, Leahy

, Lin

, Tong

. Brief report: A pilot study of the use of a virtual reality headset in autism populations. J Autism Dev Disord. 2016; 46(9):3166–3176; doi: 10.1007/s10803-016-2830-5

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

3.62 MB

0.00 MB