Job interview training targeting nonverbal communication using an android robot for individuals with autism spectrum disorder

Abstract

Job interviews are significant barriers for individuals with autism spectrum disorder because these individuals lack good nonverbal communication skills. We developed a job interview training program using an android robot. The job interview training program using an android robot consists the following three stages: (1) tele-operating an android robot and conversing with others through the android robot, (2) a face-to-face mock job interview with the android robot, and (3) feedback based on the mock job interview and nonverbal communication exercises using the android robot. The participants were randomly assigned to the following two groups: one group received a combined intervention with “interview guidance by teachers and job interview training program using an android robot” (n = 13), and the other group received an intervention with interview guidance by teachers alone (n = 16). Before and after the intervention, the participants in both groups underwent a mock job interview with a human interviewer, who provided outcome measurements of nonverbal communication, self-confidence, and salivary cortisol. After the training sessions, the participants who received the combined interview guidance by teachers and the job interview training program using an android robot intervention displayed improved nonverbal communication skills and self-confidence and had significantly lower levels of salivary cortisol than the participants who only received interview guidance by teachers. The job interview training program using an android robot improved various measures of job interview skills in individuals with autism spectrum disorder.

Keywords

android robot autism spectrum disorder cortisol job interview nonverbal communication

Introduction

Approximately 500,000 individuals with autism spectrum disorder (ASD) will enter adulthood over the next 10 years (Shattuck et al., 2012). The low employment rate of individuals with ASD (56%) has become a major social issue worldwide (Taylor & Seltzer, 2011). Job interviews are typically required for obtaining competitive employment, but these interviews may be significant barriers for individuals with ASD (Higgins, Koch, Boughfman, & Vierstra, 2008; Strickland, Coles, & Southern, 2013). Social competence during a job interview is a specific skill that is critical for a positive occupational outcome (Strickland et al., 2013).

Individuals with ASD use less nonverbal communication (e.g., eye contact, facial expressions, gestures, and body language), which is directly connected to poor performance during job interviews (Strickland et al., 2013). Nonverbal communication is as important as, or even more important than, verbal communication in the job interview setting. According to a previous study, 55% of first impressions are based on nonverbal communication, and only 7% of first impressions are based on the actual verbal content (Albert, 1971). Certain nonverbal mistakes “(e.g., individuals with ASD not looking the interviewer in the eye and not making adequate facial expressions)” can ruin the chances of receiving a job offer, even if the answers to the interview questions are impressive.

There are few evidence-based interventions for individuals with ASD, so many tutors find it difficult to teach the nonverbal communication skills that are essential for successful job interviews (Kandalaft, Didehbani, Krawczyk, Allen, & Chapman, 2012). Thus, a more effective intervention with a greater substantial impact on job interview competence in individuals with ASD is urgently needed. Given the recent rapid technological advances, interventions using advanced technology could be effectively harnessed to provide innovative clinical treatment for individuals with ASD.

Recently, an advanced project using Internet-accessed training and virtual reality has demonstrated that these tools have preliminary efficacy in improving job interview skills in individuals with ASD (Smith et al., 2015; Smith et al., 2014; Strickland et al., 2013). However, in these projects, the individuals could not provide subtle nonverbal cues through their avatars. The physical presence of a robot allows for a more engaging and enjoyable interaction than an avatar on a monitor (Lee, Jung, Kim, & Kim, 2006; Wainer, Feil-seifer, Shell, & Mataric, 2006). The most obvious and unique attribute of a robot is its physical embodiment, and during human-robot interactions, the individual’s impression of the helpfulness and trustworthiness of the robot as well as its enjoyability is significantly affected by the physical embodiment of the robot (Wainer et al., 2006; Wainer, Feil-Seifer, Shell, & Mataric, 2007). The appearance and movements of an android robot resemble those of an actual human (Kumazaki, Warren, Corbett, et al., 2017; Kumazaki, Warren, Muramatsu, et al., 2017; Yoshikawa, Matsumoto, Sumitani, & Ishiguro, 2011). Android robots exhibit facial expressions (e.g., smiling, nodding, and brow movements) during speech and can provide subtle nonverbal cues. One benefit of android robots is that they are similar to humans. Therefore, there is a possibility that generalization may be more easily achieved. Creating intelligent three-dimensional (3D) learning environments using android robots may represent another, potentially more powerful, avenue for enhancing skills with generalization to real-world settings (Warren et al., 2015).

Individuals with ASD are impaired in their ability to attribute mental states to themselves and to others, that is, they lack what has been labeled the Theory of Mind (ToM) (Baron-Cohen, Leslie, & Frith, 1985). Therefore, they do not understand the effect of their behavior on others (Strickland et al., 2013), which is associated with their low motivation to acquire nonverbal communication skills (Chevallier, Kohls, Troiani, Brodkin, & Schultze, 2012). Moreover, a previous study has shown that explicit instruction in emotion recognition can improve an individual’s performance on tasks that require them to attribute mental states to self and others (Golan & Baron-Cohen, 2006). In designing an intervention to help individuals with ASD to learn nonverbal communication, it is important to be concerned not only with teaching the appropriate nonverbal skills required for such an interaction (e.g., making eye contact and using appropriate language) but also with improving understanding of the importance of nonverbal communication.

A previous study reported a case report that tele-operating an android robot could promote an understanding of nonverbal communication (Kumazaki, Muramatsu, et al., 2017). Exposure to a tele-operated android robot before a mock job interview can help participants understand the reason for nonverbal behaviors, rendering the intervention more effective.

Imitating human body motion is also difficult for individuals with ASD. Individuals with ASD have been shown to improve their performance in imitation tasks in experiments using a robot (Cook, Swapp, Pan, Bianchi-Berthouze, & Blakemore, 2013; Pierno, Mari, Lusher, & Castiello, 2008). During a nonverbal communication exercise, presenting nonverbal expression using an android robot may facilitate the acquisition of nonverbal expression.

To facilitate job interview skill training in young adults with ASD, we developed a job interview training program using an android robot (JUA). The JUA consists of the following three stages: (1) tele-operating an android robot and conversing with other individuals through the android robot, (2) a face-to-face mock job interview with an android robot, and (3) feedback based on the mock job interview and nonverbal communication exercise using the android robot.

A previous study shows that the mere exposure to the android robot-mediated interview procedures contributed to a trend of self-reported increases in confidence and, importantly, corresponding reductions in the biological indicators of stress/anxiety during the mock job interview (Kumazaki, Warren, Corbett et al., 2017). JUA differs from the method of a previous study because JUA aims to improve nonverbal communication (i.e., the JUA method adds the tele-operating of the android robot to converse with others and feedback regarding the mock job interview and nonverbal communication exercise using the android robot). In this study, we examined the acquisition of nonverbal communication, self-confidence, and metrics of stress using JUA. We assessed the changes in self-confidence and the metrics of stress after experiencing JUA by measuring self-reports of self-confidence and the salivary cortisol levels. A higher self-confidence in one’s ability to perform a job interview is associated with greater performance during the job interview (Hall, Jackson, Gradt, Goetz, & Musu-Gillette, 2011; Kumazaki, Warren, Corbett et al., 2017; Tay, Ang, & Van Dyne, 2006). Salivary cortisol provides a reliable, non-invasive metric of stress (Corbett, Schupp, & Lanni, 2012) that has been used previously to measure the response to social situations (Corbett et al., 2017; Corbett et al., 2012; Corbett et al., 2014). Cortisol is also associated with self-confidence (van Eck, Berkhof, Nicolson, & Sulon, 1996). In addition, the importance of measuring both the physiological arousal level and self-reported assessment of psychiatric state (e.g., self-confidence, irritability, and anxiety) (Kumazaki, Warren, Corbett et al., 2017; Mikita et al., 2015; Simon & Corbett, 2013) in individuals with ASD has been recently emphasized. Therefore, we assessed both self-reports and the physiological measures of arousal in saliva to obtain a more objective view of self-confidence in individuals with ASD.

Method

Participants

Participants were recruited from our institute well known in Japan for specializing in developmental disorders and related conditions. All procedures involving human participants were conducted in accordance with the ethical standards of the institutional and/or national research committee and the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards. After receiving a complete explanation of the study, all participants and their guardians agreed to participate in the study. All participants provided written informed consent. The inclusion criteria for the participants were as follows: (1) age of 18–27 years of age, (2) IQ ⩾ 70, and (3) unemployed workers who were actively seeking employment and have undergone consecutive mock job interviews before but could not improve their nonverbal communication skills. To exclude other psychiatric diagnoses, the Mini-International Neuropsychiatric Interview (M.I.N.I.) (Sheehan et al., 1998) was administered. The participants were diagnosed by psychiatrist with more than 10 years of experience in ASD using the criteria in the Diagnostic and Statistical Manual of Mental Disorders (5th ed., text rev.; DSM-V; American Psychiatric Association, 2013) and standardized criteria taken from the Diagnostic Interview for Social and Communication Disorders (DISCO) (Leekam, Libby, Wing, Gould, & Taylor, 2002) at the time of enrolment in the study. The DISCO is reported to have good psychometric properties. It also contains items on early development, and a section on activities of daily life, thereby giving the interviewer an idea of the individual’s level of functioning in several different areas, not only social functioning and communication (Wing, Leekam, Libby, Gould, & Larcombe, 2002). All participants who were diagnosed with childhood autism, atypical autism, or Asperger’s syndrome with DISCO were included in this study.

All participants completed the Autism Spectrum Quotient—Japanese version (AQ-J; Wakabayashi, Tojo, Baron-Cohen, & Wheelwright, 2004), which was used to evaluate the ASD-specific behaviors and symptoms. The AQ-J is a short questionnaire with five subscales (i.e., social skills, attention switching, attention to detail, imagination, and communication). Previous studies using the AQ-J have been replicated across cultures (Wakabayashi et al., 2007). The AQ is also sensitive to the broader autism phenotype (Wheelwright, Auyeung, Allison, & Baron-Cohen, 2010). IQ was measured using the Wechsler (2008) Adult Intelligence Scale—Fourth Edition.

The severity of the social anxiety symptoms was measured using the Liebowitz Social Anxiety Scale (LSAS) (Liebowitz, 1987). This clinician-administered scale consists of 24 items, including 13 items that describe performance situations and 11 items that describe social interaction situations. Each item is separately rated for “fear” and “avoidance” using a 4-point categorical scale. According to receiver operating curve analyses, an LSAS score of 30 is predictive of minimal symptoms and is the best cut-off value for distinguishing individuals with from those without social anxiety disorder (Mennin et al., 2002).

Procedures

Initially, participants were given a document containing recruitment information from which they could select jobs including data entry clerk, shelf stacker in a supermarket, custodian, kitchen assistant in a restaurant, nursing assistant, and paper delivery person. We interviewed many staff members working in an employment support facility for individuals with ASD and chose six jobs that are popular among such individuals. Participants were asked questions related to the job they selected during an interview. Questions asked during the interview were about the participants, their reasons for applying for the job, their expertise, their disabilities, their weaknesses, and details about their duties after joining the company. These questions were selected from a textbook which was made with reference to past job interview tests of individuals with ASD and used in schools for them. Then, the individuals were randomly assigned to the two groups (Figure 1). The subsequent trial procedures were conducted from day 1 to day 7. On days 1 and 7, the participants in both groups participated in an approximately 20-min mock job interview with a human interviewer at the same time of day. During this mock job interview, the human interviewer followed a specific interview script and protocol across all interviews. From day 2 to day 6, the participants in the combination group received JUA after the interview guidance by teachers (IGT). The participants in the control group received only IGT.

Figure 1.

Participant flowchart.

The android robot used in this study was Actroid-F (Figure 2) (Kokoro Co Ltd Hamura, Tokyo, Japan), which is a female humanoid robot with an appearance similar to that of a real person (Kumazaki, Warren, Corbett et al., 2017; Kumazaki, Warren, Muramatsu, et al., 2017; Yoshikawa et al., 2011). Its artificial body has the same proportions, facial features, hair color, and hairstyle as a human. The voice of the android robot is similar to that of an actual person. To elicit the belief that the robots behaved and responded autonomously, we adopted a remote-control system similar to that conventionally used in robotics studies (Nishio, Taura, Sumioka, & Ishiguro, 2013). The Actroid-F incorporated changes in facial expression (i.e., smiling, nodding, and brow movements) during speech.

Figure 2.

Example of a typical interaction between the participants and Actroid-F (android robot) during the mock job interview.

As described in the introduction, JUA consists the following three stages: (1) tele-operating an android robot and conversing with others (Figure 3), (2) a face-to-face mock job interview conducted by an android robot, and (3) feedback regarding the mock job interview and nonverbal communication exercise using the android robot. The participants tele-operated the android robot for approximately 5 min before the mock job interview that was conducted by the android robot, which required approximately 15 min. All scripts were prepared before the mock job interview. When a button is pushed by a teacher who is familiar with this system, the android robot begins to speak according to previously prepared scripts. After the mock job interview was conducted by the android robot, the participants received feedback regarding the mock job interview and a nonverbal communication exercise, which was provided by the android robot and which required approximately 10 min.

Figure 3.

Example of participants operating the android robot.

In the IGT, the participants received instruction about nonverbal communication in job interviews from teachers. During these sessions, materials regarding the most commonly asked questions in real job interviews were provided. Participants were given access to feedback about their nonverbal communication. The approximate duration of each IGT was approximately 30 min per day.

After the human interviewer sessions (on days 1 and 7), all participants completed questionnaires scored using Likert-type rating scales regarding their self-confidence in their performance. The ratings ranged from 0 (not at all comfortable) to 6 (very comfortable). In addition, after the intervention (on day 6) all participants answered yes/no question, namely “Do you want to receive the intervention again?”

We rated the job interview performance of the participants in the mock job interviews with a human interviewer on days 1 and 7. The interviews were scored using a 7-point Likert-type scale related to the interview performance, and the higher scores reflected a greater performance in nonverbal communication (i.e., posture, gaze, voice volume, nodding, and facial expressions). The ratings ranged from 0 (very poor) to 6 (very excellent). Two trainers independently rated the scores via real-time observation. Both raters received training (approximately 2 h) on scoring by watching videos of the interview scenes. The primary rater was blinded to the participants’ group assignments. The other rater participated throughout the experiment and was aware of the participants’ group assignments. The primary rater and the other rater attained a high degree of reliability (intra-class coefficient (ICC) = 0.91). The score used in this study was the score derived by the primary rater.

In addition, to evaluate the physiological responses, the participants provided salivary cortisol samples on days 1 and 7. Notably, there is an approximately 20-min time lag between the occurrence of an event and the detection of event-related changes in cortisol in saliva (Kirschbaum & Hellhammer, 1989). During the peer interaction, four salivary cortisol samples were obtained from each subject in intervals of 20 min as follows: S1 (baseline), S2 (immediately after the mock job interview), S3 (20 min after the mock job interview), and S4 (40 min after the mock job interview). The S2 measurements, which were collected immediately after the mock job interview, represent the circulating cortisol levels at the start of the paradigm, while the S3 measurements represent the cortisol levels at the end of the peer interaction, and the S4 measurements represent the cortisol levels after the peer interaction (Figure 4). The samples were obtained from each participant at the same time each day to control for potential diurnal variation. The participants were asked to not eat for 1 h prior to the interview. Immediately before and following the mock job interview paradigm, the participants were each assigned to an individual room and sat with a research assistant for the cortisol sampling. In addition to providing the cortisol samples, the participants relaxed for 30 min before and 40 min after the mock job interview. After providing the S4 salivary cortisol samples, the participants left the room.

Figure 4.

Timeline of events on days 1 and 7.

Salivary cortisol measurements

The saliva samples (0.5–2.0 mL) were collected directly into sterile plastic tubes (15 mL) by passive drool and immediately frozen on dry ice. The samples were stored at −80°C until use. After thawing at room temperature, the saliva samples were centrifuged at 1500g for 10 min at 4°C to remove the large precipitants. Tween 20 was used to block nonspecific reactions and accurately measure the volume of the viscous saliva samples. The determination of the saliva cortisol level was performed using a cortisol enzyme immunoassay kit (Salimetrics, State College, PA, USA). The samples (25 μL) were treated according to the manufacturer’s instructions. The measurements were performed in duplicate. The optical density of the samples and standards were measured at wavelengths of 450 nm using a microplate reader (Model 680, Bio-Rad, Richmond, CA, USA). The sample concentrations were calculated using MATLAB 7 according to the relevant standard curve (Tsuji et al., 2015).

Statistical analysis

We performed the statistical analyses using SPSS version 24.0 (IBM, Armonk, NY, USA). The descriptive statistics of the sample were calculated. The differences in age, IQ, and AQ-J and LSAS scores between the groups were analyzed using an independent samples t-test. The difference in the sex ratio was analyzed using the chi-square test. To investigate the improvement comparing the difference between the two groups (i.e., combined IGT and JUA vs IGT alone), a one-way analysis of variance (ANOVA) was performed to analyze the nonverbal communication score, self-confidence rating, and rate of change in salivary cortisol with one group factor using baseline data as covariate. We employed an alpha level of 0.05 for these analyses.

Results

In this study, no participant was paid to undergo the intervention. The duration of the IGT intervention and the JUA intervention was 30 min each. The approximate duration of the “combined IGT and JUA intervention” was about 60 min. In the JUA intervention, participants attended one mock interview conducted by the android robot every day. In the “combined IGT and JUA intervention” group, all participants completed an intervention every day from day 2 to day 6 (i.e., they attended a total of five mock job interviews conducted by the android robot). Interviews were conducted according to a prepared script. The mock job interview conducted by the android robot did not increase in its difficulty. In “the IGT only group,” 13 participants completed an intervention every day from day 2 to day 6. In both the “combined IGT and JUA intervention” and “IGT only” groups, no participant attended other mock interviews outside the study.

Demographic data

In total, 29 individuals with ASD participated in the study. In the “combined IGT and JUA intervention” group, all participants completed the trial procedures without technological challenges or distress that could lead to session termination. In response to the question, “Do you want to receive the intervention again?” all participants indicated “yes.” In contrast, in the IGT alone group, three participants dropped out of the study because they struggled to maintain their motivation. In response to the question, “Do you want to receive the intervention again?” seven participants (53.8 %) indicated “yes.” The IGT and JUA combination group included 13 participants (9 males) with a mean age of 21.9 ± 2.6 years, an average full IQ score of 86.5 ± 12.4, an AQ-J score of 30.8 ± 7.2, and an LSAS score of 43.9 ± 12.5. The IGT alone group included 13 participants (12 males) with a mean age of 21.9 ± 2.5 years, an average full IQ score of 86.9 ± 12.1, an AQ-J score of 30.3 ± 7.5, and an LSAS score of 43.8 ± 9.1. No significant differences were observed between the groups in the mean age (p = 1.000), sex ratio (p = 0.135), average IQ score (p = 0.949), total AQ-J score (p = 0.874), or total LSAS score (p = 0.585). Details regarding the participants who completed the trial are presented in Table 1.

Table 1.

Descriptive characteristics of the participants in the “IGT and JUA combination” and “IGT alone” groups.

Characteristics	IGT and JUA group (n = 13) (M, SD)	IGT group (n = 13) (M, SD)	Statistics
Characteristics	IGT and JUA group (n = 13) (M, SD)	IGT group (n = 13) (M, SD)	t or chi-square (χ²)	df	p
Age in years	21.9 (2.6)	21.9 (2.5)	t = 0.000	24	1.000
Sex (male:female)	9:4	12:1	χ² = 2.229	1	0.135
Full scale IQ	86.5 (12.4)	86.9 (12.1)	t = 0.064	24	0.949
AQ-J	30.8 (7.2)	30.3 (7.5)	t = 0.160	24	0.874
LSAS	43.9 (12.5)	43.8 (9.1)	t = 0.036	24	0.972

M: mean; SD: standard deviation; AQ-J: Autism Spectrum Quotient-Japanese version; LSAS: Liebowitz Social Anxiety Scale; IGT: interview guidance by teachers; JUA: job interview training using an android robot.

In the AQ-J, higher scores reflect a greater number of ASD-specific behaviors.

Parentheses indicate standard deviations.

Nonverbal communication

The one-way ANOVA revealed that there was a significantly greater improvement in posture (F = 18.56; df = 1, 23; p < 0.001; η² = 0.447), gaze (F = 6.89; df = 1, 23; p < 0.001; η² = 0.750), voice volume (F = 13.64; df = 1, 23; p = 0.001; η² = 0.372), nodding (F = 70.01; df = 1, 23; p < 0.001; η² = 0.753), and facial expression (F = 59.62; df = 1, 23; p < 0.001; η² = 0.722) in the combined IGT and JUA group than that in the IGT only group. Details regarding the nonverbal communication scores are described in Table 2.

Table 2.

Means and standard error of the mean of the “IGT and JUA combination” and “IGT alone” groups in nonverbal communication at post-intervention (day 7).

Outcome	IGT and JUA (M, SEM)	IGT alone (M, SEM)	Statistics
Outcome	IGT and JUA (M, SEM)	IGT alone (M, SEM)	F	P
Posture	5.46 (0.46)	4.00 (0.47)	18.56	<0.001**
Gaze	5.92 (0.33)	4.31 (0.38)	6.89	<0.001**
Voice volume	5.23 (0.43)	4.15 (0.47)	13.64	0.001**
Nodding	5.92 (0.24)	4.08 (0.42)	70.01	<0.001**
Facial expressions	5.77 (0.20)	4.31 (0.33)	59.62	<0.001**

M: mean; SEM: standard error of the mean; IGT: interview guidance by teachers; JUA: job interview training using an android robot.

p < 0.01.

Self-confidence

The one-way ANOVA demonstrated that the self-confidence of the combined IGT and JUA group improved significantly than the IGT only group (F = 5.67; df = 1, 23; p = 0.026; η² = 0.198). Details regarding the self-confidence scores are described in Table 3.

Table 3.

Means and standard error of the mean of the “IGT and JUA combination” and “IGT alone” groups in the confidence rating scale at post-intervention (day 7).

Outcome	IGT and JUA (M, SEM)	IGT alone (M, SEM)	Statistics
Outcome	IGT and JUA (M, SEM)	IGT alone (M, SEM)	F	p
Confidence rating scale	4.23 (0.34)	3.84 (0.45)	5.671	0.026*

M: mean; SEM: standard error of the mean; IGT: interview guidance by teachers; JUA: job interview training using an android robot.

p < 0.05.

Salivary cortisol

According to the one-way ANOVA, the improvement exhibited by the combined IGT and JUA group in the S4/S1 (F = 7.91; df = 1, 23; p = 0.010; η² = 0.256) was more than that of the IGT only group. In addition, the trend in improvement in the S3/S1 (F = 3.79; df = 1, 23; p = 0.064; η² = 0.141) of the combined IGT and JUA group was more than that in the IGT only group. However, no improvement was observed in the rate of change in the salivary cortisol level in the S2/S1 (F = 0.08; df = 1, 23; p = 0.786; η² = 0.003). Details regarding the rate of change in the salivary cortisol levels are described in Table 4.

Table 4.

Means and standard error of the mean of the “IGT and JUA combination” and “IGT alone” groups in the rate of change in salivary cortisol at post-intervention (day 7).

Salivary cortisol	IGT and JUA (M, SEM)	IGT alone (M, SEM)	Statistics
Salivary cortisol	IGT and JUA (M, SEM)	IGT alone (M, SEM)	F	p
S2/S1	1.08 (0.06)	1.12 (0.05)	0.08	0.786
S3/S1	0.99 (0.07)	1.16 (0.08)	3.79	0.064
S4/S1	0.90 (0.07)	1.21 (0.08)	7.91	0.010**

M: mean; SEM: standard error of the mean; IGT: interview guidance by teachers; JUA: job interview training using an android robot.

p < 0.01.

Discussion

In this study, we assessed the efficacy of JUA that used an android robot in various measures. The completion rate suggests that participants who received both IGT and JUA were able to continue to participate in the program without losing motivation and had better outcomes than did the participants in the control group (IGT alone). In this study, the motivation of all participants was originally high because they volunteered to participate in this study, and using an android robot may have sustained their motivation and achieved greater efficacy.

The JUA intervention contributed to improvement in various nonverbal communication skills. Previous intervention studies using virtual reality could not show clear improvement in nonverbal communication skills, which was mainly because the researchers could not provide subtle nonverbal cues using the avatar and prompt feedback for the participants’ response during the interview (Smith et al., 2014; Strickland et al., 2013). An android robot is realistic to a certain extent and can provide subtle nonverbal cues and prompt feedback to the participants using a remote-control system, which may improve nonverbal communication. In addition, our new method of tele-operating may promote an understanding of nonverbal communication and contribute to improvement.

Young adults with ASD can self-report psychiatric symptoms, including anxiety (Tsuji et al., 2015). These individuals may be more accurate reporters of their own mood dysregulation than their caregivers (Hurtig et al., 2009). Thus, the self-reporting questionnaire assessing self-confidence is highly reliable and showed a trend toward improvement in self-confidence for a job interview.

In this study, we also evaluated a physiological measure, that is, level of cortisol in saliva, to obtain a more objective view of self-confidence in individuals with ASD. Group differences were observed in the rate of change in the salivary cortisol level between the IGT and JUA combination group and the IGT alone group in S3/S1 and S4/S1, which was not the case in S2/S1. Because individuals with ASD display significantly greater physiological arousal in response to social interactions relative to the general population (Corbett et al., 2010; Corbett et al., 2012; Corbett et al., 2014; Schupp, Simon, & Corbett, 2013) and an approximately 20-min time lag exists between the occurrence of an event and the detection of an event-related change in cortisol in saliva (Kirschbaum & Hellhammer, 1989), the rate of change (S3/S1 and S4/S1) in saliva cortisol reflects that the additional JUA intervention reduced stress during and after the mock job interview.

Several limitations in our study should be acknowledged. First, the number of participants was relatively small. Future studies involving larger sample sizes are needed to provide more meaningful data regarding the potential use of our system. Second, we did not collect employment outcome data for the participants in this study. The next step is to establish evidence supporting the generalizability of the acquired job interview skills to daily life. The ultimate goal of the program is to enhance communication skills in daily life and to make participants more competitive when searching for employment or volunteering positions. To implement our system, an employment support facility is required. Knowing that the cost of providing care for individuals with ASD is very huge (Buescher, Cidav, Knapp, & Mandell, 2014) supporting these individuals to be in a competitive position is of great importance to the economy. In order to examine whether our program can attain this goal, future studies involving employment support facilities with long-term longitudinal designs are needed.

This is the first study to evaluate the effect of an intervention for nonverbal communication skills using an android robot in a variety manner for exercise in job interviews. Our intervention could improve nonverbal communication in individuals with ASD. Given the promising results of this study and to draw definitive conclusions regarding the efficacy of an android robot for job interview training, further studies involving individuals with ASD in larger, more diverse samples using a longitudinal design are warranted.

Supplemental Material

AUT827134_Lay_Abstract – Supplemental material for Job interview training targeting nonverbal communication using an android robot for individuals with autism spectrum disorder

Supplemental material, AUT827134_Lay_Abstract for Job interview training targeting nonverbal communication using an android robot for individuals with autism spectrum disorder by Hirokazu Kumazaki, Taro Muramatsu, Yuichiro Yoshikawa, Blythe A Corbett, Yoshio Matsumoto, Haruhiro Higashida, Teruko Yuhi, Hiroshi Ishiguro, Masaru Mimura and Mitsuru Kikuchi in Autism

Footnotes

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by the ERATO ISHIGURO Symbiotic Human-Robot Interaction Project, the Center of Innovation Program from the Japan Science and Technology Agency, JST, Japan, and Grants-in-Aid for Scientific Research from the Japan Society for the Promotion of Science (grant/award no.: “18H02746”).

References

Albert

(1971). Silent messages: Implicit communication of emotions and attitudes. Belmont, CA: Wadsworth Publishing.

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Arlington, VA: American Psychiatric Publishing.

Baron-Cohen

Leslie

A. M.

Frith

(1985). Does the autistic child have a “Theory of Mind”? Cognition, 21, 37–46.

Buescher

A. V.

Cidav

Knapp

Mandell

D. S.

(2014). Costs of autism spectrum disorders in the United Kingdom and the United States. Journal of the American Medical Association Pediatrics, 168, 721–728. doi:10.1001/jamapediatrics.2014.210

Chevallier

Kohls

Troiani

Brodkin

E. S.

Schultze

R. T.

(2012). The social motivation theory of autism. Trends in Cognitive Sciences, 16, 231–239.

Cook

Swapp

Pan

Bianchi-Berthouze

Blakemore

S. J.

(2013). Atypical interference effect of action observation in autism spectrum conditions. Psychological Medicine, 44, 731–740. doi:10.1017/s0033291713001335

Corbett

B. A.

Blain

S. D.

Ioannou

Balser

(2017). Changes in anxiety following a randomized control trial of a theatre-based intervention for youth with autism spectrum disorder. Autism, 21, 333–343. doi:10.1177/1362361316643623

Corbett

B. A.

Gunther

J. R.

Comins

Price

Ryan

Simon

. . . Rios

(2010). Brief report: Theatre as therapy for children with autism spectrum disorder. Journal of Autism and Developmental Disorders, 41, 505–511. doi:10.1007/s10803-010-1064-1

Corbett

B. A.

Schupp

C. W.

Lanni

K. E.

(2012). Comparing biobehavioral profiles across two social stress paradigms in children with and without autism spectrum disorders. Molecular Autism, 3(1), Article 13. doi:10.1186/2040-2392-3-13

10.

Corbett

B. A.

Swain

D. M.

Coke

Simon

Newsom

Houchins-Juarez

. . . Song

(2014). Improvement in social deficits in autism spectrum disorders using a theatre-based, peer-mediated intervention. Autism Research, 7(1), 4–16. doi:10.1002/aur.1341

11.

Golan

Baron-Cohen

(2006). Systemizing empathy: Teaching adults with Asperger syndrome or high-functioning autism to recognize complex emotions using interactive multimedia. Development and Psychopathology, 18, 591–617.

12.

Hall

N. C.

Jackson Gradt

S. E.

Goetz

Musu-Gillette

L. E.

(2011). Attributional retraining, self-esteem, and the job interview: Benefits and risks for college student employment. The Journal of Experimental Education, 79, 318–339. doi:10.1080/00220973.2010.503247

13.

Higgins

K. K.

Koch

L. C.

Boughfman

E. M.

Vierstra

(2008). School-to-work transition and Asperger syndrome. Work, 31, 291–298.

14.

Hurtig

Kuusikko

Mattila

M.-L.

Haapsamo

Ebeling

Jussila

. . . Moilanen

(2009). Multi-informant reports of psychiatric symptoms among high-functioning adolescents with Asperger syndrome or autism. Autism, 13, 583–598. doi:10.1177/1362361309335719

15.

Kandalaft

M. R.

Didehbani

Krawczyk

D. C.

Allen

T. T.

Chapman

S. B.

(2012). Virtual reality social cognition training for young adults with high-functioning autism. Journal of Autism and Developmental Disorders, 43, 34–44. doi:10.1007/s10803-012-1544-6

16.

Kirschbaum

Hellhammer

D. H.

(1989). Salivary cortisol in psychobiological research: An overview. Neuropsychobiology, 22, 150–169. doi:10.1159/000118611

17.

Kumazaki

Muramatsu

Yoshikawa

Matsumoto

Miyao

Ishiguro

. . . Kikuchi

(2017). Tele-operating an android robot to promote the understanding of facial expressions and to increase facial expressivity in individuals with autism spectrum disorder. The American Journal of Psychiatry, 174, 904–905. doi:10.1176/appi.ajp.2017.17030257

18.

Kumazaki

Warren

Corbett

B. A.

Yoshikawa

Matsumoto

Higashida

. . . Kikuchi

(2017). Android robot-mediated mock job interview sessions for young adults with autism spectrum disorder: A pilot study. Frontiers in Psychiatry, 8, Article 169. doi:10.3389/fpsyt.2017.00169

19.

Kumazaki

Warren

Muramatsu

Yoshikawa

Matsumoto

Miyao

. . . Kikuchi

(2017). A pilot study for robot appearance preferences among high-functioning individuals with autism spectrum disorder: Implications for therapeutic use. PLoS ONE, 12(10), e0186581. doi:10.1371/journal.pone.0186581

20.

Lee

K. M.

Jung

Kim

S. R.

(2006). Are physically embodied social agents better than disembodied social agents?: The effects of physical embodiment, tactile interaction, and people’s loneliness in human–robot interaction. International Journal of Human-Computer Studies, 64, 962–973. doi:10.1016/j.ijhcs.2006.05.002

21.

Leekam

S. R.

Libby

S. J.

Wing

Gould

Taylor

(2002). The Diagnostic Interview for Social and Communication Disorders: Algorithms for ICD-10 childhood autism and Wing and Gould autistic spectrum disorder. Journal of Child Psychology and Psychiatry, 43, 327–342.

22.

Liebowitz

M. R.

(1987). Social phobia. Modern Problems of Pharmacopsychiatry, 22, 141–173.

23.

Mennin

D. S.

Fresco

D. M.

Heimberg

R. G.

Schneier

F. R.

Davies

S. O.

Liebowitz

M. R.

(2002). Screening for social anxiety disorder in the clinical setting: Using the Liebowitz Social Anxiety Scale. Journal of Anxiety Disorders, 16, 661–673.

24.

Mikita

Hollocks

M. J.

Papadopoulos

A. S.

Aslani

Harrison

Leibenluft

. . . Stringaris

(2015). Irritability in boys with autism spectrum disorders: An investigation of physiological reactivity. Journal of Child Psychology and Psychiatry, 56, 1118–1126. doi:10.1111/jcpp.12382

25.

Nishio

Taura

Sumioka

Ishiguro

(2013). Teleoperated android robot as emotion regulation media. International Journal of Social Robotics, 5, 563–573. doi:10.1007/s12369-013-0201-3

26.

Pierno

A. C.

Mari

Lusher

Castiello

(2008). Robotic movement elicits visuomotor priming in children with autism. Neuropsychologia, 46, 448–454. doi:10.1016/j.neuropsychologia.2007.08.020

27.

Schupp

C. W.

Simon

Corbett

B. A.

(2013). Cortisol responsivity differences in children with autism spectrum disorders during free and cooperative play. Journal of Autism and Developmental Disorders, 43, 2405–2417. doi:10.1007/s10803-013-1790-2

28.

Shattuck

P. T.

Roux

A. M.

Hudson

L. E.

Taylor

J. L.

Maenner

M. J.

Trani

J. F.

(2012). Services for adults with an autism spectrum disorder. Canadian Journal of Psychiatry, 57, 284–291.

29.

Sheehan

D. V.

Lecrubier

Sheehan

K. H.

Amorim

Janavs

Weiller

. . . Dunbar

G. C.

(1998). The Mini-International Neuropsychiatric Interview (M.I.N.I.): The development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. The Journal of Clinical Psychiatry, 59(Suppl. 20), 22–33.

30.

Simon

D. M.

Corbett

B. A.

(2013). Examining associations between anxiety and cortisol in high functioning male children with autism. Journal of Neurodevelopmental Disorders, 5(1), 32. doi:10.1186/1866-1955-5-32

31.

Smith

M. J.

Fleming

M. F.

Wright

M. A.

Losh

Humm

L. B.

Olsen

Bell

M. D.

(2015). Brief report: Vocational outcomes for young adults with autism spectrum disorders at six months after virtual reality job interview training. Journal of Autism and Developmental Disorders, 45, 3364–3369. doi:10.1007/s10803-015-2470-1

32.

Smith

M. J.

Ginger

E. J.

Wright

M. A.

Taylor

J. L.

Humm

L. B.

. . . Fleming

M. F.

(2014). Virtual reality job interview training in adults with autism spectrum disorder. Journal of Autism and Developmental Disorders, 44, 2450–2463. doi:10.1007/s10803-014-2113-y

33.

Strickland

D. C.

Coles

C. D.

Southern

L. B.

(2013). JobTIPS: A transition to employment program for individuals with autism spectrum disorders. Journal of Autism and Developmental Disorders, 43, 2472–2483. doi:10.1007/s10803-013-1800-4

34.

Tay

Ang

Van Dyne

(2006). Personality, biographical characteristics, and job interview success: A longitudinal study of the mediating effects of interviewing self-efficacy and the moderating effects of internal locus of causality. Journal of Applied Psychology, 91, 446–454. doi:10.1037/0021-9010.91.2.446

35.

Taylor

J. L.

Seltzer

M. M.

(2011). Employment and post-secondary educational activities for young adults with autism spectrum disorders during the transition to adulthood. Journal of Autism and Developmental Disorders, 41, 566–574. doi:10.1007/s10803-010-1070-3

36.

Tsuji

Yuhi

Furuhara

Ohta

Shimizu

Higashida

(2015). Salivary oxytocin concentrations in seven boys with autism spectrum disorder received massage from their mothers: A pilot study. Frontiers in Psychiatry, 6, Article 58. doi:10.3389/fpsyt.2015.00058

37.

van Eck

Berkhof

Nicolson

Sulon

. (1996). The effects of perceived stress, traits, mood states, and stressful daily events on salivary cortisol. Psychosomatic Medicine, 58, 447–458.

38.

Wainer

Feil-seifer

D. J.

Shell

D. A.

Mataric

(2006). The role of physical embodiment in human-robot interaction. Roman 2006, Hatfield, UK. doi:10.1109/roman.2006.314404

39.

Wainer

Feil-Seifer

D. J.

Shell

D. A.

Mataric

M. J.

(2007). Embodiment and human-robot interaction: A task-based perspective. RO-MAN 2007, Jeju, South Korea. doi:10.1109/roman.2007.4415207

40.

Wakabayashi

Baron-Cohen

Uchiyama

Yoshida

Tojo

Kuroda

Wheelwright

(2007). The autism-spectrum quotient (AQ) children’s version in Japan: A cross-cultural comparison. Journal of Autism and Developmental Disorders, 37, 491–500. doi:10.1007/s10803-006-0181-3

41.

Wakabayashi

Tojo

Baron-Cohen

Wheelwright

(2004). The Autism-Spectrum Quotient (AQ) Japanese version: Evidence from high-functioning clinical group and normal adults. Shinrigaku kenkyu, 75, 78–84.

42.

Warren

Zheng

Das

Young

E. M.

Swanson

Weitlauf

Sarkar

(2015). Brief report: Development of a robotic intervention platform for young children with ASD. Journal of Autism and Developmental Disorders, 45, 3870–3876. doi:10.1007/s10803-014-2334-0

43.

Wechsler

(2008). WAIS-IV administration and scoring manual. San Antonio, TX: Psychological Corporation.

44.

Wheelwright

Auyeung

Allison

Baron-Cohen

(2010). Defining the broader, medium and narrow autism phenotype among parents using the Autism Spectrum Quotient (AQ). Molecular Autism, 1, Article 10.

45.

Wing

Leekam

S. R.

Libby

S. J.

Gould

Larcombe

(2002). The diagnostic interview for social and communication disorders: Background, inter-rater reliability and clinical use. Journal of Child Psychology and Psychiatry, 43, 307–325.

46.

Yoshikawa

Matsumoto

Sumitani

Ishiguro

(2011). Development of an android robot for psychological support in medical and welfare fields. 2011 IEEE International Conference on Robotics and Biomimetics, Karon Beach, Phuket, Thailand. doi:10.1109/robio.2011.6181654

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.50 MB