Abstract
BACKGROUND:
Job interviewing serves a key function in the process of gaining employment. To date, no research has evaluated whether a literacy-based behavioral intervention (LBBI) might be used to improve the essential vocational skill of job interviewing.
OBJECTIVES:
The purpose of this study was to explore the effectiveness of a LBBI for the acquisition of job interviewing skills.
METHODS:
Using a combined experimental design (non-concurrent multiple probe across participants with an embedded A1B1A2B2 design) three young adults with intellectual disability who received an in-person and remote LBBI were evaluated on their correct and independent performance on a task analysis created for job interviewing. Generalization and maintenance were also assessed.
RESULTS:
The results showed that all three students mastered job interviewing skills with 100% accuracy and maintained the acquired skills after the intervention was removed.
CONCLUSIONS:
The study showed that LBBI is an effective tool to teach and practice job interviewing skills, and can be implemented face-to-face and virtually. Educators, vocational rehabilitation professionals, and families can use this tool in a wide range of settings.
Keywords
Introduction
Employment has been identified as an essential domain of quality of life for individuals with developmental disabilities (Wehman et al., 2018). The effects of unemployment are much greater than lack of income and can have a significant negative effect on happiness and life-satisfaction (Stam et al., 2016). Employment settings allow for the formation of meaningful friendships and community participation. Approximately one in four young adults with intellectual disability (ID) (23.8%) and one in three young adults with autism spectrum disorder (ASD) (32.3%) or multiple disabilities (37.5%) have never held paid employment within 8 years of exiting high school (Gilson et al., 2017). Individuals with intellectual and developmental disabilities can be successfully employed in competitive, integrated positions and substantially increase earnings and their quality of life (Butterworth et al., 2017; Wehman et al., 2014; Migliore et al., 2012), however, the current rate of individually integrated paid employment for adults with intellectual and developmental disabilities is less than 10% (Hiersteiner et al., 2016). Early interventions such as employment-related programs provided in high school result in higher rates of employment, higher wage earnings, more hours worked per week, and higher job satisfaction for students with disabilities (Mazzotti et al., 2016; Carter et al., 2012).
To obtain and maintain competitive employment, it is necessary to demonstrate the ability to acquire and maintain job skills with a certain level of proficiency and independence. Successfully navigating a job interview is an important first step in demonstrating the ability to gain employment (Lindsay & DePape, 2015). Unfortunately, a job interview can become a major obstacle to gaining employment for individuals with intellectual and developmental disabilities because they often have challenges with self-advocacy and social skills (Brooks et al., 2015; Shogren et al., 2017). Social skills such as being assertive but not aggressive, identifying overt and subtle cues in specific environments or situations, taking turns in conversation, using appropriate facial expressions and eye contact, staying on topic during conversations, sharing only relevant information, and being a good listener are vital for a successful job interview (Walker et al., 2016).
First impressions can also have a profound impact on an individual’s interview performance; these impressions are often based on one’s ability to appear confident yet humble during conversations. Conversational and behavioral fluencies are often difficult for individuals with ID to attain in natural environments (Walker et al., 2016; Hawkins, 2004). Job interviews also can be anxiety-provoking social situations that many individuals with ID will avoid (Burke et al., 2018; Brazeau et al., 2017). For those who do interview, displaying anxiety during an interview often results in reduced confidence in the candidate’s self-efficacy, and has a negative impact on subsequent interviews (Tross & Maurer, 2008).
For individuals with intellectual and developmental disabilities who pursue employment, a wide range of complex job interview skills have been ta-ught using different interventions. Interventions to teach job interview skills include video modeling (Allen et al., 2010; Goh & Bambara, 2013; Mechling & Ortega-Hurndon, 2007), web-based training (Strickland et al., 2013), virtual reality simulations (Burke et al. 2018; Walker et al., 2016), and role play using an android robot resembling a real interviewer (Kumazaki et al., 2019). Even though these approaches were effective, the feasibility and practicality of using them in typical community work settings requires further research.
One strategy that has been used to teach acquisition skills to adolescents and adults with ID is literacy-based behavioral interventions (LBBIs; Pistorio et al., 2018). LBBI is an umbrella term for interventions that use social narratives (Collet- Klingenberg & Franzone, 2008), comic strips (Daly & Ranalli, 2003), social scripts (Krantz & McClannahan, 1998), picture activity schedule books (Spriggs et al., 2007), and Social Stories™ (Gray, 1998). LBBI storybooks have been used to teach safety skills (Kearney et al., 2018), independent living skills (Brady, Hall et al., 2016), adaptive daily living skills and routines (Brady, Honsberger et al., 2016), and self-regulation skills (Hall et al., 2019). LBBIs are a behavior-based intervention that can be delivered as a tabletop storybook (Kearney et al., 2018). LBBIs can be delivered by teachers, counselors, job coaches, parents, and even peers. LBBIs are comprised of a task analysis of the skill, pairing each step in the task analysis to a single page with a matching picture with a short, simple sentence. The mediator of the LBBI uses a pause-point-practice-praise technique. The mediator asks the learner to pause after reading the sentence on the page, point to the picture, practice the behavior in the picture, and then provides praise to the learner. Traditionally, if the learner makes an error, the mediator refers the learner back to the storybook to re-read the step.
To date, there are only two studies that apply LBBIs to prepare adults with ID for employment (Bucholz et al., 2008; Honsberger et al., 2019). Buholz and colleagues used LBBIs in two different experiments. In the first experiment they used LBBIs to increase productivity in adult employees with ID in assembly and packing tasks. In the second experiment they used an LBBI to teach an employee with ID to transition from break time back to work time. Honsberger and colleagues (2019) extended the employability focus by using an LBBI to teach new skill acquisition. In this study, a peer delivered the LBBI to teach high school students with ASD various skills needed to run a food truck. LBBI appears to have other implications for employment training. This includes teaching job interview skills in a variety of settings. To date, however, no research exists on the use of LBBIs to teach job interviewing skills for individuals with ID. Therefore, in this study we posed the following research questions: Will a LBBI increase the acquisition of job interviewing skills by adults with ID? If participants acquire the job interviewing skills, will they maintain these skills after the LBBI is removed?
Method
Participants
The participants in this study were three college students (aged 20 to 22) enrolled in an inclusive postsecondary comprehensive transition program at a university in the southeastern United States. All students were diagnosed with ID based on the Wechsler Adult Intelligence Scale – Fourth Edition (WAIS-IV) (Weschler, 2008). All students also were assessed with the Job Observation Behavior Scale (JOBS; Rosenberg & Brady, 2002) to establish their (a) Qua-lity of Performance (QoP) and (b) Type of Support (ToS) measures on the three JOBS subscales: Work Required Daily Living Activity (DLA), Work Req-uired Behavior (BEH), and Work Required Job Duties (JD). One student had a secondary diagnosis of ASD. All students provided written and verbal consent to participate in the study. The investigators received formal approval from the university’s institutional review board prior to initiating the study.
David
David was a 20-year-old Hispanic male diagnosed with ID and ASD. He had limited job shadowing and volunteer experience at a local restaurant through his previous high school program, but he had never been interviewed before, nor had he previously been employed. David had a WAIS full-scale IQ score of 65. David’s JOBS scores showed a DLA subscale score of 35 of 65 points (QoP) and 22 of 65 points (ToS), a BEH score of 19 of 40 points (QoP) and 15 of 40 points (ToS), and a JD score of 20 of 45 poi-nts (QoP) and 17 of 45 points (ToS). The JOBS scores indicate that on Quality of Performance ratings, David scored near the mean for the high school students’ comparison group, but substantially below the mean for the entry-level employee comparison group on the DLA subscale, and below or substantially below the mean for both high school students and entry-level employees on both other subscales. On Type of Support ratings, David scored substantially below the mean for both high school students and entry-level employees on all JOBS subscales, indicating a need for intermittent to continuous support from a job coach or supervisor.
Judith
Judith was a 22-year-old Caucasian female diagnosed with ID and Down syndrome. Judith had volunteer experience with a community center, but she had never been interviewed before, nor had she previously been employed. Her WAIS full-scale IQ score was 53. Judith’s JOBS scores were 37 (QoP) and 37 (ToS) for the DLA, 22 (QoP) and 20 (ToS) for the BEH, and 25 (QoP) and 30 (ToS) for the JD. These scores indicate that on Quality of Performance ratings, Judith scored slightly above the mean for high school students on the DLA subscale, but matched the mean for high school students on the two other subscales. Her QoP scores for the BEH and JD subscales were at or slightly below the mean for both high school students and significantly below the mean for entry-level employees. On Type of Support ratings, Judith scored slightly above the mean on JOBS DLA and JD subscales for high school students, but below the mean for this comparison group on the BEH subscale. Her ToS ratings on all three subscales was substantially below the mean when compared to the entry-level employee comparison group, indicating a need for frequent to continuous support from a job coach or supervisor.
Sharon
Sharon was a 22-year-old Caucasian female diagnosed with ID. Sharon had previously had a paid job, and had participated in one job interview. She expressed that she felt a great deal of anxiety during that interview, and was interested in learning to improve her ability for future interviews. Sharon’s WAIS full-scale IQ score was 68. Her JOBS scores were 33 (QoP) and 30 (ToS) for the DLA subscale, 18 (QoP) and 20 (ToS) for the BEH, and 23 (QoP) and 21 (ToS) for the JD. These scores indicate that on Quality of Performance ratings, Sharon scored near the mean for the high school students’ comparison group, but substantially below the mean for the entry-level employee comparison group on the DLA subscale. For the other JOBS subscales, Sharon’s QoP scores were slightly to substantially below the means for both the high school students and entry-level employee comparison groups. On Type of Support ratings, Sharon scored substantially below the mean for both the high school students and entry-level employee comparison groups on all JOBS subscales, indicating a need for frequent to continuous support from a job coach or supervisor.
Setting
Initially, all intervention and observation sessions were conducted in-person in the College of Education building of a public university in the southeastern United States. Baseline and intervention sessions were conducted in a meeting room consisting of a small table with chairs on opposite sides. There was a one-way mirror on one wall, adjacent to an observation room. During each baseline interview, the student and the investigator designated as “employer” were present in the meeting room. The student sat across the table facing the employer, and a data collector was positioned in the observation room. During the intervention sessions the student sat next to a different investigator designated as “interventionist.”
Mid-way through the study, the 2020 COVID-19 global pandemic occurred, resulting in colleges and universities in the state transitioning to fully remote, distance learning. Because in-person contact among investigators and students was no longer safe nor av-ailable, the research team implemented a series of methodological modifications that would enable a continuation of the study. Specifically, the investi-gators modified the study to continue virtually, with future baseline and intervention sessions delivered remotely via the smartphone videoconferencing app-lication WhatsApp. During this modification, the setting for the study was individualized for each participant. Investigators implemented the study protocol from their individual home offices, and students participated from a workspace in their individual homes. During the remote portion of the study, each baseline, follow-up, and generalization session was conducted as a group video call with three people: the investigator designated as “employer”, the student, and the data collector. The intervention sessions also were conducted as a group video call with three people: the investigator designated as “interventionist”, the student, and the data collector. Thus, the setting for every session included three individuals, and was delivered remotely to the individual students at their own homes.
To minimize setting distractions (at the students’ homes and investigators’ offices), a remote interaction protocol was developed. The investigator serving as “host” of the software application set the parameters of the software so that a student could only see and hear the investigator designated as employer or interventionist when the data collector’s camera and microphone were blind to the student. This reduced the probability of the data collector serving as an unintended reactive influence on the student’s performance.
Behavioral measures, data collection, and interobserver agreement
The dependent measure for this study was the percentage of steps performed correctly and independently in a task analysis of job interviewing skills (see Table 1). Interviewing was selected because it is a complex, anxiety-producing skill and many people with ID lack key interviewing skills such as communication, self-advocacy, and articulating details about themselves (Winsor et al., 2017). Target behaviors were divided into three categories: greeting, answering four interview questions, and closing. The interview questions used in this study were selected utilizing a curriculum to teach workforce readiness skills to youth with disabilities, “Skills to Pay the Bills: Mastering Soft Skills for Workplace Success” (U.S. Department of Labor, Office of Disability Employment Policy, 2012) and the Career One Stop website (U.S. Department of Labor Employment and Training Administration, 2019).
Job interviewing task analysis: Face-to-face & virtual
Job interviewing task analysis: Face-to-face & virtual
The task analysis for the in-person, face-to-face portion of the study was comprised of 17 steps. The task analysis for virtual job interviewing, was modified to remove 5 steps that required face-to-face interaction. Specifically, steps including ‘follow the interviewer’, ‘sit where directed’, and ‘shake interviewer’s hand’ were removed. The remaining 12 steps were kept unchanged (see Table 1). Because data were analyzed as percentage of steps, this allowed us to evaluate students’ performance with this modification in the task analysis. Each step in the task analysis had agreed upon acceptable responses from the students. For example, a silent smile for the greeting was not acceptable, the students were expected to verbally greet the interviewer in a professional manner, such as “hello” or “good afternoon” (rather than with an unprofessional “what’s crackin”).
To collect data, each student was individually ob-served while performing the job interviewing skills in a simulated interview. Steps had to be performed in a strict sequential order. For example, students were required to greet the interviewer at the beginning of the interview. However, the order of the answers to a particular question did not have to follow the same order provided in the task analysis. For example, in response to the question “Tell me about yourself”, the student could give the three responses in any order. Each step in the task analysis was scored by a data collector as (a) correct or (b) incorrect. A step was recorded as correct if the student performed it as identified in the task analysis within 30 seconds. An incorrect response was recorded if student did not correctly perform a step or omitted a step in the task analysis. Only steps that were both correct and provided without assistance were included in the graphed results and used for instructional decisions.
During the in-person, face-to-face portion of the study, one data collector was stationed in the observation room. Data were collected live, and data col-lectors were trained to use the data sheets prior to the study. During the virtual sessions, the data collector was part of the video call, however, the observer’s camera and microphone were blind to the student. The data collectors were experienced researchers including a faculty member and two graduate students enrolled in doctoral programs in special education and counselor education.
To establish interobserver agreement (IOA), two data collectors recorded data independently and at the same time during 81% of all sessions. Agreement was determined by counting the steps of the task analysis scored the same by both data collectors, then dividing that number by the total number of steps observed, then multiplying by 100. These agreement sessions were conducted during 71% of baseline, 83% of intervention, and 50% of follow-up sessions for David (76% overall); 63% of baseline, 71% of intervention, and 50% of follow-up sessions for Judith (67% overall); and 57% of baseline, 100% of intervention, and 50% of follow up sessions for Sharon (79% overall). The results of the IOA assessment showed substantial agreement between the data collectors. Agreement across all students and sessions (combined) was 97%. For individual students, agreement was 98% for David, 96% for Judith, and 98% for Sharon.
LBBI storybook
Following the development of the task analysis for the job interview, an LBBI storybook was created using a Microsoft PowerPoint® presentation. The storybook contained a total of 23 pages including a front cover page, 17 matching pages with each step of the task analysis, four pages that illustrate an employer’s four interview questions, and a closing page. Each page contained one simple sentence written in first person, and one matching photograph. For example, on the second page, the step “I shake interviewer’s hand” was written, and a matching photograph of the employer extending her hand out was shown (see Fig. 1). The photographs were taken by the investigators from the student’s point of view, prior to baseline, at the same location the interviews were conducted. The words on each page were printed in Franklin Gothic Book size 44 font, and the photographs were approximately 6×8 inches. The length of the sentences ranged from four to nine words on each page. The LBBI paper book was created by printing the pages and assembling them into a one-inch, three-ring binder. See Fig. 1 for a sample page from the LBBI storybook.
Sample page from LBBI storybook. Note: LBBI = Literacy-based behavioral intervention.
After the study transitioned from face-to-face to virtual sessions, the LBBI storybook was modified by removing five pages that required face-to face interactions; these pages represented the same five steps deleted from the in-person task analysis. The same Microsoft PowerPoint® presentation used for the LBBI storybook was used for the e-book. The e-book contained a total of 18 pages including a front cover page, 12 matching pages with each step of the modified task analysis, four pages that illustrate an employer’s four interview questions, and a closing page.
Experimental design and procedures
Due to the presence of the 2020 COVID-19 pandemic during this study, we were forced to examine experimental design features that would allow us to address our research questions, while respecting the principles and traditions of behavior analytic experimentation. In addressing the inherent flexibility of single subject research, researchers from Sidman (1960) to Baer et al., (1987) to Kennedy (2005) have addressed the sine qua non of experimental design. To understand experimental effects, a study must (a) have convincing demonstrations of experimental control, (b) establish a functional relation between independent and dependent variables, (c) demonstrate an adequate number of replications, and (d) show predictability in the data patterns. Ken-nedy summarized numerous studies in which design alterations have been made to seize opportunities to investigate phenomena brought on by serendipitous and unplanned opportunities, including stepping away from “rote and inflexible” “cookbook” design features and to construct “a good design ... that answers the question convincingly” (Baer et al., 1987; p. 319).
For this study, we implemented a combined design approach (Kennedy, 2005) to capture the opportunity to study the efficacy of the LBBI under rapidly changing conditions. A multiple probe design across participants was used to determine the effects of the LBBI on the acquisition of job interviewing skills. The design used multiple probes during baseline to prevent repeated exposure to inaccurate practice of the skills before the intervention was introduced. When the COVID-19 forced the closure of on-campus interventions, we delayed the experimental procedures for the third participant, thus implementing a non-concurrent multiple baseline design for this participant. In addition, for David and Judith, we were able to return to baseline following their initial, in-person intervention. Although neither were able to demonstrate mastery after exposure to the in-person LBBI due to the abbreviated time before the campus closure, both showed clear effects when examining the level and trend of their data. For both, we followed the in-person intervention with a return to baseline, but this time the baseline and second intervention were modified to the remote application of the LBBI, thus consisting of an A1-B1-A2-B2 design for these participants.
Finally, we added two design features to assess maintenance and generalization. Follow-up observations were held after the intervention was removed with each student to assess whether the experimental effects and learning would maintain when the students were no longer coached to use the LBBI. Additionally, we conducted a generalization probe for each student during intervention to determine if the students would use their new interview skills with a different investigator who played the role of a novel employer.
Baseline
During all baseline sessions an investigator designated as “employer” conducted the job interviews with the student face-to-face and virtually. Prior to the initial face-to-face baseline sessions, David and Judith were told that they will practice interviewing for a job and were provided with information about the interview time and location, and the employer’s name. The face-to-face baseline sessions began at the designated waiting area where the employer and the student met. The employer said “Hello, please follow me” and they walked to the interview room. The employer conducted the job interview following the sequence outlined in the task analysis. The student’s performance was observed by both employer and the data collector from the observation room. The decision to move from baseline to intervention occurred after at least three data points demonstrated a low rate of accurate skill performance. Once the face-to-face interview intervention stopped due to COVID-19, the transition time for David and Judith to the virtual LBBI was three weeks. David and Judith returned to baseline using the remote modification prior to introducing the modified remote intervention. For Sharon, baseline observations were only conducted using the remote baseline modification.
Intervention
For David and Judith, the intervention sessions were delivered face-to-face and virtually. Aside from the very first teaching session, all data were collected at the beginning of each session, prior to the delivery of the LBBI. The participants did not receive any additional interview training prior to data collection, nor were any additional data collected after the interview training. This guaranteed that there was at least 24 hours between intervention and data collection, eliminating the possibility that performance improvement might be due to immediate practice effects. During face-to-face intervention sessions, the interventionist sat next to the students, and placed the storybook on a table in front of them so they could both see each page. The interventionist read the sentence aloud on each page and pointed to the picture. Next, the student read each page aloud and then performed the step independently without looking at the storybook. If the student made an error on a step, the correction was prompted by the interventionist and the student was asked to repeat the step. While performing a behavior such as “shaking hand”, a behavioral rehearsal was provided by the interventionist first by pointing to the picture and then illustrating the behavior with the student. Next, the student was asked to perform the behavior independently.
For the virtual intervention sessions, the interventionist, the student, and the data collector connected through WhatsApp using their personal smartphones. The student could see and hear only the interventionist. During the virtual intervention sessions, the interventionist showed each page of the LBBI e-book by sharing her screen and reading the sentences aloud. Next, the student was asked to repeat the sentence first by reading, and then performed the step independently. If the student made an error on a step, a correction was prompted by the interventionist and the student was asked to repeat the step. All three students participated in the remote LBBI.
Generalization
The generalization sessions followed the same procedure as the baseline sessions, but the interviews were conducted by a different investigator to determine whether the students would use their new interview skills with a different employer. During these sessions, there was no teaching or coaching provided; the new investigator greeted the student, asked four interview questions in the task analysis, and made the closing. The investigator collected data based on how many steps were performed independently and accurately. There was one generalization probe during intervention for each student.
Follow-up
After students achieved mastery (defined as 100% of steps completed correctly and independently for five consecutive sessions), the intervention was stopped and the LBBI was removed to see if the gains would maintain without the intervention. The procedure for the follow-up sessions were same as baseline sessions; the employer conducted the job interview following the sequence outlined in the task analysis. Follow-up sessions were conducted for David and Judith 20 and 21 days after the intervention was removed; a follow-up observation for Sharon was held 15 days after the intervention was removed.
Treatment fidelity
To evaluate the degree to which the intervention was implemented as intended, data were collected on treatment fidelity for each student. Prior to the study, a training session was conducted to teach the data collector to collect fidelity data. The training required the data collector to watch the interventionist perform the actions on each page of the LBBI (without any student present). The data collector then practiced coding the treatment fidelity data sheets, first with the investigator and then independently. The data collector and investigator then discussed the scores until no disagreements were noted.
To collect the fidelity data, the data collector used a four-item checklist during baseline and a seven-item checklist during intervention. The baseline fidelity checklist was used to ensure the LBBI was not implemented but that the cue to begin the interview was given, and the interview was terminated as planned. The intervention fidelity checklist ensured that the interventionist used the LBBI accurately and foll-owed the pause, point, practice, and praise procedure. During the face-to-face sessions, the data collector watched the live sessions from the observation room. For the virtual sessions, the data collector was part of the video call with the student and interventionist. Throughout the study, fidelity data were collected during 83% of the sessions. A review of the fidelity results indicate that the protocol was followed with 100% fidelity during each observation.
Data analysis
First, data were analyzed by the investigators using traditional visual analysis procedures. This included calculating measures central tendency and deter-mining ranges for each participant during baseline, intervention, and follow-up conditions. These measures helped us establish direction, trend, and level of data within each condition, and were used to justify condition changes (Kratochwill et al., 2013). Next, the investigators conducted a post-hoc analysis of graphed data to establish the size of the effect of data changes. A Tau-U coefficient was calculated to determine true effect size. Tau-U provides an aggregate effect size based on the weighted average of each students’ baseline and intervention changes. To determine the Tau-U coefficient, the authors used the Tau-U web-based calculator (Vannest et al., 2016). This calculation includes only baseline to intervention effect of Sharon and second baseline to second intervention effect for David and Judith.
Results
The effects of the in-person face-to-face LBBI, as well as the remote LBBI are shown in Fig. 2.
Percentage of steps performed correctly and independently.
David originally scored relatively high during the initial baseline (56% accuracy). This score was influenced by David asking the interviewer questions about the position. After the second baseline session he no longer asked the questions and stabilized at 34% correct and independent. His in-person baseline scores ranged from 34% – 56%, averaging 41% correct and independent. Once the in-person intervention was introduced, David displayed little growth until his fifth intervention session, when he had a noticeable increase from 43% to 65% accuracy. Following this, David increased dramatically to 100% correct and independent during the seventh intervention session. Before an eighth intervention session could be delivered, the COVID-19 pandemic resulted in the state and campus closure. Three weeks passed before investigators were able to re-start the study. Because there was no opportunity for in-person, on-campus contact, we shifted David’s experimental protocol to the modified LBBI delivered remotely. During David’s remote baseline, the accuracy of his interviewing regressed, but was stable at 67% correct and independent. When the intervention was re-introduced virtually, David immediately increased to 100% accuracy, and maintained this performance for five consecutive sessions, plus one generalization probe. A follow-up probe occurred 20 days after the intervention was removed and he maintained his 100% correct and independent responding.
Judith
During Judith’s initial in-person baseline, her data were stable, with a range of 21% – 26% correct and independent, and an average of 25% accuracy across the five baseline sessions. During her first three in-person intervention sessions, Judith had a slow, increasing trend. By the fourth intervention Judith increased from 39% to 60% accuracy. By her sixth intervention session she had reached 95% accuracy, at which point data collection stopped due to COVID-19. Judith’s remote baseline condition was introduced three weeks after her last face-to-face intervention. Judith’s remote baseline condition was stable at 67% for three sessions. When her remote intervention was introduced, Judith showed a modest increase in accuracy for first three sessions, followed by a substantial increase to 100% accuracy. She maintained 100% accuracy for five consecutive sessions, plus one generalization probe. A follow-up probe was conducted 21 days later, and Judith maintained 100% correct and independent responding.
Sharon
Sharon’s first baseline session did not occur until virtual baselines were under way for David and Judith. Sharon’s first virtual baseline resulted in 33% accuracy, followed by an increase in accuracy during the second and third sessions. Baseline data stabilized at 58% correct and independent for four consecutive sessions before introducing the virtual LBBI. During Sharon’s first intervention sessions, she showed an increasing trend, then decreased from 83% to 58% accuracy during her fourth and fifth intervention session. During her sixth intervention session, Sharon’s accuracy increased to 100% and remained there for five consecutive sessions, plus one generalization session. A follow-up probe 15 days after the intervention was removed, showed that Sharon responded with 100% accuracy.
Post-hoc analysis and effect size
The Tau-U post-hoc analysis showed an overall effect size of .878, suggesting a robust effect size across students (Parker et al., 2011). It should be noted that this coefficient only reports the weighted average of David and Judith’s second baseline to their second intervention, and Sharon’s baseline to intervention.
Discussion
The results of this study indicated that an LBBI is an effective tool to teach job interviewing skills to adults with ID. All three students received the LBBI and mastered job interviewing skills with 100% accuracy. In addition, all students maintained these skills with 100% accuracy during follow-up sessions 15 to 21 days after removing the intervention. The acquisition pattern of learning gains in this study is similar to previous LBBI studies (Brady, Hall et al., 2016; Brady, Honsberger et al., 2016; Bucholz et al. 2008; Hall et al., 2019; Kearney et al., 2018; Pistorio et al., 2018).
To support skill acquisition, the investigators modified the language used in the LBBI for each par-ticipant. Initially, some students had difficulty with longer phrases or those that were not a part of their everyday repertoire; therefore, phrases were edited to reflect more natural, conversational language. For example, the response “I know how to manage my time” was changed to “I am always on time.” Using a simpler sentence structure helped students under-stand the terminology and provide expected respo-nses, thus supporting acquisition of interview skills.
Additionally, investigators found that some participants tried to memorize certain responses or mixed up responses for some of the interview questions but did not actually understand the meaning of the questions. To improve comprehension, some questions and res-ponses were revised, and required explicit teaching. For example, to understand the question “What are your professional skills?”, the investigators taught the students the definition of “professional” and provided examples for the phrase “professional skills.” Similarly, to explain the question, “Why should we hire you?”, we defined the word “hire” and provided examples of reasons why a business would want to hire a new employee. The LBBI thus became the mec-hanism that supported the direct teaching of these questions and responses, helped students differentiate between the questions being asked, and increased their ability to respond accurately.
Preparing young adults with ID to transition into employment is critical given the disparities in post-school competitive employment faced by youth with ID (Gold et al., 2013). When adequate support and training are provided, there is a significant long-term potential for securing competitive employment (Siperstein et al., 2014; Wehman et al., 2014). However, there is a need for effective interventions for employment transition programs that can be easily integrated into community work sites. There are a limited number of evidence-based practices to support transition services (Bishop-Fitzpatrick et al., 2013; Roux et al., 2015; Taylor et al., 2012), and few of these programs can be easily administered with fidelity (Schultz et al., 2011; Seaman & Cannella-Malone, 2016). This study provides evidence that an LBBI has several advantages as an intervention to teach employment skills. First, LBBIs are cost-effective interventions that do not require special software or equipment to create or deliver. LBBI storybooks can be created using Microsoft PowerPoint® presentations and can be used in printed format as a tabletop activity, or as an e-book. LBBI e-books can be delivered through phone, tablet, or computer (Pistorio et al., 2018). Second, the LBBI offers high accessibility in multiple environments such as classroom, home, or workplace. This is a unique benefit for users, considering some job interview interventions (e.g., video modelling or virtual reality simulations) are limited to settings that are specially equipped. Finally, an LBBI is an efficient tool that can be implemented by various individuals who play vital employment preparation roles for individuals with ID including families, special educators, transition specialists, and vocational rehabilitation professionals.
In an increasingly digital world, employers are hiring and conducting daily business virtually. Digital tools are key to engaging in numerous community living activities, finding and obtaining employment, developing social networks, and scheduling work or home tasks (Anderson & Perrin, 2017). Although it was not our original intent, when we incorporated a virtual instructional format in this study, we discovered a serendipitous use of the LBBI for participating students. Using a smartphone app to deliver the LBBI, all three students adapted to the change with no additional training.
Interviewing for employment is an anxiety provoking situation. In this study, the LBBI was effective for teaching students how to effectively interview with an employer; it also provided a safe environment for them to practice the new skill. Practicing the new skill after reaching a mastery criterion is an essential component of this intervention because students are given the opportunity to observe themselves mastering a difficult skill several times. This pattern appears to be theoretically consistent with Self Efficacy Theory (Bandura, 1997) which postulates that people with high levels of confidence in their ability to perform well are more likely to persist in their efforts to increase skills in that particular area. Individuals with a strong sense of efficacy are more likely to feel confident in their abilities to accomplish a difficult task and see this as a challenge to be conquered, rather than a task that should be avoided (Bandura, 1994). On the other hand, individuals with a low sense of efficacy tend to focus on their personal deficits and view difficult tasks as threats. They are more likely to avoid these tasks because of their fear of failure (Bandura, 1994).
Limitations and future research
Although this study supports the use of an LBBI to teach interview skills to college students with ID, some limitations should be noted. Replication with other adult populations is necessary; our study only included adults in the same inclusive postsecondary education program, and all had similar educational backgrounds and experiences. Additionally, all three students share similar characteristics (diagnosed with ID, similar ages, and had similar requisite skills). Because most adults with ID do not have access to inclusive college programs, we recognize that students in this study are not representative of the general population of people with ID.
Second, while social responses to interview questions are a critical skill for individuals with ID, other valuable interviewing skills also affect applicants’ access to employment, and we did not explore these variables. For example, we did not address issues of proper dress, personal grooming, hygiene, or punctuality. LBBIs have shown their value in improving independent daily living skills and routines in adolescents and adults with disabilities (Brady, Hall et al., 2016; Brady, Honsberger et al., 2016). We believe LBBIs might be valuable as a strategy for enhancing these and related variables that contribute to interview efficacy.
Finally, the disruption caused by the 2020 COVID-19 pandemic had obvious effects on this research. Because of the mandated campus closure and the switch to remote learning, the LBBIs were delivered in both face-to-face and virtual formats. The speed of the switch required investigators to implement rapid changes to our research protocols. Fortunately, we were able to implement experimental procedures to address our research questions. For example, by adopting combined experimental designs (Kennedy, 2005) we were able to use the flexibility inherent in single subject research to create designs to understand experimental effects while meeting the robust expectations for behavior analytic experiments. However, there were limitations. We were unable to implement the multiple probe design as typically described, and ideally, we would have extended the in-person LBBI a few more days to assess whether David and Judith would achieve mastery of the new skill. Yet, the combined designs provide strong evidence of experimental control. By comparing baseline and intervention performance among the participants, the LBBI demonstrated a functional relation with the acquisition of the interviewing skills. Across the study, there were 8 replications of this phenomenon. Thus, the data pattern for each student demonstrated predictability of the experimental effects.
An additional limitation of the study was our in-ability to program for generalization across conditions. In this study we did not include generalization probes during baseline, upon initiation of the intervention, nor follow-up. This precludes any comparison of the students across conditions. Our observation of spontaneous use of the interviews with a novel interviewer at the end of the study is encouraging and suggests that the LBBI might be effective for promoting such generalization. This should be explored in future research and the design implemented to specifically probe for such evidence of generalization. We also anticipate that future studies will examine generalization questions involving different types of interviews, different types of employment settings, and different types of job tasks. We also anticipate the need to examine other forms of generalization such as transitioning from in-person to virtual employment scenarios.
Replications of this study should be conducted with different populations, including participants wi-th other disability characteristics, ages, race and ethnic identities, and educational experiences. Given the recent importance of remote learning, researchers need to study the impact of remote coaching and instruction on these adults, including other college students with disabilities. Finally, additional research is needed on various delivery methods of the LBBI instructional package. This includes other methods of virtual delivery, and other skills sets needed for competitive employment.
Conclusion
This study extends the literature by demonstrating the effects of LBBIs designed to teach job interviewing skills to young adults with ID. In this study, an LBBI was implemented both face-to-face and virtually. In the post-pandemic education landscape, there will be an increasing need for similar interventions that improve employability of college students and other adults with ID. This study showed that LBBI can be seamlessly incorporated into virtual learning. The consistent use of LBBI could play a significant role in increasing the number of adults with disabilities gaining meaningful employment.
Conflict of interest
None to report.