A packaged intervention to improve job performance of a competitively employed young adult with autism spectrum disorder

1.1 Evidence-based practices as job supports for employees with ASD

Evidence-based practices (EBPs) in the field of special education are interventions that have been deemed to have sufficient experimental support to justify practical application (Cook et al., 2009; Horner et al., 2005). Implementing existing EBPs for individuals with ASD in the context of CIE settings may be one potential solution to enrich their employment experiences and outcomes. Specifically, using EBPs for employees with ASD may be a viable on-the-job training tactic because they frequently require more intensive training and support compared to employees without disabilities (Schall & McDonough, 2010). In addition, transition-aged youth with ASD who received on-the-job training and were four times more likely to obtain employment than those with no job-specific training (Kaya et al., 2016). Several job training strategies have been found to be beneficial for employees with ASD, including workplace accommodations, natural supports, and regularly scheduled on-site training sessions (Cannella-Malone & Schaefer, 2015). Traditional training methods (e.g., lengthy, didactic instruction) will likely not be sufficient for many employees with ASD (APA, 2013; Hendricks, 2010). Fortunately, several specialized interventions have been found effective for teaching a variety of skills to individuals with ASD. In particular, the National Autism Center (2009) and National Professional Development Center on ASD (Odom et al., 2010) conducted exhaustive literature reviews to identify and describe EBPs for learners with ASD. Wong and colleagues (2015) broadened and updated these previous reviews by identifying 27 EBPS for learners with ASD up to age 22. Existing educational interventions with a strong literature base may prove useful for implementing job skills training for employees with ASD in CIE settings. The interventions used in the current study are reviewed below.

Technology-aided instruction and interventions (TAII) incorporate technology as the central feature of the intervention (Wong et al., 2015). Many businesses and employers utilize TAII to train their employees (regardless of disability status). Video modules are an example of TAII in which learners typically progress through content at their own pace. In a review of literature, TAII was found effective to develop a variety of skills (e.g., academic, communication, vocational) for transition-aged youth with ASD (Odom et al., 2015). Several TAII using audio and video were included in two recently conducted systematic reviews on vocational interventions for young adults with ASD (Anderson et al., 2017; Seaman & Cannella-Malone, 2016) and found overall positive results. However, social validity of technology-based interventions was often underreported (Anderson et al., 2017) and additional research is needed in CIE settings.

Applied behavior analysis (ABA) is the process of systematically applying interventions to improve socially significant behaviors (Baer et al., 1968; Cooper et al., 2020). Although there are common misconceptions regarding the use of ABA (Trump et al., 2018), many teaching procedures grounded in principles of ABA have a well-established literature base to enhance a variety of outcomes for individuals with ASD. For example, two basic principles of ABA are prompting and reinforcement. Prompts are typically delivered before the target behavior and can be verbal instructions, modeling, or physical guidance (Cooper et al., 2020). Reinforcement is a consequence following a response that has the effect of increasing some dimension of future behavior (Cooper et al., 2020). In particular, positive reinforcement entails the delivery of a desired stimulus in order to increase a target behavior. Wehman, Schall, McDonoughand colleagues (2014) provided systematic instruction using behavior analytic practices as part of their randomized controlled trial examining an autism-specific employment model, Project SEARCH Plus ASD Supports. They found 87% of participants who received the employment model obtained employment compared to 6.25% of control group participants. Accordingly, interventions using principles of ABA appear to have relevance and applicability in employment settings.

Social narratives are written stories that describe situations by providing relevant cues and instruction for the learner (Wong et al., 2015). Social narratives are usually written in first person from the perspective of the learner and provide appropriate responses in specific situations. Social Stories™ (Gray, 2000) and literacy-based behavioral interventions (Bucholz et al., 2008) are examples of interventions using social narrative techniques. Several reviews have examined the efficacy of social narratives to improve a variety of outcomes for learners with ASD, such as reducing aggressive behavior and increasing social and communication skills (e.g., Karkhaneh et al., 2010; Qi et al., 2018). Social narratives are generally more effective when read immediately prior to the target situation (Kokina & Kern, 2010). For example, employees with developmental disabilities increased their on-task work behavior and more quickly returned from breaks after reading social stories (Bucholz et al., 2008). Although social narratives are considered an EBP for learners with ASD (Wong et al., 2015), additional research focused on using social narratives to enhance job skills is needed.

1.2 Purpose

Supporting employees with ASD with individualized, on-the-job training is advantageous to their successful employment (Wehman, Schall, Carr, et al., 2014; Wehman, Schall, McDonough et al., 2014). The purpose of this study was to investigate the effects of an intervention package on the work performance of a competitively employed young adult with ASD. The intervention package was designed in collaboration with the young adult’s employer using identified EBPs according to the Wong et al. (2015) review. In addition, individuals naturally found in CIE settings, such as co-workers, may assist with implementation of individually-designed job supports. Indigenous intervention agents likely have higher social validity (Barton, Meadan-Kaplansky, et al., 2018) than university researchers serving as intervention agents, which is particularly relevant in CIE settings where presence of outside professionals may be stigmatizing and distracting. Successful competitive employment is often contingent on co-workers and employers who provide a supportive workplace environment to the employee with ASD (Hendricks, 2010). Therefore, we anticipated incorporating a co-worker as an intervention agent would enhance the social validity of the intervention package. The following questions guided our research: (1) To what extent does an intervention package consisting of video modules, training trials with prompting and reinforcement, and social narratives improve the work performance of a competitively employed young adult with ASD? (2) Can a co-worker implement the intervention package with fidelity as a means to generalize to an indigenous intervention agent? (3) What is the social validity of the intervention package as reported by the employee ASD and his employer?

2.1 Participant and setting

A university institutional review board approved this study prior to participant recruitment. Young adults with ASD were recruited from a non-profit organization that provided social skills classes and licensed therapy to children and adolescents with ASD including young adults up to age 26. Inclusion criteria for young adult participants included: (a) diagnosis of ASD as confirmed by educational or medical record, (b) employment status in a CIE setting according to the Workforce Innovation and Opportunity Act definition, (c) need for workplace supports as identified by the employer and during preliminary observations. The employer also had to provide signed consent and agree for observations to be conducted during the employee’s regularly scheduled work shifts. If a young adult and employer consented to participate, employers were asked to nominate a co-worker to support the employee with ASD. Approximately twenty young adults and their families received recruitment information, but many young adults were actively seeking employment, unemployed, or participating in other postsecondary experiences (e.g., attending college). Only two young adults with ASD and their employers consented to participate in this study. Given the highly individualized nature of each young adult’s employment needs, two different studies were designed and implemented (see Bross et al., 2019 if interested in the other study). The young adult in the current study received a $100 stipend for participation, but the employer and co-worker did not receive a stipend.

Darius was a 20-year-old African American male employed at a large grocery store in a Midwestern city. Darius qualified for special education services under the disability category of ASD as confirmed by his final Individual Education Program (IEP). He graduated high school at 18 years of age by meeting IEP objectives in the areas of functional writing, reading fluency, and pragmatic language skills. Darius was in general education classes less than 40% of his school day and spent the majority of his school day in a self-contained special education classroom. Although Darius reportedly read at a 5th grade level and performed math skills at a 2nd grade level, his IEP did not report a co-morbid diagnosis of intellectual disability. Darius’ transition goals related to working with a state vocational rehabilitation (VR) agency to maintain CIE and live independently. Darius’ final IEP did not include formal diagnostic or assessment information.

Darius had worked at the grocery store as a courtesy clerk for approximately 1.5 years at the time of the study. He lived at home with his mother. The grocery store was located within walking distance of his home, and walking was his primary mode of transportation to work. Darius’ mother was his legal guardian and expressed an interest for Darius to participate in the study because his hours had recently been reduced from 20 to six hours per week due to reported unsatisfactory work performance by his employer. Darius had previously received adult agency services primarily in the form of a job coach but was not working with an adult agency at the time of the study. Darius’ primary job duty was bagging groceries. All data collection and training sessions occurred during Darius’ regularly scheduled work shifts, which were twice a week for three-hour shifts.

2.2 Dependent variable

The dependent variable was identified through a collaborative process involving Darius, his employer (i.e., grocery store head manager), two shift leaders, several cashiers, and the first author. In-vivo behavioral observations were conducted by the first author to identify Darius’ strengths and support needs before baseline data collection. The first author also interviewed Darius’ manager, shift leaders, and cashiers to obtain information about his current work performance. Darius’ shift leaders reportedly monitored Darius throughout his shift and gave frequent verbal directions. Darius worked with 6–8 different cashiers whose schedules varied. Some cashiers gave him verbal instructions as needed, while others complained or reported frustration due to his alleged off-task behavior, bagging errors (e.g., produce bagged with raw meat), and slow speed of bagging. Cashiers reported Darius frequently left the bagging area without telling them where he was going, typically to either pace or move to areas near the register (e.g., movie rental kiosk). Darius was allowed one 15-minute break during his three-hour shift and was observed to frequently return late from his break.

All staff interviewed described Darius’ off-task behavior as the largest detriment to his successful employment. Therefore, the primary dependent variable was identified as on-task work behavior specific to Darius’ courtesy clerk duties. The primary courtesy clerk duty was bagging groceries, which was operationally defined as Darius moving his hands and placing groceries into a bag. Additional on-task courtesy clerk duties included moving customer carts to the end of the conveyer belt, returning carts to the front of the grocery store, placing shopping baskets onto a rack, and returning items to aisles when requested by cashiers or customers. Behavior inconsistent with the operational definition of the dependent variable were considered off-task and scored accordingly. Examples of off-task behaviors exhibited by Darius during the preliminary observations included staring, pacing, leaving the bagging area, looking at movies on the movie rental kiosk, and standing still but not bagging.

2.3 Independent variable

The independent variable was an individually-designed intervention package with three components: (a) corporate-created video modules, (b) bagging training trials, and (c) social narratives. The interventions were selected in collaboration with Darius’ manager and shift leaders. The first author initially served as the interventionist, and then a male co-worker served as the interventionist during the generalization condition. The intervention package was implemented at the beginning of Darius’ work shift and is further described below. Darius’ manager agreed he could participate in the intervention package for purposes of the study after clocking in (i.e., paid training).

2.4 Experimental design

An ABAB withdrawal design was used to evaluate the efficacy of the intervention package. A withdrawal design was selected because we predicted work performance would return to baseline levels without the supports provided as part of the intervention package. The withdrawal design demonstrates experimental control when the level and trend of the target behavior improves during the intervention condition and returns to baseline level when the intervention is removed (Gast et al., 2018). Confidence in a functional relation is strengthened when the magnitude of change in the dependent variable is abrupt and immediate upon implementation of the independent variable (Barton, Lloyd, et al., 2018).

Due to the manager’s dissatisfaction with Darius’ work performance and reductions in his number of hours worked per week, we decided against implementing prolonged withdrawal phases with five or more data points as recommended by Kratochwill et al., 2013. Phase lengths in withdrawal designs may be shortened if ethical to do so (Cooper et al., 2020; Gast et al., 2018). Given each data point represented one full work shift and Darius only worked two shifts per week, we were concerned about further reductions in weekly hours and potential job termination if pro-longed withdrawal conditions were used. Thus, we decided to use shortened withdrawal conditions if we observed immediate returns to baseline performance after withdrawing the intervention package.

Data were analyzed using tools of visual analysis (Barton, Lloyd, et al., 2018; Cooper, et al., 2020). Specifically, we used level, immediacy of change, and overlap to conduct within condition visual analyses. Mean and range percentages of on-task work performance were calculated for each condition to describe the level. Immediacy of change was evaluated by looking for large changes in level of responding in adjacent conditions (Barton, Lloyd, et al., 2018). Overlap was reported by describing the percentage of non-overlapping data (PND; Scruggs et al., 1987) between adjacent conditions as a tool of visual analysis. In addition, data paths in each condition with five or more data points were analyzed for trend (i.e., accelerating, decelerating, or zero celerating) and stability using a stability envelope (Barton, Lloyd, et al., 2018). Data paths were considered stable if all data points in a condition fell within 30% of the median. Finally, summative visual analysis was conducted at the conclusion of the study to determine a potential functional relation if three demonstrations of effect were present (Barton, Lloyd, et al., 2018).

2.5 Procedures

2.6 Interobserver reliability and procedural fidelity

To ensure the reliability of dependent measures, the first and third authors collected data to calculate interobserver agreement (IOA) for all conditions. The third author was trained to decipher between on-task and off-task behavior during two in-vivo training sessions that occurred at the grocery store with Darius performing his regular job duties prior to baseline data collection. Subsequently, a minimum of 20% of sessions during baseline, intervention, withdrawal, and generalization conditions were conducted for reliability checks. Specifically, IOA occurred during the following session numbers: 3, 9, 12, 14, and 20. Results were subjected to a point-by-point agreement method using the following formula: (Agreements)/(Agreements + Disagreements)   × 100 = PercentofAgreement (Cooper et al., 2020). Mean IOA of the behavioral observations was 93% (range = 89–97%) across all conditions of the study.

Procedural fidelity data was collected to measure consistent implementation of the intervention package. A 20-step procedural fidelity checklist was developed and completed during each intervention session. Average procedural fidelity of implementation of the intervention package during the primary intervention conditions was 93% (range = 80–100%). A similar procedural fidelity checklist was developed for the co-worker as the interventionist. Average procedural fidelity of the generalization condition was 93% (range = 90–100%). To ensure reliability of the procedural fidelity data, a second rater also scored adherence to intervention procedures for a minimum of 20% of the intervention sessions during the same sessions IOA occurred for the dependent variable. Interrater reliability of procedural fidelity was determined using the same point-by-point agreement method and formula described above. Reliability of procedural fidelity of the intervention package across all conditions and interventionists was 100%.

2.7 Social validity

Darius and his employer completed an eight-item social validity questionnaire at the conclusion of the study regarding their satisfaction with the intervention package. The questionnaires used a four-point Likert-style scale with anchors of strongly agree, agree, disagree, and strongly disagree. The statements assessed the likeability and usefulness of each intervention component. Example employee questionnaire statements included, “The extra training I received at the beginning of my work shift helped me do my job better,” and “I liked doing the video modules on the computer when I first arrived at work.” Some statements provided an opportunity for Darius or the employer to express a preference between interventionists (“e.g., “The bagging training trials with the researcher were effective” or “The bagging training trials with the co-worker were effective”). The questionnaire also asked if Darius and his employer believed he received adequate support at work. Finally, participants were asked to describe the most and least effective components of the package, plus additional comments or suggestions for improvement. The first author read the questionnaire to Darius and served as a scribe for his verbal responses while the employer independently wrote his responses.

4.1 Limitations and suggestions for future research

A main limitation of this study relates to our decision to use shortened withdrawal conditions with less than three data points, the minimum number generally considered appropriate to assess trend. Current guidelines in single case research design standards indicate five data points per condition is preferred or recommended, depending on variability of the data (Kratochwill et al., 2013). Variable data generally warrants the need for additional data (Ledford et al., 2019). However, the data appeared stable and consistent changes in levels between each condition warranted sufficient confidence in a functional relation. Also, given the employer’s reported dissatisfaction with Darius’ work performance, we deemed in unethical to allow low work performance for an extended period of time to further increase confidence in the apparent functional relation. Nonetheless, it is possible that Darius’ performance during withdrawal data collection sessions would have improved spontaneously if we implemented longer withdrawal conditions.

An additional limitation is we were unable to collect maintenance data because Darius obtained a different job with more hours and better wages shortly after the study concluded. Future researchers should investigate whether individualized workplace interventions can produce durable changes in job performance that lead to job maintenance and advancement, including type of job, hours worked per week, and job responsibilities. Future researchers should also examine jobs matched to an individual’s interests as a means to enhance job performance. Darius’ reported dissatisfaction with his job may be a confounding variable that contributed to his low work performance.

Finally, the packaged nature of our intervention precludes any examination of effects relative to package components. It may be that one component of the intervention accounted for the majority or entirety of the observed changes. Future researchers might use a component analysis to evaluate which intervention components are essential, secondary, and superfluous. If future studies can establish which interventions work best for specific individuals and conditions, then implementation of EBPs in CIE settings may become more feasible and sustainable.

4.2 Implications for practice

Employers may consider applying established EBPs to improve job performance of employees with ASD, but more systematic replication is needed beyond the procedures conducted in this study, particularly given there was only one participant. A more formalized process is needed that allows employers to independently (1) identify employee skills needed for improvement, (2) select and learn to apply EBPs that are likely to improve targeted job skills, and (3) evaluate intervention effects and adapt when necessary. Accordingly, a more immediate solution in need of investigation may entail training employers, co-workers, job coaches, or other available personnel to utilize this process for employees with ASD. Studies that investigate a formalized process like this may result in better service delivery and outcomes. Employers may also consider providing incentives to employees who volunteer to support their fellow co-workers with ASD.

An additional implication for practice is job coaches, VR counselors, and other employment professionals should consider using, augmenting, or adapting existing training materials prior to creating specialized training materials only for employees with disabilities. For example, Darius had previously watched the corporate-created video modules when he was first hired but was not consistently performing the job duties described in the modules. The video modules were incorporated into the intervention package because they were used by all employees and, therefore, offered a natural and inclusive training method for Darius. Employees with ASD are likely to benefit from repeated practice (APA, 2013; Nicholas et al., 2015) and continued access to training materials, and it may be unnecessary to create specialized training materials if existing materials can be adapted or modified.

4.3 Conclusion

Obtaining employment is a critical benchmark during the transition to adulthood for individuals with disabilities including ASD (Gerhardt & Lainder, 2011). In particular, CIE within the local labor market increases community engagement and is superior to non-competitive employment experiences (Cimera et al., 2012). Young adults with ASD face unique barriers to positive CIE experiences (Lipscomb et al., 2017) and implementing existing EBPs may be a viable on-the-job training method to alleviate some of those barriers. Employers and co-workers can also assist with the implementation of individually-designed interventions for employees with ASD. This study should be considered preliminary given there was only one participant, replication would be challenging, and the limitations described above. Additional research examining the efficacy of specific EBPs to enhance competitive employment outcomes for young adults with ASD is clearly needed.