Teaching young adults with visual impairment and intellectual/developmental disability how to determine appropriate job fit

1.1 Dual diagnosis: Visual impairment and intellectual/developmental disabilities

Just like individuals with visual impairments, the individuals with intellectual/developmental disabilities (IDD) historically have been woefully un- or under-employed (Domin et al., 2020). Only 10% of people with IDD are paid members of the integrated work force (Wehman et al., 2018). A dearth of pre-employment training, lack of career preparation, and limited work experiences are major reasons for poor employment outcomes for adults with IDD (Carter et al., 2012; Clark et al., 2019; Kearney & Torres, 2022; Southward & Kyzar, 2017). Individuals with multiple impairments have more complex developmental problems than individuals with a single impairment. For example, there is a high prevalence of visual impairments in individuals with severe or profound intellectual disability (Dijkhuizen, 2019). However, there is limited research in the special education or vocational rehabilitation literature addresses the unique needs and challenges of individuals with both visual impairments and IDD. In particular, given the high unemployment rates of young adults with visual impairments and IDD, there is an urgent need to identify services, interventions, teaching tools, and resources that improve their employment outcomes.

1.2 Employment decision making

Job searching is a pre-employment skill that involves intentional, dynamic, and self-regulated goal-directed processes (Cmar & Steverson, 2021). It is a direct method of countering unemployment (Kearney & Torres, 2022). To master job searching skills, job seekers must know how to determine whether the jobs they identified during their search are a good fit for them (Torres et al., 2021). Employment decision making is a selection process for one of several potential employment options and requires problem-solving and choice-making skills (Wehmeyer & Field, 2007). To become fluent in identifying options, evaluating jobs, and decision making, adults with disabilities need to practice these skills in option-rich environments (Bambara, 2004).

Interventions to teach employment decision-making also have the potential to address many common barriers for young adults with disabilities; these barriers include unclear goals, unrealistic expectations, lack of vocational information, and lack of self-awareness (Agran & Krupp, 2011; Germeijs & DeBoeck, 2003; Kaur & Kaur, 2017; Ochs & Roessler, 2001). The following methods have been used to teach choice-making and decision-making to young adults with disabilities: task analysis (Belfiore et al., 1994; Cooper & Browder, 1998; Lattimore et al., 2003), picture books (Faw et al., 1996), forced-choice preferences (Browder et al., 1998), computer assisted instruction (Mazotti et al., 2010), multimedia stories (Richter & Test, 2011), and Self-Determined Learning Model of Instruction (SDLMI; Burke et al., 2019; Shogren et al., 2019). These methods were found to be effective, however, literature highlights the lack of intervention research (Raley et al., 2018).

1.3 What ELSE about this job?

“What ELSE about this job?” is a multicomponent intervention to teach students employment decision-making skills. The intervention utilizes a four-step mnemonic device ‘ELSE’ with each letter representing a different category to evaluate the job fit:

E: What is the EDUCATION level required for this job?

This four-step process helps students discriminate between appropriate and inappropriate jobs based on their personal circumstances. Torres and colleagues (2021) developed this intervention to be implemented through remote audio coaching with college students with IDD. Their research indicated that all students who received the intervention substantially increased their ability to determine appropriate job fit. The effect size suggested that the intervention was highly effective for all students. In another study, investigators evaluated this intervention using the range-bound changing criterion design to assess students’ stepwise progression through skill acquisition (Brady et al., 2022). Investigators used the intervention to teach students with IDD to assess whether preferred jobs were a good fit for them, and to provide reasons for their decisions. This study also found that “What ELSE about this job?” substantially increased students’ employment decision-making skills. In addition, students generalized those skills to a novel job coach who was not part of the intervention, and maintained the skills after the intervention was removed.

1.4 Purpose of the study

The purpose of this study was threefold: First, we sought to replicate the Torres et al. (2021) study that combined a mnemonic strategy with remote audio coaching (RAC) to teach employment decision-making skills virtually. The effectiveness of using a mnemonic teaching strategy has been well establish in special education literature. However, there is limited information available on the remote delivery of this method. Second, we investigated the effectiveness of the intervention with a unique population, young adults with both visual impairment and IDD. We made adaptations to the implementation process so that the intervention does not rely on the visual methods in teaching. This is one of the first intervention studies addressing employability skills of young adults with dual diagnoses of visual impairment and IDD. Finally, the interventionist in this study was a field practitioner who worked for a community-based agency providing vocational rehabilitation services to adults with vision loss. The interventionist did not have experience with the intervention. This study also evaluated the practicality and feasibility of the utilization of the intervention by field practitioners. We asked the following research questions:

What are the effects of the ELSE mnemonic on the acquisition of employment decision-making skills for young adults with visual impairment and IDD?

2.1 Participants

Three young adults participated in this study. Hugo, Edward, and Paul were all active clients of an agency servicing adults with blindness or other visual impairment in the southeastern region of the United States at the time of this study. Each participant had a primary diagnosis of visual impairment and a secondary intellectual and/or developmental disability (IDD) based on educational and medical records. Each participant had an employability assessment completed with the Job Observation Behavior Scale: Opportunity for Self Determination (JOBS:OSD; Brady et al., 2006). JOBS:OSD assesses individuals’ perception of their work performance (WP), and the degree to which they can work with varying types of support (TS). The JOBS:OSD assessment had a series of analyses done that found the assessment to be both reliabile and valid (Brady et al., 2008), and is useful in determining the level of support needed at a job compared to individual’s actual job performance (Brady et al., 2008).

Each participant had been selected to participate in a job development program through the agency. Each participant received employment services with the agency through a work experience program, which is how they met the interventionist, who served as their job coach at the agency. Neither the agency nor the job coach had previously been involved with the participants’ educational programming while they attended high school. Services related to their visual impairment were provided through the county’s department of education by a public school teacher of students with visual impairments (TVI), independent of the agency.

These three participants were chosen for this study due to their lack of employment history, as well as their limited experiences in job searching. The information regarding each participants’ transition services received while in their secondary education program was retrieved from personal records available to the job coach at the time of the study. Clarification and further details concerning medical diagnosis for developmental disability were provided by legal guardians for Edward and Hugo. Each participant self-reported on the type of career-related experience they had participated in as part of the agency’s work experience placement process. All three participants had some previous work-related volunteer experience in integrated community-based settings, but only Paul had any competitive work experience prior to participating in this study. Participation in the study required consistent attendance, consistent access to an electronic device, and independent use of an electronic device (e.g., phone and/or laptop). Hugo and Edward provided written and verbal consent prior to participation. Paul provided written and verbal assent and his guardian provided written consent prior to participation. The study received approval from the University’s institutional review board before the researchers contacted the participants.

2.2 Setting

This study took place completely virtually. All sessions were conducted through Zoom, a popular videoconferencing application freely available to download online. The research team used a paid, professional Zoom account in order to prevent any privacy issues. All participants had previously downloaded Zoom to their smartphones or laptops for other purposes prior to this study and were experienced Zoom users. Participant caregivers were asked not to assist during the sessions; there were no known instances of caregiver interference during any session.

Each session consisted of the participant and interventionist logged onto Zoom and was recorded for primary and secondary data collectors to review later that day. A job coach attended the generalization sessions live. When the job coach attended, she kept her camera and microphone off, unless she was speaking directly to the participant, in order to minimize any distractions.

2.3 Materials

Two of the participants used their personal smartphones during the study for all the sessions. Hugo used a desktop computer primarily and his personal smartphone for a couple sessions. The Career One Stop website was used by the research team to share career videos with the participants. A script to describe the ELSE mnemonic was created to teach participants to determine appropriate job fit.

The independent variable was remote audio coaching (RAC), coupled with the ELSE mnemonic script. The interventionist was a field practitioner currently working for the agency as a job coach. She had interacted with all of the participants previously for their work experience placement, job coaching and/or through the agency’s transition summer camp program. She had a graduate degree in special education and was trained by the lead researcher on RAC, the ELSE mnemonic, and the accompanying script twice for a total of 30 minutes prior to implementing the intervention.

When implementing the independent variable, the interventionist shared her screen and used RAC to coach each participant through the ELSE mnemonic script. She then played a pre-selected video from Career One Stop, once the video finished playing, she said, “What ELSE about this job?” As the participant worked his way through the mnemonic, the interventionist provided no verbal feedback unless an error was made. If the participant made an error, the interventionist used RAC to correct the step. For example, if the participant forgot to state the education level required the interventionist used her microphone to say, “E stands for education –what is the education level required for this job?”

The goal was to teach students the decision making model using the ELSE mnemonic and to teach each ELSE concept (education / likes / skills / earnings) and their relation to employment decisions. Other than Education level, for items Likes, Skills, and Earnings, students reflect on their preferences (based on their personal likes / perception of the skill / thought of earning).

2.5 Dependent variable and data collection

The research team developed a task analysis to teach how to determine appropriate job fit using the ELSE mnemonic. The task analysis involved 12 total steps, two of which could be omitted during a session depending upon the participant’s response to the job presented that session. Participants responded verbally. See Table 2 for task analysis. The dependent variable was the percentage of steps the participant independently provided a verbal response to in the task analysis. The data collector marked each item in the 12-step-task analysis “independent and correct” or “incorrect” if the student covers (+) or misses (–) each step (without any prompts).

Each session, regardless of condition, started with a data collection probe. This data collection probe involved the data collector taking data on each participant’s ability to complete the steps in the task analysis independently, prior to teaching the lesson that day. During intervention sessions the intervention took place immediately following the data collection probe, ensuring a minimum of 24 hours between the application of the intervention and the corresponding data collection probe. This eliminated the threat of artificially exaggerated scores from immediate practice effects. Data were translated to percentages by dividing the number of steps completed accurately by the entire number of steps and multiplying the total by 100. Accuracy or correctness of the step completion was determined by whether students remembered, covered, and followed the steps as outlined in the task analysis.

The lead researcher served as the primary data collector during all sessions. She had a graduate degree in special education and had extensive experience working with adults with developmental disabilities. A second researcher, who had a graduate degree in vocational rehabilitation counseling and was experienced in working with people with visual impairment and blindness, collected data for interobserver agreement (IOA) and treatment fidelity purposes. The interventionist recorded all sessions and sent to the data collectors for review later that same day.

The secondary data collector watched 41% of the recorded sessions to determine IOA, observing sessions in baseline, intervention, and follow-up, as well as being present live during generalization probes. Generalization probes involved the data collector gathering data on each participant’s ability to complete the steps in the task analysis independently with a job coach the participant had never met before, prior to teaching the lesson that day. Agreement was calculated by counting the number of steps in the task analysis both observers agreed on, dividing the agreements by the total number of possibilities for agreement, then multiplying by 100. Agreement on Hugo’s observations averaged 96%, agreement on Edward’s observations averaged 97%, and agreement for Paul’s observations averaged 97%. Protocol required the primary data collector and secondary data collector to meet and discuss IOA discrepancies if IOA was scored as less than 80% in any given session. However, no session had IOA that low, so discrepancies were not discussed between the data collectors.

The research team also collected data on treatment fidelity. A 4-step fidelity checklist was used during baseline, follow-up, and generalization conditions to determine that the experimental procedures were followed appropriately by the interventionist, and ensuring that the intervention was not implemented during these conditions. A 5-step fidelity checklist was used during intervention to determine if the interventionist used the ELSE mnemonic script and RAC as intended. The secondary data collector gathered treatment fidelity data for all participants during 41% of all baseline, intervention, follow-up, and generalization sessions. While following the 5-step checklist, the secondary data collector determined the interventionist implemented the experimental protocol with 100% fidelity across all conditions.

2.7 Experimental design and procedures

A multiple probe across participants design was used to determine the effect of the intervention on participants’ ability to determine appropriate job fit. The probe design limits extended, inaccurate practice of a new skill (Kennedy, 2005). By introducing the intervention in a systematic and staggered way, researchers demonstrate that the increase in the skill is due to the introduction of the intervention instead of by chance or unanticipated elements.

2.8 Data analysis

Data were first analyzed through established visual inspection procedures. We determined measures of central tendency and ranges for each participant during each of the conditions of baseline, intervention, and follow-up. We calculated level, trend, and variability of the data across conditions to determine condition changes (Kratochwill et al., 2013). Following the visual inspection, we calculated Tau-U to establish an omnibus effect size based on the weighted average of each participant’s baseline and intervention changes (Parker et al., 2011). According to Rakap (2015), an effect of 65% or less would be a weak effect, an effect between 66% and 92% would be considered a medium effect, and an effect of 93% or higher would be classified as a strong effect. We used the Tau-U web-based calculator as a post-hoc analysis to determine the true effect size (Vannest et al., 2016).

3.1 Hugo

OPEN IN VIEWER

Fig. 1

Percentage of correct and independent responses.

During Hugo’s first baseline session he completed none of the steps in the task analysis accurately. During sessions two and three he correctly completed one step in the task analysis, but during session four he dropped back down to no steps correct. In the generalization probe conducted during baseline he completed one step correctly in the task analysis. Intervention was implemented in session six, but his skill level did not increase after the first intervention session. However, after the second session his skill level increased substantially, jumping from 8% correct to 75% correct. Hugo had a steady, accelerating trend and reached 100% skill acquisition by session 11. Hugo reached mastery criteria after a total of eight intervention sessions. However, his generalization probe during intervention was only at 25%. Hugo’s follow-up probe occurred 13 days after intervention. Both his follow-up probe and generalization probe demonstrated skill maintenance of 100%.

During Edward’s first baseline session he completed none of the steps in the task analysis correctly. During the following session, Edward increased to 8% skill accuracy and stabilized at 8% throughout the rest of the baseline sessions. The generalization probe during baseline was also 8% accurate. Intervention was implemented in session ten, and he had a slight increase in skill accuracy after the first intervention session, from 8% in baseline to 17%. He continued to have a steady, increasing trend throughout intervention, reaching 100% skill accuracy on session 15 and reaching mastery criteria after only seven intervention sessions. His generalization probe during baseline was 73% accurate. Edward’s follow-up probe occurred eight days after intervention. Both the follow-up probe and generalization probe demonstrated skill maintenance of 100%.

3.3 Paul

During Paul’s first and second baseline sessions he completed 8% of the steps in the task analysis accurately. He then increased to 17% skill accuracy and eventually his baseline levels stabilized there. Paul’s data during intervention demonstrated some variability, but had an overall increasing trend. Paul reached 100% skill accuracy on session 18, but did not hit mastery criteria until session 23. The generalization probe that occurred during baseline had 33% skill accuracy, which was relatively low, yet still higher than his baseline sessions. Paul’s follow-up probe occurred 28 days after intervention due to the end of the semester and participant availability. The follow-up data had some variability, with a dip to 70% skill accuracy and 20% skill accuracy during the generalization probe.

3.4 Post hoc analysis and effect size

Tau-U results showed a weighted effect of 98% for the baseline and intervention contrast. This is classified as a strong effect size (Parker et al., 2011).