Abstract
Keywords
Job search skills training has been identified as a facilitator of employment for adults with visual impairments (Silverman et al., 2019). Some evidence supports the benefits of employment programs for adults with visual impairments that include content on job search skills (McMahon et al., 2013; Wittich et al., 2013), but few of these programs offer comprehensive instruction in job-seeking skills (Cervenka, 2020). Research indicates that transition-age youths with visual impairments who received comprehensive job search skills training had significant increases in job search behavior, knowledge, and self-efficacy (Cmar & McDonnall, 2019, 2021). However, no studies have focused on the efficacy of job search skills training for adults with visual impairments.
Job search interventions (i.e., training programs that focus on helping people obtain employment) are effective at improving employment outcomes for other populations (Liu et al., 2014). Findings from a meta-analysis revealed that job search intervention participants had 2.67 times higher odds of employment than control participants; these interventions had more substantial effects on people with certain characteristics, such as disabilities and health conditions (Liu et al., 2014). Research has also documented positive long-term effects of job search interventions that have strong theoretical and empirical foundations (Malmberg-Heimonen et al., 2019) and interventions that combine job search skills training with therapeutic components, including coping skills, social support, and self-efficacy enhancement (Hult et al., 2020).
Previous job search interventions have been designed for in-person delivery in a group setting (Cmar & McDonnall, 2019; Curran et al., 1999).
by answering questions on this article. For more information, visit:☑Earn CEs Online
To address the need for programs targeting job-seeking skills for adults with visual impairments, we adapted an existing job search intervention for use with this population (Cmar & Antonelli, in press). Our intervention is based on the JOBS program (Curran et al., 1999), a theory-driven, evidence-based practice designed for face-to-face implementation with job seekers without disabilities (Price & Vinokur, 2014). Using the Planned Adaptation approach (Lee et al., 2008), we made numerous adaptations to JOBS, including tailoring the content for adults with visual impairments and altering the procedures for implementation via videoconferencing. Importantly, we retained the intervention's core components, including its group processes that focus on enhancing self-efficacy and creating a supportive learning environment (Vuori et al., 2005). A critical consideration is whether the intervention's distance format is conducive to developing social support and a positive group atmosphere.
As a precursor to investigating the intervention's efficacy in improving outcomes for adults with visual impairments, we conducted a pilot study to test the adapted program with this new population. The study's purpose was to evaluate the feasibility and acceptability of the job search intervention with a group of adults with visual impairments. For this study, we defined feasibility as whether the intervention can be successfully delivered as intended and acceptability as the intervention's suitability from the participants’ perspective (Feeley et al., 2009).
Method
Participants
To enter the study, participants had to be age 18 years or older, have blindness or low vision, live in the United States, be unemployed but seeking work, be able to participate in verbal conversation, and have access to Internet service and technology to connect with the videoconferencing platform. We also broadly assessed job readiness, requiring participants to have a system for accessing printed materials, have basic computer skills, and not have current or planned involvement in postsecondary education or vocational training. Fourteen people enrolled and met all requirements for participation; however, two withdrew on or before the first day of the intervention, resulting in a sample size of 12 adults from 8 states. Participants’ ages ranged from 22 to 63 years (M = 45.33, SD = 14.99). Five participants received Supplemental Security Income, five received Social Security Disability Insurance, and nine received vocational rehabilitation services. For those reporting income (n = 11), one reported annual income of $100,000 or more and 10 reported income of less than $40,000. Additional demographic information is provided in Table 1.
Demographic Characteristics.
Procedure
The institutional review board at Mississippi State University reviewed the study protocol and granted it an exempt determination. We recruited participants nationally via our website, listservs, social media, professional and consumer organizations, professional contacts of the research team and colleagues, and an online participant registry. We used an accessible online prescreening survey to assess eligibility. After reviewing prescreening survey responses to identify potentially eligible candidates, a researcher scheduled a scripted phone call with each candidate to discuss the study and its requirements. Individuals who were still eligible, interested, and available for the study completed an electronic informed consent form. Following their consent, participants completed a baseline survey addressing demographic questions and some additional measures being piloted for a future efficacy study.
We provided the adapted training program over a 5-day period (Monday through Friday) via the Zoom videoconferencing platform. Each day included two 2-h sessions (morning and afternoon) with an intervening 2-h break. Two professionals from the vision rehabilitation field facilitated the program; both had experience providing training or instruction and familiarity with the JOBS method. One trainer held certifications as a Certified Rehabilitation Counselor and Certified Psychiatric Rehabilitation Practitioner, and the other was a teacher of students with visual impairments, Certified Orientation and Mobility Specialist, and Certified Low Vision Therapist. Two researchers attended the sessions as observers to record intervention fidelity; one of the two researchers served as an alternate trainer on 2 days of the training during which a primary trainer was unavailable. We used Zoom's recording features to audio- and video-record the sessions.
The training opened with group introductions and an overview of the program history. The training curriculum was divided into topics covered in specific sections across the 5 days. Topics included overall preparation for the job search, identifying and overcoming perceived obstacles to employment (e.g., transportation, employer attitudes), disclosing one's visual impairment, and planning for setbacks. The curriculum also covered interacting with employers, such as understanding employers’ perspectives, addressing employers’ concerns, making initial contacts with employers, and requesting interviews. Other topics were resume design, interview preparation, using concrete examples of one's strengths and skills, and finding job leads, including networking and informational interviews. Interview topics included comparing effective and ineffective interviews, responding to behavioral interview questions, controlling the interview direction, and closing the interview. The training closed with a group appreciation and graduation exercise.
The trainers followed a detailed trainer's manual when facilitating the sessions, which included brief lectures, group discussions, brainstorming, role-play examples, and small-group practice exercises. Some of these activities incorporated virtual polls and breakout rooms. Participants received supplemental and reference materials in handouts of electronic Word documents, shared through a cloud-based storage service (i.e., Google Drive) or by email upon request. Trainers took “live” notes in a shared Word document during specified activities. Participants had access to the document through Google Drive, and the trainers shared their screen for low vision participants and read the notes aloud. At the end of Days 1–4, participants were given homework assignments to begin or complete for the following day. After each day's sessions, participants completed a short survey to assess cognitive load. On Day 5, this survey included additional measures of participant engagement and acceptability. Participants who completed the study received an electronic $20 gift card to thank them for their participation.
Measures
Feasibility
Feasibility measures included attendance, intervention fidelity, and cognitive load. Two researchers monitored fidelity by observing the Zoom sessions and documenting activity duration and completion, adherence to the protocol, and implementation issues. Both researchers were very familiar with the intervention and had a prominent role in the adaptation process. One researcher observed 100% of the sessions and one observed 60% of the sessions.
The cognitive load instrument was the Multidimensional Cognitive Load Scale for Physical and Online Lectures (MCLS-POL; Andersen & Makransky, 2021), which is based on Leppink et al.'s (2013) Cognitive Load Scale. The MCLS-POL includes 18 items divided into 6 scales representing different dimensions of cognitive load. The three-item Intrinsic Load scale measures perceived difficulty of the content (e.g., the topics covered in the sessions were very complex). The four Extraneous Load scales focus on elements other than the content that increase cognitive load, such as instructions and distractions. The Extraneous Load-Instructions scale has four items (e.g., low-quality audio made the instructions hard to follow); Extraneous Load-Noises has three items (e.g., distractions in the environment made learning ineffective); Extraneous Load-Media has two items (e.g., my activities on my phone/computer made it difficult to focus on the learning content); and Extraneous Load-Devices has two items (e.g., technical issues made learning ineffective). The four-item Germane load scale measures factors that promote learning (e.g., the sessions really enhanced my knowledge and understanding of job-seeking). The MCLS-POL has evidence of construct validity, external validity, and reliability (Andersen & Makransky, 2021). At the end of each day's sessions, participants rated the items on a 5-point scale (1 = strongly disagree, 5 = strongly agree), adapted from the original 10-point scale for clarity and ease of navigation for screen reader users. We calculated a composite score for each scale using the mean of the items representing each dimension. Higher scores on the Intrinsic Load and Extraneous Load scales represent higher demand on working memory, whereas higher scores on the Germane Load scale represent enhanced learning.
Acceptability
We used quantitative and qualitative measures to assess participants’ perceptions of the program. The quantitative items (listed in Table 2) were administered on Day 5 and covered the following domains: engagement, trainer behavior, group processes, relevance, interactivity, and virtual format. The engagement measure included items from the Involvement subscale of the User Engagement Scale (O’Brien & Toms, 2010). Participants reflected on their level of engagement during the program by rating their agreement on a 5-point scale (1 = strongly disagree, 5 = strongly agree). The trainer behavior, group processes, and relevance items were based on the JOBS program evaluation (Curran et al., 1999; Vinokur & Price, 1999), and the interactivity and virtual format items were created for this study. Participants rated their experiences with each aspect of the program on a 5-point scale (1 = not at all, 5 = a great deal).
Descriptive Statistics for Acceptability Ratings.
The qualitative measures comprised five open-ended questions. These questions covered (a) the effects of the virtual format on participants’ learning (Days 1–2), (b) the sessions’ content and format (Days 3–4), (c) the most helpful parts of the program (Day 5), (d) the least helpful parts of the program (Day 5), and (e) suggestions for improving the program (Day 5).
Data Analysis
We calculated descriptive statistics to examine participant characteristics, attendance, cognitive load, and acceptability ratings. To evaluate intervention fidelity, we compared the actual versus planned duration and content of each session and calculated the overall duration and amount of content covered. We compiled all implementation issues documented during the sessions, referring to the video recordings for more context when needed. For open-ended questions requesting participant feedback on the training experience from the daily surveys, two researchers independently reviewed responses and categorized them as positive, negative, neutral, or suggestion for improvement. Researchers then met to discuss and resolve any discrepancies until reaching a consensus.
Results
Feasibility
Participants attended an average of 9.17 (SD = 1.59) out of 10 sessions. Two participants attended 6 sessions, one attended 8 sessions, and nine attended all 10 sessions. Reasons for participants’ absences included family emergencies, a job interview, and personal issues. Participants used various devices to access the sessions and materials. Six participants used a computer; three used a computer plus a smartphone, tablet, or both; one used a smartphone and tablet; and the remaining two did not report this information. Overall, intervention fidelity was high. The trainers adhered to the protocol and delivered the intervention as directed, aside from a few slight deviations. The total intervention duration was 19 h and 41 min, and the trainers covered 95.5% of the planned content. Some activities took slightly less or more time than anticipated, and a few activities exceeded the allotted time considerably. Those discrepancies led to minor adjustments (e.g., shortening discussions) and decisions to skip two activities.
The trainers did not encounter any implementation issues that prohibited the successful delivery of the intervention. Background noise (e.g., JAWS, barking dogs) occurred occasionally when participants’ microphones were unmuted, and one participant called into a session by phone due to lack of Internet connectivity, but these issues did not interrupt or delay the activities. Most activity delays resulted from troubleshooting participants’ technical difficulties, which were most common and impactful on Day 1. For example, several participants had trouble navigating between open windows on their computers, unmuting their microphones, entering and exiting breakout rooms, accessing the Zoom polls, downloading handouts from Google Drive, and opening and saving the handouts. The trainers and researchers assisted with these tasks as needed; however, the participants were instrumental in helping each other by sharing Zoom keyboard shortcuts, JAWS commands, and other technology tips. This informal, unprompted peer-to-peer technical support started early on Day 1 and continued throughout the program. Other minor issues were evident during the activities. For example, some activity instructions were unclear and required clarification by the trainers. Additionally, the disability disclosure activity, despite being productive, was too open-ended, and the discussion did not flow very smoothly.
Figure 1 provides average cognitive load scale scores for Days 1–5. The low Intrinsic and Extraneous Load scores indicate that participants did not find the content overly complex and that instructional and environmental factors did not adversely impact their learning. The high Germane Load scores indicate that the sessions enhanced participants’ knowledge and understanding of the content. As depicted in Figure 1, cognitive load remained relatively stable across the program with some small fluctuations. For instance, Extraneous Load-Noises and Extraneous Load-Media exhibited a slight reduction between Days 1 and 2 before leveling off afterward. Conversely, Germane Load had a gradual upward trend, despite a slight dip on Day 4.
Average Cognitive Load Scores for Each Day.
Acceptability
Table 2 shows that acceptability ratings were high for all domains. All participants agreed or strongly agreed that the program was fun and that they felt involved in the experience; 90% agreed or strongly agreed that they were really drawn into the experience. Participants rated the trainers favorably in all areas; trainer behavior ratings were the highest for listening to participants’ comments, responding with specific examples, and providing equal opportunities to share ideas. The participants felt comfortable participating and indicated that the content and discussions were relevant, the activities were interactive, and the virtual format enhanced their learning. They felt that other participants shared their experiences and concerns, faced similar problems, would listen to them, and would not criticize their ideas.
In line with the quantitative ratings, responses to the open-ended questions were predominantly positive. Regarding how the virtual format affected their learning, participants mentioned the benefits of attending from their own homes and being free from distraction, and they felt that it did not differ much from in-person learning. One participant commented, “I loved it! I learned more effectively within this format.” Most negative comments about the virtual format related to technology issues (n = 3), for example, “A little difficult at times because of my computer…” and one comment focused on content, “My struggle was the material as it seemed to be geared toward a group that had never been on a job interview before.” All other comments regarding the content and format of the sessions were positive. One participant stated, “I found the sessions very useful. Especially the breakout room with the informational interview,” and another commented, “It was very informative. I love it; it has opened my eyes and has made me a better interviewee.”
Participants’ responses about the most and least helpful parts of the program were also primarily positive, with only two responses identifying an unhelpful aspect. One response about the most helpful parts was, “I love the handouts, and the way the instructors walked us through the process.” Another participant stated, “For myself, the greatest benefit was to interact with others who were having almost identical experiences in the job market.” Comments on the least helpful aspects were, “Maybe the technology issues. Preparing for computer use, like downloading the handouts, for many, was difficult for a variety of reasons” and “What I didn’t find useful in the program is the length of each session.”
Several participants provided suggestions for improving the program. Recommendations included gauging participants’ technology skills and providing them with tutorials or reference materials ahead of time, for example, “Adding references to using JAWS with Zoom as part of handouts in case someone needs tips. Also, could add a reference list for helpful JAWS keystrokes. Could save time during meetings.” One participant recommended using more diverse role-play examples reflecting different experience levels and creating dual programs, “One for individuals just entering the workforce and one for experienced workers trying to re-enter the workforce.” Other suggestions included conducting a similar program for employers, providing a list of companies or websites at which to apply, and extending activity time or program duration. One participant stated, “I liked this format. Perhaps a little more time in the breakout rooms, especially for the longer activities,” and another suggested, “Extend it by a few weeks, and cut down on the hours of daily sessions.”
Discussion
The purpose of this study was to pilot-test an adapted group job search intervention for adults with visual impairments. We evaluated the feasibility and acceptability of implementing the intervention through Zoom videoconferencing software. Most participants attended all scheduled sessions, the trainers adhered to the protocol, and participants did not experience high cognitive load. Some technical difficulties and other issues occurred, but their overall impact on program implementation was minor. Participants rated all aspects of the program highly, including their level of engagement, their interactions with the trainers and other participants, the relevance and interactivity of the content, and the virtual format. Participants’ comments supported their overall satisfaction with the program despite the technology-related delays. These findings support the feasibility and acceptability of the intervention, indicating that it can be delivered successfully via videoconferencing and is suitable for adults with visual impairments.
Several aspects of our intervention and training procedures align with strategies for optimizing intervention fidelity (Carroll et al., 2007). First, the trainer's manual provided detailed implementation guidelines, including scripts, discussion prompts, and step-by-step activity instructions. When possible, the core components of the intervention were incorporated into the scripts and instructions (Curran et al., 1999). Second, the trainers were familiar with the training techniques from previous training or experience. Third, the trainers participated in practice sessions, during which they rehearsed all aspects of the intervention with the videoconferencing platform and received feedback from the researchers. Those factors, combined with the controlled research setting, likely contributed to the high degree of fidelity documented in this study.
Several participants had technical issues during the program, including difficulties using Zoom features, downloading handouts, and managing files on their computers. These issues interrupted the flow or timing of some activities but did not seem to diminish participants’ overall acceptance of the intervention. Technical issues were most prevalent on Day 1 and decreased over time, as found for other group interventions conducted via videoconferencing (Banbury et al., 2018). Participants’ reduction in technical difficulties corresponded with their slight decrease in cognitive load after Day 1, as they became more familiar with the procedures and gained experience with the videoconferencing platform.
The acceptability results and observation notes indicate the replication of the positive, supportive group atmosphere that is central to the JOBS method. This finding coincides with studies of group videoconferencing interventions for other populations that documented high social support, engagement, bonding, and group cohesiveness among participants (Banbury et al., 2018). As found by Lopez et al. (2020), the participants particularly enjoyed the breakout rooms, which facilitated interactions and connections between group members on an individual level. The positive group processes may have mitigated the negative aspects of videoconferencing, as a higher sense of group belongingness is associated with reduced videoconferencing fatigue (Bennett et al., 2021).
Our findings shaped additional program adaptations and logistical changes, some of which aimed to reduce participants’ technology issues. For example, we decided to offer future participants a pre-intervention Zoom orientation, provide a document containing helpful Zoom keyboard shortcuts to participants, and send handouts via email instead of sharing them through a cloud-based storage service. Activity-related adaptations include editing instructions to improve clarity, incorporating reminders of Zoom keyboard shortcuts into the instructions, replacing virtual polls with virtual hand-raising, and revising the disability disclosure activity to provide more structure. Another logistical change focused on enhancing referent power, an essential component of the JOBS method (Curran et al., 1999). One way that trainers build referent power during the program is through self-disclosure of their job-seeking experiences, which was not ideal with sighted trainers who did not have lived experience as job seekers with visual impairments. Based on that point and one participant's inquiry about the trainers’ vision, we recommend having at least one person with a visual impairment on the training team for future program implementation.
Limitations and Future Research Directions
A limitation of this study is that participation required a reliable Internet connection and access to a computer with a microphone and camera or another device that could run the videoconferencing software. Furthermore, our evaluation of job readiness in the prescreening survey relied on self-report, and some participants appeared to have over-reported their skills. Those participants struggled with navigating the technological aspects of the training despite rating themselves highly on their technology skills. Although our results provide initial evidence that the JOBS method can transfer well to a synchronous distance environment, comparing face-to-face and distance implementation was beyond the scope of this study. Future research involving in-person delivery of the intervention would be beneficial for investigating how attendance, fidelity, cognitive load, and acceptability may differ between the two formats. It would also be useful to investigate the feasibility of implementing the intervention with shorter videoconferencing sessions over multiple weeks and how that change may affect participation rates and group processes. This study focused on the feasibility and acceptability of the intervention but did not evaluate its ability to improve short-term and long-term participant outcomes, which is a critical direction for future research. Accordingly, a randomized controlled trial is in progress to assess the efficacy of the intervention in improving various outcomes, including job search self-efficacy, resume quality, and employment. This study will include a longitudinal component to examine the effectiveness of the program over time.
Implications for Practice
Our findings have implications for the provision of group videoconferencing interventions to people with visual impairments in practice settings. The following considerations can help practitioners prepare themselves and the participants for a productive learning experience. Determine hardware requirements, software requirements, and prerequisite skills for participants (Gustavson et al., 2021); consider options such as loaning equipment (e.g., laptop computer with screen reader, webcam, mobile hotspot) to broaden participation. Decide how to distribute handouts and other materials (e.g., email, mail, cloud-based storage, or flash drive), considering participants’ experience with those methods and the format of the materials and handouts. Provide handouts, including basic keyboard commands for accessing the videoconferencing platform, to participants in advance and ensure that instructors and support staff are familiar with the commands. Ensure that instructors have ample opportunities to practice implementing all aspects of the intervention in the videoconferencing platform (Gustavson et al., 2021). Offer participants an orientation session that includes troubleshooting audio, video, and connectivity issues and practice using the videoconferencing platform with support (Gustavson et al., 2021; Lopez et al., 2020; Marhefka et al., 2020). Allocate extra time when scheduling activities in anticipation of technical difficulties, particularly during the initial sessions (Gustavson et al., 2021). Delineate staff roles and responsibilities for providing technical support, managing breakout rooms, monitoring the chat, and related tasks (Banbury et al., 2018; Gustavson et al., 2021). Consider assigning these technical responsibilities to staff members who will not have a lead instructional role.
Several strategies can be beneficial for promoting group cohesion during videoconferencing interventions. These strategies also apply to in-person groups but may require more deliberate planning for successful implementation with videoconferencing groups. First, establish and reinforce ground rules and norms for acceptable behavior to lay the groundwork for respectful interactions and a supportive atmosphere (Bennett et al., 2021; Marhefka et al., 2020). These rules and norms may include expectations for muting when not speaking, keeping cameras on, refraining from multitasking, participating actively, identifying oneself by name when speaking, respecting privacy, and maintaining confidentiality. Second, model supportive behavior during discussions and activities (Curran et al., 1999). Third, provide equal opportunities for participants to contribute to discussions and share their ideas. Consider calling on each person by name to ensure that everyone has a chance to speak. The chat feature provides an additional avenue for interaction and may encourage participation from people who are less comfortable speaking aloud in a group setting (Lopez et al., 2020). Finally, build in unstructured time for informal interactions before sessions, after sessions, or during breaks.
Footnotes
Acknowledgment
The contents of this manuscript were developed under a grant from the U.S. Department of Health and Human Services, NIDILRR grant 90RTEM0007. However, these contents do not necessarily represent the policy of the Department of Health and Human Services and should not indicate endorsement by the Federal Government.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Institute on Disability, Independent Living, and Rehabilitation Research (grant number 90RTEM0007).
