Abstract
BACKGROUND:
Peer-facilitated interventions have been successfully used in chronic disease management. Less is known about their application in vocational rehabilitation.
OBJECTIVE:
To pilot-test an online peer discussion forum to improve exploration of employment options in job-seekers with long-term multiple sclerosis (MS).
METHODS:
Twenty-nine persons with relapsing/progressive MS (4–27 years since diagnosis) were recruited online and randomly assigned to an 8-week discussion forum led by peers and a moderator, MS JobSeek (n = 14), or an active control group (n = 15). Forum engagement and user satisfaction were examined, as were pre-post self-evaluations of job search behaviour, efficacy, and quality of life.
RESULTS:
A high (43%) attrition rate was noted, although intervention completers acknowledged and valued online mentors’ guidance and expertise. Group and time effects were not significant (p < 0.05), with both MS JobSeek and control participants reporting positive (e.g., increase in job search activities) and negative (e.g., lowered self-efficacy) change.
CONCLUSIONS:
The pilot findings suggest that online peer support is feasible, however more can be done to promote peer-to-peer communication. Recommendations for improving forum engagement and satisfaction in a future controlled trial, including strategies to better support mentees’ individualised needs, are offered.
Introduction
Peer mentors with lived experience are an integral part of community-based supports for persons living with a chronic neurodegenerative condition, such as multiple sclerosis (MS) (Uccelli et al., 2004). By sharing their knowledge and experiences, mentors can promote self-confidence and personal control (self-efficacy) (Vassilev et al., 2019). From a social network perspective, peer-based interventions can expand and diversify a mentee’s social connections and opportunities for participation (Chronister, 2019). These same psychosocial outcomes are critical to successful return-to-work for persons living with MS; a cohort with one of the highest unemployment rates (Raggi et al., 2016). Specifically, those with high self-efficacy beliefs are likely to judge themselves as capable of engaging in work (Plow et al., 2015). Similarly, social support has been identified as an important predictor of employment following a diagnosis of MS (Dorstyn et al., 2017).
To date, evidence for the benefits of peer mentoring in vocational rehabilitation remains limited. ‘Job club’ style interventions have been successfully implemented in the general population, although not always rigorously evaluated (Moore et al., 2016). Such interventions have typically supplemented professional efforts in vocational rehabilitation, making it difficult to isolate the specific effects of peer support alone. Among the few studies that have focused on vocational mentoring, scant details on intervention implementation (e.g., peer selection and training) have been provided (Frentzel et al., 2021; Lindsay et al., 2016; McEnhill et al., 2012). Nonetheless there is some evidence that having several mentors is necessary to ensure that the support is both socially and temporally immediate (Lorig et al., 2006) as well as an effective catalyst for behaviour change –both indirectly (i.e., by positively influencing attitudes about work) and directly (i.e., providing work opportunities; Brown et al., 2010).
Logistical barriers to community participation, such as finance and accessibility, also need to be considered when developing a peer-based intervention (Jalili et al., 2020). In this context, the affordability of broadband connections in the home provides the opportunity to explore online mentoring as an avenue for peer support. Online mentoring offers advantages over in-person mentoring, including access to mentors from different backgrounds and participation opportunities for those individuals that are hard-to-reach (i.e., rural or home-bound adults; Lindsay & Cagliostro, 2020). Indeed, preliminary research highlights the social benefits of peer mentorship delivered via interactive websites and email to prepare youth with a disability for further education and work (Lindsay et al., 2018; McEnhill et al., 2012). Anecdotally, peer led forums to improve post-stroke community engagement and participation, including employment, have also been successful (e.g., enable me, The Stroke Foundation https://enableme.org.au/). Whether the same benefits apply to job-seekers living with MS remains unknown.
Current study
Overall, peer mentoring is a well-recognised but under-researched concept in the vocational rehabilitation literature. Research evidence is needed to guide providers in the use of peer mentoring for employment support. The present study addresses this research gap by developing a peer-facilitated discussion forum to help job-seekers with MS build a social and professional network of work-related contacts. To maximise access, we delivered this targeted support online as a Phase I randomized controlled trial; an important first step to establish whether this novel intervention is feasible and to inform the design of a full-scale trial RCT (Eldridge et al., 2016b).
Methods
Trial registration
This trial was registered with the Australian and New Zealand Clinical Trials Registry (ANZCTR12619001537145) and MS Research Australia Clinical Trials Network (http://www.mstrials.org.au/trials/all) and follows the CONSORT reporting guidelines for pilot and feasibility studies (Eldridge et al., 2016a).
Sample
Inclusion criteria consisted of: (i) adults of working age (18-64 years) with relapsing-remitting or progressive forms of MS, (ii) English fluency, (iii) access to a desktop pc, tablet or smartphone with an Internet connection and (iv) computer literacy. Employment status was an additional criterion. Mentors needed to be currently employed. Participants were either unemployed and wanting to return to the workforce or in precarious employment (i.e., temporary/casual work). Individuals who indicated they were unable to work due to health issues (i.e., medically retired) were excluded.
Recruitment and group allocation
Nine state-based and national MS organisations in Australia were contacted, with seven agreeing to assist with recruitment. Recruitment occurred in two phases over five months. Each MS society first contacted potential peer mentors. Six people with MS responded to the study invitation and were subsequently screened, with five completing training to become mentors (from 28 March to 2 May 2020). This was followed by a second stage of recruitment (from 26 April to 6 July 2020) of a participant sample via social and digital media (i.e., Facebook, Twitter, e-newsletters, listservs). Of 43 persons who provided informed consent, 29 were deemed eligible and randomly assigned to one of two interventions: (1) an online peer discussion forum: MS JobSeek, or (2) an active control consisting of the provision of written information resources (see Fig. 1). The advent of the COVID pandemic in Australia at the commencement of the study slowed recruitment, with no further enrolments three months into the study.
CONSORT flowchart of participation.
Randomisation without stratification was performed using a computer-generated schedule by a statistician otherwise not involved in the trial, with group allocation (1 : 1 ratio) undertaken by a research assistant (fifth author, E.P). All participants were offered a $AUD 20 shopping voucher in lieu of their time, with mentors provided a $AUD 50 voucher.
Consistent with treatment fidelity guidelines developed for web-based interventions (Bellg et al., 2004), a protocol manual to support the MS JobSeek intervention was written. The research team adapted the manual content from a face-face peer mentoring scheme developed by the second author (M.O) for the Leukaemia Foundation of Australia (‘Blood Buddies’ program). The manual comprised four PowerPoint modules, each with a brief multiple-choice assessment task to test understanding of the content. In addition to providing an orientation to the study, the modules outlined the roles and responsibilities of a peer mentor, the nature of effective online communication, and debriefing and support processes (see Table 1). 1
Structure and content of training modules
Structure and content of training modules
*Content modified for participant modules.
Prior to enrolment, mentors were screened for eligibility via a brief telephone interview (mean duration: 25 minutes) undertaken by a psychologist (M.O). Screening questions focused on motivation, mental health history, ability to commit to the role, balancing the mentor role with their health and employment, willingness to undertake training, and any concerns they may have had about the role. Following this, all mentors were emailed the protocol manual. Mentors were asked to read each module and successfully complete the associated assessment task, administered via SurveyMonkey software, within two weeks of receipt. Mentors were also provided written instruction on how to use the MS JobSeek platform.
Discussion forum
The primary intervention involved a secure online discussion forum moderated by the same psychologist that conducted the screening interviews (M.O) and hosted on Invision Power Services Inc. software (http://www.invisionpower.com). The forum was accessible through different reading devices (i.e., laptop, smartphone, tablet).
After completing baseline assessments, intervention participants were emailed two training modules, adapted from the protocol manual developed for mentors. The first module provided an orientation to the study and details of the online forum whilst the second focused on safe online communication (see Table 1). Participants received a further email 1-week thereafter to confirm that they had read the modules, along with login details to the discussion forum. Given that individual participants had staggered start times, mentors monitored the forum over an extended (4-month) period to ensure that each individual had equal access to their support.
Participants were encouraged to login at least once per week for eight weeks, for 5 to 10 minutes each, at their convenience. They were not required to log in at the same time. The first time they logged in, mentors and participants were invited to introduce themselves to the discussion forum. Peer mentors subsequently posted questions or topics of interest to the group, including examples from their own work experiences, barriers and facilitators to successful return-to-work, and practical advice about the job search process. Mentors interacted with participants through group discussion, although both groups had the opportunity to privately email each other and the moderator.
The discussion platform included several built-in features to promote engagement. First, all participants were provided a user name and profile (pseudo-nyms were encouraged to preserve anonymity). Second, an ‘activity feed’ notified users which discussion topics had new posts, with a selection of recent posts automatically emailed to participants each week. Third, participants could interact by ‘liking’ or commenting on the activity feed. Finally, the moderator (M.O) used the dashboard to view participant activity and email those who failed to login for more than one week, encouraging them to join the discussion.
Active (information) control
On enrolment, control participants were emailed a 7-module written resource that they could view at their own pace during the study. A follow-up email was sent to confirm receipt. The same modules were available on the MS JobSeek forum for intervention participants to view and download. This purposefully developed and pilot-tested resource (see Dorstyn et al., 2018) included an introduction module and six information modules covering the following job-search and career-planning skills: 1) job statistics and facts for persons with MS; 2) finding the right job; 3) resume writing and application letters; 4) successful job interviews; 5) maintaining work; and 6) career development and planning. Each module included online activity worksheets, primarily for self-information, and case examples to illustrate practical aspects of return-to-work. Participants did not need to read all seven modules - rather, they selected the modules relevant to their work experience (i.e., new vs. experienced job-seekers) and accessed them. The modules were all designed to be brief, with each taking no more than 20 minutes to complete. Control participants were offered the opportunity to access the MS JobSeek forum after completing their final assessments.
Data collection
Baseline sociodemographic (i.e., age, sex, education level, time since last employment, current employment status) and illness information (MS subtype, Patient Determined Disease Steps –Hohol et al., 1995; Learmonth et al., 2013) were obtained. Forum engagement (i.e., dropout rate) and usage (i.e., discussion thread topics, number of logins, individual posts, page views, module downloads) were also examined.
In addition, three well-established measures were administered pre- and post-intervention. The Job Search Behaviour Scale (JSBS; Blau, 1994) examines the time and frequency with which individuals have engaged in job search activities (i.e., past two months) as well as their future intention to search for a job (i.e., in the next two months). Cronbach’s alpha reliability estimates for subscale and total scores were high in the current study (range: 0.82–0.86). The Job Search Self-efficacy Scale (JSSE) rates both job-search behaviours (e.g., “Prepare a sales pitch that will attract the interest of employers.”) and outcomes (e.g., “Obtain more than one good job offer”). Consistent with previous studies, the JSSE subscales and total score demonstrated excellent internal consistency in this study (α= 0.88–0.94; Saks & Ashforth, 2000; Saks, Zikic, & Koen, 2015). Quality of life perceptions were then examined using the Patient-Reported Outcomes Measurement Information System Global Health scale (PROMIS Global), considered a sensitive tool in MS research (Amtmann et al., 2011; Hays et al., 2009). Both individual-item and composite (physical health) 2 scores were considered, with higher scores representing better functioning (Tractenberg et al., 2013). The PROMIS Global-10 demonstrated good reliability in the current study (physical subscale α= 0.79).
At the 8-week assessment MS JobSeek participants also completed the four-item Client Satisfaction Questionnaire (CSQ © Attkisson & Greenfield, 1996; Larsen et al., 1979). Responses for individual items range from 1–4, with higher scores indicating higher satisfaction. A final open-ended question provided further opportunity to provide feedback about the forum.
Data analysis
The two randomised groups were compared on each repeated measure using an intent-to-treat approach and linear mixed-effects models, with adjustments for baseline age, education level, and time since last employed (Detaille et al., 2009; Hammond et al., 2000). In addition, standardised mean differences with 95%confidence intervals and p values were calculated for each group at each time point. Within-group differences were also examined using Reliable Change Indices (RCI) to evaluate the degree of individual pre-post change for both job search behaviour (JSBS) and self-efficacy (JSSE). An RCI greater than 1.96 is considered to represent reliable change rather than the result of measurement error alone (Jacoben & Truax, 1991). RCIs were calculated using the Excel utility developed by the Center for Clinical Informatics (https://www.psychoutcomes.org/Brown et al., 2015).
Results
Sample characteristics
Mentors included four women and one man with an average age of 42 (range: 33–51 years) and chronic MS (range: 4.5–27 years). All were working full-time in a managerial, professional, or clerical-administrative role. Mentor characteristics matched those of the larger participant sample, which primarily included women (n = 22, 76%) who had been living with MS for 9.5 years (range: 4–15 years), held a post-secondary or tertiary qualification and lived with mild to moderate disability (PDSS≥5; see Table 2).
Sample baseline characteristics by group and overall
Sample baseline characteristics by group and overall
*Could select more than one, total can add to > 100%.
Dropout rates were higher in the MS JobSeek group (43%vs. 7%), with six participants not completing post-assessments. Of these, two cited co-occurring health issues whilst a third had sourced work and no longer required employment support. Among the eight participants that did complete all assessments, five logged into the discussion forum each week, although did not necessarily comment on the material posted (Mdn = 8 posts per person, range = 6–29 posts). Despite the challenge of engaging participants, both the moderator and peer mentors maintained an active involvement in the forum: the moderator initiated 45%of the discussion posts (see Table 4), whilst each mentor had at least 13 posts (range = 5–23 posts). Popular topics focused on opportunities to upskill (e.g., further training, casual employment), challenges in finding a job –including maintaining motivation to job search, and general job search strategies (e.g., resume writing, phone interviews).
Discussion topics and associated level of engagement
Discussion topics and associated level of engagement
*Included multiple views by the same participants, mentors and moderator.
In addition to forum posts, 50%(4/8) of MS JobSeek participants indicated that they had read at least one of the seven information modules posted online. User-end data suggested higher rates of participation, with 55 total downloads recorded. Among controls, 57%(8/14) indicated that they had read the written modules emailed at study commencement.
Median scores on the Client Satisfaction Questionnaire suggested mixed responses to MS JobSeek (Mdn = 10, IQR = 8–11). Whilst the forum content did not always meet individuals’ specific job search needs (n = 4), the majority (n = 7) indicated that they would continue to access the forum in future. Open-ended responses highlighted co-occurring health issues and worry related to the COVID pandemic as key barriers to forum engagement: “I have been more anxious and worried about going out this past week. I have been limiting my movements as the new (COVID) cases have come up here.” (Participant 4). Notably, peer support and advice were seen as beneficial: “It has been good to participate and hear similar stories from others...I felt that I’m not the only one that is going through this.” (Participant 5). Notably, no intervention participants raised concerns or criticisms about the forum activity –suggesting that the group was effectively monitored.
Table 3 reports the linear mixed effect models for each repeated measure, with all mean values standardised. No statistically significant group by time effects were noted, with both groups reporting small pre-post differences for each measure.
Linear mixed effect models for the group by time effect for each repeated measure
Linear mixed effect models for the group by time effect for each repeated measure
∧P-value for interaction between group and time.
Individual-level RCI scores highlighted variable patterns. Two MS JobSeek participants reported reliable improvement: one had actively engaged in the job search process and intended to continue doing so in the near future (JSB total RCI > 15), whilst both reported confidence in their ability to continue the job search (JSE total RCI > 10). Conversely, three assigned to MS JobSeek –none of whom engaged with the forum (Mdn = 0 posts, range 0–3) –indicated they were less likely to seek work in the next two months, with two additionally reporting lower levels of self-efficacy. Among the control group, seven reported reliable change: two had improved at the 8-week assessment (greater frequency and intensity of current job search activities and/or enhanced efficacy), whereas five reported a decline (reduced job-seeking efforts or self-efficacy). Variation was also seen in employment outcomes: one MS JobSeek participant had sourced full-time work at the 8-week assessment, a second had commenced voluntary (unpaid) work, whilst a control participant had transitioned from casual to part-time hours.
This study reports the pilot findings of an online discussion forum moderated by peers and a psychologist for persons with chronic MS experiencing either long-term unemployment, or unstable (temporary) employment. Whilst engagement and participation in the forum varied, feedback received suggests that this may have been a reflection of co-occurring stressors in participants’ lives. Those who accessed the forum did report positive experiences with MS JobSeek, indicating that the peer support was relevant to them.
The non-significant group differences are perhaps expected given that the control condition involved a previously tested resource associated with small to medium and positive changes in work expectations and self-efficacy for persons with MS (Dorstyn et al., 2018), potentially contributing to skewed results in the current study. A future trial might involve an untreated comparison (i.e. wait-list control) to determine if a novel, experimental condition such as MS JobSeek has an effect.
Our pilot study was also dramatically impacted by the timing of the COVID pandemic in Australia, with attrition rates and examined psychosocial variables likely affected. During this time, employment opportunities as well as usual levels of physical and social activity substantially reduced for the entire population. In saying this, COVID also fostered opportunities for alternative means of working and education –although the positive benefits of COVID have been largely unexplored for disability groups (Lebrasseur et al., 2021). Importantly, the findings show real promise as a foundation to continue to refine and develop MS JobSeek as a supplemental service to individuals with MS. The need to further develop remote interventions is all the more evident post COVID-19, with many living with MS facing a complex employment situation necessitating maintenance of quality and continuity of vocational support services (Bishop & Rumrill, 2021; Koch et al., 2021).
There were, however, challenges in maintaining engagement with job-seekers in the online environment. For MS JobSeek, guided support by a trained health professional was critical to facilitating online peer and mentor interactions. The forum moderator played a distinct role by encouraging discussion and ensuring that participant posts and discussion threads received replies. The moderator also provided clinical expertise, by responding to participants’ information needs and providing a supportive, assistive role –aspects that are important when developing an online community (Huh et al., 2013).
The present findings highlight pertinent issues for future development and evaluation of online vocational mentoring. In particular, our discussion forum relied on one telecommunication modality. There is suggestion that a combination of media tools (i.e., phone, email, video, bulletin boards) and intervention features (e.g., structured programs and activities for mentors and mentees combined with informal discussion; Kolakowsky-Hayner et al., 2012) are necessary to support accessibility and longer-term maintenance of an online peer support group (Shpigelman & Gill, 2013). Additionally, the minimum requirements for MS JobSeek participation (i.e., 5 mins per week over 8 weeks) may have not been sufficient to produce behavioural change. In saying this, online forums involving more frequent email exchanges over a longer duration (i.e., 12 weeks) run the risk of reduced interest and interaction over time (Lindsay & Cagliostro, 2020). Regardless of duration, participation withdrawal needs to be factored in the design and planning of a future MS JobSeek trial. A post-hoc power calculation suggests that approximately 52 participants (i.e., 26 per group) would be required to detect a medium and statistically significant group difference (α set at 0.05, power at 0.8; Faul et al., 2007). Based on the 43%attrition rate for the current pilot trial, this would involve recruiting at least 92 participants (i.e., 46 per group) at the outset.
Interviews with mentors and mentees would also be useful, to examine acceptability of the MS JobSeek intervention as well as identify design features (e.g., appearance and navigation of online content) that could be modified to enhance engagement. Similarly, the quality of the online interactions could be analysed. Such analyses would help to determine the degree to which mentors and moderator were consistent in their messages (e.g., the importance of reinforcing potential job leads from both and social connections; Hoye et al., 2009; Wanburg et al., 2020). Whilst monitoring of treatment fidelity was problematic, as is the case for most web-based interventions (Eaton et al., 2011), future trials might consider having mentors record or self-rate the specific skills and strategies that they applied from the MS JobSeek protocol manual (Eaton et al., 2011).
Finally, it would be beneficial for future research to explore how MS JobSeek could be tailored to meet individuals’ personal vocational goals, interests and needs. Individual matching of mentors and mentees based on shared interests and experiences - a strategy successfully used in general mentoring programs (DuBois et al., 2002) - is one option. Such matching would allow group-based support to be supplemented with one-on-one mentoring: a combination deemed effective for other chronic disability groups (e.g., spinal cord injury; Jones et al., 2019).
Limitations
Valuable lessons were learned during the implementation and evaluation of our pilot trial. First, the intervention may not have been of sufficient length to lead to individual-level changes across the assessed measures, particularly in time-dynamic constructs such as job search effort and self-efficacy (Wanburg et al., 2020). Second, the lack of a mental health outcome requires attention in future research, given that job-seekers experiencing heightened distress may avoid job search activities (Kanfer et al., 2001) and because group participation can have negative mental health effects for some (Uccelli et al., 2004). Finally, our findings were limited to individuals with long-term disability. Future research might consider an online forum specifically targeted to those newly diagnosed with MS –a subgroup that can be hesitant to access support (Garabedian et al., 2020).
Conclusion
Online interventions such as MS JobSeek can help to increase job-seekers’ access to peer mentoring opportunities. Both end-user data and self-reports obtained in this pilot study suggest that such an intervention is feasible, as individual participants did engage with mentors. Future research can enhance MS JobSeek by comparing and contrasting the impact of different intervention formats (i.e., duration, use of different media tools) and by monitoring mentor training.
Footnotes
Acknowledgments
The authors are grateful to the MS organisations, peer mentors and individuals who dedicated their time to this study. The authors also acknowledge the contribution of statistician Kelly Hall from Adelaide Health Technology Assessment at the University of Adelaide.
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical declarations
The procedures involving experiments on human subjects are done in accord with the ethical standards of the University of Adelaide Human Research Ethics Committee (project no. H-2019-206) and the Helsinki Declaration of 1975.
Funding
This work was supported by an Incubator Grant from Multiple Sclerosis Research Australia (Grant number 18_0581).
Informed consent
Written informed consent was obtained from all subjects before the study.
Training material available on request from corresponding author.
An error in the online administration of the GLOBAL PROMIS-10, where Item 4 was inadvertently omitted, precluded calculation of a mental health composite score.