Virtual Reality Job Interview Training for Adults Receiving Prison-Based Employment Services: A Randomized Controlled Feasibility and Initial Effectiveness Trial

Abstract

Returning citizens struggle to obtain employment after release from prison and navigating job interviews is a critical barrier they encounter. Implementing evidence-based interview training is a major gap in prison-based vocational services. We conducted a randomized controlled trial (RCT) to evaluate the feasibility and initial effectiveness of Virtual Reality Job Interview Training (VR-JIT) within two prisons. Forty-four male returning citizens were randomized to receive service-as-usual (SAU) with VR-JIT (SAU + VR-JIT, n = 28) or SAU (n = 16). Participants reported VR-JIT was highly acceptable and usable. SAU + VR-JIT, compared with SAU, had significant improvements (with large effect sizes) in interview skills, interview training motivation, and interview anxiety (all p < .05; $η_{p}^{2}$ > .15), and greater employment by 6-month follow-up (odds ratio [OR] = 7.4, p = .045). VR-JIT can potentially help fill a major gap in prison-based services. Future research is needed to validate VR-JIT effectiveness and evaluate VR-JIT implementation strategies within prisons.

Keywords

virtual reality job interviews smart decarceration employment vocational rehabilitation

Introduction

Every year, more than 700,000 returning citizens are released from state or federal facilities in the United States (U.S. Department of Justice, 2021). The Bureau of Justice Statistics reports 44% of people released from state prisons across 30 states were rearrested within the first year (Alper & Durose, 2018). These data suggest justice facilities need to do more to provide rehabilitation and ensure successful reentry into communities. One area of rehabilitation that deserves more attention is employment readiness. Unemployment is among the eight notable criminogenic risk factors related to recidivism (Andrews & Bonta, 2017; Visher et al., 2017).

Gainful employment addresses many criminogenic risk factors by enabling returning citizens to network within their communities, secure housing, and pay their rent and utilities (Petersilia, 2005; Visher & Courtney, 2006). Despite evidence suggesting that employment reduces recidivism, research suggests current employment ranges from 28% to 45% within 1 to 4 years after release (Seim & Harding, 2020; Visher et al., 2011). Furthermore, the employment rate was significantly lower among African American compared with White returning citizens (Seim & Harding, 2020; Visher et al., 2011). Given the increased risk for unemployed returning citizens to recidivate, developing services to increase employment at reentry is a public safety priority.

A meta-analysis of prison-based education, vocation, and work programs found that returning citizens who engage in prison-based educational or vocational services have higher employment rates and are less likely to recidivate upon release (Wilson et al., 2000). However, only half of all state facilities have vocational services (Stephan, 2008). Job interview preparation is one of the most common vocational services provided by state facilities as decades of research suggest job interview skills influence employment in the general population (Macan, 2009). However, despite researchers attempting to develop job interview skills training for returning citizens since the 1970s (e.g., Twentyman et al., 1978), the job interview remains a critical barrier to employment among returning citizens as the field still lacks an evidence-based practice that enhances job interview skills (Flake, 2015). Hence, job interview preparation is highlighted as a critical need by the United States National Institute of Justice (Wells, 2014).

Relatedly, a job acquisition framework suggests that job preparation activities (e.g., job interview training) reduce anxiety, which in turn enhances access to employment (Corbiere et al., 2011). This framework is especially relevant for returning citizens who lack adequate interview skills to successfully navigate job interviews (e.g., Pham et al., 2017; Ricciardelli & Mooney, 2018). Furthermore, interviewing for a job provokes anxiety in most people (McCarthy & Goffin, 2004), which is exacerbated for returning citizens who need to effectively discuss their prior conviction but lack the skills to do so (Ricciardelli & Mooney, 2018). Thus, stronger interview skills would likely reduce the anxiety surrounding a job interview, as anxiety itself can disrupt answering interview questions due to memory lapse, stuttering, and other physical distractions (e.g., excessive sweating; McCarthy & Goffin, 2004).

Moreover, returning citizens with greater motivation to engage in job preparation have stronger employment outcomes (Hunter & Boyce, 2009). Thus, it is critical that job preparation activities such as job interview training are perceived as acceptable to optimize motivation to engage with them. That said, job interview preparation activities typically occur when an educator helps develop job interview skills through mock interviews and class-based didactics. Such mock interviews occur in nearly all vocational programs whether in prisons, mental health services, or special education so that service recipients are better prepared to answer questions regarding their work skills and ability to work hard (Hurtz & Donovan, 2000).

Although mock job interview training is commonly offered by vocational services, few studies have evaluated the effectiveness of these methods when delivered within prisons (Ellison et al., 2017; Newton et al., 2018). Meanwhile, clients receiving community-based vocational services are typically limited to vocational staff conducting one or two mock interviews (due to large caseloads and time required to conduct them) before a client’s real-life job interview. Thus, clients may need additional training to practice interviewing and enhance their interview skills. Moreover, agencies seldom train vocational staff on how to conduct mock interview training sessions. For example, vocational staff may not consistently provide feedback to clients regarding their interview responses, interviewing anxiety, or confidence about interviewing. In addition, vocational staff have existing relationships with clients that likely buffer the feelings of anxiety one has when interviewing with a hiring manager in the community.

Recently, the criminal justice system has advanced the integration of technology (e.g., computer-simulated learning, video games) into their practices to alleviate the challenges faced with training and supervising staff and returning citizens (Cornet & Van Gelder, 2020; Ticknor, 2019). Notably, virtual reality (i.e., a two- [2D] or three-dimensional [3D] virtual environment accessed using a computing device [nonimmersive] or head-mounted display [immersive]; Paes et al., 2017) has emerged as a versatile technology that can address several novel aspects of correctional services (e.g., substance abuse, aggression regulation, risk assessment; Cornet & Van Gelder, 2020; Ticknor, 2019). For example, virtual reality has been effective at treating anxiety in clinics and has recently been piloted to treat anxiety within prisons (Peters, 2018). Meanwhile, research suggests virtual reality may be feasible for delivering cognitive behavioral therapy to treat the cognitive distortions of returning citizens (Ticknor, 2019). Moreover, multiple technology-based job interview simulators have emerged to try and fill a gap in evidence-based practice by helping adults with psychiatric and developmental disabilities prepare for job interviews (e.g., Burke et al., 2018; Smith, Ginger, Wright, Wright, Humm, et al., 2014; Smith, Ginger, Wright, Wright, Taylor, et al., 2014; Strickland et al., 2013). However, most have been limited to small pilot or noncontrolled studies. Meanwhile, the nonimmersive, Virtual Reality Job Interview Training (VR-JIT) and its adaptation for youth have emerged from these initial studies with the strongest empirical evidence to date in more rigorous studies (Smith, Sherwood, et al., 2021; Smith, Sherwood, et al., 2022; Smith, Smith, et al., 2021; Smith, Smith, et al., 2022

Specifically, VR-JIT is a computerized job interview simulator that targets interview skills using SIMmersion LLC’s (www.simmersion.com) proprietary PeopleSim® technology. PeopleSim controls the personality of the virtual hiring manager, Molly Porter, with an algorithm that uses probabilities to facilitate Molly’s emotions, questions she asks, and responses to her questions. The result is that Molly comes across to trainees as realistic, authentic, and engaging during an experience that presents as if you are being interviewed over a remote meeting platform (e.g., Zoom; Smith, Ginger, Wright, Wright, Humm, et al., 2014). This approach creates strong social presence (i.e., sense of being with another person; Aragon, 2003) for the Molly character that also aligns with best practices for immersive high-fidelity simulations (Motola et al., 2013).

The efficacy of VR-JIT was demonstrated in a series of five randomized controlled trials (RCTs) in lab settings, where the VR-JIT groups, compared with control groups, improved their interview skills and interview confidence, and increased their access to employment (e.g., odds ratio [OR] = 9.6, OR = 8.7; Smith, Fleming, Wright, Jordan, et al., 2015; Smith, Fleming, Wright, Roberts, et al., 2015). This efficacy was demonstrated among multiple vulnerable adult populations, including individuals with substance abuse histories, serious mental illness, and veterans with posttraumatic stress disorder (PTSD; Smith et al., 2016; Smith, Ginger, Wright, Wright, Humm, et al., 2014; Smith, Humm, et al., 2015). In addition, two recent RCTs demonstrated the real-world effectiveness of VR-JIT at improving job interview skills, reducing job interview anxiety, and increasing access to employment. The first RCT was among adults with serious mental illness in the Individual Placement and Support (IPS) model of supported employment (OR = 3.0; Smith, Smith, et al., 2022). The second RCT was among transition age youth with autism using an adapted VR-JIT in special education preemployment transition services (OR = 12.4; Smith, Sherwood, et al., 2021). However, research has not yet evaluated the potential effectiveness of VR-JIT implemented in prisons to support returning citizens as they prepare to reenter their communities.

Based on the Proctor et al.’s (2011) implementation outcome taxonomy (i.e., a framework to bring consistency to implementation science research domains), the current study tested the feasibility and initial effectiveness of VR-JIT when implemented by local staff in prison-based vocational services. Regarding feasibility, we hypothesized that participants will perceive VR-JIT as acceptable and usable and they will actively engage the intervention. Regarding initial effectiveness, we hypothesized that trainees receiving VR-JIT in addition to employment services-as-usual (SAU), compared with participants receiving SAU only, would demonstrate improved interview skills, interview training motivation, interview anxiety, and employment rates by a 6-month follow-up. Finally, we explored the correlates of VR-JIT performance and effectiveness outcomes to inform a future validation trial.

Method

Trial Design

This RCT evaluated SAU + VR-JIT compared with an SAU group at an unbalanced randomized ratio of 2:1, which was done to gather sufficient feasibility data. There were no changes to the methods after the trial commenced. Aligned with best practices of community-based participatory research (Newman et al., 2011), we convened a stakeholder advisory board that included criminal justice scientists, state-level corrections assistant education manager, and a reentry coordinator for a local county sheriff’s office who was a returning citizen. This advisory board was instrumental in developing the overall study design.

Participants

Study participants (ages 26–58) were recruited from vocational rehabilitation programs in two prisons (one minimum security, one medium security). Employment readiness counselors at the prisons (that we reference in this paper as prison staff) used study inclusion criteria to prescreen potential participants who were then invited to a recruitment presentation by research staff. Inclusion criteria included the following: (a) within 3 months of their earliest possible release date, (b) high school diploma or equivalency, or (c) at moderate-to-high risk for reoffending violent crimes (assessed by the Michigan Department of Corrections [MDOC]) via the Correctional Offender Management Profiling for Alternative Sanctions (COMPAS; Brennan et al., 2009). Exclusion criteria included the following: (a) the presence of an uncorrected hearing or visual problem that interfered with using VR-JIT, and (b) a medical illness that compromised cognition (e.g., moderate-to-severe traumatic brain injury). The University of Michigan’s Institutional Review Board approved the study protocol, and all participants provided informed consent. The study occurred from September 2019 to June 2020, with follow-up completed by January 2021. Data were remotely collected after prison access was restricted in March 2020 due to the coronavirus pandemic.

Sample Size and Blinding

For this feasibility RCT, a sample size of 60 was determined by budget limitations and reviewed and approved by the study’s data safety and monitoring board (DSMB). A random number generator determined the randomly generated eight blocks of nine participants at a 2:1 ratio (n = 6 SAU + VR-JIT to n = 3 SAU only). This block size of 9 was chosen because the prison computer labs housed six computers that could be used for VR-JIT. As noted in our CONSORT (Figure 1), 48 participants met inclusion criteria, provided informed consent, and completed pretest study measures. Given that four participants dropped out prior to randomization, we randomized 44 participants to the SAU + VR-JIT (n = 28) and SAU only (n = 16) groups. The DSMB approved the termination of trial enrollment after 44 participants completed the study protocol due to the stoppage of VR-JIT delivery and SAU because of the coronavirus pandemic, and after reviewing the study’s feasibility for recruiting and retaining participants in the trial. Twenty-three participants were enrolled at the minimum security prison and 21 participants were enrolled at the medium security prison. Participants and VR-JIT implementers (i.e., prison staff) were not blinded to group assignment due to the real-world implementation design.

Figure 1:

CONSORT Diagram

Interventions

The MDOC Vocational Villages are separate residential programs within the prisons where returning citizens live, work, and study together prior to their release. Returning citizens must meet a high standard of behavioral and academic milestones (e.g., at least 6 months free of Class I misconduct [such as possession of a weapon or assault and battery], must have completed all core and academic programming) for entry into the Vocational Villages. The main components of the Vocational Village curriculum are the 13 vocational trade training programs (e.g., CDL/forklift, carpentry, automotive) that returning citizens can access. Specifically, returning citizens select one primary vocational trade where they earn legitimate trade credentials through coursework and hands-on training that are transferable to the workforce upon release. Returning citizens completed a 15-hr preemployment preparation workshop designed to enhance employability skills related to the job search, completing job applications, cover letters, resume writing, and job interviewing (Washington, 2018). Returning citizens self-reported they completed approximately one to two mock interviews (M = 1.22, SD = 1.95) with employment readiness instructors or peers based on their need and an assessment of their skill. One participant completed an outlier of 36 mock interviews (data not included). Data were only available for 24 of 44 participants due to participant attrition (n = 11) and an administrative delay (n = 9) to include the measure. All participants completed 1 to 2 hr of classroom instruction on interview skills led by employment readiness instructors. Additional Vocational Village details can be found at https://www.michigan.gov/corrections/0,4551,7-119-33218_75514—,00.html.

VR-JIT

As noted in the introduction, VR-JIT is a computerized job interview simulator (see Figure 2) and is delivered via the internet (or locally installed, no head mount required). Trainees interview with the hiring manager Molly via speech recognition software across three levels of difficulty (i.e., easy, medium, and hard) that display her different personalities (i.e., friendly, professional, inappropriate). Prior to interviewing with Molly, trainees review an e-learning curriculum that introduces eight job interview skills based on the literature (e.g., hard worker, sounds easy to work with; Huffcutt, 2011). In addition, VR-JIT helps trainees practice disclosing a prior conviction in a judgment-free environment.

Figure 2:

Interface for Virtual Reality Job Interview Training

During the virtual interviews, trainees receive four levels of feedback: (a) real-time nonverbal cues from an on-screen coach (see Figure 2), (b) a real-time and a postinterview transcript featuring feedback on each statement that trainees speak to the virtual hiring manager, (c) a postinterview performance assessment on how well they addressed each of the eight job interview skills highlighted in the e-learning, and (d) a numerical score from 0 to 100 (see Supplemental Figures 1–3, available in the online version of this article). A more detailed description of VR-JIT can be found in Smith, Ginger, Wright, Wright, Humm, et al. (2014).

Study Procedures

Participants attended Pretest Visit 1 where they completed self-report surveys about their background, employment, and criminal justice (i.e., times arrested, duration in jail or prison) histories. During Pretest Visit 2, participants completed a mock job interview with a research team member that was recorded on video, followed by a series of self-report measures to assess their job interview skills, motivation, and anxiety. At the end of Pretest Visit 2, participants were randomly assigned to SAU + VR-JIT or SAU. At the posttest visit, all participants completed the same self-reports and mock interview. Also, SAU + VR-JIT self-reported their perceptions of VR-JIT acceptability and usability. Due to the coronavirus pandemic, some participants completed their pretest (n = 4) and posttest (n = 10) mock interviews over the phone.

All prison-based study visits were completed in a semi-private space with a closed door that had a window and prison staff were in the next room to meet minimal security standards. In conjunction with the pretest visits, the research team accessed a public MDOC records database (https://mdocweb.state.mi.us/otis2/otis2.aspx) to collect criminal justice history data for each participant (i.e., MDOC number, earliest release date, and offense type, for example, violent or not). The study team conducted a 6-month postrelease follow-up phone interview with participants to establish the feasibility of following participants to obtain their employment outcomes. Participants were sent postcard reminders at 3 and 5 months postrelease before their 6-month follow-up date. If participants were nonresponsive via phone and email after 4 weeks, the study team terminated contact efforts. Concurrently, postrelease employment records were collected from MDOC. At the time of release, all participants received a copy of their consent form and an estimated date for their 6-month follow-up phone interview.

Research Visit Training Fidelity

To prepare for data collection at two prisons, the research team completed approximately 10 hr of training: They reviewed study assessments, practiced delivery, and completed at least two mock study visits during which their performance was evaluated by a master’s level research supervisor. Staff members completed approximately one to two mock study visits prior to subsequent data collection to prevent drift. To facilitate adherence to research visit fidelity, research staff used a checklist to complete all study visit procedures.

VR-JIT Orientation Training for Research and Prison Staff

Research staff were trained to prepare prison employment readiness counselors to implement VR-JIT for returning citizens. The first author led the research team’s VR-JIT orientation using the VR-JIT implementation fidelity checklist that identified each component of the intervention. Then, research staff engaged VR-JIT to learn to navigate its interface and completed three virtual interviews. Next, research staff role-played teaching VR-JIT to one another using the fidelity checklist. Concurrently, the first author observed the role-plays and used the checklist to monitor the fidelity of the role-played orientation. If required, feedback was provided and additional role-plays were completed until staff felt prepared to conduct an orientation session with the prison employment readiness counselors. Research staff then followed the aforementioned procedures to prepare the employment readiness counselors to orient study participants on how to use VR-JIT.

VR-JIT Implementation Procedures

After the completion of pretest visits, one employment readiness counselor at each prison led VR-JIT implementation. During Session 1 (approximately 60 min), the counselor used the aforementioned fidelity checklist (monitored by research staff) to teach the various components of VR-JIT. Then, the SAU + VR-JIT group practiced using the e-learning, job application, and virtual interviewing components. The two prison staff led a total of seven cohorts of VR-JIT implementation that ranged from four to nine VR-JIT trainees per cohort. An eighth cohort was randomized to VR-JIT but never used VR-JIT due to the coronavirus pandemic.

We assessed adherence to the VR-JIT implementation protocol at the staff and participant levels. Regarding staff-level adherence, employment readiness counselors completed all required VR-JIT orientation training requirements. Also, the research team supervised the fidelity of the first two implementation sessions at each prison, which obtained 100% fidelity. Then, one prison staff implemented VR-JIT independently for three additional cohorts and provided recordings of these orientations. The staff member’s fidelity ratings (in order) were 95%, then 75%, and then 60%. The staff was retrained due to a drop in fidelity to an unacceptable rating.

Regarding participant-level adherence, we monitored the frequency of participants who progressed from easy-to-medium-to-hard interviews and completed at least four hard interviews. Notably, five of 28 (17.9%) participants randomized to SAU + VR-JIT did not use VR-JIT due to the stoppage of Vocational Village activities during the coronavirus pandemic. Two of the remaining 23 (8.7%) VR-JIT participants dropped out of the study (but did not withdraw their consent). Thus, 21 participants engaged in VR-JIT and required between three and 10 sessions to complete their training. Fourteen of 21 (67.7%) VR-JIT participants successfully progressed through the recommended easy-to-medium-to-hard difficulty levels on the virtual interviews. Meanwhile, six of 21 (28.6%) participants progressed through easy, medium, and three of four required hard interviews. Data on easy, medium, and hard completions were lost for one participant (4.3%) who completed 15 total interviews.

Study Measures

Background Characteristics

Demographic, employment, and criminal justice histories

Participants completed a brief survey about their backgrounds (e.g., age, race, education) and employment histories (i.e., prior employment). Participants also completed a survey about their criminal justice involvement (e.g., number of times arrested? What is the total length of time spent in jail/prison?). The 6-month phone interview asked participants whether they obtained employment and what their employment start date, wages earned, and hours worked were.

Mental health

Participants self-reported whether they had ever received a Diagnostic and Statistical Manual of Mental Disorders (5th ed.; DSM-5; American Psychiatric Association, 2013) diagnosis (e.g., depression, bipolar disorder), and we assessed their current psychological distress using the 10-item version of the Symptom Checklist (SCL; Rosen et al., 2000) adapted from the Symptom Checklist-90 (Derogatis & Savitz, 1999). The SCL-10 consists of a total score of 10 items summed on a 5-point scale that assessed how often participants had experienced various types of distress (e.g., feeling afraid, difficulty making decisions) over the past 30 days (0 = not at all or 0 days to 4 = extremely or 20+ days). Internal consistency was good (α = .87).

Feasibility Outcomes

Feasibility was assessed via VR-JIT acceptability, usability, and engagement. We assessed participant VR-JIT acceptability using an adapted version of the Treatment Acceptability Rating Form (Reimers & Wacker, 1988). The adapted form used a total score from a summed five-item self-report on a 7-point scale (e.g., 1 = extremely unenjoyable to 7 = extremely enjoyable). Sample items reflected the enjoyability, ease of use, and helpfulness of VR-JIT. Internal consistency was acceptable (α = .70).

We assessed participant VR-JIT usability using an adapted version of the Systems Usability Scale (Brooke, 1996). The adapted form used a total score from a summed six-item self-report scale ranging from 1 = not at all to 4 = very much. Sample items reflected readiness to use VR-JIT and ability to use VR-JIT. Internal consistency was unacceptable (α = .21) and may reflect a lack of variation in the summed score as prior studies of VR-JIT using this same scale reported good internal consistencies (Smith, Sherwood, et al., 2021; Smith, Sherwood, et al., 2022; Smith, Smith, et al., 2021).

We assessed VR-JIT engagement with variables generated by the intervention. Specifically, VR-JIT automates the total number of completed virtual interviews, the total number of minutes engaged with VR-JIT, and the total number of e-learning minutes. VR-JIT performance was automated via scores from 0 to 100 that participants received for each completed virtual interview.

Effectiveness Outcomes

Employment

A competitive job (i.e., located in an integrated community setting that pays at least minimum wage and was not set aside for persons with disabilities or other needs) during the 6-month follow-up was coded 1 = “yes” and 0 = “no.” Competitive employment outcomes were provided to the study team by the MDOC Employment and Opportunities Unit. Employment data were validated via a pay stub provided to a parole agent who enters the data into the database.

Job interview skills (performance-based)

We assessed participant job interview skills at each pre- and posttest visit via a single mock interview using a version of the Mock Interview Rating Scale (MIRS; Smith, Ginger, Wright, Wright, Humm, et al., 2014) that was adapted for justice-involved populations (J-MIRS). The J-MIRS provides eight different job scenarios from which participants can select for their mock interviews with research staff. Participants had 5 min to review the scenario prior to research staff asking 14 standard interview questions along with four to eight random interview questions from a list of 70 available questions.

Prior to rating the J-MIRS, study team raters (blinded to study group assignment) watched the video and then rated a single item “likeliness to be hired” (scaled from 1 = unlikely to 5 = very likely) to reflect a global job interview skill rating. Then, raters rewatched the video and rated nine job interview skills based on the literature (Huffcutt, 2011), which used an anchored 5-point Likert-type scale. Items included the following: (a) comfort level, (b) discuss prior conviction, (c) hard worker, (d) works well with others, (e) communicates in a positive way, (f) sounds honest, (g) sounds interested, (h) sounds professional, and (i) overall rapport. Each item was rated using a set of qualitative anchors to guide the rating (e.g., a rating of 1 for comfort level is anchored by a participant being observed as “highly anxious” and “loses train of thought”). A total score was computed by summing the nine job interview skills. The four raters were master’s degree students with prior experience conducting real-world job interviews. The raters were trained using 10 gold standard role-play videos to learn how to use the rating anchors before independently rating study videos. Raters compared their ratings on the 10 gold standard videos, and reviewed and discussed the ratings with the study role-play trainer. Also, raters jointly coded videos at regular reliability sessions with the role-play trainer to review and discuss rating inconsistencies (i.e., ratings more than one point apart) and reach a consensus score. Raters coded the same videos for 15% of the sample for reliability (one-way random effects ICC across all nine items = .96).

Job interview skills (self-report)

Participants self-reported their skill or comfort level across nine job interview skills after completing the mock job interviews. This measure was adapted from prior research and we used a total summed score from nine items using a 7-point Likert-type scale (1 = extremely skilled to 7 = extremely unskilled). Sample items included “making a good first impression,” “telling the interviewer about your strong points,” and “asking questions to learn more about the job” (Smith, Ginger, Wright, Wright, Humm, et al., 2014). Internal consistency was strong at pretest (α = .90) and posttest (α = .94).

Job interview training motivation

Participants self-reported their motivation to engage in job interview practice and we used a total summed score from the seven-item interest/enjoyment subscale via the Intrinsic Motivation Inventory (IMI; McAuley et al., 1989). The IMI items are based on the theory of self-determinism and are answered using a Likert-type rating scale (1 = not at all true to 7 = very true) to assess motivation for targeted behaviors. Sample items included “I enjoyed preparing to find a job very much,” “Preparing to find a job was fun to do,” and “I would describe practicing job interview role-plays as very interesting.” Internal consistency was strong at pretest (α = .87) and acceptable at posttest (α = .78).

Job interview anxiety

Participants self-reported their job interview anxiety via an adapted version of the Personal Report of Public Speaking Apprehension (PRSPA; McCroskey, 1970). We replaced the phrase “public speaking” with “job interviewing” for all 34 items, whose Likert-type scale was from 1 = strongly disagree to 5 = strongly agree. Scores were computed using the PRSPA’s validated two-step process reflecting where the total score = (72 − [total of Step 2 items] + [total of Step 1 items]). Internal consistency was strong at pretest (Step 1 items, α = .97 and Step 2 items, α = .84) and posttest (Step 1 items, α = .96 and Step 2 items, α = .88).

Data Analysis

Participants were analyzed in the group to which they were assigned (i.e., intent-to-treat). Independent two-sample t tests and chi-square analyses evaluated potential between-group differences at pretest related to background characteristics. VR-JIT feasibility measures were evaluated using descriptive statistics: acceptability (total score), usability (total score), and VR-JIT engagement (i.e., total completed virtual interviews, total minutes engaged with virtual interviews, total minutes engaged with e-learning). Meanwhile, VR-JIT performance was measured as the mean (SD) of the virtual interview scores.

Repeated measures analysis of variance (RM-ANOVA) evaluated whether VR-JIT was associated with changes in job interview skill (performance- and self-reported), interview training motivation, and interview anxiety. Specifically, we examined whether there was a significant group-by-time interaction and generated partial eta-squared ( $η_{p}^{2}$ ) effect sizes. We included the following covariates in the RM-ANOVA models given their prior associations with interviewing and employment: criminal justice history (e.g., time served in prison [or jail], arrest count, and having committed a violent crime as a proxy for risk), age at release, psychological distress, and race (Bushway et al., 2007; Decker et al., 2015; Holzer et al., 2006; Leasure & Kaminski, 2020; Metcalf et al., 2001; Ramakers et al., 2014; Seim & Harding, 2020; Turney et al., 2013). Also, 56.3% of the SAU group and 42.9% of the SAU + VR-JIT group received job offers between pretest and posttest from community employers due to the SAU model at the Vocational Villages. These offers could have biased participant engagement in SAU or VR-JIT with unintentional effects on interview skills, interview training motivation, and interview anxiety. Thus, we included the receipt of a job offer prior to posttest as a fixed-effect covariate.

For employment outcomes, we first used chi-square analyses to evaluate whether VR-JIT was associated with a higher rate of obtaining competitive employment by 6-month follow-up. Second, we conducted a logistic regression to evaluate the aforementioned relationship while covarying for the above noted factors and pretest job interview skill given that multiple participants exercised these skills prior to their posttest and not all participants completed posttest assessments. We used a t test to evaluate between-group differences among the employed participants for number of days to obtain employment, hourly wage, and hours worked per week. Notably, all effectiveness outcomes were evaluated with one-tailed tests given the previously demonstrated efficacy and effectiveness of virtual interview training (e.g., Smith, Ginger, Wright, Wright, Humm, et al., 2014; Smith, Sherwood, et al., 2021; Smith, Smith, et al., 2022). Finally, we used Pearson correlations (or point biserial with dichotomous variables) to explore whether VR-JIT performance was correlated with the effectiveness outcomes.

Missing Data and Outliers

Data were reviewed and no missing data at the item level were observed. As noted in Figure 1, 11 participants did not complete posttest measures due to early release or the coronavirus pandemic and one participant’s posttest data were administratively withdrawn due to several outlier values and the participant’s disclosure of anxiety about failing to be released. Two posttest role-plays (one per group) were inaudible and could not be scored. We did not impute these data and only analyzed the raw available data. One participant’s interview anxiety data were an outlier and were not analyzed. Outliers were determined using the interquartile range method (Ghasemi & Zahediasl, 2012).

Results

Participant Characteristics

Table 1 displays the characteristics of the study sample (n = 44). Groups did not differ with respect to their demographic characteristics (all p ≥.10). In addition, groups did not differ regarding their clinical, employment, and justice-involved backgrounds (all p ≥.10).

Table 1:

Background Characteristics of Study Sample

	SAU Group (n = 16)	SAU + VR-JIT Group (n = 28)	Test Statistic (t or χ²)	p
Mean Age (SD)	39.1 (8.9)	38.1 (6.1)	0.4	.66
Race
Black/African American (%)	56.3	46.4
White (%)	37.5	46.4
Latinx (%)	6.3	0.0	3.3	.35
More than one race (%)	8.0	7.1
Current grade
High school grad/GED equivalent (%)	43.8	46.4
Some college (%)	31.3	14.3	2.8	.43
Technical school graduate (%)	25.0	32.1
Associates degree (%)	0.0	7.1
Prior employment (%)	93.8	96.4	0.2	.68
Self-Reported mental health disorders
Depressive disorder (%)	25.0	17.9	0.3	.57
Anxiety disorder (%)	25.0	7.1	2.8	.10
Bipolar disorder (%)	18.8	7.1	1.4	.24
Posttraumatic stress disorder (%)	12.5	7.5	0.4	.55
Psychological distress (M, SD)	3.7 (6.1)	2.1 (3.3)	1.1	.27
Criminal justice history
Total arrest count (M, SD)	8.3 (9.4)	8.0 (4.9)	0.1	.90
Total years in prison/jail (M, SD)	8.8 (6.6)	6.3 (4.2)	1.5	.13
Primary offense was violent (%)	37.5	39.3	<0.1	.91
Risk for non-violent re-offense
Medium (%)	43.7	46.4	<0.1	.86
High (%)	56.3	53.6
Risk for violent re-offense
Medium (%)	50.0	60.7	0.5	.49
High (%)	50.0	39.3

Note. SAU = service-as-usual; VR-JIT = Virtual Reality Job Interview Training; GED = general educational development.

Feasibility Outcomes

VR-JIT was highly acceptable (M = 31.1, SD = 3.6, range = 21–35; max = 35) and highly usable (M = 23.6, SD = 0.7; range = 22–24; max = 24). Regarding VR-JIT engagement, participants completed M = 13.3 (SD = 2.8, range = 9–18) virtual interviews across M = 5.00 (SD = 1.8, range = 3–10) sessions. They also spent M = 218.7 (SD = 59.8, range = 75–350) minutes engaging with the virtual interviewer and M = 22.3 (SD = 18.2, range = 4–70) minutes engaging in the e-learning. VR-JIT performance (i.e., means score across all virtual interviews) was M = 92.1 (SD = 3.6, range = 86–98).

Effectiveness Outcomes

Effectiveness results are displayed in Table 2. We observed a significant time-by-group interaction characterized by a large effect size when evaluating the performance-based job interview skills total score, F(1, 18) = 3.4, p = .04, $η_{p}^{2}$ = .16, where SAU + VR-JIT were rated as having improved overall job interview skills between pretest and posttest compared with SAU. The individual job interview skills “communicates in a positive way” and “sounds interested” were both characterized by significant group-by-time interactions, F(1, 18) = 10.5, p = .005 and F(1, 18) = 4.7, p = .02, respectively, and large effect sizes ( $η_{p}^{2}$ = .37 and $η_{p}^{2}$ = .21, respectively). The remaining interview skills were characterized as nonsignificant (p > .08) medium-to-large effect sizes (.06 < $η_{p}^{2}$ < .15; hard worker; discuss prior conviction). The group-by-time interaction for self-reported job interview skills was nonsignificant, F(1, 21) = 0.1, p = .77, $η_{p}^{2}$ < .01. However, the model revealed a significant time-by-job offer interaction, characterized by a large effect size. Specifically, participants who obtained a job offer between pretest and posttest self-reported having stronger job interview skills between pretest and posttest, F(1, 21) = 5.3, p = .03, $η_{p}^{2}$ = .21.

Table 2:

Estimated Means (Standard Error) for Effectiveness Outcomes.

Effectiveness outcomes	SAU group		SAU + VR-JIT group		F	p	$η_{p}^{2}$
Effectiveness outcomes	Pretest, M (SE)	Posttest, M (SE)	Pretest, M (SE)	Posttest, M (SE)	F	p	$η_{p}^{2}$
Job interview skills (performance-based)	31.8 (2.3)	31.6 (1.7)	29.9 (1.5)	33.2 (1.1)	3.4	.04	.16
Likeliness to be hired (Global Rating)	3.3 (0.4)	3.6 (0.4)	3.0 (0.3)	3.2 (0.3)	0.1	.71	.01
Comfort level	3.9 (0.3)	4.2 (0.3)	3.8 (0.2)	3.7 (0.2)	1.9	.09	.09
Discuss prior conviction	2.9 (0.4)	2.6 (0.3)	2.9 (0.3)	3.2 (0.2)	1.1	.15	.06
Hard worker	4.0 (0.3)	4.0 (0.4)	3.6 (0.2)	4.0 (0.2)	1.3	.07	.13
Works well with others	3.9 (0.4)	3.7 (0.4)	3.4 (0.2)	3.8 (0.2)	1.2	.06	.14
Communicates in a positive way	3.6 (0.2)	3.3 (0.3)	3.1 (0.2)	3.7 (0.2)	10.5	.005	.37
Sounds honest	3.2 (0.4)	3.5 (0.3)	3.2 (0.3)	3.9 (0.2)	0.4	.53	.02
Sounds interested	2.4 (0.4)	2.0 (0.4)	2.3 (0.3)	3.0 (0.3)	4.7	.02	.21
Sounds professional	4.1 (0.5)	4.1 (0.3)	3.9 (0.3)	4.4 (0.2)	1.0	.16	.06
Overall rapport	3.8 (0.5)	4.2 (0.4)	3.6 (0.3)	3.6 (0.3)	0.5	.24	.03
Job interview skills (self-report)	48.0 (2.1)	49.8 (2.5)	47.1 (1.5)	49.6 (1.7)	0.1	.77	<.01
Job interview training motivation	37.9 (2.1)	36.1 (1.7)	40.9 (1.5)	43.0 (1.2)	4.6	.04	.19
Job interview anxiety	77.7 (6.0)	80.1 (5.6)	81.8 (4.7)	72.4 (4.5)	5.1	.02	.21

Note. SAU = service-as-usual.

The group-by-time interaction for job interview training motivation was significant, F(1, 19) = 4.6, p = .04. The interaction was characterized by a large effect size ( $η_{p}^{2}$ = .19). Specifically, the SAU + VR-JIT group, compared with the SAU group, had increased enjoyment of practicing job interview training between pretest and posttest. A significant group-by-time interaction was characterized by a large effect size for job interview anxiety where SAU + VR-JIT had reduced job interview anxiety between pretest and posttest compared with SAU, F(1, 19) = 5.1, p = .02, $η_{p}^{2}$ = .21. Unless otherwise noted, time-by-race and time-by-job offer interactions within the aforementioned outcome models were nonsignificant (all p > .10).

Our 6-month follow-up efforts with returning citizens yielded a very low response rate (25%). As a result, we only analyzed the employment outcomes that were provided by MDOC. We observed that SAU + VR-JIT did not differ from SAU with respect to competitive and integrated employment (82.1% vs. 68.8%; χ² = 1.0, p = .15). The logistic regression (Wald = 11.6, p = .001; Nagelkerke R² = .439) revealed that SAU + VR-JIT had significantly better odds of obtaining a competitive job within 6 months compared with the SAU (OR = 7.4, 95% confidence interval [CI] = [1.1, 51.4], p = .045), after covarying for age, race, time served in prison (or jail), arrest count, prior violent crime committed, psychological distress, and pretest interview skill. Significant covariates in this model included psychological distress (OR = 0.7, 95% CI = [0.6, 0.98], p = .04) and total length of time in prison or jail (OR = 1.4, 95% CI = [1.0, 1.9], p = .04). Remaining covariates were nonsignificant (all p > .10). In addition, the SAU + VR-JIT and SAU groups did not differ with respect to (a) who interviewed prior to obtaining the job (95.7% vs. 100%; χ² = 0.5, p = .50; respectively), (b) highest wage earned per hour (M = 15.7, SD = 4.0 vs. M = 17.1, SD = 3.1; t = 1.5, p = .13), (c) hours worked per week (M = 39.6, SD = 2.1 vs. M = 41.0, SD = 3.2; t = 1.0, p = .32), and (d) number of days to obtain their job (M = 42.8, SD = 26.2 vs. M = 58.0, SD = 55.2; t = −0.7, p = .27).

Exploratory Correlations

The exploratory correlations between VR-JIT and effectiveness outcomes are reported in Supplemental Table 1 (available in the online version of this article). Higher mean VR-JIT performance correlated with higher performance-based job interview skills (total score) at posttest (r = .50, p = .026). Higher mean VR-JIT performance also correlated with the individual interview skills “discuss prior conviction” (r = .45, p = .045) and “sounds professional” (r = .50, p = .026) at posttest.

Discussion

The findings from this initial study of VR-JIT in prisons revealed feasible implementation by prison staff and were initially characterized as highly acceptable and highly usable by returning citizens. Participants completed a mean of 13 virtual interviews and 3.5 hr of interactions with the virtual hiring manager, and 70% completed the recommended progression from easy to hard interviews (including four hard interviews) with the remaining 30% completing three of the four required interviews at the hard level. Also, participants reported a strong mean acceptability score of 31.1 (total possible = 35) and strong mean usability score of 23.6 (total possible = 24). Importantly, the acceptability of an intervention is critical to facilitate its effectiveness (Elliott, 2017). Overall, VR-JIT acceptability and usability is consistent with prior research implementing VR-JIT in noncriminal justice settings (Smith, Sherwood, et al., 2021; Smith, Sherwood, et al., 2022; Smith, Smith, et al., 2021).

The study results support the initial effectiveness of VR-JIT at enhancing job interview skills among adult male returning citizens. Specifically, SAU + VR-JIT improved their overall mock job interview performance between pretest and posttest compared with SAU. Medium-to-large effect size improvements were observed in overall skill, comfort level, discussing a prior conviction, hard worker, works well with others, communicates in a positive way, sounds interested, and sounds professional. Overall, the results are consistent with prior RCTs of VR-JIT efficacy and effectiveness in non-justice-involved populations (e.g., Smith, Ginger, Wright, Wright, Humm, et al., 2014; Smith, Sherwood, et al., 2021; Smith, Smith, et al., 2022). Thus, this study provides initial evidence that VR-JIT could potentially fill a critical gap in evidence-based practices needed to support returning citizens preparing to reenter the community. Also, the SAU + VR-JIT group had greater motivation to prepare for job interviews compared with the SAU group. These results suggest that engaging with virtual interviews could be a more enjoyable method of practice. An alternative interpretation could be that participants using VR-JIT could be more motivated to practice virtual interviews as it enabled them to add something new to their routine.

The SAU + VR-JIT group reported reduced job interview anxiety beyond the SAU group, which was characterized by a large effect size. Although the current study was not fully powered, this finding suggests VR-JIT could potentially serve as an effective exposure therapy given job interviewing is anxiety-provoking among many people (McCarthy & Goffin, 2004), and especially for returning citizens who are preparing for a job search with the added pressure of discussing their prior conviction (Ricciardelli & Mooney, 2018). Notably, this finding is consistent with two RCTs that found using VR-JIT in mental health services and its adapted version for youth in special education was associated with reduced job interview anxiety (Smith, Sherwood, et al., 2021; Smith, Smith, et al., 2022).

In terms of real-world outcomes, SAU + VR-JIT had a higher unadjusted rate of obtaining competitive and integrated employment by 6-month follow-up compared with SAU (82.1% vs. 68.8%, p = .15), but the finding did not obtain significance. However, participants randomized to SAU + VR-JIT had greater odds of obtaining competitive employment (OR = 7.4, p = .04) after adjusting for factors associated with obtaining employment among justice-involved adults (e.g., race, psychological distress, time served in prison [or jail], violent crime committed; Bushway et al., 2007; Decker et al., 2015; Holzer et al., 2006; Leasure & Kaminski, 2020; Metcalf et al., 2001; Ramakers et al., 2014; Seim & Harding, 2020; Turney et al., 2013). These results are consistent with recent effectiveness studies of VR-JIT (Smith, Sherwood, et al., 2021; Smith, Sherwood, et al., 2022; Smith, Smith, et al., 2021; Smith, Smith, et al., 2022) and suggest a fully powered RCT evaluating VR-JIT within prison-based vocational services is warranted.

In addition, we observed that the 6-month employment rate for the Michigan Department of Corrections Vocational Villages SAU (i.e., control) group was high (68.8%), which is consistent with their internal records that approximately 67% of returning citizens in 2021 who paroled obtained employment (R. McGeorge, personal communication, October 6, 2021). Although the Vocational Villages themselves have not yet been the focus of a rigorously designed RCT, these are stronger employment outcomes compared with a comprehensive employment reentry program that reported peak 12-month employment at 59% (Cook et al., 2015). Meanwhile, the exploratory correlations revealed potential initial mechanisms of action for obtaining employment. Specifically, VR-JIT performance was correlated with multiple posttest job interview skills (i.e., total score, discussing prior conviction, sounds professional). These results suggest that VR-JIT may be an effective intervention for enhancing these job interview skills—findings that are consistent with prior research using VR-JIT and its adapted version (Smith et al., 2017; Smith, Sherwood, et al., 2021). Ideally, we would evaluate the correlations between VR-JIT performance, pre-to-post change outcomes (e.g., interview skills), and employment. However, the observed employment rate of 82.1% in the VR-JIT group is near ceiling and lacks variation, and related correlations may not be reliable.

Limitations

The study results should be considered within the context of its limitations. First, this study focused on evaluating VR-JIT within a prison-based vocational rehabilitation program. Notably, returning citizens engaged in the Vocational Villages are required to meet high thresholds of good behavior targets and completion of internal academic programming to be selected, and most study participants served fewer than 10 years in prison. Thus, it is unclear how our results might generalize to other correctional settings or to returning citizens who served a longer duration, and whose behavior and academic programming accomplishments are not as exemplary. Second, the Vocational Villages conducted some of their typical mock interview training with participants prior to VR-JIT implementation. Thus, we did not capture the potential effects of the usual “mock job interview training” administered within SAU between pretest and posttest. The study groups received different amounts of mock interview training and future studies could consider controlling the dose of training received. Third, there was a low response rate (25%) when the research team collected data from returning citizens at follow-up. Thus, we analyzed the MDOC-reported employment outcomes that were available for all participants. Fourth, this sample has limited statistical power, and as such, the results should be interpreted conservatively. Finally, 61% of participants reentered their communities during the coronavirus pandemic and they all had at least 2 months of their follow-up occur during this period of time. Thus, the effects of the pandemic on study outcomes are unclear and the results should be interpreted with caution.

Future Directions

Although the current study demonstrated the feasibility and initial effectiveness of VR-JIT in prison-based vocational services, future studies are needed to evaluate the intervention in several areas. First, a fully powered trial using the effect sizes from this study is needed to validate the effectiveness of VR-JIT in prisons and to evaluate whether VR-JIT engagement correlates with employment outcomes and potential mechanisms. Second, future studies will need to engage in research follow-up assessment strategies beyond our methods (i.e., reminder postcards at 3 and 5 months, outreach to schedule beginning 2 weeks prior to date) to facilitate higher response rates at 6-month follow-up (e.g., outreach to participants 2 to 4 weeks after release, followed by regular outreaches, increase incentive payment). Third, the subsequent larger study could consider whether the effects of VR-JIT may be moderated by social and cognitive ability, which were not evaluated in the current study. Fourth, given the evidence of effectiveness for VR-JIT across multiple trials (Smith, Sherwood, et al., 2021; Smith, Sherwood, et al., 2022; Smith, Smith, et al., 2021; Smith, Smith, et al., 2022), an appropriate future study would be a cluster randomized trial design that also tests implementation strategies and uses a multilevel implementation evaluation (e.g., qualitative interviews with returning citizens, staff, and administrators) of VR-JIT within prison-based vocational rehabilitation programs. Such a trial would both establish the effectiveness of VR-JIT, compared with SAU, for obtaining employment and identify an optimal implementation approach for sustaining the intervention (Damschroder et al., 2009; Owens et al., 2014). Finally, future studies could consider assessing interview anxiety using physiological measures such as the galvanic skin response.

We offer three lessons learned during this study that may inform the design and implementation of future RCTs in prisons: first, allow sufficient time for research staff to be fully oriented to prison procedures; second, obtain a physical authorization letter from prison-level (or state-level) corrections administrators that research staff can carry to provide tangible authorization for bringing in and removing study materials. Such a letter will expedite potential security concerns upon prison entry and exit. Third, recruit a stakeholder advisory board to inform the study design. Although we included a returning citizen on the panel for this study, future RCTs may consider including one or more returning citizens from the targeted prison setting as their intimate knowledge of prison procedures will be critical to the success of the study design.

Conclusion

Results suggest VR-JIT is highly acceptable and highly usable among returning citizens. They adhered to and performed well with the training. Prison-based staff implemented VR-JIT with fidelity after being trained by research staff. Results also suggest preliminary effectiveness of VR-JIT for prison-based services as trainees increased their job interview skills, reduced their job interview anxiety, and had greater odds of competitive employment within 6 months of their community reentry, compared with returning citizens receiving SAU.

Supplemental Material

sj-docx-1-cjb-10.1177_00938548221081447 – Supplemental material for Virtual Reality Job Interview Training for Adults Receiving Prison-Based Employment Services: A Randomized Controlled Feasibility and Initial Effectiveness Trial

Supplemental material, sj-docx-1-cjb-10.1177_00938548221081447 for Virtual Reality Job Interview Training for Adults Receiving Prison-Based Employment Services: A Randomized Controlled Feasibility and Initial Effectiveness Trial by Matthew J. Smith, Brittani Parham, Jamie Mitchell, Shannon Blajeski, Meghan Harrington, Brittany Ross, Jeffery Johnson, Daphne M. Brydon, Jennifer E. Johnson, Gary S. Cuddeback, Justin D. Smith, Morris D. Bell, Robert Mcgeorge, Kyle Kaminski, Aaron Suganuma and Sheryl Kubiak in Criminal Justice and Behavior

Footnotes

AUTHORS’ NOTE:

The authorship team would like to acknowledge the research staff at the University of Michigan’s Level Up: Employment Skills Simulation Lab (), and the teams at the Michigan Department of Corrections Vocational Villages at the Parnall Correctional Facility and Richard A. Handlon Correctional Facility. The authors would also like to thank the study participants for volunteering their time. All authors contributed to the study design and reviewed the protocol. M.J.S and B.P conducted the literature review, conducted the analysis, and drafted the manuscript. Remaining authors provided feedback, contributed to writing, and revised the manuscript. All authors listed meet the authorship criteria according to the latest guidelines of the International Committee of Medical Journal Editors. This study was supported by a grant to Dr. Matthew J. Smith from the Michigan Institute for Clinical & Health Research (UL1TR002240). The University of Michigan, University of Michigan School of Social Work, and Dr. Matthew Smith will receive royalties on sales of an adapted version of Virtual Reality Job Interview Training (VR-JIT) that will focus on meeting the needs of transition-age youth with autism spectrum disorders. Dr. Smith’s research on the effectiveness of the adapted version of VR-JIT is independent of the data reported in this manuscript that reports on the original version of VR-JIT. No other authors report any conflicts of interest.

ORCID iDs

Matthew J. Smith

Brittani Parham

Brittany Ross

Supplemental Material

Supplemental Figures 1–3 and Supplemental Table 1 are available in the online version of this article at

Matthew J. Smith, PhD, LCSW, is an associate professor in the School of Social Work at the University of Michigan and director of the Level Up: Employment Skills Simulation Lab. His research is focused on developing and evaluating the effectiveness and implementation of technology-based interventions to enhance equity in employment for underserved and marginalized groups.

Brittani Parham, MSW, is a PhD student in social work and psychology at the University of Michigan. Her research focuses on evaluating programs and developing interventions that engage vulnerable populations within criminal justice settings. She has experience working as a Juvenile Justice Youth Counselor and Adult Pretrial Probation Officer for the city of Richmond, VA.

Jamie Mitchell, PhD, is an associate professor of social work at the University of Michigan and assistant director of Clinical Research Participation at Rogel Cancer Center. Her research domain focuses on addressing disparities in health and research participation among Black adults with an emphasis on men.

Shannon Blajeski, PhD, is a research associate in the Level Up Lab at the University of Michigan School of Social Work. Her primary research interest is to develop interventions that improve real-world outcomes for young adults with first episode psychosis who are not engaged in education, employment, or training.

Meghan Harrington, BA, is a clinical subjects coordinator in the Level Up Lab at the University of Michigan School of Social Work. Her role includes data collection, processing, and analysis.

Brittany Ross, MS, manages the Level Up Lab at the University of Michigan School of Social Work. She has worked several years as a research specialist, including research design and data base management, and manages multiple federal and private grant-funded research projects.

Jeffery Johnson, LLMSW, builds coordinated care networks as a community engagement manager at Unite Us. He previously served as a community research coordinator in the Level Up Lab where he collected, processed, and analyzed study data.

Daphne M. Brydon, PhD, LMSW, is a clinical social worker and researcher in private practice. Her research focuses on the development of trauma-informed behavioral health interventions and practice for individuals involved in the criminal legal system.

Jennifer E. Johnson, PhD, is the C. S. Mott endowed professor of public health, professor of OBGYN, and professor of psychiatry and behavioral medicine at Michigan State University. She is a licensed clinical psychologist who conducts policy-relevant implementation and effectiveness trials of mental health and substance use interventions for vulnerable populations, including perinatal women and individuals involved in the criminal justice system.

Gary S. Cuddeback, PhD, is a professor and associate dean for research at the School of Social Work at Virginia Commonwealth University. Dr. Cuddeback’s research is focused on mental health services and intervention research and individuals with mental illnesses, especially those who are involved with the justice system.

Justin D. Smith, PhD, is faculty in the Department of Population Health Sciences, Division of Health System Innovation and Research, Spencer Fox Eccles School of Medicine at the University of Utah. He is an implementation scientist and clinical psychologist whose research aims to achieve health equity through delivery of evidence-based interventions to diverse communities and patient populations.

Morris D. Bell, PhD, ABPP, is emeritus professor, Department of Psychiatry, Yale University School of Medicine and senior research career scientist for the Department of Veterans Affairs Rehabilitation Research and Development Service. He is the originator and scientific proponent of the Virtual Reality Job Interview Training program.

Robert McGeorge is the assistant education manager for the Michigan Department of Corrections. He has been highly involved with the creation and implementation of the Vocational Village model now used in three locations across Michigan and replicated by several other states.

Kyle Kaminski is the offender success administrator for the Michigan Department of Corrections (MDOC). In this role, he oversees prisoner programming, education, reentry, and education efforts on a statewide basis in Michigan. He has worked for the MDOC for more than 7 years after starting his career in the Michigan Legislature.

Aaron Suganuma, MSW, is the reentry coordinator for the Washtenaw County Sheriff’s Office. He is actively engaged in direct service and advocacy efforts related to criminal justice. He holds a master’s degree in social work from Eastern Michigan University and completed his associate’s degree while serving 4.5 years in prison.

Sheryl Kubiak, PhD, is the dean of Wayne State University School of Social Work and director of the Center for Behavioral Health and Justice. Kubiak’s 30-year career has focused on improving identification, services access, and intervention focused on the prevention or reduction of consequences associated with behavioral health disorders for those involved in the criminal/legal system.

References

Alper

Durose

M. R.

(2018). 2018 update on prisoner recidivism: A 9-year follow-up period (2005-2014) (NCJ 250975). Office of Justice Programs, Bureau of Justice Statistics. https://www.bjs.gov/index.cfm?ty=pbdetail&iid=6266

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). American Psychiatric Publishing.

Andrews

D. A.

Bonta

(2017). The psychology of criminal conduct (6th ed.). Routledge. https://doi.org/10.4324/9781315677187

Aragon

S. R.

(2003). Creating social presence in online environments. New Directions for Adult and Continuing Education, 100, 57–68. https://doi.org/10.1002/ace.119

Brennan

Dietrich

Ehret

(2009). Evaluating the predictive validity of the COMPAS risk and needs assessment system. Criminal Justice and Behavior, 36, 21–40. https://doi.org/10.1177/0093854808326545

Brooke

(1996). System Usability Scale (SUS): A “quick and dirty” usability scale. In Jordan

P. W.

Thomas

B. A.

Weerdmeester

A. L.

(Eds.), Usability evaluation in industry (pp. 189–194). Taylor & Francis. https://doi.org/10.1201/9781498710411

Burke

S. L.

Bresnahan

Epnere

Rizzo

Partin

Ahlness

R. M.

Trimmer

(2018). Using virtual interactive training agents (ViTA) with adults with autism and other developmental disabilities. Journal of Autism and Developmental Disorders, 48(3), 905–912. https://doi.org/10.1007/s10803-017-3374-z

Bushway

Stoll

M. A.

Weiman

D. F. E.

(2007). Barriers to reentry? The labor market for released prisoners in post-industrial America. Russell Sage Foundation.

Cook

P. J.

Kang

Braga

A. A.

Ludwig

O’Brien

M. E.

(2015). An experimental evaluation of a comprehensive employment-oriented prisoner re-entry program. Journal of Quantitative Criminology, 31(3), 355–382. https://doi.org/10.1007/s10940-

10.

Corbiere

Zaniboni

Lecomte

Bond

Gilles

P. Y.

Lesage

Goldner

(2011). Job acquisition for people with severe mental illness enrolled in supported employment programs: A theoretically grounded empirical study. Journal of Occupational Rehabilitation, 21(3), 342–354. https://doi.org/10.1007/s10926-011-9315-3

11.

Cornet

L. J. M.

Van Gelder

J. L.

(2020). Virtual reality: A use case for criminal justice practice. Psychology, Crime & Law, 26(7), 631–647. https://doi.org/10.1080/1068316X.2019.1708357

12.

Damschroder

L. J.

Aron

D. C.

Keith

R. E.

Kirsh

S. R.

Alexander

J. A.

Lowery

J. C.

(2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4, 50. https://doi.org/10.1186/1748-5908-4-50

13.

Decker

S. H.

Ortiz

Spohn

Hedberg

(2015). Criminal stigma, race, and ethnicity: The consequences of imprisonment for employment. Journal of Criminal Justice, 43(2), 108–121. https://doi.org/10.1016/j.jcrimjus.2015.02.002

14.

Derogatis

L. R.

Savitz

K. L.

(1999). The SCL-90-R, Brief Symptom Inventory, and Matching Clinical Rating Scales. In Maruish

M. E.

(Eds.), The use of psychological testing for treatment planning and outcomes assessment (2nd ed., pp. 679–724). Lawrence Erlbaum.

15.

Elliott

S. N.

(2017). The social validity of “acceptability of behavioral interventions used in classrooms”: Inferences from longitudinal evidence. Behavioral Disorders, 43(1), 269–273. https://doi.org/10.1177/0198742917739021

16.

Ellison

Szifris

Horan

Fox

(2017). A rapid evidence assessment of the effectiveness of prison education in reducing recidivism and increasing employment. Probation Journal, 64(2), 108–128. https://doi.org/10.1177/0264550517699290

17.

Flake

(2015). When any sentence is a life sentence: Employment discrimination against ex-offenders. Washington University Law Review, 93, 46–102. https://doi.org/10.2139/ssrn.2523179

18.

Ghasemi

Zahediasl

(2012). Normality tests for statistical analysis: A guide for non-statisticians. International Journal of Endocrinology and Metabolism, 10(2), 486–489. https://doi.org/10.5812/ijem.3505

19.

Holzer

H. J.

Raphael

Stoll

M. A.

(2006). Perceived criminality, criminal background checks, and the racial hiring practices of employers. The Journal of Law & Economics, 49(2), 451–480. https://doi.org/10.1086/501089

20.

Huffcutt

A. I.

(2011). An empirical review of the employment interview construct literature. International Journal of Selection and Assessment, 19(1), 62–81. https://doi.org/10.1111/j.1468-2389.2010.00535.x

21.

Hunter

Boyce

(2009). Preparing for employment: Prisoners’ experience of participating in a prison training programme. The Howard Journal of Criminal Justice, 48(2), 117–131. https://doi.org/10.1111/j.1468-2311.2008.00551.x

22.

Hurtz

G. M.

Donovan

J. J.

(2000). Personality and job performance: The big five revisited. Journal of Applied Psychology, 85(6), 869–879. https://doi.org/10.1037/0021-9010.85.6.869

23.

Leasure

Kaminski

R. J.

(2020). The impact of a multiple conviction record on hiring outcomes. Crime & Delinquency, 67(6–7), 1022–1045. https://doi.org/10.1177/0011128720973150

24.

Macan

(2009). The employment interview: A review of current studies and directions for future research. Personnel Psychology, 55(1), 1–81. https://doi.org/10.1016/j.hrmr.2009.03.006

25.

McAuley

Duncan

Tammen

V. V.

(1989). Psychometric properties of the Intrinsic Motivation Inventory in a competitive sport setting: A confirmatory factor analysis. Research Quarterly for Exercise and Sport, 60(1), 48–58. https://doi.org/10.1080/02701367.1989.10607413

26.

McCarthy

Goffin

(2004). Measuring job interview anxiety: Beyond weak knees and sweaty palms. Personnel Psychology, 57(3), 607–637. https://doi.org/10.1111/j.1744-

27.

McCroskey

J. C.

(1970). Measures of communication-bound anxiety. Speech Monographs, 37, 269–277. https://doi.org/10.1080/03637757009375677

28.

Metcalf

Anderson

Rolfe

(2001). Barriers to employment for offenders and ex-offenders: Part one. National Institute of Economic and Social Research, Department for Work and Pensions.

29.

Motola

Devine

L. A.

Chung

H. S.

Sullivan

J. E.

Issenberg

S. B.

(2013). Simulation in healthcare education: A best evidence practical guide. AMEE Guide No. 82. Medical Teacher, 35(10), e1511–e1530. https://doi.org/10.3109/0142159X.2013.818632

30.

Newman

S. D.

Andrews

J. O.

Magwood

G. S.

Jenkins

Cox

M. J.

Williamson

D. C.

(2011). Community advisory boards in community-based participatory research: A synthesis of best processes. Preventing Chronic Disease, 8(3), A70. http://www.cdc.gov/pcd/issues/2011/may/10_0045.htm

31.

Newton

Day

Giles

Wodak

Graffam

Baldry

(2018). The impact of vocational education and training programs on recidivism: A systematic review of current experimental evidence. International Journal of Offender Therapy and Comparative Criminology, 62(1), 187–207. https://doi.org/10.1177/0306624X16645083

32.

Owens

J. S.

Lyon

A. R.

Brandt

N. E.

Warner

C. M.

Nadeem

Spiel

Wagner

(2014). Implementation science in school mental health: Key constructs in a developing research agenda. School Mental Health, 6(2), 99–111. https://doi.org/10.1007/s12310-013-9115-3

33.

Paes

Arantes

Irizarry

(2017). Immersive environment for improving the understanding of architectural 3D models: Comparing user spatial perception between immersive and traditional virtual reality systems. Automation in Construction, 84, 292–303. https://doi.org/10.1016/j.autcon.2017.09.016

34.

Peters

(2018, November/December, 18–21). Finding the tools for wellness: A conversation with Colette S. Peters. Corrections Today. https://www.aca.org/common/Uploaded%20files/Publications_Carla/Docs/Corrections%20Today/2018%20Articles/CT_Nov-Dec_2018_Finding-the-Tools-for-Wellness_Colette_S_Peters.pdf

35.

Petersilia

(2005). When prisoners come home: Parole and prisoner reentry. Oxford University Press.

36.

Pham

Y. K.

Unruh

D. K.

Waintrup

Sinclair

Johnson

M. D.

Alverson

C. Y.

(2017). Taking responsibility: Preparing young offenders to handle disclosure on the job. Beyond Behavior, 26(1), 36–41. https://doi.org/10.1177/1074295617694409

37.

Proctor

Silmere

Raghavan

Hovmand

Aarons

Bunger

Griffey

Hensley

(2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services, 38(2), 65–76. https://doi.org/10.1007/s10488-010-0319-7

38.

Ramakers

Apel

Nieuwbeerta

Dirkzwager

Van Wilsem

(2014). Imprisonment length and post-prison employment prospects. Criminology, 52(3), 399–427. https://doi.org/10.1111/1745-9125.12042

39.

Reimers

T. M.

Wacker

D. P.

(1988). Parents’ ratings of the acceptability of behavioral treatment recommendations made in an outpatient clinic: A preliminary analysis of the influence of treatment effectiveness. Behavioral Disorders, 14(1), 7–15. https://www-jstor-org-443.web.bisu.edu.cn/stable/23886135

40.

Ricciardelli

Mooney

(2018). The decision to disclose: Employment after prison. Journal of Offender Rehabilitation, 57(6), 343–366. https://doi.org/10.1080/10509674.2018.1510866

41.

Rosen

C. S.

Drescher

K. D.

Moos

R. H.

Finney

J. W.

Murphy

R. T.

Gusman

(2000). Six- and ten-item indexes of psychological distress based on the symptom checklist-90. Assessment, 7(2), 103–111. https://doi.org/10.1177/107319110000700201

42.

Seim

Harding

D. J.

(2020). Parolefare: Post-prison supervision and low-wage work. Journal of the Social Sciences, 6(1), 173–195. https://doi.org/10.7758/rsf.2020.6.1.08

43.

Smith

M. J.

Bell

M. D.

Wright

M. A.

Humm

Olsen

Fleming

M. F.

(2016). Virtual Reality Job Interview Training and 6-month employment outcomes for individuals with substance use disorders seeking employment. Journal of Vocational Rehabilitation, 44, 323–332. https://doi.org/10.3233/JVR-160802

44.

Smith

M. J.

Fleming

M. F.

Wright

M. A.

Jordan

Humm

L. B.

Olsen

Bell

M. D.

(2015). Job offers to individuals with severe mental illness after participation in virtual reality job interview training. Psychiatric Services, 66(11), 1173–1179. https://doi.org/10.1176/appi.ps.201400504

45.

Smith

M. J.

Fleming

M. F.

Wright

M. A.

Roberts

A. G.

Humm

L. B.

Olsen

Bell

M. D.

(2015). Virtual reality job interview training and 6-month employment outcomes for individuals with schizophrenia seeking employment. Schizophrenia Research, 166(1–3), 86–91. https://doi.org/10.1016/j.schres.2015.05.022

46.

Smith

M. J.

Ginger

E. J.

Wright

M. A.

Humm

L. B.

Olsen

Bell

M. B.

Fleming

M. F.

(2014). Virtual reality job interview training in individuals with psychiatric disabilities. Journal of Nervous and Mental Disease, 202, 659–667. https://doi.org/10.1097/NMD.0000000000000187

47.

Smith

M. J.

Ginger

E. J.

Wright

M. A.

Taylor

J. L.

Humm

L. B.

Olsen

D. E.

Bell

M. D.

Fleming

M. F.

(2014). Virtual reality job interview training in adults with autism spectrum disorder. Journal of Autism and Developmental Disorders, 44(10), 2450–2463. https://doi.org/10.1007/s10803-014-2113-y

48.

Smith

M. J.

Humm

L. B.

Fleming

M. F.

Jordan

Wright

M. A.

Ginger

E. J.

Wright

Olsen

Bell

M. D.

(2015). Virtual reality job interview training in veterans with posttraumatic stress disorder. Journal of Vocational Rehabilitation, 42, 271–279. https://doi.org/10.3233/JVR-150748

49.

Smith

M. J.

Sherwood

Ross

Oulvey

E. A.

Monahan

J. A.

Sipovic

J. E.

Atkins

M. S.

Danielson

E. C.

Jordan

Smith

J. D.

(2022). Scaling out virtual interview training for transition age youth: A quasi-experimental hybrid effectiveness-implementation study. Career Development and Transition for Exceptional Individuals. https://doi.org/10.1177/21651434221081273

50.

Smith

M. J.

Sherwood

Ross

Smith

J. D.

Dawalt

Bishop

Humm

Elkins

Steacy

(2021). Virtual interview training for autistic transition age youth: A randomized controlled feasibility and effectiveness trial. Autism, 25(6), 1536–1552. https://doi.org/10.1177/1362361321989928

51.

Smith

M. J.

Smith

J. D.

Blajeski

Ross

Jordan

Bell

M. D.

McGurk

Mueser

K. T.

Burke-Miller

Oulvey

E. A.

Fleming

M. F.

Nelson

Brown

Prestipino

Pashka

Razzano

L. A.

(2022). An RCT of virtual reality job interview training for individuals with serious mental illness in IPS supported employment. Psychiatric Services. https://doi.org/10.1176/appi.ps.202100516

52.

Smith

M. J.

Smith

J. D.

Fleming

M. F.

Jordan

Brown

C. H.

Humm

L. B.

Olsen

Bell

M. D.

(2017). Mechanism of action for obtaining job offers after virtual reality job interview training. Psychiatric Services, 68(7), 747–750. https://doi.org/10.1176/appi.ps.201600217

53.

Smith

M. J.

Smith

J. D.

Jordan

Sherwood

McRobert

Ross

Oulvey

E. A.

Atkins

M. S.

(2021). Virtual reality job interview training in transition services: Results of a single-arm, noncontrolled effectiveness-implementation hybrid trial. Journal of Special Education Technology, 36(1), 3–17. https://doi.org/10.1177/0162643420960093

54.

Stephan

(2008). Census of state and federal correctional facilities (Report No. NCJ 222182). Bureau of Justice Statistics. U.S. Department of Justice. https://bjs.ojp.gov/content/pub/pdf/csfcf05.pdf

55.

Strickland

D. C.

Coles

C. D.

Southern

L. B.

(2013). JobTIPS: A transition to employment program for individuals with autism spectrum disorders. Journal of Autism and Developmental Disorders, 43(10), 2472–2483. https://doi.org/10.1007/s10803-013-1800-4

56.

Ticknor

(2019). Virtual reality and correctional rehabilitation: A game changer. Criminal Justice and Behavior, 46(9), 1319–1336. https://doi.org/10.1177/0093854819842588

57.

Turney

Lee

Comfort

(2013). Discrimination and psychological distress among recently released male prisoners. American Journal of Men’s Health, 7(6), 482–493. https://doi.org/10.1177/1557988313484056

58.

Twentyman

C. T.

Jensen

Kloss

J. D.

(1978). Social skills training for the complex offender: Employment seeking skills. Journal of Clinical Psychology, 34, 320–326. https://doi.org/10.1002/1097-4679(197804)34:2<320::AID-JCLP2270340211>3.0.CO;2-Y

59.

U.S. Department of Justice. (2021). Returning citizens & reentry. U.S. Attorney’s Office. https://www.justice.gov/usao-ndwv/returning-citizens-reentry

60.

Visher

C. A.

Courtney

(2006). Cleveland prisoners’ experiences returning home. The Urban Institute. https://www.urban.org/research/publication/cleveland-prisoners-experiences-returning-home

61.

Visher

C. A.

Debus-Sherrill

Yahner

(2011). Employment after prison: A longitudinal study of former prisoners. Justice Quarterly, 28(5), 698–718. https://doi.org/10.1080/07418825.2010.535553

62.

Visher

C. A.

Lattimore

P. K.

Barrick

Tueller

(2017). Evaluating the long-term effects of prisoner reentry services on recidivism: What types of services matter? Justice Quarterly, 34(1), 136–165. https://doi.org/10.1080/07418825.2015.1115539

63.

Washington

H. E.

(2018). Creating offender success through education: The Michigan department of correction’s efforts to offer a comprehensive approach to prisoner education and employment. Advancing Corrections: Journal of the International Corrections and Prisons Association, 6, 130–142.

64.

Wells

(2014, November/December 18–19). Training and preparing inmates for post-prison employment. Corrections Today. https://www.ojp.gov/pdffiles1/nij/248574.pdf

65.

Wilson

D. B.

Gallagher

MacKenzie

(2000). A meta-analysis of correctional-based education, vocation, and work programs for adult offenders. Journal of Research in Crime & Delinquency, 37, 347–368. https://doi.org/10.1177/0022427800037004001

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

1.33 MB