Abstract
The application of mainstream intimate partner abuse (IPA) services developed for locally born residents to immigrants and refugee survivors does not take their unique needs, risk factors, and protective factors into consideration and has been documented to cause harm. Given the high rate of immigration to Canada and that IPA is one of the most impactful social determinants of immigrant and refugee health, it is imperative to develop and validate culturally safe and effective programs for Canadian immigrant and refugee survivors. This project’s purpose was to provide an early-stage evaluation of the acceptability, feasibility, and potential effectiveness of a newly developed intervention called the abbreviated Roots of Safety (a-RoS) program using quantitative methods. a-RoS is a two session, culturally safe, survivor-centered, safety-focused, strengths-based, and solution-oriented IPA program for immigrant and refugee women survivors of IPA. Information was analyzed from 22 consecutively admitted survivors who participated in the program and 65 staff who were trained to deliver the program. The results indicated that the program was feasible to implement and very acceptable to staff and survivors. Additionally, there was a large statistically significant increase in immigrant/refugee survivors’ understanding of their internal tools (knowing how they wanted to increase safety and having the tools to achieve their goals). There were also increases across 10 domains in the survivors’ perceived risk and protective factor for IPA. The a-RoS program can be considered a promising practice in need of larger-scale and more rigorous evaluation and research.
Keywords
Intimate partner abuse (IPA) refers to abuse caused by a current or former intimate companion or spouse and includes but is not limited to physical violence, criminal harassment, sexual assault, and emotional/psychological abuse (cf. Government of Canada, 2022). Although the results of a nationally representative sample of 26,000 Canadians showed similar rates of IPA perpetrated against men and against women (Canadian Centre for Justice Statistics, 2000; Lysova et al., 2019), women in Canada are more than twice as likely to be seriously injured as a result of IPA (27% vs. 12%) and much more likely to fear for their or their children’s lives (30% vs. 6%; Statistics Canada, 2002). Nevertheless, these rates mask important subgroup differences. For example, Du Mont and Forte (2012) used population-based sampling and found, contrary to popular stereotypes, that rates of IPA are lower overall among Canadian immigrant women (17.5%) compared to their locally born counterparts (20.3%). In addition, rates of IPA appear higher among immigrant/refugee women relocating from low-to-middle-income countries to Canada compared to immigrants/refugees from high-income countries, with IPA rates increasing for immigrants the longer they reside in Canada (Brownridge & Halli, 2002). Moreover, immigrant women who experience IPA are more likely to have children living at home, resulting in a disproportionate familial impact of their IPA (Du Mont & Forte, 2012).
IPA significantly impairs the ability of immigrants and refugees to become effectively settled in Canada (Okeke-Ihejirika et al., 2020), and the Mental Health Commission of Canada lists exposure to violence among the 15 most detrimental social determinants of their overall health (Kwame, 2016). Given the high migration rate to Canada (Government of Canada, 2022), it is especially important to develop and validate effective IPA programs for Canadian immigrants and refugees. With women representing an increasing proportion of immigrants and refugees in Canada (Okeke-Ihejirika et al., 2020), IPA against immigrant and refugee women is becoming profoundly impactful in this national context.
However, the vast majority of Canadian IPA policies center on a Western paternalistic “rescue and prosecute” strategy that promotes problem-focused methods of action and coping that are often not welcomed by immigrants and refugees (Okeke-Ihejirika et al., 2020). The majority of Canadian IPA policies and services also take more of an individualistic stance and do not tend to focus on nor try to protect the integrity of the larger family unit (including the perpetrator). This individualistic and paternalistic emphasis hinders many immigrant and refugee survivors’ from reporting the IPA to authorities and discourages them from seeking professional help (Du Mont & Forte, 2012; Okeke-Ihejirika et al., 2020). Overall, many mainstream Canadian IPA policies have harmed for immigrant and refugee women and have even prompted an increase in IPA or unintentionally helped perpetuate it (Okeke-Ihejirika et al., 2020; Pottie et al., 2011).
Policies inform the implementation and uptake of services. Overall, immigrant and refugee women in Canada underutilize IPA services (Government of Canada, 2018; Tappis et al., 2016). They possess many characteristics related to their relocation and some distinct risk factors that increase their vulnerability to IPA that locally born women do not tend to have (Choi et al., 2016). The occurrence and impact of IPA and how it manifests are, in some ways, quite distinctive for immigrant and refugee communities compared to the locally born population, including racialized and ethnic minority ones (Choi et al., 2016; Okeke-Ihejirika et al., 2020). Therefore, it should come as no surprise that professional services cannot just adopt mainstream IPA procedures—these are often not culturally safe 1 (Curtis et al., 2019) and are typically less effective with immigrant/refugee women or often not effective at all (Okeke-Ihejirika et al., 2020; Simon-Kumar et al., 2017).
Immigrant and refugee survivors of IPA, especially those who are English-language learners, frequently report that existing IPA services are culturally inappropriate or unsafe and usually ineffective because they are designed for locally born (or Westernized) survivors of IPA (Curtis et al., 2019; Du Mont & Forte, 2012; Okeke-Ihejirika et al., 2020; Simon-Kumar et al., 2017). In addition to these claims of cultural bias, immigrant and refugee survivors of IPA often also report that locally born community members and service providers (a) tend to over-pathologize immigrant and refugee survivors of IPA and (b) over-attribute the abuse to the cultural beliefs, gender roles, and cultural practices that they arrived with from their origin countries (Okeke-Ihejirika et al., 2020). This state of affairs led Okeke-Ihejirika et al. (2020) in their scoping review of IPA with Canadian immigrants to conclude, “the majority of existing services and policies. . .are not well suited to immigrant women’s needs and may undermine women’s capacity to find satisfying solutions” (p. 788). They went on to say, “immigrant’s women’s recourse to emotion-focused strategies, including their reluctance to access help, has primarily to do with the failure of services and policies to provide assistance that corresponds with their specific situations” (p. 802). This is consistent with the Mental Health Commission of Canada’s position that names cultural incompatibility as the main barrier to accessing of services for immigrants and refugees and calls for more culturally adapted practices (Kwame, 2016).
It is not enough to provide culturally adapted and culturally competent care, which is the current emphasis of many IPA services geared toward immigrant and refugee survivors (Curtis et al., 2019). The common three-pillar model of cultural competence for mental health professionals (self-awareness of oneself as culturally conditioned, knowledge about other cultures, culturally adapted skills and interventions) perpetuates othering of those from different cultures, often based on cultural stereotypes and racial/ethnic/cultural essentialism and implies that cultural competence can be eventually achieved with sufficient self-reflection, knowledge, and skills-acquisition (Curtis et al., 2019). But even in light of keen awareness of one’s own specific cultural conditioning, ample cultural knowledge and a culturally adapted intervention skill set, the neglect of power, privilege, and decolonization in the provider–survivor and survivor–country relationship will only constrain the ability to provide culturally safe services to immigrant and refugee survivors (Curtis et al., 2019).
Based on the literature reviewed, what is needed are IPA services that provide culturally safe care for immigrants and refugees, as defined by clients and cultural communities, rather than those that focus on culturally competent service providers offering potentially culturally mismatched services (Curtis et al., 2019). These culturally safe IPA services should foreground self-determination even if it contradicts Western norms and values (including a wish to maintain the collectivistic family unit if desired) and emphasize the decolonial reduction of power differentials within society, including between service providers/organizations and immigrant/refugee survivors. In addition, these services should not only tolerate or accept different cultural beliefs and practices but strive to amplify, celebrate, and promote the cultural identity and well-being of immigrant and refugee survivors (especially those that are incompatible with Western cultural beliefs and practices) while maintaining their physical and emotional safety. The immigrants/refugees’ religion and culture should be mobilized against IPA as much as possible (including culturally appropriate, emotion-focused coping and action strategies; Okeke-Ihejirika et al., 2020) rather than rejected, and the disguised motivation should not be to save the immigrant/refugee from the survivor’s oppressive culture (Simon-Kumar et al., 2017)—like a White Savior (Jailani, 2016). Furthermore, the services should address risk factors and protective factors more characteristic of the immigrant/refugee survivor experience, such as but not limited to lack of knowledge of local government systems, limited English language skills, deportation risks, poverty/financial hardship, local absence of close family members, limited access to cultural community members and resources, and cultural stigma (Sabri, Nnawulezi, et al., 2018; Sabri, Simonet, & Campbell, 2018).
Some pragmatic concerns should also be considered. First, to increase uptake of services (in light of the issues noted) and acknowledging the limited staff, graduate-trained professionals, and financial capacity of many organizations that provide community or IPA services for immigrant and refugee survivors, there is a need for a brief intervention that can be administered by a wide range of service providers, especially paraprofessionals and those without graduate-level training in counseling or equivalent (such as settlement workers). That way, even if the survivor wishes not to or is unable to avail themselves of professional counseling services afterward, demonstrated benefit can still be obtained. Second, the lack of convincing evidence for the effectiveness of IPA programs for immigrant and refugee survivors is contributed to by the fact that most research on immigrant or refugee survivors of IPA in Canada is based on small-scale qualitative interviews or focus groups, with very little quantitative research conducted (which can provide more objective assessments), which is a conspicuous gap in the literature (Okeke-Ihejirika et al., 2020).
Purpose of this Evaluation
The purpose of this early-stage evaluation was to assess the acceptability for implementation (perceived usefulness to staff and confidence in implementing the program), feasibility (ability to enroll and retain clients), and potential effectiveness of the abbreviated Roots of Safety (a-RoS) program using quantitative methods and judge whether it meets the criteria for a promising practice likely to develop into a best practice for an IPA service/program for underserved IPA survivors (Status of Women Canada, 2018). In addition to employing quantitative means, which are being called for in order to provide a more expansive understanding of IPA services in Canada for immigrants and refugees, particularly around objective effectiveness (Okeke-Ihejirika et al., 2020), this study responds to the call of the Mental Health Commission of Canada to provide more research on the mental health outcomes and needs of Canadian racialized refugees and immigrants separate from those of racialized groups born in Canada. They call for this because amalgamating both groups and focusing only on race or ethnicity as an identifying factor ignores highly impactful subgroup differences; thus, they believe this leads to an inaccurate understanding of both racialized immigrants/refugees and racialized but locally born individuals (Kwame, 2016).
Method
Participants
This program evaluation involved the participation of both immigrant/refugee women who were survivors of IPA and the program staff who delivered the program. The participating survivors were 22 women consecutively enrolled in the a-RoS program by a particular nonprofit organization that provides settlement and counseling services to immigrants and refugees. Using Cohen’s (1992) criteria, this sample size is sizeable enough to be able to detect a large effect (d = 0.8) for the standardized outcome measure with a very high statistical power of over 0.9 for a single-group pre/post-test design. Eighty-three staff members also participated by providing the intervention and/or providing feedback on the program and their training to administer it.
Characteristics of IPA Survivors
The demographic questions used in the evaluation were collaboratively decided upon between the evaluators and the community organizations and were selected from the organization’s intake form to prevent additional workload for staff. Given the varied amount of missing data per descriptive variable, participant characteristics reported are based upon valid percentages, that is, on those who provided that data in order to provide information about the sample’s extent of variability. Percentages do not always add up to 100.0% due to rounding to one decimal place. All participants listed “female” as their biological sex, and all identified as women. Participants ranged in age from 20 to 53 years old (M = 38.5, SD = 8.7). The participants’ country of origin included Afghanistan (6.3%), Canada (6.3%), 2 China (6.3%), Kenya (6.3%), Pakistan (6.3%), Syria (6.3%), Taiwan (6.3%), India (18.8%), Iran (18.8%), and Iraq (18.8%). In terms of first language, it was 25.0% Arabic, 25.0% Farsi, 18.8% Punjabi, 12.5% Mandarin, 6.3% Swahili, 6.3% Urdu, and 6.3% English. They had zero to six children (M = 1.3, SD = 1.6). About 45.5% were government-assisted refugees, 27.3% were family class immigrants, and 27.3% were refugee claimants. Participants came to Canada between 1999 and 2020, with the mean year of entry of 2011 (SD = 7.9 years). With respect to the participants’ highest educational background, 58.3% completed a university degree, 33.3% had completed high school, and 8.3% reported “other.” About 45.5% identified as Muslim, 18.2% as Hindu, 18.2% as Christian, and 18.2% as nonreligious. About 90.9% reported an annual personal income below $CAD25,000. For their current relationship status, 36.4% were separated, 36.4% divorced, 18.2% single, and 9.1% married. In terms of comfort speaking English, 35.7% were very comfortable, 21.4% comfortable, 14.3% slightly comfortable, and 28.6% uncomfortable.
Characteristics of Staff
Demographic information was obtained from 65 settlement-related workers or program managers who were provided training in a-RoS (out of 83 who attended the training; 78.3% response rate), immediately after training was completed. In order to minimize the administrative burden on staff and minimize staff-expressed fears of identification in anonymous surveys by managers, only information about cultural background and type of staff position was collected (and individual questionnaires were not shared with managers). These variables were prioritized to confirm the program’s implementation was agency-wide and that staff from various cultural backgrounds received the training to ensure greater program accessibility to a wider range of immigrants and refugees from different cultural backgrounds. With respect to their cultural background, 37.7% of staff identified as Asian, 24.6% identified as Middle Eastern, 6.6% as Punjabi Sikh, 4.9% as Arabian, 3.3% as Canadian, 3.3% as Persian, 3.3% as Indo-Canadian, and 16.4% as “other.” Their job positions were counsellor (23.4%), settlement worker (20.3%), program coordinator (12.5%), newcomer information and support worker (10.9%), case manager (10.9%), manager (7.8%), and “other” (14.1%).
Questionnaire information was also obtained from the same pool of trained staff at the end of the evaluation timeframe (after approximately 13 months of program implementation), and 23 responded (27.7% response rate). With respect to the cultural background of those staff who completed this latter questionnaire, 47.6% identified as South Asian, 38.1% as Middle Eastern, 4.8% as European, 4.8% as African, and 4.8% as Canadian. Their job positions were counsellor (26.1%), settlement worker (26.1%), case manager (8.7%), and program coordinator (8.7%), with 26.1% reporting “other” job positions.
Procedures
Program Development
Through frequent discussion between staff and managers at a particular nonprofit organization, it was determined that there was an unmet need for a culturally safe IPA program that would be able to attract and retain a larger number of local immigrants and refugees who could benefit from such services. For example, although the organization already had IPA counseling programs, they were underattended based on the extent of IPA being reported to or suspected by settlement workers. Fortuitously, at that time, there was a call for funding from Status of Women Canada (2018) to develop and evaluate promising practices to address gaps in support to women survivors of IPA based upon the guiding principles of survivor engagement, being trauma/violence-informed, and being culturally safe. National funding was applied for and obtained to develop such a program specifically to address perceived gaps in culturally safe services for immigrant and refugee women, which included first-language service provision. In the grant application, it was successfully argued that adapting an existing evidence-based program for a different presenting issue would be more likely to become a promising and then best practice for IPA for immigrant and refugee survivors.
The program proposed as the basis for adaptation for immigrant and refugee survivors of IPA was the Signs of Safety program. Signs of Safety is a well-established and evidence-based child protection approach developed in the 1990s in Australia and now utilized in numerous countries in the world. It is designed to address child abuse and child protection in a collaborative, family-focused, and safety-centered manner (Baginsky et al., 2019; Turnell & Murphy, 2017). Over approximately a 1-year period, program managers and senior staff intensely worked in consultation with a certified Signs of Safety program consultant and a contracted psychologist (first author) to adapt the Signs of Safety program for use with immigrant and refugee survivors of IPA. The contracted psychologist also developed and oversaw first-language focus groups in English, Arabic, Farsi, Chinese, and Punjabi with 16 survivors of IPA and with program managers and settlement workers to better identify specific local gaps to inform development of a-RoS. These focus groups specified lack of staff knowledge about and use of culturally safe interventions as well as survivors’ fears about safety planning as two of the biggest limitations of the existing programming available to immigrant and refugee survivors of IPA. In addition, two newly created advisory boards, one involving survivors of IPA who had previously received services at the nonprofit organization and one involving managers from nearby nonprofit organizations that served immigrants/refugees, provided ongoing consultation. Finally, to address potential implementation barriers, regular and ongoing consultations were conducted with the managers of all settlement programs within the organization.
The Program
The overarching aim of a-RoS is to create or improve safety for immigrant and refugee survivors and their children by (a) empowering women and helping them self-assess their safety level, identify their safety needs, and generate client-defined solutions and culturally safe safety planning and (b) addressing risk factors for IPA and strengthening protective factors. The a-RoS program is a survivor-centered, safety-focused, solution-oriented, and strengths-based brief (two-session) program. The program is survivor-centered by adopting and working within the survivors’ worldview and cognitive framework. The program is safety-focused by emphasizing and prioritizing protection of the survivors’ physical and emotional integrity throughout the entire intervention, which is centered around safety planning. The program is solution-oriented by being constantly action-oriented and taking a pragmatic focus on what can be done now or next. Lastly, the program is strengths-based because it transparently draws upon existing survivor resilience, assets, and resources and sets out to increase protective factors against IPA. a-RoS is designed to be administered by general immigration/refugee workers or other paraprofessionals without a graduate degree in a mental health or social services field, ideally in the survivor’s first language, but with supervision or consultation available from a licensed mental health professional. The a-RoS program is manualized as a set of two one-on-one sessions to be administered in person or online. Each session is expected to last 1 to 2 hours, based on the client’s needs. The intended time between the first session and second session is nonstandardized and based on client desire and availability. Due to available staff language fluency, a-RoS was offered in English, Punjabi, Chinese, Spanish, Arabic, Farsi, Korean, and French.
Rather than take a protectionist or paternalistic stance (e.g., try to “save” the survivor), staff delivering the program aimed to develop a more egalitarian working relationship and promote survivor self-determination and choice in risk assessment, safety planning, and solution implementation, even if the worker disagreed with the survivor (within legal reporting and ethical requirements). They strived to be nonjudgmental and nonpathologizing in a culturally safe manner. This could include, for example, appreciating aspects of the survivor’s culture (assuming she does), attempting to mobilize the survivor’s culture and cultural leaders in solutions/safety planning as welcomed by the survivor, and validating any survivor’s wish to maintain the family unit within legal child abuse reporting requirements. A key undergirding focus of the intervention is trying to reduce risk factors and strengthening protective factors for IPA.
Program Implementation
Women who participated in the program were selected based on their responses during a standardized, organization-wide needs assessment questionnaire administered by an intake settlement worker. All women seeking services from the nonprofit organization who directly reported IPA, or who screened positive on the Abuse Assessment Inventory administered in English, Chinese, Arabic, or Spanish (Escribà-Agüir et al., 2016; Khawaja & Hammoury, 2008; McFarlane & Parker, 1994; Tiwari et al, 2007) or translated on the spot to the client’s first language, were automatically admitted to the a-RoS program.
The intervention began with the worker asking select nonjudgmental and culturally safe questions about IPA (without calling it abuse or partner violence, in case the survivor does not understand it as such or is not ready to name the IPA or disclose it) and about risk/protective factors with a focus on appreciative inquiry (Baginsky et al., 2019; Turnell & Murphy, 2017). There was an attempt to match survivors with staff who spoke the same languages. If that was not possible, all possible attempts were made to translate or provide translated questionnaires/measures to survivors and work with translators. The assigned staff assisted survivors in completing various questionnaires (described under “Measures”), which assessed different aspects of their situations. These questionnaires were not only part of evaluating the program but also helped the staff offer more individualized support. In probing about current/past IPA, many questions were focused on objective outcomes and behaviors (e.g., specific injuries, specific weapons used, specific safety behaviors). This mapping of the survivor’s circumstances was further promoted by reviewing the survivor’s answers on the questionnaires administered prior to start of the service. Their responses were used to map out past/present harms, complicated factors, worries, strengths, existing safety, and empowerment goals. This mapping of the survivor’s circumstances was documented on an assessment and planning graphical form (called a Three Columns Framework; Baginsky et al., 2019; Turnell & Murphy, 2017), which is organized around three questions: (a) What are we worried about? (b) What’s working well? (c) What needs to happen?
The Three Columns Framework form was then collaboratively reviewed to help empower the survivor to conduct a risk assessment of herself and her children and create a self-determined plan for safety and empowerment. In support of the above, the worker verbalized a genuine worry statement about the client based on the material in the form using “I” language (e.g., “I am worried. . .”) if the woman could not verbalize her own comparable worry statement. Then the survivor was asked to rate the situation on a subjective Empowerment Scale (“On a scale of 1–10, where 10 means everyone knows the women and children are safe enough for the family to continue independently to support their safety and well-being and 1 means that things are so bad that the women and the children cannot live at home, where do we rate this situation?”). The worker then collaborated with the survivor to identify realistic and achievable goals (“next steps”), which served as a preliminary empowerment plan. At the end of the intervention, survivors completed the same questionnaires again, assessing for any change in their situations, mood, life functioning, and protective/risk factors, and received feedback. The staff considered the information and offered information about cultural and wraparound resources or services that were relevant to the survivor. These could include referral to a general or IPA counseling program or a cultural or community resource designed to reduce risk IPA factors or increase protective factors (such as reduce social isolation/loneliness, increase social support, increase community participation, improve English language skills, improve financial situation or provide monetary resources, etc.).
Training to Administer the Program
Prior to training, individual briefings were conducted with managers of all settlement programs, all of whom previously provided ongoing implementation consultation during the development of a-RoS. Therefore, program managers were already quite familiar with a-Ros prior to the commencement of official training. All settlement workers and their program managers were first provided with two full-days of in-person training focused on skills development particular to the specific techniques and strategies drawn from Signs of Safety (e.g., appreciative inquiry questioning, Three Columns Framework for safety assessment) with substantial opportunity for partner-to-partner role-plays. This was followed by a more procedural 2-hour virtual training. In the latter training, they were introduced to the a-RoS intervention in its entirety and provided instructions on all worksheets and evaluation forms associated with the intervention, including how to administer and score measures. They were also introduced to the consultation team, which included two registered counselors in the organization who provided ongoing consultation and retraining, as needed, to the staff (and these were the two individuals who delivered the training). The individual consultation meetings occurred as requested for individual staff. Group consultation meetings (with all staff of each settlement program) occurred monthly.
Program Evaluation Implementation
The funding obtained by the nonprofit organization to deliver the program required them to work with an external program evaluation team as a condition. The three authors of this article comprised the main program evaluation team through a contract awarded to the first author and were tasked with evaluating both staff training outcomes and client outcomes. During the first session of the a-RoS program, participant demographic information was obtained via questionnaire. A standardized outcome measure was administered prior to and right after completion of the intervention, and a questionnaire about changes in risk and protective factors was also administered at the end of the intervention. The piloting of the program (and data collection) was done during a 13-month time frame. The community organization was responsible for collecting and providing the data to the evaluators.
Measures
Movers
The Measure of Victim Empowerment Related to Safety (MOVERS) was used to assess the objective effectiveness of the program, particularly because one of the program’s two main objectives was to promote survivor empowerment. MOVERS was created through a university–community collaboration that included researchers, survivors of IPA, and staff from numerous IPA programs across the United States. It is a 13-item measure with a 5-point Likert scale (ranging from “never true” to “always true”) that measures IPA survivor safety-related empowerment (“the extent to which a survivor has the internal tools to work toward safety, knows how to access available support, and believes that moving toward safety does not create equally challenging problems”; Goodman, Thomas, & Heimel, 2015, p. 5) with safety loosely defined as a life free from physical and emotional abuse (Goodman, Bennett Cattaneo, et al., 2015; Goodman, Thomas, & Heimel, 2015). It is available in multiple languages, including but not limited to Arabic, Chinese, Farsi, French, Korean, English, and Spanish (these were the versions used in this evaluation; Goodman, Bennett Cattaneo, et al., 2015; Goodman, Thomas, & Heimel, 2015). MOVERS has been recommended not only for individual client assessment and progress tracking but also for program evaluation (Goodman, Thomas, & Heimel, 2015). MOVERS is composed of three subscales: Internal Tools, Expectations of Support, and Trade-Offs.
Due to staff concerns about extra time being added to an already extensive intake and program implementation, only the Internal Tools and Expectations for Support subscales were used (10 items). These two subscales were selected for two reasons: (a) practical implementation reasons, to put the focus on two domains that we believed were more amenable to brief intervention and within the competence of support workers without graduate training in a mental health profession, and (b) the strongest psychometric support was demonstrated for these two subscales in past research. The Internal Tools subscale measures the extent to which a survivor has developed a set of safety-related goals and a belief in their ability to accomplish them. The Expectations for Support subscale measures the survivor’s sense that support is available and accessible.
The MOVERS has demonstrated factorial validity in past research (Goodman, Bennett Cattaneo, et al., 2015; Goodman, Thomas, & Heimel, 2015). Discriminant validity was also demonstrated for the Internal Tools and Expectations of Support subscales with respect to self-efficacy (highest correlation with Internal Tools) and social support (highest correlation with Expectations of Support; Goodman, Bennett Cattaneo, et al., 2015; Goodman, Thomas, & Heimel, 2015). The test developers reported in their validation sample that the highest internal consistency reliability was demonstrated for Internal Tools (α = .88) and Expectations of Support (α = .83; Goodman, Bennett Cattaneo, et al., 2015; Goodman, Thomas, & Heimel, 2015). In independent research using a pre-test/post-test evaluation design, these subscales have demonstrated even higher internal consistency: Internal Tools: α = .94 at intake and α = .82 at follow-up; Expectations of Support: α = .8 at intake and α = .72 at follow-up (Rivera & Prendergast, 2018).
In a functional and feasibility assessment of the measure, Rivera and Prendergast (2018) asked survivors who completed MOVERS if the questions were confusing or hard to understand; 70.3% reported “not at all confusing,” and only 5.1% said “very confusing.” They also found excellent completion rates (98.1% at intake), with 91.9% of clients completing the measure in its entirety (i.e., without overlooking or intentionally skipping a question) at intake and 90.0% at follow-up. These researchers also concluded that the subscales of MOVERS were very sensitive to picking up changes and progress in survivors based on their analysis. As a result of these findings and its alignment with survivors’ subjective understandings of their experience, MOVERS is the only IPA outcome measure recommended by the Colorado Domestic Violence Program and the Center for Policy Research (Rivera & Prendergast, 2018).
Survivor Risk/Protective Factors Questionnaire
The authors created a face valid questionnaire for this study to assess subjective experiences of effectiveness related to the program’s second key objective to increase protective factors and decrease risk factors. This questionnaire asked about changes (since the start of the program) in four protective and six risk factors for IPA. These factors were identified through a locally conducted needs assessment 3 and cross-referenced through a review of extant literature (e.g., Sabri, Nnawulezi, et al., 2018; Sabri, Simonet, & Campbell, 2018) and were deemed to be just as important to address by consulted immigrant/refugee survivors as the IPA directly. The four protective factors asked about were (high) self-confidence, (high) community participation, (high) social support, and (high) family unity. The six risk factors were (high) social isolation, (high) loneliness, (low) English language ability, (poor) financial situation, (high) family conflict, and perceived stigma. At the end of the intervention, the participants were asked if the protective or risk factor was “better,” the “same,” or “worse” since participating in the program.
Staff Training Questionnaire
The authors created a Staff Training Questionnaire using face validity intentions and used it to assess the acceptability of the program and its implementation to service providers. It comprised four questions about (a) how confident they were in using the program procedures after the training, (b) whether the training increased their understanding of immigrant/refugee survivors’ experience of culturally safe resources, (c) whether the training increased their understanding of gaps in culturally safe resources for immigrant/refugee survivors of IPA, and (d) whether the training increased their understanding of IPA protective and risk factors for immigrant/refugee survivors. The confidence question was rated on a 5-point scale (“extremely,” “very,” “moderately,” “slightly,” “not at all”). The other three questions were rated on a 4-point scale (To a great extent, Somewhat, Very little, and Not at all).
Staff Post-Implementation Questionnaire
The authors also created a Staff Post-Implementation Questionnaire using face validity intentions and used it to assess the program’s acceptability. This was administered after the implementation of the intervention at the end of the evaluation time frame. It comprised four questions asking about (a) how confident they felt using the program procedures, (b) whether the program increased their understanding of immigrant/refugee survivors’ experience of culturally safe resources, (c) whether the program increased their understanding of gaps in culturally safe resources for immigrant/refugee survivors of IPA, and (d) whether the program increased their understanding of IPA protective and risk factors for immigrant/refugee survivors. Item scaling mirrored that of the training questionnaire.
Data Analysis
Attendance and completion rates were calculated to assess feasibility. Percentages were used to summarize the responses to acceptability variables. Percentages and a paired t test were used to assess effectiveness using pairwise deletion. IBM SPSS version 29 was used for frequency analysis and t tests to analyze the pre and post measures.
Results
Feasibility and Acceptability
All (100%) of the survivors whose responses initiated the application of the a-RoS program accepted participation in it, and all the survivors who started the program completed it. The average time between the first and second sessions was approximately 3.8 months. However, most survivors (66.7%) had their second sessions with 2 months of their first sessions. Training outcome results are presented in Table 1, describing the impact on confidence in using a-RoS, understanding of culturally safe resources, understanding of gaps in culturally safe services for immigrant/refugee survivors of IPA, and understanding of risk and protective factors for immigrant/refugee survivors.
Staff Responses to Training and Post-Implementation Feedback Questionnaire
Note. Percentages do not always add up to 100.0% due to rounding to one decimal place. RoS = Roots of Safety.
Effectiveness
Descriptive statistics for the two subscales of the abbreviated MOVERS and the results of the paired-samples t test are provided in Table 2. After the women participated in the a-RoS intervention, they reported a statistically significant large increase in their understanding of their internal tools (knowing how they wanted to increase safety and having the tools to achieve their goals). They also reported a statistically nonsignificant small increase in their expectations of support—that is, perceptions that the support they needed, existed and that they could access it.
One-Group t Test for Two Subscales of the Abbreviated MOVERS
Note. MOVERS = Measure of Victim Empowerment Related to Safety.
p < .05.
In terms of risk factors, 18 of the women (78.3%) completed these questions on the after-intervention questionnaire. Out of those who completed the questionnaire, 61.1% of participants felt that their social isolation had improved, 59% felt less lonely, 13.6% reported a better financial situation after the intervention, 31.8% reported less family conflict, 22.3% experienced less stigma related to their IPA situation, and 9.1% reported that their language ability had improved since starting the intervention. In terms of protective factors, 17 women (73.9%) completed these questions on the after-intervention questionnaire. Out of these, 76.5% had higher self-confidence than before they started the program, 17.6% had increased their community participation, 64.7% had experienced better social support in their lives, and 17.6% reported better family unity.
Discussion
The a-RoS program was adapted from an evidence-based intervention for child protection. It responds to various calls for culturally safe immigrant/refugee IPA services that center self-determination and are specifically tailored to the distinct manifestation of IPA in immigrant/refugee communities and the unique needs, risk factors, and protective factors of immigrant and refugee survivors. Responding to barriers to uptake and cost that have hindered development and expansion of IPA services specifically tailored to immigrant and refugee survivors, the a-RoS program can be administered by settlement workers or IPA program staff who are not licensed mental health professionals, although oversight/supervision/consultation with such professionals is highly recommended. It also requires only two sessions between a program staff member and a survivor for implementation. This early-stage program evaluation provides evidence for the acceptability, feasibility, and effectiveness of the a-RoS program.
Feasibility, Acceptability, and Effectiveness
Largely owing to its length (only two sessions) and initiation protocols (procedures get automatically initiated upon a positive screen), the program was feasible to implement and complete with immigrant/refugee survivors of IPA. Recruitment and retention data speak affirmatively to its high feasibility and acceptability to survivors. All of the survivors who admitted IPA during a general intake session or who screened positive on the Abuse Assessment Inventory accepted participation in a-RoS, and all the survivors who started the program completed it.
The program was also acceptable to staff in terms of its usefulness to them and their confidence in their ability to implement it. Staff confidence was high right after training and continued to increase over 13 months of offering the program. At the time of this evaluation, 100% of the staff felt “very confident” or “moderately confident” in their ability to implement the program. There was strong consensus that the program increased staff members’ understanding of culturally safe resources for immigrant/refugee survivors of IPA, from “somewhat” to “a great extent.” There was 100% consensus that the program increased their understanding of gaps in services (not just IPA services but wraparound services) specifically for immigrant/refugee IPA survivors, with about 43% reporting it did “to a great extent.” There was about 99% consensus that the program increased staff members’ understanding of risk and protective factors specific to immigrant/refugee survivors of IPA. The staff found the program highly valuable and that it added to their past knowledge about how best to understand and intervene with immigrant/refugee survivors of IPA.
In terms of the program’s objective effectiveness, there was a large and statistically significant increase in the survivors’ internal tools, that is, their knowing how they wanted to increase safety and believing they had the tools to achieve it. Descriptively, these particular survivors also reported a statistically nonsignificant small increase in their expectations of receiving support. In terms of the program’s subjective effectiveness for risk factors, after completing the program, the survivors reported improvement in all six risk factors (social isolation, loneliness, financial situation, language ability, family conflict, and perceived stigma), ranging from 61% for social isolation to 9.1% for language ability. The program had a bigger experienced impact on psychological and familial/social risk factors (social isolation, personal loneliness, family conflict) than on ones that require a long-term investment such as altering stigma within her culture, language skills, and financial situation. In terms of protective factors, the survivors reported improvement in all four factors (confidence, community participation, social support, and family unity), but most strongly with respect to self-confidence and social support. It is impressive that a two-session, culturally safe and very cost-effective intervention delivered by paraprofessionals (low cost of nongraduate-school-trained staff members, number of sessions) can result in large experienced changes in various risk and protective factors and an objective increase in direction about how to create safety for oneself and one’s children. Research by Rivera and Prendergast (2018) with IPA survivors concluded that improvements in internal tools (as measured by MOVERS) are most associated with crisis intervention, safety planning, advocacy, counseling, mobile advocacy, and immigration assistance. This implies which aspects of the a-RoS program might have most contributed to the large increase in the sense of internal tools to navigate IPA (i.e., safety planning, advocacy, immigration assistance).
Strengths and Limitations of This Program and the Evaluation
The key strength of this evaluation was its adaptation of an existing effective program based on local needs and the concerns of immigrant and refugee survivors of IPA. The local needs-based assessment completed prior to implementation of this program identified survivors’ fear of safety planning and the staff’s lack of formal training in a culturally adapted and culturally safe IPA approach as two of the main shortcomings of current interventions to support immigrant and refugee survivors of IPA. By designing a program based on the results of a local needs assessment, and tailoring the intervention based on the survivors’ needs, a-RoS was able improve the women’s knowledge about their existing resources, thus empowering them to take steps according to their own goals. The results from the staff training validated the major knowledge gaps identified in the needs assessment. The trainings provided for a-RoS helped to close this knowledge gap, ultimately improving service delivery for immigrant and refugee women. Additionally, the implementation of program-wide screening for IPA allowed for broader and more equitable service delivery for immigrant and refugee women requiring services from this community organization. Moving forward, cross-collaboration with other settlement agencies can increase access and services and harmonize the intake and intervention process for survivors.
A novel advantage of conducting a program evaluation like this is that it underscores key prevention domains (especially those related to risk and protection factors) that are not a formal part of many IPA intervention programs. There was also no selection or self-selection bias in the sample, which increases confidence in the widespread applicability and generalizability of the evaluation findings. All immigrant/refugee women seeking services at the particular nonprofit community organization were potential participants, all that screened positive during the project time frame were admitted to the a-RoS program, and all completed it.
In terms of limitations, this evaluation only examined short-term outcomes, and future evaluation should assess the longer-term outcomes of a-RoS. Also, staff members’ fidelity to the RoS program was not formally examined, although cases were discussed at weekly staff or monthly supervision/consultation meetings, which implies that some degree of conformity to procedures was necessary because incorrect procedures were corrected when noticed. In addition, although the sample size (N = 22) was small, the evaluation was still highly statistically powered to detect a large effect of a-RoS; however, statistically significant medium and small impacts of the program are likely to go undetected, which might be the case with Expectations for Support. Regardless, finding at least one objective large effect size (on MOVERS Internal Tools) is a strong justification for expansion and larger-scale evaluation of the program because it has the potential to make a large difference. Nevertheless, the use of a single-group uncontrolled design, which is how services typically are offered in real community practice (no randomization to intervention, no comparison group) and is very externally valid, precludes any definitive conclusion that the program itself fully caused or contributed to the improvements observed. It is therefore still possible that the benefits observed might be fully or partially related to factors unrelated to a-RoS (e.g., life events unrelated to the intervention). However, the extremely short duration of the intervention (two sessions, generally over 1–2 months) makes these alternative explanations less likely. In addition, due to the small sample size, this evaluation did not attempt to look for possible subgroup differences across different types of immigrant/refugees, but past research has shown that immigrants/refugees of different national heritages can show differential effects to interventions (Okeke-Ihejirika et al., 2020). Further, it would be important to know how many of these participants were prompted to subsequently enter full-scale IPA programs or seek out professional counseling programs. Due to the end of the program funding period and lack of agency resources to self-fund such efforts outside the typical scope of a nonprofit services institution, this could not be formally investigated. Finally, the outcome measures used in this evaluation are susceptible to social desirability and response bias, and no formal means were used to attempt to detect honesty or response bias. This is a bigger threat to the survivor outcome measures, as the measures were usually but not always administered by the worker providing the services, but less of a concern with the staff outcomes (acceptability of RoS), which were answered anonymously, and not a concern at all to the assessment of feasibility (which was based on observable data about recruitment and retention).
Future Evaluation
The promising results of this early-stage evaluation substantiate the need for a larger-scale and more rigorous evaluation of a-RoS using a larger group of survivors, a comparison group (either a waitlist-control or services-as-usual group), and ideally randomization, although this is often infeasible or impractical in a community organization setting (but more acceptable in a research lab setting). Future evaluation should also incorporate fidelity checks and a larger battery of standardized outcome measures assessing a variety of functioning and outcomes. Future evaluation should also attempt to determine if the program outcomes are different for newcomers from different parts of the world and track whether their participation prompts future participation in full-service IPA programs or counseling.
Conclusion
Cultural incompatibility and lack of safety are the main barriers to uptake of formal support services for immigrants and refugees in Canada (Kwame, 2016), and many conventional IPA services developed for locally born residents have often proven ineffective or even harmful for immigrant and refugee survivors (Okeke-Ihejirika et al., 2020). a-RoS appears to be an acceptable, feasible, and likely effective brief intervention for immigrant/refugee survivors of IPA. Based on the results of this evaluation, the a-RoS program can be considered a promising practice for an IPA program for underserved immigrant and refugee survivors that shows tremendous potential for developing into a best practice, according to the Status of Women Canada (2018) criteria. Assuming the intervention is further evaluated and continues to demonstrate effectiveness, widespread adoption of a-RoS (such as can be accomplished by a government policy mandating evidence-based programs) can contribute to reducing IPA among immigrant/refugee survivors in Canada. This will inevitably help immigrant/refugee survivors become more effectively settled in Canada. In addition, because immigrant/refugee survivors are more likely to have children living at home than locally born survivors (Du Mont & Forte, 2012), a-RoS holds the potential to have a larger impact on the effect of IPA on children in Canada. Given the continued influx of immigrants and refugees in Canada, and noting that most are coming from non-Western countries (Statistics Canada, 2022), there will be an increasing need for a decolonized, culturally responsive, and culturally safe approach to IPA that is demonstrably effective for immigrant/refugee women. Without greater adoption of culturally adapted and culturally safe approaches like a-RoS, IPA prevalence and frequency in Canada, as a whole, would be expected to increase corresponding to the increase in migration to Canada from non-Western countries.
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was based on a program grant funded by Status of Women Canada with funding awarded to DIVERSEcity Community Resources Society.
