Unpacking Multiagency Structured Professional Judgment Risk Assessments for Family Violence

Abstract

Assessing the risk for future harm is a crucial task for agencies managing Family Violence (FV) cases. The Integrated Safety Response (ISR) is a multiagency collaboration of such agencies operating in two areas of New Zealand, and one of the first steps in their process is to perform a risk assessment. However, in these assessments, it is unclear whether the factors ISR triage team members select are the basis for their overall risk categorization (low, medium, or high), and if those factors are risk factors (i.e., empirical predictors of outcomes). Therefore, in this study we documented the factors ISR triage teams recorded during their risk assessments for 842 FV cases and examined the relationship of those factors with the risk categories. We then investigated whether those factors and the risk categories were indeed capable of predicting FV-related outcomes (recurrence and physical recurrence). We found most of the triage teams’ recorded factors were associated with the risk categories, but fewer than half of the factors were associated with FV-related outcomes. Moreover, the risk categories predicted FV-related outcomes better than chance, but their predictive ability varied across subgroups, performing poorly for aggressors who were Māori or women, and for non-intimate partner cases. We concluded that the ISR triage teams’ risk assessment protocol may benefit from increased structure and validation.

Keywords

family violence intimate partner violence risk factors responsivity field validity risk triage

Agencies responding to Family Violence (FV) cases commonly use risk assessments because they aid practitioner judgments about the likelihood of future harm and can be used to inform their efforts to prevent that harm (Garrington & Boer, 2020; Henning et al., 2021; Robinson & Clancy, 2020; Stanley & Humphreys, 2014). The Integrated Safety Response (ISR) is a multiagency collaboration of organizations responsible for managing FV cases reported to police in two areas of New Zealand (Integrated Safety Response, 2018). Every FV episode reported to New Zealand Police is referred to ISR for further case management, and one of the first actions the ISR takes is to perform a risk assessment. However, it is unclear whether the factors ISR triage team members identify are the basis for the categorical risk level they allocate to cases (low, medium, or high), as well as whether those factors are risk factors (i.e., empirical predictors of criminal outcomes; Bonta & Andrews, 2016). Therefore, in this study we document the factors ISR triage teams recorded for 842 cases and examine the relationship of those factors with the risk categories. We also analyze whether those factors and the risk categories were indeed capable of predicting two FV-related outcomes (recurrence and physical recurrence).

The RNR Model and Risk Assessment by FV Practitioners

Agencies providing FV services often use risk assessments to guide their decision-making (Henning et al., 2021; Robinson & Clancy, 2020; Spivak et al., 2020; Stanley & Humphreys, 2014). In criminal justice settings, risk assessments are designed to predict the likelihood of an individual committing a criminal act in the future (Prins & Reich, 2021). This process improves the ability of law enforcement and other agencies to protect victims and target limited resources at the people who need them most. The risk principle of the Risk Needs Responsivity (RNR) model for correctional treatment states that treatment intensity should be matched to the level of risk offenders pose; with high-risk offenders having the greatest “room for improvement,” thus also receiving the greatest level of intervention (Bonta & Andrews, 2016; Polaschek, 2012). In correctional settings, the RNR principles are commonly used to design and evaluate treatment programs for individual offenders, but the model is also relevant for a wider range of interventions, including family-based interventions (Polaschek, 2016). Translated into the FV context, the risk principle simply indicates that families or whānau¹ with the greatest risk-related needs should be prioritized for effective interventions, and a higher level of service and support (Polaschek, 2016).

Risk assessment instruments are comprised of various risk factors, with the instruments’ ability to predict the criminal outcome being drawn from the statistical association those factors have with that outcome. Importantly, this association is not necessarily causal (Kropp, 2004). Risk factors are understood as measurable characteristics that can be used to predict peoples’ likelihood of engaging in criminal behavior (Bonta & Andrews, 2016; Polaschek, 2012). In addition, victim-focused risk factors can be used to predict peoples’ likelihood of experiencing victimization (Spencer et al., 2019). Many of these characteristics are known as static risk factors, which cannot be changed easily once they have occurred (e.g., age at first offense).

On the other hand, some characteristics used in risk assessments to predict criminal behavior are dynamic risk factors: changeable characteristics of the person or their circumstances (e.g., current substance use; Bonta & Andrews, 2016), with changes in these offending-related needs being followed by changes in the measured criminal outcome (Douglas & Skeem, 2005; Polaschek, 2012). These factors are often psychologically meaningful; they may be causal or act as proxies for the underlying causal mechanisms that lead to criminal behavior (Mann et al., 2010); thus, they are often chosen as targets for change during treatment (Prins et al., 2021).

In line with this idea, a second purpose of risk assessments is to identify information that can be used to inform service provision and case management (Butters et al., 2021; Garrington & Boer, 2020; Kropp, 2008). The responsivity principle from the RNR model states that interventions should be delivered in a way that maximizes people’s ability to engage and benefit from them (Bonta & Andrews, 2016). Agencies play a pivotal role in selecting which interventions families receive (e.g., alcohol or drug rehabilitation, parenting support, financial planning, or nonviolence programs) and providing those interventions in an engaging and accessible manner. The process of assessing risk, which involves reviewing case information, identifying changeable needs that are linked to recurrence, and thinking through future scenarios where harm may occur, serves the dual function of assisting practitioners to plan their response (Butters et al., 2021; Garrington & Boer, 2020; Kropp, 2008).

Types of Risk Assessments

Risk assessment processes differ in the extent to which they are structured. Actuarial instruments are the most structured; they contain a list of risk factors selected for their ability to predict the outcome. These risk factors are scored and combined algorithmically to produce an overall risk score (Geurts et al., 2021). Contrastingly, in unstructured clinical judgment risk assessments, practitioners subjectively evaluate case information and use their expertise in a nontransparent way to identify what they regard as important information, before forming an intuitive impression of the likelihood of future harm (Hilton et al., 2006). Structured Professional Judgment (SPJ) risk assessments sit between these two contrasting methods of assessing risk. SPJ risk assessments typically include empirically derived risk factors, but allow discretion (Hart et al., 2016); practitioners can include information they deem relevant from outside the instrument’s list of risk factors and decide themselves how all the pertinent information (i.e., the identified risk factors) should be weighted to influence the final assessment (Hilton et al., 2006; Kropp, 2004, 2008).

Within SPJ, assessment protocols vary in their procedures and in their predictive validity (Garrington & Boer, 2020; Geurts et al., 2021). Some SPJ risk assessment instruments contain a list of items that are rated (e.g., 0, 1, or 2; absent, somewhat present, or present, respectively; Douglas et al., 2013; Garrington & Boer, 2020; Kropp & Hart, 2015). After completing these ratings, practitioners select a categorical label to communicate the level of risk they think is present in the case (e.g., low, moderate, or high). For research, ratings on SPJ instruments are often summed to produce a total score, and when used this way (i.e., actuarially), their predictive validity is similar to that of actuarial instruments (van der Put et al., 2019). When the risk categories produced using SPJ instruments are used to predict outcomes they tend to have poorer predictive validity than actuarial instruments (e.g., DASH; Richards, 2009; Turner et al., 2019), but this may be in part due to the reduced variance inherent in a three-item scale (e.g., low = 1, medium = 2, or high = 3; Jolliffe Simpson et al., 2021).

Some of the most well-known actuarial or SPJ risk assessment instruments were created to predict reassault, rearrest, or reconviction in intimate partner violence (IPV) cases, and include the Revised Domestic Violence Screening Instrument (Stansfield & Williams, 2014), the Ontario Domestic Assault Risk Assessment (Hilton & Eke, 2017; Hilton et al., 2004), the Spousal Assault Risk Assessment Guide (Kropp & Hart, 2015), and other instruments to predict the risk of intimate partner homicide (e.g., Danger Assessment; Messing et al., 2017). Few validated risk assessment instruments exist for other types of family relationships; a small number have been created for child maltreatment (e.g., California Family Risk Assessment, Dankert & Johnson, 2014; Checklist of Child Safety, van der Put et al., 2016), and some others can be used with all types of FV (e.g., New Zealand Police Dynamic Risk Assessment; Jolliffe Simpson et al., 2021; Victoria Police Screening Assessment for Family Violence Risk; Spivak et al., 2020). In addition to assessing FV risk, some instruments (e.g., SARA; Kropp & Hart, 2015) explicitly inform case management. But despite the availability of these instruments, practitioners often rely on their expertise when assessing risk for FV cases, opting instead to use a less-structured approach (Walklate & Mythen, 2011).

The Integrated Safety Response

Practitioners working in New Zealand’s ISR multiagency crisis response service for FV routinely perform risk assessments based on the SPJ approach. The ISR includes representatives from organizations such as Police, District Health Boards, and nongovernmental FV service providers (e.g., whānau support, nonviolence programs, and women’s refuge). Importantly, Indigenous service providers are partner agencies in the ISR and deliver many of the interventions offered to families. Māori (Indigenous people of New Zealand) are overrepresented in ISR cases, making up around one-fifth of ISR cases but one-tenth of the population in the Canterbury area, and over half of ISR cases but one-fifth of the population in the Waikato area² (Dobbs & Eruera, 2014; Mossman et al., 2017). Accordingly, responsiveness to Māori is an important indicator of the ISR’s performance (Mossman et al., 2017).

One of the first steps in the ISR’s process is for a triage team to assess risk for future FV and assign interventions. Triage teams meet daily to discuss recent cases reported in the ISR catchment area, assess the likelihood and severity of future FV, and collaboratively make a risk categorization (low, medium, or high) using a risk assessment guide (see “Method” section and Supplemental Materials; Integrated Safety Response, 2018). During their assessments, the triage teams do not score the items in the risk assessment guide; instead, they freely select risk factors from the guide and from information held by their respective agencies and record the factors they think are most relevant for the case in a free-text field in a centralized database. In essence, they use SPJ to perform an idiographic formulation of what underpins risk for the case in question.

This Study

In this study, we seek to better understand both the process and the predictive validity of the ISR triage teams’ risk assessments. Importantly, triage teams record factors after agreeing upon a risk categorization for a case, introducing the possibility that they may retrospectively justify their categorization, rather than using the factors prospectively to inform their judgment of risk. Put simply, it is unclear whether the factors that triage teams select are the basis for the risk category they allocate. Therefore, we will document the factors triage teams recorded for 842 cases and examine their frequencies and the relationship of those factors with the risk categories the teams assigned.

In addition, because the triage teams can freely select factors, it is unclear whether the factors chosen are risk factors that can be used to predict the likelihood of future FV (Bonta & Andrews, 2016), and the risk categories’ predictive validity has not previously been explored in research. Accordingly, we will examine the association of the factors with FV-related outcomes (recurrence and physical recurrence) and appraise the discriminative ability of the risk categories for those same outcomes. Because FV cases involve heterogeneous aggressors and family relationships, we will explore the risk categories’ discriminative ability for different subgroups of cases according to the relationship of the aggressor to the victim and by aggressor characteristics.

Method

Data Source

This study used archival data sourced from the ISR’s Family Safety System database for a larger study on FV in New Zealand (Jolliffe Simpson et al., 2021). The ISR is a FV multiagency crisis response service including representatives from New Zealand Police, Ara Poutama Aotearoa (Department of Corrections), District Health Boards, Oranga Tamariki (child protection services), Accident Compensation Corporation, and nongovernmental organizations that provide services to families and whānau experiencing FV (Mossman et al., 2017). All FV episodes reported to police are referred to ISR for further triaging, risk assessment, and case management, enabled by information sharing between the organizations involved. A FV episode was defined as an event involving any form of harm to a person who is in, or has been in, a family relationship with the aggressor (Integrated Safety Response, 2019).

Procedure

New Zealand Police extracted the dataset containing information about each FV episode reported in November and December 2018 in Waikato and Canterbury from the Family Safety System database. We coded the risk factors recorded in ISR case notes for episodes reported in the 2-week period between 1 and 14 November, 2018, which covered 842 cases. We also coded FV-related recurrence outcomes using criteria (see below) from information in the archive about episodes reported in the 24 weeks following the index episode where the index aggressor was again recorded as an aggressor or mutual participant³ (i.e., was not the reported victim).

Risk Factors

Triage teams assessed each case and made a risk categorization (low, medium, or high) assisted by a risk assessment guide (Integrated Safety Response, 2018). The guide included 46 risk factors relating to the victim, aggressor, relationship, children or young people, and practice considerations (see supplemental materials; Integrated Safety Response, 2018). The ISR risk assessment guide is based on risk factors and lethality indicators drawn from the international research literature, but it has not yet been validated or evaluated. Triage teams did not score items per se; instead, team members identified relevant risk factors from the information held by their respective agencies and shared that information to guide group decision-making. After a triage team agreed on a risk category, they recorded the risk factors present in the case in a free-text field and used those risk factors to form a case management plan. Although it is likely that the triage teams prioritized some risk factors over others, it was not possible for us to establish risk factors’ weights from the recorded information, but only their presence. It is the presence of these risk factors that we focus on in the present study.

We initially developed a coding scheme to organize the risk factors listed in the free-text field in the Family Safety System database into 74 dichotomous variables. As expected, many of these variables came from the risk assessment guide; but some recorded factors were not listed in the guide, and we coded these as well. We then removed the variable any mention of child because it overlapped two other variables (vulnerable child and non-biological child). The first author coded the presence or absence of each risk factor for all cases, and to examine inter-rater reliability a postgraduate student with experience in both FV practice and the Family Safety System independently coded 199 cases (23.5% of the sample). We calculated intraclass correlation coefficients and used guidelines from Koo and Li (2015) to interpret them: <.50 = poor, .50–.74 = moderate, .75–.89 = good, and .90–1.0 = excellent. Of the 73 risk factors we examined, 10 had perfect agreement, 21 had excellent agreement, 24 had good agreement, 11 had moderate agreement, and 2 had poor agreement (see supplemental materials for the intraclass correlations). A further five variables were low in frequency and absent in all cases coded for inter-rater reliability, but we left them, because infrequent variables still have the potential to be good risk predictors, and to avoid artificially reducing the heterogeneity of the risk factors. We therefore chose to retain all variables except those with poor agreement, leaving 71 variables for analysis.

FV-Related Outcomes

We recorded two FV-related outcomes. Recurrence was a binary indicator defined as any subsequent FV-related call for police service within 24 weeks after the index episode, where the index aggressor was an aggressor or mutual participant (i.e., not the victim). Physical recurrence was a binary indicator for whether physical harm was recorded in the police report for any episode reported during the 24 weeks following the index episode.

Analytic Plan

We conducted analyses in R (R Core Team, 2013) and in IBM SPSS Statistics v. 27. We used descriptive statistics and cross-tabulations to describe the sample and identify statistically significant associations between demographic characteristics and the ISR triage teams’ risk categories. Next, we used descriptive statistics and cross-tabulations to identify associations between the recorded factors and risk categories.

To discern whether the factors were risk factors (i.e., empirically associated with the FV-related outcomes), we analyzed their associations with recurrence and physical recurrence with point biserial correlations. Then we calculated the area under the receiver operating characteristics curve (AUC) for the risk categories predicting recurrence or physical recurrence over the following 24 weeks. AUC statistics communicate the probability that a randomly selected person with a given outcome (i.e., recurrence or physical recurrence) would have a higher risk classification than a randomly selected person without that outcome. AUCs can range from 0 to 1; values near .5 indicate discriminative ability similar to chance, and values closer to 1 indicate better accuracy (Helmus & Babchishin, 2016). The AUC is one of many statistics used to evaluate predictive accuracy and has several advantages, including being relatively resilient to the base rate of the outcome, and being nonparametric (thus appropriate for ordinal categories; Helmus & Babchishin, 2016). We repeated the AUC analyses for different subgroups of cases according to the relationship of the aggressor to the victim and by aggressor characteristics.

Results

The average age of aggressors was about 33 years with a small proportion being under 18 years old (n = 67, 8.0%), whereas the average age of victims was about 35 years and a similarly small proportion of victims were under 18 years old (n = 55, 6.5%). Around three-quarters of aggressors were men, and around three-quarters of victims were women. Most aggressors and victims were Māori or New Zealand European. Aggressors were most commonly the current partners of victims (half of the sample) followed by former partners, parents, siblings, or other family members. Almost 60% of cases in the sample were in the Waikato region, with the remainder in Canterbury.

Table 1 shows that over half of the cases were classified by an ISR triage team as being at medium risk of future FV outcomes, a third were low risk, and a small proportion were classified as high risk (6.7%). The final two columns of the table show χ² statistics for the relationship between demographic variables and the risk categories, with statistically significant associations at α = .05 present for almost all variables. For example, one in three aggressors were women in the low-risk category, and this proportion became smaller in the higher risk categories. The proportion of victims who were men followed a similar pattern.

Table 1.

Descriptive Statistics for the Sample Overall (N = 842) and by ISR Risk Category.

		Risk Category f (%)
Variable	f (%)	Low	Medium	High	χ²	p
Variable	f (%)	(n = 316)	(n = 470)	(n = 56)	χ²	p
Aggressor gender^a					30.67	<.001
Gender diverse	2 (0.2)	1 (0.3)	1 (0.2)	0 (0.0)
Female	203 (24.1)	107 (33.9)	92 (19.6)	4 (7.1)
Male	637 (75.7)	208 (65.8)	377 (80.2)	52 (92.9)
Aggressor ethnicity					11.70	.020
European	360 (42.8)	144 (45.6)	193 (41.1)	23 (41.1)
Māori	373 (44.3)	119 (37.7)	227 (48.3)	27 (48.2)
Other	109 (12.9)	53 (16.8)	50 (10.6)	6 (10.7)
Relationship to victim					30.49	<.001
Current partner	418 (49.6)	139 (44.0)	242 (51.5)	37 (66.1)
Former partner	177 (21.0)	56 (17.7)	108 (23.0)	13 (23.2)
Parent	42 (5.0)	18 (42.9)	24 (57.1)	0 (0.0)
Child	108 (12.8)	51 (47.2)	51 (47.2)	6 (5.6)
Sibling	51 (6.1)	26 (8.2)	25 (5.3)	0 (0.0)
Other	46 (5.5)	26 (8.2)	20 (4.3)	0 (0.0)
Victim gender					24.54	<.001
Female	648 (77.0)	219 (69.3)	375 (79.8)	54 (96.4)
Male	194 (23.0)	97 (30.7)	95 (20.2)	2 (3.6)
Victim ethnicity					17.12	.002
European	388 (46.1)	150 (47.5)	210 (44.7)	28 (50.0)
Māori	351 (41.7)	111 (35.1)	216 (46.0)	24 (42.9)
Other	103 (12.2)	55 (17.4)	44 (9.4)	4 (7.1)
ISR area					16.50	<.001
Waikato	480 (57.0)	198 (41.3)	263 (54.8)	19 (4.0)
Canterbury	362 (43.0)	118 (32.6)	207 (57.2)	37 (10.2)
Recurrence	389 (46.2)	108 (34.2)	248 (52.8)	33 (58.9)	30.18	<.001
Physical recurrence	174 (20.7)	41 (13.0)	113 (24.0)	20 (35.7)	22.41	<.001
		Risk Category M (SD)
	M (SD)	Low	Medium	High
Aggressor age	33.03 (12.79)	30.54 (12.75)	31.50 (10.65)	31.12 (10.01)
Victim age	34.57 (14.07)	37.55 (16.10)	33.54 (12.29)	32.12 (11.52)

Note. f = frequency; ISR = Integrated Safety Response.

Gender diverse aggressor was excluded from χ² test.

There was also an association between the risk categories and ethnicity, whereby a greater proportion of aggressors who were Māori were in the medium- and high-risk categories, a greater proportion of victims who were Māori were in the medium-risk category, and a greater proportion of victims who were European were in the high-risk category, compared with other ethnicities. Cases in the high-risk category almost exclusively involved IPV, and other family relationships were overrepresented in the low-risk category. A greater proportion of cases in the Canterbury region had higher risk categorizations than in Waikato.

Finally, Table 1 shows the proportion of cases with recurrence or physical recurrence in the 24 weeks following the index episode. Overall, almost half of cases had a recurrence reported to police during this period, with one in five cases experiencing a recurrence in which the aggressor physically harmed the victim. The proportion of cases with recurrence and physical recurrence differed by risk category, with those proportions increasing as the risk category increased.

Table 2 shows the 71 factors coded from the triage teams’ risk assessments, which we organized inductively, based on the sections in the ISR risk assessment guide (see Supplemental Materials; Integrated Safety Response, 2018). Unsurprisingly, the most common factors recorded were types of harm in the index episode: verbal abuse (43.2%), physical harm (31.2%), followed by aggressor FV/criminal history (27.4%). Of note, many other factors were low in frequency, with only 14 factors being present in more than 15% of cases. On an average, there were 6.51 factors present in each case (range = 1–23, SD = 3.50).

Table 2.

ISR Factor Frequency by ISR Risk Category.

		Risk Category f (%)
	f (%)	Low	Medium	High	χ²	p
Aggressor variables
Alcohol use	229 (27.2)	85 (26.9)	129 (27.4)	15 (26.8)	0.03	.983
Breaching/noncomplying with conditions/orders	78 (9.3)	4 (1.3)	65 (13.8)	9 (16.1)	38.79	<.001
Drug use	151 (17.9)	31 (9.8)	108 (23.0)	12 (21.4)	22.76	<.001
FV/criminal history	231 (27.4)	40 (12.7)	168 (35.7)	23 (41.1)	56.19	<.001
Mental health issues	134 (15.9)	48 (15.2)	76 (16.2)	10 (17.9)	0.31	.859
Suicide/self-harm	90 (10.7)	23 (7.3)	63 (13.4)	4 (7.1)	8.22	.016
Victim variables
Alcohol use	99 (11.8)	43 (13.6)	52 (11.1)	4 (7.1)	2.41	.300
Drug use	39 (4.6)	10 (3.2)	26 (5.5)	3 (5.4)	2.47	.291
Fear	97 (11.5)	10 (3.2)	76 (16.2)	11 (19.6)	35.24	<.001
FV/criminal history	212 (25.2)	39 (12.3)	150 (31.9)	23 (41.1)	46.47	<.001
Mental health issues	64 (7.6)	16 (5.1)	41 (8.7)	7 (12.5)	5.65	.059
Suicide/self-harm	24 (2.9)	4 (1.3)	18 (3.8)	2 (3.6)	4.60	.100
Relationship variables
Cohabiting	44 (5.2)	17 (5.4)	23 (4.9)	4 (7.1)	0.54	.765
Custody issues	38 (4.5)	16 (4.7)	22 (4.7)	1 (1.8)	1.04	.595
Denial of end of relationship	29 (3.4)	10 (3.2)	18 (3.8)	1 (1.8)	0.75	.688
Financial stress	92 (10.9)	32 (10.1)	56 (11.9)	4 (7.1)	1.50	.472
Gang affiliations	40 (4.8)	6 (1.9)	26 (5.5)	8 (14.3)	17.57	<.001
Housing problems	70 (8.3)	23 (7.3)	40 (8.5)	7 (12.5)	1.76	.416
Infidelity	30 (3.6)	5 (1.6)	23 (4.9)	2 (3.6)	6.03	.049
Isolation	64 (7.6)	19 (6.0)	36 (7.7)	9 (16.1)	6.86	.032
On/off relationship	39 (4.6)	7 (2.2)	29 (6.2)	3 (5.4)	6.76	.034
Property dispute	28 (3.3)	14 (4.4)	14 (3.0)	0 (0.0)	3.30	.192
Protection order	13 (1.5)	3 (0.9)	9 (1.9)	1 (1.8)	1.18	.554
Separation	157 (18.6)	54 (17.1)	98 (20.9)	5 (8.9)	5.50	.064
Situational stress	163 (19.4)	72 (22.8)	84 (17.9)	7 (12.5)	4.73	.094
Unemployment	42 (5)	11 (3.5)	31 (6.6)	0 (0.0)	7.02	.030
Child/family variables
Breakdown in family dynamics	29 (3.4)	13 (4.1)	16 (3.4)	0 (0.0)	2.43	.297
Collusive family	6 (0.7)	0 (0.0)	6 (1.3)	0 (0.0)	4.78	.091
Complex family dynamics	21 (2.5)	5 (1.6)	15 (3.2)	1 (1.8)	2.14	.344
History of victimization/cumulative harm	35 (4.2)	2 (0.6)	29 (6.2)	4 (7.1)	15.89	<.001
Pregnancy or recent birth	40 (4.8)	6 (1.9)	30 (6.4)	4 (7.1)	9.16	.010
Non-biological children	79 (9.4)	20 (6.3)	52 (11.1)	7 (12.5)	5.67	.059
Vulnerable children	100 (11.9)	24 (7.6)	66 (14.0)	10 (17.9)	9.56	.008
Vulnerable elderly person	11 (1.3)	3 (0.9)	6 (1.3)	2 (3.6)	2.54	.280
Harm variables
Controlling behavior	176 (20.9)	40 (12.7)	117 (24.9)	19 (33.9)	23.27	<.001
Electronic/social media harm	25 (3)	5 (1.6)	17 (3.6)	3 (5.4)	3.90	.142
Harassment	9 (1.1)	2 (0.6)	6 (1.3)	1 (1.8)	1.03	.597
Harmed/threatened to harm, intimidated children, family members, or pets	28 (3.3)	1 (0.3)	26 (5.5)	1 (1.8)	16.43	<.001
Injuries	86 (10.2)	13 (4.1)	65 (13.8)	8 (14.3)	20.54	<.001
Intimidating behavior	123 (14.6)	30 (9.5)	87 (18.5)	6 (10.7)	13.05	<.001
Items thrown	42 (5)	8 (2.5)	33 (7.0)	1 (1.8)	9.34	.009
Nonfatal strangulation	43 (5.1)	0 (0.0)	25 (5.3)	18 (32.1)	101.52	<.001
Physical harm	263 (31.2)	47 (14.9)	187 (39.8)	29 (51.8)	66.40	<.001
Property damage	132 (15.7)	40 (12.7)	82 (17.4)	10 (17.9)	3.49	.174
Psychological abuse	33 (3.9)	6 (1.9)	23 (4.9)	4 (7.1)	6.16	.046
Sexual assault	5 (0.6)	0 (0.0)	4 (0.9)	1 (1.8)	3.76	.152
Stalking	31 (3.7)	8 (2.5)	17 (3.6)	6 (10.7)	8.99	.011
Threats	106 (12.6)	22 (7.0)	67 (14.3)	17 (30.4)	26.34	<.001
Verbal abuse	364 (43.2)	175 (55.4)	171 (36.4)	18 (32.1)	30.79	<.001
Weapons	73 (8.7)	13 (4.1)	52 (11.1)	8 (4.3)	13.92	.001
Episode variables
Aggressor not spoken to	106 (12.6)	34 (10.8)	68 (14.5)	4 (7.1)	3.98	.137
Breach of protection order	28 (3.3)	4 (1.3)	20 (4.3)	4 (7.1)	7.97	.019
Conflicting stories	19 (2.3)	0 (0.0)	18 (3.8)	1 (1.8)	12.63	.002
Delay in police attendance	30 (3.6)	13 (4.1)	17 (3.6)	0 (0.0)	2.35	.308
Episode occurred in public	61 (7.2)	16 (5.1)	37 (7.9)	8 (14.3)	6.65	.036
Left address/not located	57 (6.8)	14 (4.4)	37 (7.9)	6 (10.7)	5.03	.081
Refusing to leave	40 (4.8)	17 (5.4)	23 (4.9)	0 (0.0)	3.09	.213
Uncooperative with police	88 (10.5)	22 (7.0)	56 (11.9)	10 (17.9)	8.47	.014
Victim not spoken to	26 (3.1)	8 (2.5)	17 (3.6)	1 (1.8)	1.08	.582
Practice considerations
Cultural barriers	9 (1.1)	1 (0.3)	6 (1.3)	2 (3.6)	5.20	.074
Denial of FV	18 (2.1)	5 (1.6)	9 (1.9)	4 (7.1)	7.28	.026
Duration of episodes	59 (7)	10 (3.2)	44 (9.4)	5 (8.9)	11.48	.003
Escalation	94 (11.2)	19 (5.7)	70 (14.9)	6 (10.7)	16.13	<.001
Frequent episodes	82 (9.7)	9 (2.8)	68 (14.5)	5 (8.9)	29.07	<.001
Intergenerational FV	22 (2.6)	2 (0.6)	19 (4.0)	1 (1.8)	8.79	.012
Lack of information	184 (21.9)	44 (13.9)	124 (26.4)	16 (28.6)	18.76	<.001
Lack of support	23 (2.7)	2 (0.6)	16 (3.4)	5 (8.9)	14.13	.001
No formal orders	39 (4.6)	14 (4.4)	23 (4.9)	2 (3.6)	0.25	.885
Nonengagement	70 (8.3)	10 (3.2)	47 (10.0)	13 (23.2)	29.06	<.001
Normalization	65 (7.7)	6 (1.9)	52 (11.1)	7 (12.5)	24.21	<.001
Unreported FV	141 (16.7)	20 (6.3)	100 (21.3)	21 (37.5)	48.82	<.001

Note. f = frequency; FV = family violence; ISR = Integrated Safety Response. See Supplemental Materials for factor definitions.

The final two columns of Table 2 show χ² statistics for the relationship between the presence of each factor and the risk categories. More than half of the χ² statistics were statistically significant at α = .05, indicating that most of the factors appeared in different proportions across the three risk categories, with the overall trend being that factors were more commonly present in higher risk cases compared with low-risk cases. These χ² statistics should be interpreted with caution because there is a high chance of false positive results given the large number tests conducted, and we include them simply to illustrate the overall pattern of results.

Table 3 shows relatively few of the factors triage teams recorded were associated with recurrence or physical recurrence. In order from largest to smallest associations, frequent episodes, financial stress, housing problems, nonengagement, aggressor drug use, controlling behavior, escalation, items thrown, victim FV or criminal history, victim mental health issues, and victim drug use were positively associated with recurrence. On the other hand, injuries, physical harm, and aggressor alcohol use were negatively associated with recurrence. Frequent episodes, financial stress, denial of FV, episode occurred in public, items thrown, on/off relationship, housing problems, escalation, lack of information, nonengagement, normalization, unemployment, and victim suicide or self-harm were positively associated with physical recurrence, whereas aggressor alcohol use and custody issues were negatively associated with physical recurrence.

Table 3.

Point Biserial Correlations Between the ISR Factors and Recurrence and Physical Recurrence.

	Recurrence		Physical Recurrence
	r [95% CI]	p	r [95% CI]	p
Aggressor variables
Alcohol use	−.07 [−.14, .00]	.047	−.07 [−.13, .00]	.048
Breaching/noncomplying with conditions/orders	.06 [−.01, .12]	.097	.03 [−.04, .10]	.398
Drug use	.08 [.01, .14]	.027	.01 [−.05, .08]	.691
FV/criminal history	.03 [−.04, .10]	.414	−.03 [−.10, .04]	.367
Mental health issues	.04 [−.03, .11]	.250	−.01 [−.08, .05]	.694
Suicide/self-harm	.04 [−.03, .11]	.226	−.03 [−.10, .03]	.322
Victim variables
Alcohol use	−.04 [−.11, .03]	.219	−.03 [−.10, .04]	.361
Drug use	.07 [.00, .13]	.049	.03 [−.04, .09]	.433
Fear	.04 [−.03, .11]	.262	−.02 [−.09, .05]	.586
FV/criminal history	.08 [.02, .15]	.016	.03 [−.04, .10]	.412
Mental health issues	.08 [.01, .14]	.028	.05 [−.01, .12]	.125
Suicide/self-harm	.03 [−.04, .09]	.428	.07 [.00, .14]	.039
Relationship variables
Cohabiting	.02 [−.05, .09]	.604	.01 [−.06, .08]	.729
Custody issues	−.03 [−.10, .04]	.395	−.07 [−.14, .00]	.047
Denial of end of relationship	−.02 [−.09, .05]	.597	−.03 [−.10, .04]	.353
Financial stress	.14 [.07, .21]	< .001	.14 [.07, .21]	<.001
Gang affiliations	.02 [−.05, .08]	.622	.02 [−.04, .09]	.488
Housing problems	.09 [.02, .16]	.008	.07 [.00, .14]	.044
Infidelity	−.01 [−.08, .06]	.749	−.02 [−.09, .05]	.582
Isolation	.07 [.00, .13]	.053	.01 [−.06, .08]	.804
On/off relationship	.06 [−.01, .12]	.102	.08 [.02, .15]	.016
Property dispute	.01 [−.05, .08]	.682	.00 [−.06, .07]	.919
Protection order	.02 [−.05, .09]	.578	−.06 [−.13, .00]	.064
Separation	.00 [−.07, .06]	.925	.03 [−.03, .10]	.320
Situational stress	.03 [−.04, .10]	.412	−.03 [−.09, .04]	.428
Unemployment	.06 [−.01, .13]	.076	.07 [.00, .14]	.038
Child/family variables
Breakdown in family dynamics	.01 [−.06, .08]	.820	−.05 [−.12, .02]	.163
Collusive family	.01 [−.06, .07]	.852	−.04 [−.11, .02]	.210
Complex family dynamics	−.04 [−.11, .03]	.232	−.06 [−.13, .00]	.068
History of victimization/cumulative harm	.03 [−.03, .10]	.328	.03 [−.04, .09]	.452
Pregnancy or recent birth	.05 [−.02, .12]	.142	.00 [−.07, .06]	.915
Non-biological children	.01 [−.06, .08]	.722	.05 [−.02, .11]	.173
Vulnerable children	.03 [−.04, .10]	.417	−.02 [−.09, .04]	.484
Vulnerable elderly person	.02 [−.05, .09]	.577	.04 [−.02, .11]	.196
Harm variables
Controlling behavior	.08 [.01, .15]	.020	.00 [−.07, .06]	.938
Electronic/social media harm	.02 [−.05, .09]	.555	−.05 [−.12, .01]	.113
Harassment	.00 [−.07, .06]	.916	−.05 [−.12, .01]	.124
Harmed/threatened to harm, intimidated children, family members, or pets	−.04 [−.11, .03]	.258	−.03 [−.10, .04]	.397
Injuries	−.10 [−.17, −.03]	.004	−.03 [−.09, .04]	.437
Intimidating behavior	.00 [−.07, .07]	.973	.02 [−.05, .09]	.534
Items thrown	.07 [.00, .14]	.036	.10 [.03, .17]	.004
Nonfatal strangulation	−.01 [−.08, .06]	.786	.00 [−.07, .07]	.965
Physical harm	−.08 [−.15, −.01]	.021	.02 [−.05, .08]	.627
Property damage	.01 [−.06, .07]	.847	−.04 [−.11, .03]	.217
Psychological abuse	.02 [−.05, .09]	.533	−.01 [−.08, .06]	.720
Sexual assault	−.01 [−.08, .06]	.781	.00 [−.07, .07]	.971
Stalking	−.02 [−.08, .05]	.628	−.02 [−.09, .05]	.526
Threats	.01 [−.05, .08]	.673	−.02 [−.08, .05]	.625
Verbal abuse	−.01 [−.08, .06]	.763	.02 [−.05, .08]	.634
Weapons	−.01 [−.08, .05]	.672	.01 [−.06, .08]	.782
Episode variables
Aggressor not spoken to	.01 [−.06, .07]	.831	.00 [−.07, .07]	.981
Breach of protection order	.07 [.00, .13]	.051	−.01 [−.08, .05]	.709
Conflicting stories	.05 [−.02, .12]	.134	.06 [−.01, .13]	.078
Delay in police attendance	.05 [−.01, .12]	.123	.06 [−.01, .13]	.081
Episode occurred in public	.04 [−.03, .10]	.309	.12 [.05, .18]	.001
Left address/not located	.02 [−.05, .08]	.647	−.01 [−.08, .06]	.792
Refusing to leave	−.02 [−.08, .05]	.631	−.05 [−.11, .02]	.192
Uncooperative with police	.05 [−.02, .12]	.152	.06 [−.01, .12]	.106
Victim not spoken to	−.01 [−.08, .05]	.686	.01 [−.06, .08]	.758
Practice considerations
Cultural barriers	.00 [−.07, .06]	.916	.06 [−.01, .13]	.077
Denial of FV	.04 [−.02, .11]	.200	.13 [.06, .19]	<.001
Duration of episodes	.03 [−.03, .10]	.311	.04 [−.02, .11]	.205
Escalation	.07 [.00, .14]	.036	.07 [.00, .14]	.041
Frequent episodes	.22 [.15, .28]	<.001	.23 [.16, .29]	<.001
Intergenerational FV	.03 [−.04, .09]	.427	.01 [−.06, .08]	.809
Lack of information	.04 [−.03, .11]	.243	.07 [.00, .14]	.040
Lack of support	.01 [−.06, .07]	.874	−.05 [−.12, .02]	.151
No formal orders	.02 [−.05, .09]	.515	.04 [−.03, .11]	.234
Nonengagement	.09 [.02, .16]	.008	.07 [.00, .14]	.044
Normalization	.04 [−.03, .10]	.304	.07 [.00, .14]	.036
Unreported FV	−.01 [−.07, .06]	.833	.02 [−.05, .09]	.515

Note. CI = confidence interval; FV = family violence; ISR = Integrated Safety Response. See Supplemental Materials for factor definitions.

Next, we examined the predictive validity of the ISR triage teams’ risk categories. Table 4 shows that overall, the risk categories predicted recurrence and physical recurrence within the 6-month follow-up period better than chance, but the AUCs were small in magnitude (Rice & Harris, 2005). Table 4 also shows that the risk categories predicted recurrence or physical recurrence for all IPV cases, and for cases between current partners, as well as for aggressors who were male, aged above 18 years, or European. Further, the risk categories predicted recurrence for cases between former partners, and when aggressors were aged below 18 years, and predicted physical recurrence for cases where parents were aggressors toward their children. No other AUCs were statistically significant.

Table 4.

AUCs for the ISR Risk Categories Predicting Recurrence and Physical Recurrence, by Aggressor–Victim Relationship, Aggressor Characteristics, and ISR Area.

		Recurrence			Physical Recurrence
	n	f (%)	AUC [95% CI]	p	f (%)	AUC [95% CI]	p
Overall	842	389 (46.2)	.60 [.56, .63]	<.001	174 (20.1)	.60 [.56, .65]	<.001
Relationship to victim
IPV	595	292 (49.1)	.61 [.56, .65]	<.001	139 (23.4)	.60 [.55, .65]	<.001
Current partner	418	204 (48.8)	.60 [.54, .65]	<.001	105 (25.1)	.61 [.54, .67]	.001
Former partner	177	88 (49.7)	.63 [.55, .71]	.002	34 (19.2)	.58 [.48, .69]	.125
Non-IPV	247	97 (39.3)	.54 [.47, .61]	.285	35 (14.2)	.57 [.47, .67]	.184
Parent	42	10 (23.8)	.58 [.38, .79]	.409	3 (7.1)	.73 [.52, .94]	.030
Child	108	51 (47.2)	.54 [.43, .65]	.493	19 (17.6)	.62 [.49, .75]	.078
Sibling	51	23 (45.1)	.61 [.45, .77]	.178	8 (15.7)	.51 [.29, .73]	.959
Other	46	13 (28.2)	.41 [.23, .59]	.338	5 (10.9)	.37 [.13, .61]	.285
Aggressor characteristic
Male	637	313 (49.1)	.60 [.55, .64]	<.001	144 (22.6)	.61 [.56, .66]	<.001
Female	203	75 (36.9)	.57 [.48, .65]	.117	29 (14.3)	.54 [.43, .65]	.486
Below 18 years	67	29 (43.3)	.64 [.51, .78]	.034	14 (20.9)	.57 [.41, .73]	.370
Above 18 years	775	360 (46.5)	.59 [.55, .63]	<.001	160 (20.6)	.60 [.56, .65]	<.001
Māori	373	199 (53.4)	.55 [.49, .61]	.107	93 (24.9)	.55 [.48, .62]	.155
European	360	159 (44.2)	.59 [.53, .65]	.004	64 (17.8)	.61 [.54, .69]	.003
ISR area
Waikato	480	232 (48.3)	.58 [.53, .63]	.001	104 (21.7)	.59 [.53, .65]	.004
Canterbury	362	157 (43.4)	.62 [.56, .68]	<.001	70 (19.3)	.62 [.55, .69]	.001

Note. AUC = area under the curve; CI = confidence interval; f = frequency; ISR = Integrated Safety Response; IPV = intimate partner violence. See Supplemental Materials for factor definitions.

Notably, the risk categories were no better than chance at predicting recurrence or physical recurrence for aggressors who were women or Māori, and for non-IPV cases, particularly cases between siblings or people with other types of familial relationships. We speculated that one reason why the risk categories performed better for IPV cases than non-IPV cases was because IPV cases may be more likely to recur with the same victim than non-IPV cases, meaning the victim-related factors recorded in assessments remain relevant. Indeed, of the 292 IPV cases with a recurrence, 266 (91.1%) of the first recurrences recorded were again IPV episodes, with most of those (87.7%) involving the same victim as the index episode. In contrast, of the 97 non-IPV cases with a recurrence, 76 (81.4%) were again non-IPV episodes, but with only 47 (48.5%) involving the same victim.

Discussion

In this study we sought to better understand both the process and the predictive validity of the ISR triage teams’ risk assessments. To fulfill this aim we first documented the factors triage teams listed during their risk assessments and examined the proportions of those factors across the risk categories the triage teams assigned. Overall, most of the 71 factors were associated with the risk categories, with many factors being more common among higher risk cases than low-risk cases. This result may suggest the factors triage teams record are the basis for their risk categorizations, but because factors were recorded after the categorization, it could also suggest that teams retrospectively assign a greater number of risk factors to cases in higher risk categories.

We then examined the association between these factors and two outcomes—recurrence and physical recurrence—to discern whether the identified factors can be used to predict the likelihood of future FV (Bonta & Andrews, 2016). We found that frequent episodes, financial stress, items thrown, housing problems, escalation, and nonengagement were risk factors for recurrence and physical recurrence, suggesting they may be particularly important. For example, when these factors co-occur, they may indicate a particularly volatile family environment in need of intervention to prevent further harm. Moreover, for recurrence, aggressor and victim drug use, victim FV or criminal history, victim mental health issues and controlling behavior were significant predictors. There were also several significant predictors for physical recurrence: denial of FV, on/off relationship, unemployment, lack of information, normalization, and episode occurred in public. But overall, fewer than half of the factors triage teams recorded were associated with recurrence or physical recurrence, in line with previous research showing that many of the items in risk assessment instruments did not predict violence (Coid et al., 2011). The findings from this study also support other research suggesting that specific outcomes may have distinct risk factors (Capaldi et al., 2019; Robinson et al., 2018).

Some of the factors triage teams recorded had negative associations with recurrence or physical recurrence, suggesting they could be protective factors or act as proxies for other mechanisms that prevented recurrence (Eisenberg et al., 2022). For physical recurrence, there were several factors with negative associations, including injuries and physical harm, which seem counterintuitive here. We speculate that their presence in the index episode may have led to ISR agencies to prioritize these cases for interventions that prevented recurrence. In addition, one of the possibilities for why aggressor alcohol use was negatively associated with physical recurrence may be that it was a situational factor that helped to precipitate the index episode and was not a stable characteristic of the aggressor (whereas chronic substance use is a well-established risk factor for FV; Capaldi et al., 2019; Timshel et al., 2017). Furthermore, the factor custody issues was a unique protective factor for physical recurrence, with one explanation being that this factor is commonly present in cases where reported episodes are limited to verbal disagreements over parenting arrangements.

Many of the factors recorded in the triage teams’ assessments may add value to those assessments by highlighting barriers to responsivity and assisting practitioners to deliver interventions in ways that maximize engagement and behavior change (Bonta & Andrews, 2016, Polaschek, 2012, 2016), in addition to (or rather than) contributing to the assessments’ ability to predict recurrence (Butters et al., 2021; Kropp, 2008). For example, nonengagement could indicate a family or whānau has not engaged with FV services in the past, causing practitioners to consider alternative ways to foster that engagement. Some factors that may also measure responsivity (e.g., unreported FV, lack of support) were more common in cases with higher risk categorizations, perhaps due to concerns about managing those cases successfully; high-risk cases tend to be more difficult to manage (Tomkins, 2020).

Finally, we examined the discriminative ability of the ISR triage teams’ risk categories and found they predicted recurrence or physical recurrence better than chance with small effect sizes (Rice & Harris, 2005). These AUCs were comparable to the average predictive ability of risk categorizations made using SPJ risk assessment instruments reported in an international meta-analysis of instruments used to predict IPV (mean AUC of .58; van der Put et al., 2019). However, when we investigated the predictive ability of the risk categories for different subgroups based on aggressor–victim relationship and aggressor characteristics, we found that they performed poorly for aggressors who were Māori, or women, and for non-IPV cases. This result echoes previous research on the predictive validity of New Zealand Police’s FV risk assessment instruments, which also performed poorly for women, Māori and non-IPV cases of FV (Jolliffe Simpson et al., 2021).

The finding that the risk categories performed more poorly for non-IPV cases than IPV cases was not surprising. By far the most dominant and widely researched form of FV is physical IPV within heterosexual adult couples, with women as the primary victims; this was also the most common type of FV in our sample. While coding factors, we noticed the triage teams’ assessments were geared toward identifying risk factors for IPV. The risk assessment guide exemplifies this, with an entire section devoted to “relationship risks” (Integrated Safety Response, 2018). Indeed, some of the risk factors that triage teams recorded only applied to IPV (e.g., infidelity, on/off relationship). This characteristic of the assessments is not necessarily bad; but it suggests that research should test existing risk factors’ predictive validity for non-IPV types of FV and answer the question of whether those cases should be assessed using procedures created for IPV cases. Moreover, different types of FV tend to co-occur (Chan et al., 2021); there are additional challenges inherent in assessing risk and responding to families experiencing both IPV and violence against other family members, compared with families experiencing IPV or non-IPV.

That the risk categories performed more poorly when aggressors were women or Māori may be grounds for concern. This result is also unsurprising given most research on risk assessment for FV focuses on men who commit IPV, and there is less research validating risk assessment instruments for women (van der Put et al., 2019). Moreover, there is a lack of research about culturally specific risk factors (Ashford et al., 2022; Mallory et al., 2016), and few instruments have been validated for specific cultural contexts (e.g., Chan, 2012). The poor performance for Māori is especially problematic when considering the ISR’s special commitment to being responsive to Māori communities and including Indigenous service providers (Mossman et al., 2017). However, because we could not account for the level, type, or effectiveness of interventions received after the index episode, it is also possible that interventions were sufficiently effective for these groups to statistically decouple the risk categorizations from the outcomes we measured. Therefore, we need to understand more about what interventions families and whānau receive, and how those services are provided, including whether provision adheres to the RNR principles of effective correctional treatment (Mossman et al., 2017). This examination will enable us to more clearly understand how the triage teams’ assessments inform responses for different groups of people experiencing FV and the extent to which those responses moderate the relationship between the risk categories and recurrence.

Limitations

A clear limitation of this study is how risk factors were recorded. Triage teams freely listed the risk factors they considered relevant to each case, rather than systematically scoring the presence or absence of risk factors from the risk assessment guide. Our observations of the triage team meetings around the time these data were collected and they suggested that most risk factors mentioned in oral discussion by triage team members were recorded in the free-text field in the Family Safety System database; but the teams did not systematically work through and acknowledge or discard each of the risk factors included in the guide. As a result, assessments may not have captured the full range of risk factors present for each case. Because the triage teams typically recorded risk factors after determining the risk category for the case, they may have retrospectively chosen risk factors to justify their categorization. While there was an association between the risk categories and many of the factors, we could not rule out this alternative explanation. This characteristic of their risk assessment process suggests the triage teams’ risk assessment process is relatively unstructured, even though they have a guide that is intended to impose some structure.

In addition, the ISR’s recording procedure meant that to generate data for research we coded the risk factors for each case from a string of text, adding a layer of interpretation and potentially introducing further error that we sought to minimize by measuring inter-rater reliability. Because the triage teams did not score items but simply noted their presence, we also could not infer items’ weightings. Many SPJ risk assessment instruments (e.g., HCR-20; Douglas et al., 2013; SARA; Kropp & Hart, 2015) allow practitioners to weight items (usually from 0 to 2), indicating the degree to which they are relevant and present for the case in question. Some research shows risk factors indicating the potential for serious harm such as weapons, injuries, escalation, and fear may be weighted more strongly by practitioners when they assess risk, compared with other risk factors like custody issues, unemployment, and non-biological children (Perez Trujillo & Ross, 2008; Robinson & Howarth, 2012; Robinson et al., 2018). This weighting of factors is an important aspect of risk assessments that we could not investigate here. Increased structure in the ISR’s risk assessment protocol such as requiring triage teams to formally identify risk factors from the risk assessment guide before their categorization may increase transparency of their decision-making and ensure risk factors are used as the basis for risk categorizations. Moreover, greater transparency would enable formal validation of the triage teams’ risk assessments and may help identify which factors triage teams consider most strongly in their assessments.

We also could not account for interventions following the index episode that may have prevented FV recurrence, and thus moderated the association between the categorizations and recurrence. For example, a high-risk family could engage intensively with effective services and reduce their level of risk substantially. By contrast, a family at medium risk may not be offered or engage with any services. As a result, the initially high-risk family may now be at lower risk than the medium-risk family, weakening the relationship between the pre-intervention risk category and FV outcomes. The ISR was developed to coordinate interventions delivered to families experiencing FV and all cases are assigned at least one intervention (Integrated Safety Response, 2019), although not all families engage with what is offered. Thus, further research that accounts for the number, intensity, and effectiveness of those interventions is necessary before we can fairly evaluate the triage teams’ risk categorizations.

Implications

The results of this research indicate that the ISR triage teams’ risk assessment protocol may benefit from increased structure and validation. We were not able to investigate the predictive validity of the guide created for triage teams’ assessments because it was not used systematically. Therefore, encouraging practitioners to complete the guide may help in improving their assessments, by (a) helping them to systematically consider the presence of all risk factors in the guide, rather than just those that come to mind; (b) creating data that can be used to validate the risk factors more rigorously than what was possible here; and (c) producing weightings that indicate which factors contributed most strongly to their risk categorization. This information would lend itself to the systematic identification of a subset of risk factors associated with recurrence, and an optimal algorithm for combining those factors thus, creating an improved model to guide ISR triage teams’ decision-making.

Conclusion

In this study we documented and examined the ISR triage teams’ factors and risk categorizations for 842 cases, to better understand both the process and the predictive validity of the teams’ risk assessments. Taken together, the results indicate ISR triage teams consider a wide array of factors when assessing risk in a relatively unstructured way. Moreover, many of the factors triage teams recorded were not associated with recurrence or physical recurrence, and the risk categories’ predictive validity varied across subgroups, performing particularly poorly when aggressors were Māori, or women, and for non-IPV cases. Therefore, the triage teams’ risk assessment protocol may benefit from increased structure and validation. Future research should also measure the interventions cases are allocated and engage with, to find whether those interventions mitigate risk for the recurrence of FV.

Supplemental Material

sj-docx-1-jiv-10.1177_08862605221147069 – Supplemental material for Unpacking Multiagency Structured Professional Judgment Risk Assessments for Family Violence

Supplemental material, sj-docx-1-jiv-10.1177_08862605221147069 for Unpacking Multiagency Structured Professional Judgment Risk Assessments for Family Violence by Apriel D. Jolliffe Simpson, Chaitanya Joshi and Devon L. L. Polaschek in Journal of Interpersonal Violence

Footnotes

Acknowledgements

We would like to thank Josephine Ryan, the Integrated Safety Response, and New Zealand Police for supplying the data for this article. We also acknowledge the University of Waikato and the William Georgetti Scholarship for the financial support provided to the first author.

Authors’ Note

Chaitanya Joshi was affiliated with the School of Computing and Mathematical Sciences at the University of Waikato at the time of the research and is currently affiliated with Department of Statistics at the University of Auckland.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interests with respect to the authorship and/or publication of this article.

Funding

The author(s) received no financial support for the research and/or authorship of this article.

ORCID iD

Apriel D. Jolliffe Simpson

Notes

Supplemental Material

Supplemental material for this article is available online.

Author Biographies

Apriel D. Jolliffe Simpson is a Teaching Fellow at Te Puna Haumaru New Zealand Institute for Security and Crime Science and a PhD Candidate at Te Kura Whatu Oho Mauri School of Psychology, University of Waikato. Her research interests include family violence, forensic psychology, crime science, and evidence-based policing.

Chaitanya Joshi is a Senior Lecturer at Rorohiko me ngā Pūtaiao Pāngara School of Computing and Mathematical Sciences, and a Research Associate of Te Puna Haumaru New Zealand Institute of Security and Crime Science, University of Waikato. His research interests include modeling complex real-life problems, including crime, as well as mathematical and computational aspects of Bayesian statistics.

Devon L. L. Polaschek is the Director of Te Puna Haumaru New Zealand Institute for Security and Crime Science, and a Professor in Te Kura Whatu Oho Mauri School of Psychology, University of Waikato. Her research interests include high risk violent and sexual offenders, family violence, hate crime and right-wing extremist violence, psychopathy, imprisonment, desistance, reintegration, and parole.

References

Ashford

L. J.

Spivak

B. L.

Ogloff

J. R. P.

Shepherd

S. M.

(2022). The cross-cultural fairness of the LS/RNR: An Australian analysis. Law and Human Behavior, 46, 214–226. https://doi.org/10.1037/lhb0000486

Bonta

Andrews

D. A.

(2016). The psychology of criminal conduct (6th ed.). Routledge.

Butters

R. P.

Droubay

B. A.

Seawright

J. L.

Tollefson

D. R.

Lundahl

Whitaker

(2021). Intimate partner violence perpetrator treatment: Tailoring interventions to individual needs. Clinical Social Work Journal, 49(3), 391–404. https://doi.org/10.1007/s10615-020-00763-y

Capaldi

D. M.

Low

Tiberio

S. S.

Shortt

J. W.

(2019). Intimate partner violence across the lifespan: Dyadic theory and risk and protective factors. In Geffner

Vieth

Vaughan-Eden

Rosenbaum

Hamberger

White

(Eds.), Handbook of Interpersonal Violence Across the Lifespan (pp. 1–25). Springer.

Chan

K. L.

(2012). Predicting the risk of intimate partner violence: The Chinese risk assessment tool for victims. Journal of Family Violence, 27(2), 157–164. https://doi.org/10.1007/s10896-012-9418-4

Chan

K. L.

Chen

(2021). Prevalence and correlates of the co-occurrence of family violence: A meta-analysis on family polyvictimization. Trauma, Violence and Abuse, 22(2), 289–305. https://doi.org/10.1177/1524838019841601

Coid

J. W.

Yang

Ullrich

Zhang

Sizmur

Farrington

Rogers

(2011). Most items in structured risk assessment instruments do not predict violence. Journal of Forensic Psychiatry & Psychology, 22(1), 3–21. https://doi.org/10.1080/14789949.2010.495990

Dankert

E. W.

Johnson

(2014). Risk assessment validation: A prospective study. http://www.evidentchange.org/sites/default/files/publication_pdf/risk-assessment-validation.pdf

Dobbs

Eruera

(2014). Kaupapa Māori wellbeing framework: The basis for whānau violence prevention and intervention. New Zealand Family Violence Clearinghouse.

10.

Douglas

K. S.

Hart

S. D.

Webster

C. D.

Belfrage

(2013). HCR-20 Assessing Risk for violence: User guide. Mental Health, Law, and Policy Institute, Simon Fraser University.

11.

Douglas

K. S.

Skeem

J. L.

(2005). Violence risk assessment: Getting specific about being dynamic. Psychology Public Policy and Law, 11, 347–383. https://doi.org/10.1037/1076-8971.11.3.347

12.

Eisenberg

M. J.

van Horn

J. E.

van der Put

C. E.

Stams

G. J. J. M.

Hendriks

(2022). Protective factors as uni-or bipolar factors and their incremental validity and accuracy in predicting general recidivism. International Journal of Law and Psychiatry, 81, 101772. https://doi.org/https://doi.org/10.1016/j.ijlp.2021.101772

13.

Garrington

Boer

D. P.

(2020). Structured professional judgement in violence risk assessment. In Wormith

J. S.

Craig

L. A.

Hogue

T. E.

(Eds.), The Wiley Handbook of What Works in Violence Risk Management (1st ed., pp. 145–162). John Wiley & Sons Ltd.

14.

Geurts

Raaijmakers

Delsing

M. J. M. H.

Spapens

Wientjes

Willems

Scholte

R. H. J.

(2021). Assessing the risk of repeat victimization using structured and unstructured police information. Crime & Delinquency. Advance online publication. https://doi.org/10.1177/00111287211047533

15.

Hart

S. D.

Douglas

K. S.

Guy

L. S.

(2016). The structured professional judgement approach to violence risk assessment: Origins, nature, and advances. In Boer

D. P.

Beech

A. R.

Ward

Craig

L. A.

Rettenberger

Marshall

L. E.

Marshall

W. L.

(Eds.), The Wiley Handbook on the Theories, Assessment, and Treatment of Sexual Offending (pp. 643–666). Wiley Blackwell. https://doi.org/10.1002/9781118574003.wattso030

16.

Helmus

L. M.

Babchishin

K. M.

(2016). Primer on risk assessment and the statistics used to evaluate its accuracy. Criminal Justice and Behavior, 44(1), 8–25. https://doi.org/10.1177/0093854816678898

17.

Henning

Campbell

C. M.

Stewart

Johnson

(2021). Prioritizing police investigations of intimate partner violence using actuarial risk assessment. Journal of Police and Criminal Psychology, 36, 667–678. https://doi.org/10.1007/s11896-021-09466-7

18.

Hilton

N. Z.

Eke

A. W.

(2017). Assessing risk of intimate partner violence. In Campbell

J. C.

Messing

J. T.

(Eds.), Assessing Dangerousness: Domestic Violence Offenders and Child Abusers. Springer. https://doi.org/10.1891/9780826133274.0006

19.

Hilton

N. Z.

Harris

G. T.

Rice

M. E.

(2006). Sixty-six years of research on the clinical versus actuarial prediction of violence. The Counseling Psychologist, 34, 400–409. https://doi.org/10.1177/0011000005285877

20.

Hilton

N. Z.

Harris

G. T.

Rice

M. E.

Lang

Cormier

C. A.

Lines

(2004). A brief accurate actuarial assessment for the prediction of wife assault recidivism: The ontario domestic assault risk assessment. Psychological Assessment, 16(3), 267–275. https://doi.org/10.1037/1040-3590.16.3.267

21.

Integrated Safety Response. (2018). Risk Assessment Quick Guide. Unpublished internal document.

22.

Integrated Safety Response. (2019). 12 week review of cases referred to the Integrated Safety Response (ISR) pilot. https://www.police.govt.nz/sites/default/files/publications/isr-12-week-case-review-report.pdf

23.

Jolliffe Simpson

A. D.

Joshi

Polaschek

D. L. L

. (2021). Predictive validity of the DYRA and SAFVR: New Zealand Police’s family violence risk assessment instruments. Criminal Justice and Behavior, 48(10), 1487–1508. https://doi.org/10.1177/0093854821997525

24.

Koo

T. K.

M. Y.

(2015). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine. https://doi.org/10.1016/j.jcm.2017.10.001

25.

Kropp

P. R.

(2004). Some questions regarding spousal assault risk assessment. Violence Against Women, 10, 676–697. https://doi.org/10.1177/1077801204265019

26.

Kropp

P. R.

(2008). Intimate partner violence risk assessment and management. Violence and Victims, 23(2), 202–220. https://doi.org/10.1891/0886-6708.23.2.202

27.

Kropp

P. R.

Hart

S. D.

(2015). SARA-V3: User manual for version 3 of the spousal assault risk assessment guide. Proactive Resolutions.

28.

Mallory

A. B.

Dharnidharka

Deitz

S. L.

Barros-Gomes

Cafferky

Stith

S. M.

Van

(2016). A meta-analysis of cross-cultural risk markers for intimate partner violence. Aggression and Violent Behavior, 31, 116–126. https://doi.org/10.1016/j.avb.2016.08.004

29.

Mann

R. E.

Hanson

R. K.

Thornton

(2010). Assessing risk for sexual recidivism: Some proposals on the nature of psychologically meaningful risk factors. Sexual Abuse: A Journal of Research & Treatment, 22, 191–217. https://doi.org/10.1177/1079063210366039

30.

Messing

J. T.

Campbell

J. C.

Snider

(2017). Validation and adaptation of the danger assessment-5: A brief intimate partner violence risk assessment. Journal of Advanced Nursing, 73, 3220–3230. https://doi.org/10.1111/jan.13459

31.

Mossman

Paulin

Wehipeihana

(2017). Evaluation of the family violence integrated safety response pilot. SUPERU. https://thehub.sia.govt.nz/assets/documents/ISR_pilot_evaluation_FINAL.pdf

32.

Perez Trujillo

Ross

. (2008). Police response to domestic violence. Journal of Interpersonal Violence, 23(4), 454–473. https://doi.org/10.1177/0886260507312943

33.

Polaschek

D. L. L.

(2012). An appraisal of the risk-need-responsivity (RNR) model of offender rehabilitation and its application in correctional treatment. Legal and Criminological Psychology, 17(1), 1–17. https://doi.org/10.1111/j.2044-8333.2011.02038.x

34.

Polaschek

D. L. L.

(2016). Responding to perpetrators of family violence. New Zealand Family Violence Clearinghouse. https://nzfvc.org.nz/sites/nzfvc.org.nz/files/NZFVC-issues-paper-11-responding-perpetrators.pdf

35.

Prins

S. J.

Reich

(2021). Criminogenic risk assessment: A meta-review and critical analysis. Punishment & Society, 23(4), 578–604. https://doi.org/10.1177/14624745211025751

36.

R Core Team. (2013). R: A language and environment for statistical computing. R Foundation for Statistical Computing. http://www.R-project.org/

37.

Rice

M. E.

Harris

G. T.

(2005). Comparing effect sizes in follow-up studies: ROC area, Cohen’s d, and r. Law and Human Behavior, 29, 615–620. https://doi.org/10.1007/s10979-005-6832-7

38.

Richards

(2009). Domestic abuse, stalking and harassment and honour based violence (DASH, 2009) risk identification and assessment and management model. https://vawnet.org/material/domestic-abuse-stalking-and-harassment-and-honour-based-violence-dash-2009-risk

39.

Robinson

A. L.

Clancy

(2020). Systematically identifying and prioritising domestic abuse perpetrators for targeted intervention. Criminology & Criminal Justice, 21(5), 687–704. https://doi.org/10.1177/1748895820914380

40.

Robinson

A. L.

Howarth

(2012). Judging risk. Journal of Interpersonal Violence, 27(8), 1489–1518. https://doi.org/10.1177/0886260511425792

41.

Robinson

A. L.

Pinchevsky

G. M.

Guthrie

J. A.

(2018). A small constellation: Risk factors informing police perceptions of domestic abuse. Policing and Society, 28(2), 189–204. https://doi.org/10.1080/10439463.2016.1151881

42.

Spencer

C. M.

Stith

S. M.

Cafferky

(2019). Risk markers for physical intimate partner violence victimization: A meta-analysis. Aggression and Violent Behavior, 44, 8–17. https://doi.org/10.1016/j.avb.2018.10.009

43.

Spivak

McEwan

Luebbers

Ogloff

(2020). Implementing evidence-based practice in policing family violence: The reliability, validity and feasibility of a risk assessment instrument for prioritising police response. Policing and Society, 31(4), 483–502. https://doi.org/10.1080/10439463.2020.1757668

44.

Stanley

Humphreys

(2014). Multi-agency risk assessment and management for children and families experiencing domestic violence. Children and Youth Services Review, 47, 78–85. https://doi.org/10.1016/j.childyouth.2014.06.003

45.

Stansfield

Williams

K. R.

(2014). Predicting family violence recidivism using the DVSI-R: Integrating survival analysis and perpetrator characteristics. Criminal Justice and Behavior, 41, 163–180. https://doi.org/10.1177/0093854813500776

46.

Timshel

Montgomery

Dalgaard

N. T.

(2017). A systematic review of risk and protective factors associated with family related violence in refugee families. Child Abuse & Neglect, 70, 315–330. https://doi.org/10.1016/j.chiabu.2017.06.023

47.

Tomkins

(2020). High-risk victims of intimate partner violence within the Integrated Safety Response pilot: An examination of psychosocial stressors and repeat victimisation. University of Waikato. https://researchcommons.waikato.ac.nz/handle/10289/14141

48.

Turner

Medina

Brown

(2019). Dashing hopes? The predictive accuracy of domestic abuse risk assessment by police. The British Journal of Criminology, 59(5), 1013–1034. https://doi.org/10.1093/bjc/azy074

49.

van der Put

C. E.

Assink

Stams

G. J. J. M

. (2016). Predicting relapse of problematic child-rearing situations. Children and Youth Services Review, 61, 288–295. https://doi.org/10.1016/j.childyouth.2016.01.002

50.

van der Put

C. E.

Gubbels

Assink

(2019). Predicting domestic violence: A meta-analysis on the predictive validity of risk assessment tools. Aggression and Violent Behavior, 47, 100–116. https://doi.org/10.1016/j.avb.2019.03.008

51.

Walklate

Mythen

(2011). Beyond risk theory: Experiential knowledge and ‘knowing otherwise’. Criminology & Criminal Justice, 11, 99–113. https://doi.org/10.1177/1748895811398456

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB