The Predictive Validity of Three Youth Violence Assessment Instruments: The SAVRY,VRS-YV,and SAPROF-YV

Abstract

This study examined the predictive validity of three assessment instruments for violent youth, the SAVRY, the VRS-YV, and the SAPROF-YV. Files relating to a sample of 233 young male offenders aged between 12 and 18 years were retrospectively reviewed to score each instrument, with reoffending data extracted from prison, community and juvenile justice records. The analysis showed that all three assessments predicted general (any) reoffending better than violent and non-violent reoffending, with higher rates of predictive validity for the SAVRY and VRS-YV Total Scores. There were, however, only small to moderate effect sizes at 1- and 3-year follow up periods for all three instruments in predicting all types of reoffending. The SAPROF-YV had a small effect size for the prediction of violent reoffending and did not add incrementally to the predictive validity (for violent reoffending) of the SAVRY or VRS-YV.

Keywords

SAVRY VRS-YV SAPROF-YV risk assessment predictive validity

Introduction

Instruments that have been specifically designed to assess the likelihood of future violence in youth are increasingly being used to guide decision making in relation to those young people who are known to have acted violently (Shepherd et al., 2014a). Given the potentially significant consequences of these decisions, for both the youths being assessed and for the community more broadly, determining how accurately these instruments can identify youths who reoffend is a critical issue for their continued use (Pedersen et., 2010). This study reports predictive validity data for three specialist youth violence assessment instruments for a sample of young offenders in Singapore.

Structured youth violence assessment instruments are specifically designed to help assessors better understand the risks that are posed by particular individuals (Skeem & Monahan, 2011). They are based on evidence that more structured approaches typically have greater predictive validity than those that rely solely on unaided clinical judgment (Brown & Singh, 2014; Guy et al., 2012). They are also focused on assessing areas of risk that are both static (factors which are typically grounded in an individual’s history and do not change over time) and dynamic (factors which are potentially modifiable through intervention). Static risk factors have been shown to have good predictive validity for recidivism (e.g., Kroner et al., 2007; Quinsey et al., 1998; Tengstrom, 2001), while dynamic factors are particularly good predictors of short-term (within 6 months) reoffending (e.g., Beech, et al., 2002; Wilson et al., 2013). Recent studies have also identified the potential benefit of assessing protective factors, defined as those factors that either promote resilience or enhance desistance from reoffending (de Vries Robbé & de Vogel, 2013). It has been suggested that this results in a more balanced view of the individual and a more positive focus for any intervention (de Vries Robbé & de Vogel, 2013; Fortune & Ward, 2017), with one recent study also concluding that patterns of risk and protective factors have a direct relationship with risk (Li et al., 2019). Furthermore, incremental predictive validity of violence risk assessments which also consider protective factors has been demonstrated in some studies with adults (de Vries Robbé et al., 2013, 2015a). There is, however, relatively little published research relating to the assessment of protective factors in young people.

Although more than 400 different violence risk assessment instruments have been identified for use with adults, youth violence risk assessment instruments are much less common. A recent systematic review of the published literature on youth violence risk assessment instruments (Koh et al., 2020) identified a total of six different instruments that have been used to assess violence risk in youth. Of these, only two have been designed to assess risk for a period longer than 24 hrs: the Structured Assessment of Violence Risk in Youth (SAVRY; Borum et al., 2006) and the Violence Risk Scale-Youth Version (VRS-YV; Wong et al., 2004–2011). Although little empirical evidence has been reported about the predictive validity of instruments that assess protective factors for youths, new instruments such as the Structured Assessment of Protective Factors-Youth Version (SAPROF-YV; de Vries Robbé et al., 2015b) have been developed.

Structured Assessment of Violent Risk in Youth

One of the most commonly used youth violence risk assessment instruments is the Structured Assessment of Violence Risk in Youth (SAVRY; Borum et al., 2002, 2006; Singh et al., 2011, 2014). Designed to assess risk of future violence of youths aged between 12 and 18 years, the SAVRY is a structured professional judgment instrument with 10 static items, 14 dynamic risk items and six protective factor items. It has been tested in a variety of countries and it is now used in various settings (e.g., community, incarcerated, education, forensic inpatient, and forensic outpatient treatment services; see Koh et al., 2020).

Results from various studies suggest that the SAVRY has a small to large effect size for the prediction of violent reoffending (Chu et al., 2016; Dolan & Rennie, 2008; Hilterman et al., 2014; Lodewijks et al., 2008; McGowan et al., 2011; Zhou et al., 2017). However, most of the studies reporting large effect sizes were conducted in Europe and North America (AUC = 0.69–0.86); with the Singaporean (Chu et al., 2016) and Chinese (Zhou et al., 2017) studies reporting smaller effect sizes (AUC = 0.63–0.65 and AUC = 0.68 respectively). The authors of the Singapore study observed that the SAVRY Total Score was lower than that of Western studies, and the SAVRY Protective Total Score was higher. They surmised that this could be the due to differences in comparison groups, differences in operationalization and enforcement of legislations, and jurisdictional differences in the seriousness of youth crime (Chu et al., 2016).

There is, therefore, a need for further research to explore the generalizability and utility of these assessment instruments in different countries and with different cultural groups. Further, although there have been several studies that have examined the predictive validity of the SAVRY, most of these studies have measured predictive validity against other general risk assessment instruments (or measures of psychopathy) which were not specifically designed to appraise violence risk. Of note, although the SAVRY was originally designed to assess violence risk, a number of studies have reported that it is also able to predict general reoffending (Catchpole & Gretton, 2003; Chu et al., 2016; Dolan & Rennie, 2008; Penney et al., 2010; Shepherd et al., 2014b). This is unsurprising since risk factors for different types of offending overlap and most offenders are generalists. Although there may be some benefit in assessing for violence risk specifically, there is additional benefit in assessing risk for other criminal behaviors as these may cause harms to other people and society. Furthermore, any form of recidivism may result in a breach of a court order; hence, it is important to assess risk for violence specifically, but also to ascertain the predictive validity of a risk assessment instrument for all or any type of criminal behavior.

The Violence Risk Scale-Youth Version

The Violence Risk Scale-Youth Version (VRS-YV; Wong et al., 2004–2011) is closely modeled on the Violence Risk Scale (VRS; Wong & Gordon, 1999–2003); a widely used actuarial violence risk assessment instrument designed for use with adults. The VRS-YV focuses on the assessment of dynamic risk factors; compared to the SAVRY, the VRS-YV only has four static items but 19 dynamic items. The VRS-YV is also designed to measure change post-treatment using a modified application of the Transtheoretical Model of Change (Prochaska et al., 1992). To date, only one study has examined the predictive validity of the VRS-YV. Stockdale et al. (2014) report that the VRS-YV demonstrated moderate to high predictive accuracy (r = .27–.48, AUC = 0.65–0.78) for general, non-violent and violent recidivism in a sample of Canadian youths.

The Structure Assessment of Protective Factors-Youth Version

The Structured Assessment of Protective Factors-Youth Version (SAPROF-YV; de Vries Robbé et al., 2015b) was designed to assess protective factors for violence risk in youth and is intended to complement violence risk-oriented assessment instruments. It is also based on an adult assessment tool, the Structured Assessment of Protective Factors (SAPROF; de Vogel et al., 2012), with pilot studies reporting that the SAPROF-YV scores are negatively correlated with SAVRY risk factors, and positively correlated with the SAVRY protective factors (de Vries Robbé et al., 2015b).

Li et al. (2019) examined the protective ability of factors on the SAPROF-YV for young offenders in Singapore. Multivariate analyses found that only four factors (future orientation, attitudes towards agreements and conditions, bonding to school/work, and pedagogical climate) predicted the non-completion of court ordered community sentences, with two items (peers and other supportive relationships) having little protective effects. These authors concluded that there is currently only indirect evidence of the validity of the SAPROF-YV. However, another study conducted in Singapore found that the SAPROF-YV was able to significantly predict general recidivism, and had incremental predictive validity when considered with a general risk/needs assessment and case management instrument (Chu et al., 2020). They suggested that the SAPROF-YV would be useful in assisting professionals in using a strength-based approach in intervention efforts.

The Singapore Context and the Use of Risk Assessment Instruments

Although the use of violence assessment instruments has grown internationally (Singh et al., 2014), most of the existing research in this area has been conducted in Western countries and there is a need for validation in other countries. Singapore is an independent-island state in South East Asia with a total population of 5.7 million (Singapore Department of Statistics, 2019). Singapore has a crime rate of 587 cases per 100,000 population, with youth arrests accounting for 8.16% of total arrests (National Youth Council Singapore, 2019; Singapore Department of Statistics, 2018). Singapore shares similarities in the way offences are defined with many of the commonwealth countries and many statues are based on English common law (e.g., The Law Revision Commission, 2012; see also Chu & Zeng, 2017). In the early to mid-2000s, youth justice agencies in Singapore began to use risk assessment instruments to assess the risk and needs of youth offenders (see e.g., Chua et al., 2014 for further information regarding use of risk assessments in Singapore). The SAVRY has been used for violence risk assessments for some time, with preliminary results suggesting moderate predictive validity (AUC_Total = 0.65 and AUC_{Risk Rating} = 0.63; Chu et al., 2016).

Study Aim

The overall goal for this study is to explore the predictive validity of the SAVRY, VRS-YV, and SAPROF-YV for various forms of re-offending over different follow up periods. This is important information for those who are seeking to identify the most parsimonious assessment battery, especially given the differences in both the structure and content of the available instruments. The SAVRY has, for example, more static risk factors and fewer dynamic risk factors than the VRS-YV, and the SAVRY includes the assessment of protective factors, whereas the VRS-YV does not. The SAPROF-YV on the other hand, only focuses on the identification of protective factors. Furthermore, determining whether a combination of risk and protective instruments leads to incremental improvement in predictive accuracy is important, as is examining the predictive validity of these instruments in a non-Western context.

Methods

Source Sample

Data was extracted for 233 male Singaporean youth aged between 12 and 18 years (M = 15.55; SD = 1.19) who had been sentenced to either probation or to a juvenile detention facility as a result of a criminal conviction. All youth had been referred for psychological assessment and/or treatment to the Clinical and Forensic Psychology Service (CFPS) at the Ministry of Social and Family Development in Singapore, between January 2008 and December 2012 following an act of violent behavior (e.g., physical assault, and making threats to harm). The overwhelming majority (96%) had been involved in behavior that contravened the legal code; 35.6% (n = 83) had been charged with a violent offense (e.g., physical assault, robbery, armed robbery, causing bodily harm, making threats to harm or kill, rioting, and unlawful use of weapon), 30.5% (n = 71) had been charged only with a non-violent offense (e.g., theft, burglary, drug use, drug trafficking, and fraud) but they had a history of violent behaviors, and 30% (n = 70) had been charged with both violent and non-violent offenses. The remainder (3.9%, n = 9) were under child protection orders but also had a history of violent behaviors.

Procedure

The retrospective nature of the study meant that this study relied on archival case file information. Approval to access the files, collate deidentified data, and report findings was obtained from a university (de-identified for blind review) Human Research Ethics Committee as well as from the Ministry of Social and Family Development in Singapore. The first author coded all three assessment instruments (i.e., the SAVRY, VRS-YV, and SAPROF-YV) using case file data. Sources included: (a) pre-sentencing reports prepared by probation officers; (b) caseworker reports; (c) charge sheets; (d) statement of facts; (e) school reports; (f) violent incident reports; and (f) violence risk assessment reports prepared by psychologists. Items were not scored if the information necessary to code was not available.¹

To check inter-rater reliability, a team of six psychologists, each trained in the scoring of the three risk assessment instruments, independently rated 36 randomly selected files. Intraclass correlation coefficients (ICCs; two-way random, single rater, absolute agreement) were then calculated for total risk scores for each of the assessment instruments. These were found to be within the acceptable range, with ICCs of 0.73 (“good”), 0.68 (“good”), and 0.48 (“fair”) for the SAVRY, VRS-YV, and SAPROF-YV respectively (see Cicchetti, 1994 for guidance on how ICCs are classified).

Instruments

Structured Assessment of Violent Risk in Youth

The SAVRY contains 24 risk items which are spread over three risk domains (Historical, Social/Contextual, and Individual) and six items in the Protective domain. Each risk item is coded on a 3-point scale (“low,” “moderate,” and “high”), with Protective items rated as either “present” or “absent,” according to specific rating guidelines. A SAVRY Risk Rating of “low,” “moderate,” or “high” risk of violence recidivism, was assigned by the rater following consideration of all risk and protective factors present. This is a professional judgment, not based on any particular score or range of scores. In the present study, the SAVRY ratings were assigned numerical values, where “0” was low, “1” was moderate, and “2” was high. The scores of the risk items were then summed to attain a SAVRY Total Risk Score. The items in the SAVRY Protective Domain were rated as either present or absent and were assigned numerical values “1”and “0” respectively, with a total score achieved by summing those factors rated as present. The SAVRY manual advises against using actuarial cut-offs in clinical decision making and the SAVRY Total Risk Score is used for research purposes only.

The Violence Risk Scale-Youth Version

The VRS-YV comprises 23 risk items each of which are rated on a 4-point scale from 0 to 3 according to the descriptions of the construct measured by each item (with higher ratings indicating stronger links between the item and violence or risk of future violence). For this study, a VRS-YV Total Score was attained by summing the scores of all the items. The manual of the VRS-YV did not provide risk bands, however, a validation paper published by the authors of the instruments (Stockdale et al., 2014) indicated that they categorized Total Scores below 35 as “low,” 35–49 as “moderate,” and above 50 as “high.” The VRS-YV Risk Ratings in this study are based on these categories. For the purposes of this study, only the pre-treatment scores and ratings were used, the treatment change scores were not considered.

The Structure Assessment of Protective Factors: Youth Version

The SAPROF-YV is a 17-item instrument that focuses on the identification and rating of protective factors for violence. It has four domains (Resilience, Motivational, Relational, and External) and items within these domains are rated on a 3-point scale (from 0 to 2), with higher ratings indicating a “clear presence” of the item. In this study, a SAPROF-YV Total Score was derived by summing the scores of the items. This was calculated for research purposes only. The overall SAPROF-YV Protection Rating is made after consideration of all the protective factors and a clinical judgment protection rating is made by the rater on a 5-point scale (“low,” “low-moderate,” “moderate,” “moderate-high,” and “high”). For the purposes of this study, the SAPROF-YV Protective Rating is then allocated into “low,” “moderate,” and “high” risk where those who were rated as “low” and “low-moderate” were allocated into the “low” risk bin, “moderate” and “moderate-high” were allocated into the “moderate” risk bin, and those who were rated as “high” were left in the “high” risk categories.

Reoffending Data

Official reoffending data were obtained for all participants after the SAVRY, VRS-YV, and SAPROF-YV instruments had been scored, with reoffending rated as “absent” or “present”. The mean follow-up period was 2,877.68 days (SD = 491.07, range = 1,584–3,623); the end of the follow-up period was December 31, 2017.

General recidivism was defined as a conviction and associated sentence leading to probation, juvenile detention or imprisonment for any offense after the initial court order. The term violent recidivism was used to refer to a conviction and associated sentence leading to probation, juvenile detention or imprisonment for any violent act (e.g., physical assault, rioting, and robbery) following the initial court order. This study adopted the definition of violence provided in the SAVRY manual, “an act of battery or physical violence that is sufficient severe to cause injury to another person or persons, regardless of whether the injury occurs. . .or a threat made with a weapon in hand” (Borum et al., 2002, p. 19). Nonviolent recidivism refers to a conviction and associated sentence leading to probation, juvenile detention or imprisonment due to nonviolent offences (e.g., theft, fraud, drug use, and drug trafficking) that were committed following the initial court order.

Plan of Analysis

The sample was first characterized using descriptive statistics, with categorical data reported as numbers and percentages, and continuous data presented in relation to the mean, standard deviation and range. The predictive validity of the SAVRY, the VRS-YV, and the SAPROF-YV was assessed using correlations and the AUC of the Receiver Operating Characteristics (ROC). AUCs of 0.56, 0.64 and 0.71 are generally be considered to represent “small,” “moderate,” and “large” effect sizes respectively (Rice & Harris, 2005). The predictive validity of the SAVRY and the VRS-YV Total Risk Scores and Risk Ratings were based on the presence of reoffending. However, as a higher protective score would arguably mean increased protection from risk of reoffending, the outcome examined for the SAVRY Protective Scores and the SAPROF-YV was the absence of reoffending. A robust non-parametric test for correlated measures (DeLong et al., 1988) was used to compare the AUCs of ROC Curves of the different instruments. Cox regression analyses were also conducted to examine the incremental predictive validity of the risk assessments (the SAVRY and the VRS-YV) with the addition of the SAPROF-YV. Most of the analyses were conducted with SPSS 21.0, with MedCalc 14.8.1 used for comparisons of the Receiver Operating Characteristic (ROC) curves.

Results

Of the 233 male youths, 138 (59.2%) reoffended; 38 (16.3%) committed only violent offenses, 46 (19.7%) committed non-violent offenses only, and 54 (23.2%) committed both violent and non-violent offenses. Reoffending rates were also recorded at 1- and 3-year follow up periods (see Table 1).

Table 1.

Recidivism Rates.

	One year	Three years
General	36 (15.5%)	86 (36.9%)
Violent	24 (10.3%)	48 (20.6%)
Non-violent	18 (7.7%)	52 (22.3%)

Correlational Analyses

Table 2 shows the correlations between the ratings for the risk and protective assessment instruments. Correlation analyses indicated robust associations between total scores on all three instruments, with significant positive correlations observed between the SAVRY and VRS-YV (Total Score and Risk Rating). Both the SAVRY and VRS-YV (Total Score and Risk Rating) were significantly negatively correlated with the SAPROF-YV (Total Score and Protective Rating). The SAVRY Protective Domain, SAPROF-YV Total Score and SAPROF-YV Protective Rating were all significantly positively correlated with each other. The SAVRY Protective Domain and the SAPROF-YV (Total Score and Protective Rating) were significantly negatively correlated with the SAVRY and the VRS-YV (Total Scores and Risk Ratings).

Table 2.

Correlations between SAVRY, VRS-YV and SAPROF-YV Ratings and Recidivism Outcomes.

	SAVRY Total score	SAVRY Risk rating	SAVRY Protective total	VRS-YV Total score	VRS-YV Risk rating	SAPROF-YV Total score	SAPROF-YV Protective rating
SAVRY Total score	–
SAVRY Risk rating	0.80**	–
SAVRY Protective total	−0.44**	−0.35**	–
VRS-YV Total score	0.86**	0.73**	−0.42**	–
VRS-YV Risk rating	0.61**	0.64**	−0.24**	0.72***	–
SAPROF-YV Total score	−0.65**	−0.48**	0.71**	−0.63***	−0.44**	–
SAPROF-YV Protective rating	−0.55**	−0.36**	0.62**	−0.57***	−0.39**	0.87**	–

p < .05. **p < .01. ***p < .001.

ROC Analysis

The predictive validity of the SAVRY and VRS-YV for general, violent and nonviolent recidivism was then examined. The Total Scores for the SAVRY and VRS-YV were associated with large effect sizes (AUCs = 0.72 and 0.71 respectively) when predicting general recidivism, and moderate effect sizes for predicting violent (AUCs = 0.64) and nonviolent reoffending (AUCs = 0.70 and 0.68). The Risk Ratings for the SAVRY had moderate effect sizes for general (AUCs = 0.65) and nonviolent (AUCs = 0.65) reoffending but had small effect sizes for violent reoffending (both AUCs = 0.58). The Risk Ratings for the VRS-YV had small effect sizes for general, violent and non-violent reoffending (AUCs = 0.60, 0.58, and 0.58 respectively). Critical ratio z analyses showed that the AUCs of the SAVRY Total Score and VRS-YV Total Score were not significantly different for general (z = 0.92, p = .36), violent (z = 0.03, p = .97), and nonviolent (z = 1.01, p = .31) reoffending. Similarly, the AUCs of the SAVRY Risk Rating and VRS-YV Risk Rating were not significantly different for violent reoffending (z = 0.21, p = .83). However, they were significantly different for nonviolent (z = 2.87, p = .00) and general (z = 2.35, p = .00) reoffending. The SAVRY Protective Domain was associated with small effect sizes for predicting the absence of general, violent, and non-violent reoffending (AUCs = 0.60, 0.56, and 0.52 respectively). For the SAPROF-YV, the Total Score and Protective Rating had moderate effect sizes (AUCs = 0.70 and 0.69 respectively) for the absence of general reoffending, and small effect sizes for the absence of violent (AUCs = 0.62 and 0.63 respectively) and non-violent (both AUCs = 0.61) reoffending (see Table 3).

Table 3.

AUC Figures for SAVRY, VRS-YV and SAPROF-YV.

	M (SD)	Range	General	Violent	Non-violent
SAVRY Total Score	21.09 (6.154)	4–39	0.72***	0.64***	0.70***
SAVRY Risk Rating	1.23 (.529)	0–2	0.65***	0.58*	0.65***
SAVRY Protective Domain	1.81 (1.512)	0–6	0.60**	0.56	0.52
VRS-YV Total Score	28.38 (8.98)	3.14–55.41	0.71***	0.64***	0.68***
VRS-YV Risk Rating	.23 (.44)	0–2	0.60**	0.58*	0.58*
SAPROF-YV Total Score	9.64 (4.88)	2–24	0.70***	0.62**	0.61**
SAPROF-YV Protective Rating	1.06 (.794)	0–4	0.69***	0.63***	0.61**

Note. Analysis for SAVRY Protective Domain and SAPROF-YV AUCs are for the prediction of the absence of reoffending.

p < .05. **p < .01. ***p < .001.

The predictive validity of the instruments was examined at 1- and 3-follow-up periods (Table 4). At 1-year follow up, almost all the instruments had small effect sizes for predicting general, violent and non-violent recidivism (AUCs = 0.50–0.63), and only the SAVRY Total Score had moderate effect size (AUC = 0.66) for non-violent reoffending. The SAVRY Protective Domain and the SAPROF-YV (Total score and Protective Rating) also had small effect sizes (AUCs = 0.56–0.58) for predicting the absence of general, violent and non-violent reoffending. At the 3-year follow up, the SAVRY Total Score had moderate effect sizes (AUCs = 0.64–0.67) and SAVRY Risk Rating had small effect sizes for predicting all types of recidivism (AUCs = 0.58–0.62). The VRS-YV Total Score had moderate effect sizes for general reoffending (AUC = 0.66), but small effect sizes for violent and non-violent reoffending (both AUCs = 0.63). The VRS-YV Risk Rating also had small effect sizes for all types of reoffending (AUCs = 0.58–0.69). The SAVRY Protective Domain and the SAPROF-YV (Total Score and Protective Rating) had similar findings, having small effect sizes for the prediction of the absence of all types of reoffending (AUCs = 0.50–0.62).

Table 4.

AUC figures for 1- and 3-year’s recidivism.

	1 year			3 years
	General	Violent	Non-violent	General	Violent	Non-violent
SAVRY
Total score	0.61*	0.56	0.66*	0.67***	0.64**	0.64***
Risk rating	0.55	0.50	0.63	0.62***	0.58	0.62**
Protective domain	0.56*	0.56	0.58	0.54*	0.56*	0.50*
VRS-YV
Total score	0.57	0.53	0.63	0.66***	0.63**	0.63**
Risk rating	0.54	0.52	0.60	0.59*	0.59	0.58
SAPROF-YV
Total score	0.59	0.59	0.58	0.62**	0.59	0.59
Protective rating	0.58	0.59	0.55	0.60**	0.60	0.56

Note. Analysis for SAVRY Protective Domain and SAPROF-YV AUCs are for the prediction of the absence of reoffending.

p < .05. **p < .01. ***p < .001.

Incremental Predictive Validity of the Risk Instruments with the Addition of SAPROF-YV Scores

Cox regression analyses were used to examine the incremental predictive validity of the risk instruments ratings (i.e., SAVRY or VRS-YV) when combined with the SAPROF-YV. The SAPROF-YV Total Score did not significantly contribute to the incremental predictive validity of the SAVRY Total Score for violent (β = −0.01, SE = 0.03, Exp(B) = 0.99 [CI = 0.93, 1.05], p = .79), non-violent (β = 0.02, SE = 0.03, Exp(B) = 1.02 [CI = 0.96, 1.08], p = .59) and general recidivism (β = −0.02, SE = 0.03, Exp(B) = 0.98 [CI = 0.93, 1.03], p = .41). The SAPROF-YV Total Score did not significantly contribute to the incremental predictive validity of the VRS-YV Total Score for violent (β = −0.02, SE = 0.03, Exp(B) = 0.99 [CI = 0.93, 1.05], p = .62), non-violent (β = −0.00, SE = 0.03, Exp(B) = .99 [CI = 0.94, 1.05], p = .90), and general recidivism (β = −0.03, SE = 0.02, Exp(B) = .97 [CI = 0.92, 1.02], p = .18). The SAPROF-YV Protective Rating did not significantly contribute to the incremental predictive validity of the SAVRY Risk Rating for violent (β = −0.01, SE = 0.14, Exp(B) = .92 [CI = 0.75, 1.29], p = .92), non-violent (β = .08, SE = 0.13, Exp(B) = 1.09 [CI = .84, 1.41], p = .53) and general recidivism (β = −0.00, SE = 0.11, Exp(B) = 0.99 [CI = .79, 1.25], p = .97). The SAPROF-YV Protective Rating did not significantly contribute to the incremental predictive validity of the VRS-YV Risk Rating for violent (β = −0.15, SE = 0.13, Exp(B) = 87 [CI = 0.67, 1.12], p = .27), non-violent (β = −0.11, SE = 0.13, Exp(B) = 90 [CI = 0.70, 1.15], p = .38) and general recidivism (β = −0.15, SE = 0.11, Exp(B) = 0.86 [CI = 0.70, 1.06], p = .15).

Discussion

This study compared the predictive validity of three different youth violence assessment instruments using data relating to a sample of Singaporean youth. The two violence risk assessment instruments, the SAVRY and the VRS-YV Total Scores, predicted all types of recidivism (general, violent and nonviolent) to a similar extent, despite differences between the instruments in the proportion of static and dynamic factors assessed and the presence of items measuring protective factors. The comparable predictive validity for violence and the significant positive correlations observed between the SAVRY and VRS-YV may therefore suggest that these two instruments are measuring a similar underlying construct. This is unsurprising considering they share common items such as antisocial attitudes, antisocial peers and substance use.

The findings of this study also confirm those of previous studies that have reported that the SAVRY Total Score has better predictive validity than Risk Ratings alone (Chu et al., 2016; Hilterman et al., 2014; Perrault et al., 2017; Viljoen et al., 2016), and this was similar for the VRS-YV Total Score and Risk Ratings. It would also appear that, despite being specifically designed for the assessment of violent reoffending, these instruments are better at predicting general reoffending than future violence in Singaporean youth. This may, of course, reflect the presence of overlapping risk factors for general offending and violent behavior, with many young offenders committing both violent and non-violent offenses during the follow-up. In addition, violence risk assessment instruments do not take into account the degree or type of violence, and the relevance of the assessed factors to the commission of violence which may impact recorded recidivism rates (Klepfisz et al., 2016). Nevertheless, within the Singapore context, the results of this study do suggest that the SAVRY and VRS-YV may be interchangeable in the assessment of future risk of violent behaviors in youth.

Adding complexity to the assessment of violence risk in youth is the inclusion of protective factors. When assessing protective factors, the SAPROF-YV did have slightly better overall predictive ability as well as at 1- and 3-year follow up periods than the SAVRY Protective Domain which may be due to the more comprehensive appraisal of protective factors in the SAPROF-YV assessment. The SAPROF-YV had better predictive ability for general recidivism than for violent recidivism, however it was noted to be lower than that of another Singapore study (Chu et al., 2020) which examined a sample of youth general offenders and found a large effect size (AUC = 0.80) for the predictive validity SAPROF-YV. These may indicate that (similar to its risk assessment counterparts) protective factors measured on the SAVRY and SAPROF-YV encompass other areas of life which also impact on general offending behavior rather than violent behavior alone; after all, many of the protective factors included in this instrument are not specific to violent behavior and potentially simply reflect opposite poles of risk (see Klepfisz et al., 2017). The lack of incremental predictive validity when the SAPROF-YV was considered with either of the risk focused instruments suggests that the consideration of protective factors may not add to the assessment of risk of future violence. This suggests that there is a need to better understand how protective factors impact on violence risk (i.e., if there are promotive, hazard or mixed protective factors directly related to violence, and/or the booster or buffering effects that specific protective factors provide; see Li et al., 2019).

Implications for Practice

There is an ongoing need to further validate these specialist violence risk and protective assessment instruments if similarities and differences in the development of violent behavior across different cultures is to be better understood. If these assessment instruments lack validity then identifying more useful approaches should be a priority. With the modest predictive ability that is evident for these assessment instruments in a non-Western context compared to many Western studies, it would appear that there may be relevant factors within non-Western contexts that may not be considered in the assessment of violent reoffending on these instruments. For instance, the cultural context in which violent behavior and offending occurs, and how these behaviors are developed and reinforced may warrant further consideration; cultural norms influence thoughts and social expectations of behavior based on shared belief systems which are highly influential in shaping individual behavior, and are standards for appropriate and inappropriate social interactions amongst people (World Health Organization [WHO], 2009). Although some individuals may have different attitudes from normative beliefs in their cultural contexts, individuals may conform to the governing social and cultural norms because they believe that they are expected to (Lilleston et al., 2017). The WHO (2009) identified a variety of cultural and social norms in various countries which impacts the use of violence, for instance, it is suggested that “physical punishment is an acceptable or normal part of rearing a child (e.g., Turkey, Ethiopia)” and “physical violence is an acceptable way to resolve conflicts within a relationship (e.g., South Africa, China)” (p. 5). The acceptance of such violent behaviors modeled through developmental years is thought to increase the risk of developing attitudes which condone violence (see Akers, 1998; Bandura, 1971). The lack of tolerance for different cultures or social groups, where one social group propagates intense dislike and stereotyping of a perceived different group (e.g., ethnicities), can further encourage and perpetuate aggressive and/or violent behavior towards the “different” group (racist, xenophobic, and homophobic violence; WHO, 2009). Likewise, cultural norms would be expected to impact on protective factors. For instance, Taylor et al. (2004) examined differences in protective factors between Western and Chinese cultures. They found that while there were some overlapping protective factors, such as educational attainment and positive reinforcement by parents, other factors - such as the importance of interests in social reform and public cultural issues - were specific to individuals in China. They suggested that the emphasis on crime prevention as a communal approach (which tied individuals to social conformity in China) rather than the individualistic approach in Western cultures would give rise to the difference in protective factors. Such cultural norms are therefore likely to impact the risk of violence in the community.

In summary, it is important to extend the current understanding of violent behaviors internationally. The current assessment instruments (the SAVRY, VRS-YV, and SAPROF-YV) can be helpful in this endeavor. For the Singapore population, the assessment instruments can be used to identify common risk and protective factors and to build a profile of factors for targeted intervention for violent youths. By building risk and protective profiles in various cultures, known risk and protective factors which contribute to violent behaviors that emerge within different cultural contexts can be examined. Social and cultural norms thus may need to be taken into consideration with these profiles in order for future preventative programs to be effective.

Limitations

The retrospective nature of this study meant that scoring of instruments was undertaken through review of archival file data meaning that the breadth and depth of information available to score the instruments was limited. These archival data were not generated for the purpose of scoring these instruments and this may have resulted in the underestimation of the risk and protective factors present. The youths in the sample were sentenced prior to the completion of a violence risk assessment. Hence, it was not possible to determine whether there were differences in risk level pre-sentencing for those sentenced to a juvenile facility compared to those on probation. Additionally, the ICC for the SAPROF-YV in this study was only fair; although the independent raters were specifically trained they had no prior experience in using the tool as part of their routine clinical assessments. Finally, there may have been less documented information on the presence of protective factors in the 2008 to 2012 assessment period due to a more risk-focused approach at the time and this may have impacted inter-rater reliability. These problems may be alleviated through the use of prospective studies, where ratings are collected based on interviews and file records. Future research should determine whether these measures are reliable and valid for female youth.

Conclusion

This is the first study to compare the predictive validity of different assessment instruments specifically designed to assess youth violence. The results show that in the Singapore setting, the SAVRY and VRS-YV both have value in assessing risk of violent, non-violent and general (any) recidivism, with stronger results obtained when the Total Score rather than the Risk Ratings is used. The study also showed that the SAPROF-YV did not have good predictive ability, and it did not significantly add incremental predictive value when considered together with risk focused assessment instruments. Our conclusion is that more research is needed to validate and refine these tools. Youth violence assessment instruments have an important role in clinical and forensic work; youth scoring high on either the SAVRY or VRS-YV is likely to pose a risk of general or violent re-offending and benefit from treatment that targets relevant dynamic risk factors.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Li Lian Koh

Andrew Day

Notes

References

Akers

R. L.

(1998). Social learning and social structure: A general theory of crime and deviance. Northeastern University Press.

Bandura

(1971). Social learning theory. General Learning Press.

Beech

Friendship

Erikson

Hanson

R. K.

(2002). The relationship between static and dynamic risk factors and reconviction in a sample of U.K. child abusers. Sexual Abuse: A Journal of Research and Treatment, 14(2), 155–167. https://doi.org/10.1177/107906320201400206

Borum

Bartel

Forth

(2002). Manual for the structured assessment of violence risk in youth (SAVRY). University of South Florida.

Borum

Bartel

Forth

(2006). Manual for the structured assessment for violence risk in youth (SAVRY). Psychological Assessment Resources.

Brown

Singh

J. P.

(2014). Forensic risk assessment: A beginner’s guide. Archives of Forensic Psychology, 1(1), 49–59.

Catchpole

R. E. H.

Gretton

H. M.

(2003). The predictive validity of risk assessment with violent young offenders: A 1-year examination of criminal outcome. Criminal Justice and Behavior, 30(6), 688–708. https://doi.org/10.1177/0093854803256455

Chu

C. M.

Goh

M. L.

Chong

(2016). The predictive validity of SAVRY ratings for assessing youth offenders in Singapore: A comparison with YLS/CMI ratings. Criminal Justice and Behavior, 43(6), 793–810. https://doi.org/101177/0093854815616842

Chu

C. M.

Ruby

Chng

G. S.

(2020). The utility of SAPROF-YV rating for predicting recidivism in male youth under community supervision in Singapore. Criminal Justrics and Behavior, 47(11), 1409–1427. https://doi.org/10.1177/0093854820949595

10.

Chu

C. M.

Zeng

(2017). The assessment and management of youth offenders in Singapore: The implementation of the Risk, Needs, and Responsivity framework. In Chan

H. C. O.

M. Y. S.

(Eds.), Psycho-criminological perspective of criminal justice in Asia: Research and practices in Hong Kong, Singapore, and beyond (pp. 200–218). Routledge.

11.

Chua

J. R. Z. H.

Chu

C. M.

Yim

Chong

Teoh

(2014). Implementation of the Risk-Need-Responsivity framework within the juvenile justice agencies in Singapore. Psychiatry, Psychology and Law, 21, 877–889. https://doi.org/10.1080/13218719.2014.918076

12.

Cicchetti

D. V.

(1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6(4), 284–290.

13.

de Vogel

de Ruiter

Bouman

de Vries Robbé

. (2012). SAPROF. Guidelines for the assessment of protective factors for violence risk (English version, 2nd ed.). Van der Hoeven Stichting.

14.

de Vries Robbé

de Vogel

Douglas

K. S

. (2013). Risk factors and protective factors: A two-sided dynamic approach to violence risk assessment. The Journal of Forensic Psychiatry & Psychology, 24(4), 440–457. https://dx-doi-org.web.bisu.edu.cn/10.1080/14789949.2013.818162

15.

de Vries Robbé

de Vogel

Douglas

K. S.

Nijman

. (2015a). Changes in dynamic risk and protective factors for violence during inpatient forensic psychiatric treatment: Predicting reductions in post-discharge community recidivism. Law and Human Behavior, 39(1), 53–61. https://doi.org/10.1037/lhb0000089

16.

de Vries Robbé

Geers

Stapel

Hilterman

de Vogel . (2015b). SAPROF Youth Version: Guidelines for the assessment of protective factors for violence risk in juveniles. Vander Hoeven Kliniek part of De Forensische Zorgspecialisten.

17.

DeLong

E. R.

DeLong

D. M.

Clarke-Pearson

D. L.

(1988). Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach. Biometrics, 44(2), 837–845.

18.

Dolan

M. C.

Rennie

C. E.

(2008). The Structured Assessment of Violence Risk in Youth as a predictor of recidivism in a United Kingdom cohort of adolescent offenders with conduct disorder. Psychological Assessment, 20(1), 35–46. https://doi.org/10.1037/1040-3590.20.1.35

19.

Fortune

Ward

(2017). Problems in protective factor research and practice. Aggression and Violent Behavior, 32, 1–3. https://doi.org/10.1016/j.avb.2016.12.008

20.

Guy

L. S.

Packer

I. K.

Warnken

(2012). Assessing risk of violence using structure professional judgment guidelines. Journal of Forensic Psychology Practice, 12(3), 270–283. https://doi.org/10.1080/15228932.2012.674471

21.

Hilterman

E. L. B.

Nicholls

T. L.

van Nieuwenhuizen

(2014). Predictive validity of risk assessments in juvenile offenders: Comparing the SAVRY, PCL:YV, and YLS/CMI with unstructured clinical assessments. Assessment, 21(3), 324–339. https://doi.org/10.1177/1073191113498113

22.

Klepfisz

Daffern

Day

(2016). Understanding dynamic risk factors for violence. Psychology, Crime & Law, 22(1–2), 124–137. https://doi.org/10.1080/1068316X.2015.1109091

23.

Klepfisz

Day

Daffern

(2017). Understanding protective factors for violent reoffending in adults. Aggression and Violent Behaviour, 32, 80–87. https://doi.org/10.1016/j.avb.2016.12.001

24.

Koh

L. L.

Day

Klettke

Daffern

Chu

C. M.

(2020). The predictive validity of youth violence risk assessment tools: A systematic review. Psychology, Crime & Law, 26(8), 776–796.

25.

Kroner

D. G.

Mills

J. F.

Reitzel

L. R.

Dow

Aufderheide

D. H.

Railey

M. G.

(2007). Directions for violence and sexual risk assessment in correctional psychology. Criminal Justice and Behavior, 34(7), 906–918. https://doi.org/10.1177/0093854807301559

26.

Chu

C. M.

Zeng

Ruby

(2019). Risk and protective factors for probation success among youth offenders in Singapore. Youth Violence and Juvenile Justice, 17(2), 194–213. https://doi.org/10.1177/1541204018778887

27.

Lilleston

P. S.

Goldmann

Verma

R. K.

McCleary-Skills

(2017). Understanding social norms and violence in childhood: Theoretical underpinnings and strategies for intervention. Psychology, Health & Medicine, 22(1), 122–134. https://doi.org/10.1080/13548506.2016.1271954

28.

Lodewijks

H. P. B.

Doreleijers

T. A. H.

de Ruiter

Borum

(2008). Predictive validity of the Structured Assessment of Violence Risk in Youth (SAVRY) during residential treatment. International Journal of Law and Psychiatry, 31, 263–271. https://doi.org/10.1016/j.ijlp.2008.04.009

29.

McGowan

M. R.

Horn

R. A.

Mellot

R. N.

(2011). The predictive validity of the structured assessment of violence risk in youth in secondary educational settings. Psychological Assessment, 23(2), 478–486. https://doi.org/10.1037/a0022304

30.

National Youth Council Singapore. (2019). Youth statistics in brief 2019. Author. Retrieved January 30, 2020, from https://www.nyc.gov.sg/en/initiatives/resources/youth-statistics-in-brief/

31.

Pedersen

Rasmussen

Elsass

(2010). Risk assessment: The value of structured professional judgments. International Journal of Forensic Mental Health, 9, 74–81. https://doi.org/10.1080/14999013.2010.499556

32.

Penney

S. R.

Lee

Moretti

M. M.

(2010). Gender differences in risk factors for violence: An examination of the predictive validity of the Structured Assessment of Violence Risk in Youth. Aggressive Behaviour, 36, 390–404. https://doi.org//10.1002/ab.20352

33.

Perrault

R. T.

Vincent

G. M.

Guy

L. S.

(2017). Are risk assessments racially biased?: Field study of the SAVRY and YLS/CMI in probation. Psychological Assessment, 29(6), 664–678. https://doi.org/10.1037/pas0000445

34.

Prochaska

J. O.

DiClemente

C. C.

Norcross

J. C.

(1992). In search of how people change. Applications to addictive behaviors. American Psychologist, 47(9), 1102–1114. https://doi.org/10.1037//0003-066x.47.9.1102

35.

Rice

M. E.

Harris

G. T.

(2005). Comparing effect sizes in follow-up studies: ROC area, Cohen’s d, and r. Law and Human Behavior, 29(5), 615–620. https://doi.org/10.1007/s10979-005-6832-7

36.

Quinsey

V. L.

Harris

G. T.

Rice

M. E.

Cormier

(1998). Violent offenders: Appraising and managing risk. American Psychological Association.

37.

Shepherd

S. M.

Luebbers

Ogloff

J. R. P.

(2014a). Are youth violence risk instruments interchangeable? Evaluating instrument convergence in a sample of incarcerated adolescent offenders. Journal of Forensic Psychology Practice, 14, 317–341. https://doi.org/10.1080/15228932.2014.954871

38.

Shepherd

S. M.

Leubbers

Ogloff

J. R. P.

Fullam

Dolan

(2014b). The predictive validity of risk assessment approaches for young Australian offenders. Psychiatry, Psychology and Law, 21(5), 801–817. https://doi.org/10.1080/13218719.2014.904262

39.

Singapore Department of Statistics. (2018). Overall crime cases & crime rate. Author. Retrieved January 30, 2020, from https://data.gov.sg/dataset/overall-crime-cases-crime-rate

40.

Singapore Department of Statistics. (2019). Population trends. Retrieved January 30, 2020, from https://www.singstat.gov.sg/modules/infographics/population

41.

Singh

J. P.

Desmarais

S. L.

Hurducas

Arbach-Lucioni

Condemarin

Dean

Doyle

Folino

J. O.

Godoy-Cervera

Grann

Ho.

R. M. Y.

Large

M. M.

Nielsen

L. H.

Pham

T. H.

Rebocho

M. F.

Reeves

K. A.

Rettenberger

de Ruiter

Seewald

Otto

R. K.

(2014). International perspectives on the practical application of violence risk assessment: A global survey of 44 countries. International Journal of Forensic Mental Health, 13, 193–206. https://doi.org/10.1080/14999013.2014.922141

42.

Singh

J. P.

Serper

Reinharth

Fazel

(2011). Structured assessment of violence risk in schizophrenia and other psychiatric disorders: A systematic review of the validity, reliability, and item content of 10 available instruments. Schizophrenia Bulletin, 37(5), 899–912. https://doi.org/10.1093/schbul/sbr093

43.

Skeem

J. L.

Monahan

(2011). Current directions in violence risk assessment. Current Directions in Psychological Science, 20(1), 38–42. https://doi.org/10.1177/0963721410397271

44.

Stockdale

K. C.

Olver

M. E.

Wong

S. C. P.

(2014). The validity and reliability of the Violence Risk Scale-Youth Version in a diverse sample of violence young offenders. Criminal Justice and Behavior, 41(1), 114–138. https://doi.org/10.1177/0093854813496999

45.

Taylor

T. J.

Friday

P. C.

Ren

Weitekamp

E. G. M.

Kerner

(2004). Risk and protective factors related to offending: Results from a Chinese cohort study. The Australian and New Zealand Journal of Criminology, 37, 13–31. https://doi.org/10.1177/00048658040370S102

46.

Tengstrom

(2001). Long-term predictive validity of historical factors in two risk assessment instruments in a group of violent offenders with schizophrenia. Nordic Journal of Psychiatry, 55(4), 243–249. https://doi.org/10.1080/080394801681019093

47.

The Law Revision Commission. (2012). The Statues of the Republic of Singapore Criminal Procedure Code (Chapter 68). https://sso.agc.gov.sg/Act/CPC2010

48.

Viljoen

J. L.

Gray

A. L.

Shaffer

Bhanwer

Tafreshi

Douglas

K. S.

(2016). Does reassessment of risk improve predictions? A framework and examination of the SAVRY and YLS/CMI. Psychological Assessment, 29(9), 1096–1110. https://doi.org/10.1037/pas0000402

49.

Wilson

C. M.

Desmarais

S. L.

Nicholls

T. L.

Hart

S. D.

Brink

(2013). Predictive validity of dynamic factors: Assessing violence risk in forensic psychiatric in patients. Law and Human Behavior, 37(6), 377–388. https://doi.org/10.1037/lhb0000025

50.

Wong

S. C. P.

Gordon

(1999–2003). Violence risk scale. Department of Psychology, University of Saskatchewan. http://www.psynergy.ca

51.

Wong

Lewis

Stockdale

Gordon

(2004–2011). The Violence Risk Scale-Youth Version (Unpublished manuscript). University of Saskatchewan.

52.

World Health Organization (WHO). (2009). Violence prevention, the evidence: Changing cultural and social norms that support violence. Author. Retrieved on January 31, 2020, from https://www.who.int/violence_injury_prevention/violence/norms.pdf

53.

Zhou

Witt

Cao

Chen

Wang

(2017). Predicting reoffending using the Structured Assessment of Violence Risk in Youth (SAVRY): A 5-year follow-up study of male juvenile offenders in Hunan Province, China. PLoS ONE, 12(1), e0169251, 1–11. https://doi.org/10.1371/journal.pone.0169251