The SAVRY Improves Prediction of Reoffending

Criminal behavior, especially violence among adolescents, is of great public concern as crime rates are higher in adolescence than in any other age-group (e.g., Moffitt, 1993; Piquero, Farrington, & Blumstein, 2007; Sweeten, Piquero, & Steinberg, 2013). However, within the population of adolescents who engaged in criminal behavior, there is great heterogeneity with regard to the seriousness of the crimes, risk of reoffending, as well as number and patterns of risk and protective factors (Lipsey, 2009; Loeber & Farrington, 1999). Research has shown that a group of individuals account for a large part of the total number of offenses in adolescence and further on in adulthood (e.g., Bergman & Andershed, 2009; Farrington, Ttofi, & Coid, 2009). These offenders are at great risk for developing a more negative lifestyle with serious violence, economic problems, physical, as well as psychological problems (e.g., Odgers et al., 2008). Since many adolescents display criminal behavior, identifying high-risk individuals is challenging, and professional judgment alone has been shown to have low predictive validity (e.g., Hilterman, Nicholls, & Nieuwenhuizen, 2014; Lodewijks, Doreleijers, & De Rutter, 2008). Thus, assessment and decision-making instruments can potentially assist juvenile justice and the social work profession in detecting and distinguishing high-risk offenders from low-risk offenders. According to earlier research, criminal behavior can be reliably predicted and intense treatment should be given to higher risk offenders (Andrews & Bonta, 2006, 2010).

Criminal behavior during adolescence—here defined as criminal offenses according to Swedish law, committed by adolescents—is considered to be a social problem and is primarily managed within the social services in Sweden rather than within the justice system (Sarnecki & Estrada, 2006). Assessments of adolescents can be initiated by the court, due to committed crimes, or by the social services themselves. The social services are obliged by law to assess the need for support and/or protection of at-risk youth and to conduct a risk assessment of criminal behavior is an important part of that task (The National Board of Health and Welfare, 2009). During the assessment process, the social worker shall identify risk conditions and situations for the adolescence. Based on that assessment, a prognosis of the development of the adolescent is made and treatment is planned accordingly (The National Board of Health and Welfare, 2009). This procedure is in line with best practice recommendations suggested by leading researchers in the field (Andrews & Bonta, 2006, 2010; Wong & Gordon, 2006; Wong, Gordon, & Gu, 2007).

Proper risk assessment requires that risk and protective factors for reoffending are accurately identified and taken into account (Andrews & Bonta, 2006). In recent years, several structured assessment instruments based on this idea have been developed and implemented within the social services and juvenile justice, for example, Youth Level of Service/Case Management Inventory (Hoge & Andrews, 2002), Estimate of Risk of Adolescent Sexual Offense Reoffending (Worling & Curwen, 2001), and Structured Assessment of Violence Risk in Youth (SAVRY; Borum, Bartel, & Forth, 2006). In the present study, the performance of the SAVRY was evaluated in a naturalistic setting.

The SAVRY is an instrument for assessing risk for violent crimes among adolescents from 12 to 18 years (Borum et al., 2006) and comprises 24 empirically derived risk factors (e.g., past violence, negative attitudes, peer delinquency, maltreatment, and impulsivity) and 6 protective factors (e.g., prosocial activity, strong social support, and strong school attachment). For each item, a coding guideline is provided. Rather than simply summing up the factors identified, the SAVRY applies a structured professional judgment approach where the assessor interprets, weighs, and combines pattern of risk and protective factors to achieve a summary risk rating (SRR) indicating the level of risk for reoffending. The SRR is rated as low risk, medium risk, or high risk. This approach combines empirical, research-based knowledge and clinical expertise. Research has demonstrated estimates of inter-rater agreement for the SRR ranging between .35 and .95 (Catchpole & Gretton, 2003; Dolan & Rennie, 2008; Lodewijks, Doreleijers, & De Ruiter, 2008; Meyers & Schmidt, 2008; Penney, Lee, & Moretti, 2010; Spice, Viljoen, Gretton, & Roesch, 2010).

Previous studies on the SAVRY have reported predictive validity estimates ranging from area under the curve (AUC) values of .64 to .86 for violent reoffending and from .69 to .77 for general reoffending (e.g., Dolan & Rennie, 2008; Lodewijks, Doreleijers, De Ruiter, & Borum, 2008; Meyers & Schmidt, 2008; Welsh, Schmidt, McKinnon, Chattha, & Meyers, 2008). A meta-analysis of the predictive validity of the SAVRY showed moderate effect sizes r = .31 for violent reoffending and r = .38 for nonviolent reoffending for total scores (Olver, Stockdale, & Wormith, 2009). In the SAVRY manual, it is stated that SAVRY is a guideline for assessing risk of violence. Nevertheless, studies have shown that the SAVRY can be useful for assessing risk of reoffending of violent acts as well (e.g., Welsh et al., 2008).

Most research on SAVRY’s predictive validity has been based on file information only (e.g., Meyers & Schmidt, 2008) or file information combined with interviews (e.g., Dolan & Rennie, 2008). Although the majority of previous studies support the predictive validity of the SAVRY, assessments have generally been conducted by trained assessors in highly controlled research settings (e.g., Lodewijks, Doreleijers, & De Ruiter, 2008). Only a small number of the published studies have investigated how well the SAVRY can assist practitioners in their everyday task of predicting reoffending, with mixed results (i.e., Childs et al., 2013; Hilterman et al., 2014; Vincent, Chapman, & Cook, 2011; Vincent, Guy, Gershenson, & McCabe, 2012). Even though they lend some support to the predictive validity of SAVRY assessments performed by regular practitioners, they are not without limitations, such as lack of comparison groups, low number of assessors, and reoffending data solely based on official records. The many nonsignificant results (only 1 of the 12 reported outcomes pertaining violent rearrests was statistically significant in Vincent et al., 2011) and small effect sizes (AUCs ranging from .60 to .68 in Vincent et al., 2012) also call for further studies in naturalistic settings. In addition, and probably as a natural consequence of the intended purpose of the instrument, previous studies of the SAVRY have focused on accuracy in prediction of violent and general crimes rather than on specific types of crimes within these broad categories. In general practice, it may be important to assess the risk for reoffending in specific types of crimes, because the risk of certain types of reoffending (e.g., drug-related crimes) may call for certain types of intervention.

Method

Participants

Setting

Thirteen units within the social services in Stockholm County, Sweden, were recruited for the purpose of the study (Figure 1). All units were working with adolescents. All units using the SAVRY in the Stockholm County were invited to participate (n = 6). An equal number of units using either ADAD or IAU were also invited as comparisons to the SAVRY units. To be selected, the units had to be able to provide a big enough sample of juvenile delinquency cases. Five of the 13 recruited units employed the SAVRY as part of their routine investigation reports, 4 units used ADAD, and 4 units did not apply any structured instruments in their investigation reports (i.e., IAU).

OPEN IN VIEWER

Figure 1.

Flowchart of the study design.

The social workers employed at the units completed a questionnaire regarding education, years of employment, and experience (Table 1). Completed questionnaires were collected from 84% of the employees. Sample characteristics and measures were examined across the three study conditions to ensure that they were comparable. As can be seen in Table 1, there were no significant differences between conditions in terms of characteristics of the investigating social workers or the adolescents being the subject of the investigation reports at baseline. To further compare the three study conditions, we created an index based on poverty, number of immigrants, unemployment, social security, and low education for households living within the catchment area of the 13 units included in the study. The mean of the index, calculated from official statistics from 2009, did not differ among the three conditions, F(2, 12) = 1.5, p > .05.

OPEN IN VIEWER

Table 1.

Characteristics of the Social Workers and the Investigated Youths.

	SAVRY	ADAD	IAU	p
Characteristics of the social workers
N a	38	30	41
BS in social work (.63%)	90	93	95
Total years of practice (.17%)
<3	16%	40%	20%
3–5	10%	17%	12%
5–10	32%	20%	19%
>10	42%	23%	49%
Years of investigating experience (.06%)
<3	37%	67%	32%
3–5	16%	6%	12%
5–10	29%	10%	37%
>10	18%	17%	19%
Characteristics of the youth at baseline
N b	56	38	38
Age, M (SD)	16.1 (1.7)	16.1 (1.5)	16.3 (1.5)	.79
Girls	23%	16%	34%	.17
Repeated a year in school	27%	34%	45%	.22
Incomplete school grades	42%	57%	62%	.14
Previous intervention within the social services	51%	65%	65%	.28
Present out of home placement	13%	11%	16%	.79
Property crimesc	20%	29%	36%	.25
Drug sellingc	18%	37%	34%	.08
Violent crimec	64%	53%	68%	.37
Serious violentc	33%	29%	44%	.35
General recidivismc	70%	64%	82%	.20

Note. BS = bachelor in social work; SAVRY = Structured Assessment of Violence Risk in Youth; ADAD = Adolescent Drug Abuse Diagnosis; IAU = Investigation reports As Usual.

^aThe N refers to the number of social workers answering a questionnaire at the units that were included in the study. ^bThe N refers to the number of youths that were investigated. ^cSelf-reports of whether an offense in the specified crime was committed during the past year.

Two inclusion criteria were used by the social workers to identify possible cases among adolescents with criminal behavior. First, only adolescents between 12 and 20 years were eligible for the study. Second, a formal social services’ investigation concerning the adolescent had to have been initiated and disclosing a suspicion of serious criminality. The adolescents themselves could either have admitted to having committed a crime or information about a suspicion could have been provided through other sources, for example, the police or the adolescents’ parents. In the present study, serious criminality included one or more of the following crimes: assault, burglary, rape and other sexual abuse, robbery, drug trafficking, arson, kidnapping, homicide, repeated property crimes, repeated fraud, dealing in stolen property repeatedly, extortion, serious forms of counterfeiting, embezzlement, drinking and driving, threat of weapons, repeated crimes against the weapons act, and auto property crimes. This definition is similar to definitions used in previous research (Loeber & Farrington, 1999). No exclusion criteria were applied.

Cases

The social workers identified 339 adolescents who were invited to participate in the study through consecutive admission (all individuals coming to a unit during a predefined period and meeting the inclusion criteria; Figures 1 and 2). Of these, 152 declined participation and 28 did not meet the inclusion criteria. The research staff was unable to reach another 16, and 5 agreed to participate but dropped out before the initiation of the study. An additional six adolescents were excluded from the SAVRY and ADAD conditions because the social workers did not apply the designated method (SAVRY and ADAD). The final sample thus comprised 132 adolescents, from which 56 were recruited at SAVRY units, 38 at ADAD units, and 38 at IAU units (Figure 2). Thus, the participants were not randomly assigned to the three experimental conditions. The adolescents were between the ages of 12 and 20 years (M = 16.1, SD = 1.6), and 32 were girls (100 boys). A majority of the adolescents (60%) had been subjected to previous investigation reports by the social services before the current investigation. One third (35%) had repeated a year in school. At the 12-month follow-up, the dropout rate was 20% (n = 26; Figure 2). Each adolescent as well as parents or guardians provided written consent (Figure 1), and the adolescents received a gift certificate of 300 SEK (approximately 27 Euro) for their participation in the study.

OPEN IN VIEWER

Figure 2.

Flowchart of the study participants.

There were no significant differences at baseline between those who dropped out and those who participated in the follow-up regarding criminal behavior, substance use, or any of the background variables (e.g., age, sex, and school attendance).

Measures

Self-reported crime

Information on the adolescents’ criminal behavior was collected through self-reports at baseline (from December 2007 to October 2009) and again at follow-up 12 months later (from December 2008 to November 2010; Figure 1). Data were mainly collected through face-to-face interviews. Those who declined the face-to-face interview were offered to respond through self-report forms, using regular mail (10% of the cases at baseline and 22% of the cases at follow-up). The interviews were carried out by one of the two research assistants in the study, both qualified social workers (one of which is the first author of this article). No information was given to the social workers from the interviews conducted by the research assistants.

The items in the questionnaire have fixed response options and are based on 22 different offense types (Ring, 1999), which are defined as criminal offenses according to Swedish law. For each item, the respondents indicate whether they have ever committed the crime in question and if so, how many times the crime was committed during the past year. Twelve of the 22 offense types were not included in this study because they represent less serious crimes or were too infrequent in the case files. For the purpose of the present study, the remaining 10 items were grouped into four crime types. In property crimes, the items were “Have you stolen a car?” “Taken a purse, wallet, or similar objects from someone you know?” and “Broken into someone’s car, home, or other building?” For violent crimes, included items were “Started fights?” and “Beaten or injured someone?” Serious violent crimes consisted of the items “Beaten or injured someone so that they required hospitalization?” “Used violence or threatened someone to get money or other valuables?” and “Beaten or injured someone with a bat, knife, or other weapon?” Finally, in drug selling crimes, the items were “Sold marijuana or hashish?” and “Sold other drugs?” An exploratory factor analysis confirmed that the items actually were grouped as described above, with factor loadings between .44 and .94 in the designated factor with no overlapping loadings in several factors. One exception was the item “Used violence or threatened someone to get money or other valuables?” with a factor loading of .44 in serious violent crimes and .42 in property crimes. Because robbery using violence or threats legally is considered to be a serious violent crime, it was included in that factor. The factor analysis was conducted using maximum likelihood and varimax rotation, and the pattern of factor loadings were exactly the same regardless if reoffending was treated as categorical (yes/no) or continuous (number of crimes).

Identification of documented risk and protective factors in the investigation reports

Information on research-based risk and protective factors was extracted from the investigation reports (Figure 1). Each case file was reviewed according to a protocol, focusing on identifying whether these factors were documented in the file or not (documented as present, i.e., “the adolescent acts aggressive,” or absent, i.e., “the adolescent has no association with deviant peers”). Two criteria were used to decide which factors to include in the protocol. First, there had to be empirical support for a link between a factor and crime among adolescents—to maintain quality of the empirical support, we used factors presented in a systematic review of longitudinal studies investigating risk factors for offending (Loeber & Farrington, 1999). Second, the inter-rater reliability of the protocol, which was tested on 28 randomly selected case files, Cohens κ (Cohen, 1960) had to exceed 0.6. The inter-rater test was carried out by two qualified social workers trained to do the SAVRY assessments (one of them was also trained in doing ADAD assessments), they were not blinded to the study conditions. This process resulted in 23 risk factors and 7 protective factors from the protocol that was included in further analyses (Table 2). Of these 30 factors, 21 were represented in the SAVRY manual and 22 in the ADAD manual. Three of the factors that passed the two criteria for inclusion were not represented in either of the two instruments.

OPEN IN VIEWER

Table 2.

Included Risk and Protective Factors Extracted From the Investigation Reports.

Risk Factors

Protective Factors

Hyperactivity and/or attention deficits (S) Sensation seeking/risk taking (S) Aggressive behavior (S/A) Impulsive behavior (S) Low empathy (S) Favorable attitudes toward  crime (S) Early antisocial behavior  (<12 years) (S/A) Committed a violent crime  (S/A) Committed crime before  14 years of age (S/A) Committed more than one  violent crime (S/A) Criminal parents/siblings (S/A) Physical abuse by parents (S/A) Family violence (S/A) Parent–child conflicts (S/A) Parental alcohol or substance  abuse (A) Parental psychopathology (A) Family poverty (A) Unemployed parents (A) Association with deviant peers  (S/A) Use of alcohol (A) Use of drugs (A) Alcohol and/or drug abuse  (S/A) School failure (S/A)

Connectedness to parents (S/A) Connectedness to adults outside  the family (S/A) Involvement in social activities (S/A) Calm temperament (none) High empathy (none) Commitment to school (S/A) High IQ (none)

Note. S = risk/protective factor is related to one or several items that are included in the Structured Assessment of Violence Risk in Youth manual; A = risk/protective factor is related to one or several items that are included in the Adolescent Drug Abuse Diagnosis manual; none = risk/protective factor is not directly related items in any of the two manuals.

No uniform or standardized form of risk estimate (such as the SRR in the SAVRY) could be used in comparisons across study conditions. Therefore, risk for reoffending was operationalized as the documentation of empirically supported risk and protective factors in the investigation reports. Thus, in reports that included many risk factors and few protective factors, the risk for reoffending was assumed to be higher compared to reports with fewer risk factors and more protective factors. A similar argument about cumulative risk for measuring negative child development can be found in Evans, Li, and Whipple (2013). The documented risk and protective factors were summarized into an index that would represent the total risk level for each adolescent. The risk factors documented as present in an investigation were summarized, and the protective factors were subtracted from that sum. This means that the theoretical maximum score of the index was 23 (i.e., 23 present risk factors and 0 protective factors) and the minimum score was −7 (i.e., 0 present risk factors and 7 present protective factors). Before deciding to compute the index by simply subtracting the number of protective factors from the number of risk factors, this method was compared to an alternative computation used by Jessor, Van den Bos, Vanderryn, Costa, and Turbin (1995). In that method, the sum of present risk factors and the sum of present protective factors are entered separately in the regression analyses. The predictive validity in our analyses turned out to be independent of the method used to compute the index. Thus, for simplicity reasons, the index was calculated for each participating adolescent as the number of documented risk factors minus the number of documented protective factors. The professional judgment from the SAVRY assessments—the so-called SRR—was also extracted from the investigation reports. The regional ethical review board in Stockholm approved the study (DNR 2007/1154-31).

Results

Documented Research-Based Risk and Protective Factors

The extent to which the social workers documented research-based risk and protective factors (Table 2) in their investigation reports was compared between conditions (see Table 3). In investigation reports where the SAVRY was used, a significantly larger number of risk and protective factors were documented compared to investigation reports using both ADAD and IAU. The differences depended mainly on a larger number of factors documented as absent in the SAVRY assessments. There were no significant differences between the ADAD and the IAU assessments. There were also no significant differences in the summary index (the number of risk factors minus the number of protective factors described as present) between the three study conditions.

OPEN IN VIEWER

Table 3.

Risk and Protective Factors That Were Described in the Investigation Reports.

	SAVRY (S)	ADAD (A)	IAU (I)	F (2, 131)	Tukey’s	S Versus A (d)	S Versus I (d)
	N = 56	N = 38	N = 38
	M (SD)	M (SD)	M (SD)
Total no. of described factorsa	18.7 (4.5)	14.3 (3.5)	12.7 (4.6)	25.5****	S > A, S > I, A = I	1.10	1.20
No. described as present	10.4 (3.9)	8.8 (3.4)	7.7 (4.0)	5.8*	S = A, S > Ib, A = I	0.44	0.68
No. described as absent	8.4 (3.1)	5.5 (2.2)	5.0 (1.8)	25.3****	S > A, S > I, A = I	1.08	1.14
Summary indexc	5.5 (5.6)	6.1 (3.6)	5.7 (3.8)	0.14	S = A = I	−0.13	−0.04

Note. SAVRY = Structured Assessment of Violence Risk in Youth; ADAD = Adolescent Drug Abuse Diagnosis; IAU = Investigation reports As Usual. (d) = Cohen’s d.

^aThis number refers to the number of risk and protective factors, of the research-based factors presented in Table 1, that explicitly were described as either present or absent in the investigation reports. ^bThis difference was significant at the p < .05 level. All other differences were significant at the p < .0001 level. ^cThe summary index equals the number risk factors minus the number of protective factors described as present. The summary index was used to analyze the predictive validity of those factors (Table 4).

*p < .05. ****p < .0001.

Predictive Validity of the Documented Risk and Protective Factors

In investigation reports supported by the SAVRY, the risk and protective factors that were documented as present (i.e., the summary index) significantly predicted both occurrence (reoffending vs. no reoffending) and number of crimes for all types of crimes except for drug selling (Table 4). In investigation reports supported by ADAD, only 1 item was significantly predicted, “any nonviolent crime.” Risk and protective factors documented in the IAU investigations did not significantly predict any type of crime.

OPEN IN VIEWER

Table 4.

Logistic and Linear Regressions With Reoffending as Dependent Variable and Summary Index (Risk Minus Protective Factors) as Independent Variable.

	SAVRY (S)			ADAD (A)			IAU (I)
	n = 42			n = 34			n = 30
Reoffending	χ2/F	R 2	AUC	χ2/F	R 2	AUC	χ2/F	R 2	AUC	Contrastsa
Any nonviolent crime	10.2*	—	.77	12.4**	—	.86	0	—	.51	S > I, A > I
No. of nonviolent crimes	12.0*	.23	—	7.2	.18	—	0	0	—	S > I
Any property crime	12.9**	—	.84	7.5	—	.80	0.2	—	.55	S > I
No. of property crimes	14.6**	.27	—	7.0	.18	—	0.1	0	—	S > I
Any drug selling	4.2	—	.68	8.1	—	.85	0	—	.51	—
No. of drug selling crimes	5.6	.12	—	5.9	.16	—	0.4	.01	—	—
Any violent crime	10.6*	—	.77	0.4	—	.61	1.7	—	.69	ns
No. of violent crimes	41.6***	.51	—	4.3	.12	—	0.6	.02	—	S > A, S > I
Any less serious violence	10.6*	—	.77	0.4	—	.61	1.7	—	.69	ns
No. of less serious violent crimes	33.1***	.45	—	3.2	.09	—	0.4	.01	—	S > A, S > I
Any serious violence	10.2*	—	.81	2.6	—	.70	2.8	—	.69	ns
No. of serious violent crimes	18.3**	.31	—	4.1	.11	—	0.7	.02	—	S > I
Any crime	12.0**	—	.80	1.2	—	.66	2.0	—	.71	ns
Total no. of crimes	32.8***	.45	—	6.3	.17	—	0.2	.01	—	S > I

Note. SAVRY = Structured Assessment of Violence Risk in Youth; ADAD = Adolescent Drug Abuse Diagnosis; IAU = Investigation reports As Usual.

^aPlanned contrasts according to the hypotheses were conducted between predictions, given that the prediction in the ADAD (A) or SAVRY (S) groups was significant. The specified contrasts were significant at least at the p < .05 level. One-tailed tests of significance were used for directional hypotheses (i.e., S and A > I for all types of crimes and S > A for violent crimes).

*p < .0036. **p < .0007. ***p < .00007. ****= = 0.000007 (α levels adjusted with Bonferroni correction).

In direct comparisons of prediction of reoffending between the three study conditions, some significant differences emerged (Table 4). Pertaining the occurrence of reoffending, the only significant contrasts emerged for nonviolent crimes. Specifically, the SAVRY predictions were significantly stronger than the IAU predictions for all nonviolent crimes (z = 1.85, p < .05) and for property crimes (z = 2.07, p < .05). The ADAD prediction for all nonviolent crimes was also stronger than the IAU prediction (z = 2.16, p < .05).

Regarding the number of committed crimes, assessments using the SAVRY produced significantly stronger predictions than for the IAU assessments for all types of reoffending (1.91 < z < 3.01, p < .05) except for drug selling. Furthermore, the SAVRY predictions were significantly stronger than the ADAD predictions for the number of violent crimes (z = 2.24, p < .05) and specifically less serious violence (z = 2.11, p < .05). In comparisons between the ADAD and the IAU assessments, no significant contrasts emerged.

Discussion and Applications to Practice

The study examined the benefits of using structured assessment instruments, specifically the SAVRY, when the social services assess criminal behavior in adolescents. Investigation reports supported by the SAVRY were compared to investigation reports supported by another structured instrument (ADAD) and to investigation reports without support of any instrument (IAU). Our first hypothesis was partly supported, given that investigation reports from social workers applying the SAVRY, but not ADAD, contained a significantly larger number of research-based risk and protective factors compared to reports from the IAU assessments. The SAVRY seems to enable the practitioners to assess not only if relevant risk and protective factors are present but to an even greater extent to document when such factors are absent in an investigation (see Table 3). On average, 62% of the factors listed in Table 1 were addressed one way or the other in the investigations supported by the SAVRY, compared to 48% and 42% for ADAD and IAU investigations, respectively. This is an important finding, because it shows that the SAVRY can be a means to counter a tendency among social workers to focus on clients general needs and social situations rather than on empirically founded risk and protective factors (Persson & Svensson, 2012; see also Andershed & Andershed, 2015).

The fact that fewer empirically based factors were documented in investigation reports supported by ADAD than the SAVRY was surprising, given that the analyzed factors were equally well represented in both instruments (Table 2). An explanation may be that the ADAD contains as many as 150 items compared to only 30 in the SAVRY. The factors with empirical links to reoffending may have been overlooked in the assessment procedure due to the sheer amount of items and factors that are to be considered. Another explanation may be that ADAD originally was developed for assessment and treatment planning targeting adolescents with substance abuse problems. Even if ADAD is used for assessment of juvenile delinquents, the content and structure of the instrument, as well as the training in the instrument, may lead social workers to focus more on areas that are important specifically for substance abuse.

If the first aim of the study was to investigate the extent to which social workers documented the presence or absence of research-based risk and protective factors, the second aim was to examine whether they did so validly. Specifically, the aim was to examine whether the factors in Table 2, which already have been validated in a large body of highly controlled studies, would be valid if they were identified and documented by social workers in naturalistic settings. The results generally supported our hypotheses: (1) the risk and protective factors in investigation reports supported by the SAVRY as well as the ADAD predicted nonviolent reoffending significantly better than factors documented in the IAU investigation reports and (2) the factors in investigation reports supported by the SAVRY, but not ADAD and IAU, predicted violent reoffending. It is worth noting that adolescents in the three study conditions did not differ in terms of background or crimes at baseline (Table 1). Thus, the differences in predictive validity between the conditions seem to be a result of more or less accurate descriptions of risk and protective factors rather than differences in risk level between the investigated adolescents. This was also supported by the fact that the social workers in each condition described similar levels of risk at baseline (i.e., risk factors minus protective factors) in the investigation reports (Table 3). The predictive validity of the risk and protective factors in relation to violent reoffending in the SAVRY group was comparable to reports from highly controlled longitudinal studies, in which researchers have assessed the predictive validity of similar factors (e.g., Loeber et al., 2005). Thus, the results point to the conclusion that not only were the social workers using the SAVRY more accurate in their identification of risk and protective factors compared to the other conditions in the study but the validity of their assessments reached the standards that can be expected by specialized researchers.

Given that previous research has focused on the predictive validity of explicit risk ratings (e.g., SRR or SAVRY risk total which is just summing up the factors), the finding pertaining the predictive validity of the factors documented in the investigation reports was important. Several researchers point out that interventions aimed to reduce the risk of reoffending should target empirically based risk and protective factors (e.g., Andrews & Bonta, 2006, 2010), and it is therefore crucial that the factors that are explicitly expressed in investigation reports are valid.

The results regarding the predictive validity should be viewed in light of the purposes that the SAVRY and ADAD originally were developed for. This is reflected in the fact that some of the risk and protective factors being analyzed in the study were unique to one or the other instrument (see Table 2), which may have influenced the results. Related to this issue is the fact that it was the predictive validity of the summary index (present risk factors minus present protective factors) that was investigated rather than the individual factors. Therefore, differences in predictive validity across conditions do not necessarily mean that the social workers in one group were more accurate in the identification of risk and protective factors. An alternative explanation is that some particularly strong (or weak) risk and protective factor was documented more or less frequent by the social workers in the three groups. The sample size of the study does not allow for sophisticated analyses of the predictive validity of individual risk and protective factors, but a simple count points to the plausibility in this alternative explanation. In χ² tests, 9 of the 30 factors listed in Table 2 were significantly more frequent (p < .05) in investigations supported by the SAVRY and 3 of those factors were unique to the SAVRY instrument. Only two of the factors were significantly more frequent in investigations from social workers using ADAD and both were unique to the instrument. Thus, in the prediction of violent crimes, the social workers using the SAVRY may not have been more accurate in their identification of risk and protective factors than their colleagues using the other instruments. Instead, they may have focused on factors that have a particularly strong link to violent crimes. Similarly the social workers using ADAD may have focused on factors strongly related to drug selling. It was earlier pointed out that it is crucial for treatment planning that risk and protective factors in investigation reports are valid. It is however equally important that social workers focus on the most relevant factors (i.e., the strongest predictors) in their reports. It can therefore be concluded that the SAVRY was the best supporting instrument in prediction of violent crimes, regardless of the cause of the superior predictive validity. On the other hand, in assessment of nonviolent crimes, both the SAVRY and the ADAD resulted in significantly better predictions compared to the IAU. It is also worth to note that the AUC for drug selling in the ADAD was .85, which is very good but still not significant.

Few previous studies have investigated the performance of structured risk assessment instruments in naturalistic settings, not the least comparative studies. An exception is a Spanish study in which the SAVRY was compared to two other instruments as well as to unstructured assessments made by probation officers (Hilterman et al., 2014). The result showed that the SAVRY performed in parity with the other structured instruments, but the predictive validity was significantly higher than when conducting unstructured assessments, which were generally nonsignificant (AUCs between .57 and .63). Thus, both the Spanish study and the present one support the performance of the SAVRY in naturalistic settings. However, there are some important differences between the two studies. In the present study, the assessments were conducted by regular social service personnel who applied ADAD or the SAVRY as part of their routine investigations of criminal behavior among adolescents. In the Hilterman, Nicholls, and Nieuwenhuizen (2014) study, nine specially trained probation officers applied all three of the included instruments and assessed the cases only for the purpose of the study. It was required that the assessors did not have any previous professional contact with the juveniles they interviewed. Furthermore, the unstructured assessment consisted of letting the juveniles’ regular probation officers merely rate the risk of reoffending on a 3-point scale. Even if that study design controls for several biases and allows for comparisons between the included instruments, the external validity will suffer. The main purpose of this study was to investigate whether the SAVRY properly could be implemented in real-world settings and to result in meaningful advantages compared to investigations without support of any structured instrument. It was not, as in Hilterman et al. (2014), if social workers or probation officers could be trained to properly apply a structured instrument under more research like circumstances.

The predictive validity of the SAVRY SRR was similar or stronger in this study compared to previous studies in naturalistic settings (Hilterman et al., 2014; Vincent et al., 2012). The predictive validity of the SAVRY SRR in this study was also in parity with reports from studies conducted in controlled research settings (Singh, Grann, & Fazel, 2011). Previous research on the predictive validity of the SAVRY has primarily reported on the occurrence of reoffending. Therefore, an important result was that the risk and protective factors (i.e., the index) as well as the SRR explained fairly large proportions of the variance in number of offenses during the follow-up period. This is important because not only the occurrence of reoffending but also the amount of offenses have consequences both in terms of future risk for the individual (Loeber & Farrington, 1999) and the societal and personal costs related to each committed crime.

There have only been a few previous studies of the SAVRY in naturalistic settings. A strength of this study was that the assessments were conducted by regular social workers who applied the SAVRY as part of their regular work routine, as opposed to being prompted by researchers to perform the assessments. A second strength was the comparative design of the study, in which the SAVRY was compared to realistic alternative procedures (i.e., the application of other instruments or investigations guided by general guidelines only). Furthermore, a unique feature of this study was that in addition to letting the social workers report the SAVRY SRR, the study investigated how the SAVRY influenced the final investigation report (i.e., the number and validity of documented risk and protective factors). This reflects an important aspect of the impact of structured instruments in real-world settings. A final strength worth noting was the nature of the outcome measures, which included both the occurrence and the number of crimes for several types of offenses. This is in line with recent calls for greater variation in reports of outcomes in the risk assessment literature (e.g., Singh et al., 2013).

Despite the strengths of the study, the findings need to be considered in light of its weaknesses. The small sample size limited the possibility to find significant differences between and within the groups. The decision to include as many as 14 outcomes in the analyses of reoffending (Table 4) further decreased the statistical power, since Bonferroni correction was applied. The small sample size also prevented investigations of gender or age differences for example. Another limitation is that the social services units were not randomized to the three study conditions. Hence, there is a risk that the results may depend upon factors other than the use (or nonuse) of structured instruments. To minimize this risk, a fairly large number of units and social workers were included as assessors and descriptive data were compared across units (e.g., education and years of experience). Thirdly, even if researchers have pointed to the advantage of using self-reports to measure rates of reoffending (Vincent et al., 2011), it was a limitation that official data were not collected in addition to the self-reports.

Several findings in this study are of interest to practitioners within the fields of social work and criminal justice. First, in agreement with the few previous studies of the SAVRY in naturalistic settings, this study showed that regular practitioners are able to conduct valid risk assessments when using a structured instrument. This conclusion may be of special interest in Sweden, because of recent claims that Swedish social workers are unable to properly assess risk due to inadequate education and training (Persson & Svensson, 2012). Hence, our conclusion is that it is not the competence, but rather the proper methods, that are lacking. Second, the study showed that the predictive validity of the risk and protective factors that practitioners documented in their investigation reports varied to a great extent. It was apparent that social workers using the SAVRY were better at documenting factors that were empirically supported and better at identifying factors that were relevant and accurate for each investigated adolescent (i.e., predicted reoffending). Therefore, implementation of the SAVRY is useful not only to obtain a specific risk level for reoffending but may also be an effective means to improve the general quality of investigations within the social services. Third, the findings point to the advantage of using structured assessment instruments over no instrument at all. The results also indicate that it is important to match assessment instruments to the purpose of investigations and assessments within the social services.

Before the SAVRY can be classified as an evidence-based assessment, which requires empirical evidence on reliability, validity, as well as clinical utility (Mash & Hunsley, 2005), further studies of the clinical usefulness are needed. Future studies also need to consider the risk assessment of female adolescents and the risk assessments’ impact on treatment planning.