Abstract
We developed a set of risk ratios for the Violence Risk Appraisal Guide—Revised (VRAG-R) to broaden the range of risk communication options available when using this tool and to provide information needed for future efforts to apply The Council of State Governments Justice Center’s standardized five-level risk framework to the scale. A slightly reduced version of the VRAG-R normative data set was used for the analyses (N = 1,238). Contrary to previous research developing risk ratios, logistic regression provided a more accurate estimate of observed violent recidivism rates than Cox regression for both total VRAG-R scores and VRAG-R decile bins. Further analyses indicated the relationship between the VRAG-R and violent recidivism was consistent over a 15-year follow-up period. Due to the difficulties with interpreting odds ratios, the final risk ratios were computed using rate ratios derived from a logistic regression model using a 5-year fixed follow-up period. These risk ratios, and templates for how the ratios might be used in an assessment report, are presented in the appendices.
Although actuarial risk assessment tools are a central component of forensic and correctional practice, misinterpretation of risk information is common and can have serious consequences for the individuals being assessed and the wider community (Hilton et al., 2015). One of the major challenges of risk communication is knowing which information to report when an assessment is completed. For an actuarial risk assessment, there are four primary methods, or risk communication metrics, that are used to communicate the results of an assessment. Absolute recidivism estimates provide information about the likelihood of recidivism associated with each score or risk category on an assessment tool (e.g., a score of 5 is associated with a 50% likelihood of recidivism within 2 years). The other three methods—risk categories, percentiles, and risk ratios—are measures of relative risk that present an individual offender’s risk (more specifically, inferences based on groups of offenders with the same score) in comparison to other offenders. Because there are advantages and disadvantages to each method (Lehmann et al., 2016), in most cases, it is helpful to present more than one. Preliminary research suggests that combining all metrics in risk communication results in laypeople making the largest distinctions between low- and high-risk offenders (Helmus et al., 2018), ideally facilitating effective targeting of resources according to the risk principle of effective correctional intervention (i.e., more intensive resources should be targeted to higher risk offenders; Bonta & Andrews, 2016).
Risk categories and absolute recidivism estimates are the two most commonly used risk communication metrics (Blais & Forth, 2014; Chevalier et al., 2015). To some extent, these findings indicate practitioner and decision maker preference for these two metrics. However, they also reflect the fact that percentiles and risk ratios have not yet been developed for many actuarial risk assessment tools. Recently, it has been argued that relative risk metrics should be used more often and may represent the future of risk assessment (Harris, Lowenkamp, & Hilton, 2015; Helmus, 2018). To provide users with the option of using relative risk metrics, research needs to be completed developing the full range of risk communication metrics for actuarial risk assessment tools.
The Violence Risk Appraisal Guide—Revised (VRAG-R; Rice et al., 2013) is an actuarial risk assessment tool designed for use with correctional and forensic psychiatric populations. Its predecessor, the VRAG, is among the most commonly used actuarial scales for violence risk assessment (Blais & Forth, 2014; Neal & Grisso, 2014; Singh, Desmarais, et al., 2014). Existing research has provided absolute recidivism estimates based on (nine) risk bins, and percentiles for individual scores (Harris, Rice, et al., 2015), but not risk ratios. The purpose of this article is to use the VRAG-R normative data set to develop risk ratios to broaden the range of options available for risk communication when using this instrument. The development of risk ratios is also a first step toward applying The Council of State Governments Justice Center’s five-level risk framework (Hanson, Bourgon, et al., 2017) to the VRAG-R.
Risk Ratios
Risk ratios are measures of relative risk obtained from an actuarial risk assessment tool. A score on an actuarial risk assessment tool is already a measure of relative risk; if higher scores represent higher risk, an individual who scores a “4” on the tool is assumed to pose a higher risk than an individual who scores a “3.” On its own though, the score does not indicate how much riskier the first individual is than the second. Risk ratios quantify that difference. When applied to all scores on an actuarial risk instrument, risk ratios provide a quantifiable measure of the risk of offenders with that score compared with some reference group, which is commonly defined as individuals with the median score, reflecting what could colloquially be referred to as a typical, or average, offender (i.e., in the middle of the risk distribution; Babchishin et al., 2012a; Hanson et al., 2013; Lehmann et al., 2016).
Risk ratios have several advantages over other risk communication metrics. Absolute recidivism estimates have been shown to differ across samples for several scales, making it difficult to generalize this metric to new groups (Helmus, 2018); in contrast, risk ratios have been found to be consistent across samples (Helmus et al., 2012). The stability of risk ratios makes them a promising metric as a foundation for combining results across multiple risk scales (Babchishin et al., 2012b). Risk categories also have issues with inconsistency, with a lack of clarity around how category labels are developed and what they mean (Barbaree et al., 2006; Singh, Fazel, et al., 2014), with an unfortunately common result of scales providing surprisingly discordant results for the same cases (Barbaree et al., 2006; Jung et al., 2013; Mills & Kroner, 2006). Recently, Scurich (2018) has gone so far as to argue that risk categories should not be included in risk communication. Percentile ranks are useful for establishing how unusual an individual’s score is, but because these ranks are not linked to the likelihood of recidivism, it is more difficult to interpret differences between individuals (Hanson et al., 2013). Risk ratios provide a much clearer idea of how much riskier one offender is than another. This feature makes it a useful metric for matching the intensity of intervention to the level of risk (Hanson et al., 2013), a core requirement of any effective correctional intervention (Bonta & Andrews, 2016).
There are also disadvantages of using risk ratios to communicate risk. To aid the interpretation of risk ratios, it is recommended that base rates (i.e., the overall rate of recidivism) are provided for the reference group (Hanson et al., 2013). For example, knowing someone is twice as risky as an offender with the median score has a different meaning if the base rate is 5% versus 50%. Base rates, however, are susceptible to inconsistency across samples, to a greater extent than absolute recidivism estimates (because the absolute estimates take into account risk-related characteristics in a way that base rates do not; see Helmus et al., 2012). Research also indicates decision makers have difficulty interpreting the metric. One study (Varela et al., 2014) found that 80% of a sample of prospective jurors potentially misunderstood the fractions (e.g., “three fourths”) used in risk ratios, resulting in relatively lower risk offenders being considered higher risk by the jurors. However, these misinterpretation issues are not an inherent flaw of risk ratios. Rather, they indicate the need for careful explanation to ensure decision makers properly understand what the measure does and does not say about an individual. On balance, there is a strong argument for developing risk ratios as an option for communicating actuarial risk assessment estimates.
Although risk ratios appear helpful to include in applied risk assessment reports, the calculation of this risk communication metric is not as obvious as it seems. A ratio is fundamentally comparing one group to another group (in this case, a group with a particular score to a reference group), but there are three primary types of risk ratios that can form the basis of these calculations: rate, odds, and hazard. The rate ratio for two groups is calculated by dividing the recidivism rate over a fixed follow-up period for the first group by the recidivism rate of the second (reference) group. The odds ratio similarly compares two groups, but instead uses the odds of recidivism over the fixed follow-up rather than the rate of recidivism (Hanson et al., 2013). Odds and probabilities (and subsequently, ratios of them) have different interpretations and must be carefully distinguished (Helmus & Hanson, 2011). The rate ratio indicates the increased probability of recidivism (which is the number of recidivists divided by the total sample size), whereas the odds ratio indicates the increased odds of recidivism (where odds refers to the number of recidivists divided by the number of non-recidivists; see Hanson et al., 2013, p. 486, for a worked example of this difference). Hazard ratios are interpreted in the same way as rate ratios. Unlike the other two methods though (which can often be calculated by hand), hazard ratios are calculated from survival data, typically using Cox regression, when there are unequal follow-up lengths within a sample. This feature can increase the sample size available for analyses. Hazard rates can be interpreted as a measure of the recidivism rate averaged across time; more accurately, they indicate the likelihood of recidivism occurring in the next time period if it has not already occurred. Consequently, a hazard ratio can be conceptualized as similar to a rate ratio, but averaged across time.
Several factors influence which of these three methods to use to calculate risk ratios. Risk ratios make the assumption that each one-unit increase on the actuarial tool is associated with an increase in the likelihood of recidivism (Babchishin et al., 2012a). For example, on a hypothetical assessment tool, a score of 5 is assumed to indicate a higher likelihood of a recidivism than a score of 4, which in turn indicates a higher likelihood than a score of 3. The increase in likelihood is not assumed to be the same for each unit increase (Hanson et al., 2013). Continuing the previous example, if a score of 5 is associated with a 10% relative increase in likelihood of recidivism compared with a score of 4, it is not assumed that the same increase in likelihood would be observed from a score of 3 to a score of 4. However, while the relationship need not be linear, it is assumed to be relatively smooth (Hanson et al., 2013). Thus, if, for example, the increase in recidivism is quite small moving from a score of 3 to 4, but then is quite large moving from a score of 4 to 5, this could indicate that key assumptions are violated and risk ratios may not be suitable.
Different methods of analysis make different assumptions about the shape of the relationship between risk scores and recidivism, or in other words, about the increase in recidivism associated with each one-point increase on the risk scale. Hazard ratios, calculated using Cox regression, use an exponential function to model this relationship, where increases in risk get bigger and bigger. In contrast, odds ratios from logistic regression use a logistic function, which looks like a smoothed S curve, where increases in risk taper off for the highest scores. To determine which method to use, tests can determine how well the observed data fit these different functions, and whether any transformation is required. Similar fit between these methods is expected when recidivism rates are low, but differences are likely to emerge when rates are higher (Hanson et al., 2013). Specifically, due to the shape of the two functions (i.e., the ever-increasing exponential function versus the S-shaped logistic function), in samples with a high base rate, the exponential function could result in expected probabilities of recidivism greater than one. The logistic function is not susceptible to this error, as estimated probabilities are restricted to between zero and one. In some cases, this may favor the use of odds ratios. Odds ratios, however, are generally considered to be more difficult to interpret than hazard ratios, which can be interpreted as rate ratios (Babchishin et al., 2012a). Thus, hazard ratios may be favored over odds ratios when there is no clear difference in how well the two functions model the relationship between scores and recidivism.
Risk ratios have been developed for several actuarial risk assessment instruments including Static-2002R (Babchishin et al., 2012a), Static-99R (Hanson et al., 2013), and the Risk Matrix 2000 (RM2000; Lehmann et al., 2016). All three of these studies used hazard ratios derived from Cox regression analyses to develop risk ratios. Hanson et al. (2013) noted that hazard ratios were chosen because a larger sample was able to be utilized (due to the inclusion of cases with unequal follow-up lengths); the results they obtained using odds ratios were essentially identical. Babchishin et al. (2012a) and Lehmann et al. (2016) also found their data were a good fit to both logistic and exponential curves, so chose to calculate risk ratios using hazard ratios because of sample size and ease of interpretability. The sample groups in these three studies included only sexual offenders and the recidivism base rate was low, which may explain the similar results found using odds ratios and hazard ratios.
VRAG-R
The VRAG-R (Rice et al., 2013) was developed as a replacement tool for both the VRAG and the Sex Offender Risk Appraisal Guide (SORAG). The intention was to create a static actuarial instrument that was easier to score and could be used to assess the risk of future violence for both general and sexual offenders. A study to develop and validate the VRAG-R indicated the VRAG-R was an accurate predictor (area under the curve [AUC] values of around .75) of the likelihood, frequency, and severity of long-term violence in a forensic population, and predicted at least as well as the tools it was intended to replace (Rice et al., 2013). Predictive accuracy was similar when the scores were broken down into nine risk categories roughly based on score deciles (two of the middle deciles were combined, creating nine categories). Three validation studies have supported its inter-rater reliability and relative predictive accuracy (Glover et al., 2017; Gregório Hertz et al., 2021; Olver & Sewall, 2018).
The sample of 1,261 men used for the VRAG-R validation study comprised individuals included in three previous studies: Harris et al. (1993, 2003) and Quinsey et al. (1995). That sample, described by Rice et al. (2013), has some characteristics more typical of forensic psychiatric hospitals than correctional institutions (e.g., relatively high prevalence of offenders assessed to determine whether they were not criminally responsible due to mental disorder, high rates of schizophrenia and homicide offenses compared with typical offender samples). Although relative predictive accuracy has been supported in validation studies in diverse samples, these somewhat atypical features of the development sample raise questions about the generalizability of the VRAG-R recidivism norms. So far, studies examining calibration of the recidivism norms have found mixed results (Gregório Hertz et al., 2021; Olver & Sewall, 2018), an issue we will return to in the discussion.
Advancing a Standardized Language for Risk Assessment
The United States Council of State Governments Justice Center has recently proposed a standardized framework for communicating risk assessment information (Hanson, Bourgon, et al., 2017). The framework has five levels (Level 1 to Level 5) that correspond not only with likelihood of recidivism but also with information about the criminogenic needs and correctional response most likely to effectively reduce recidivism for individuals who fall in those categories. It is intended that the framework will be able to be applied to a range of risk assessment tools measuring different outcomes with different types of offenders. The framework aims to address inconsistencies in the way risk categories are currently used and communicated.
To achieve that goal, the developers of the framework argue that the full range of statistical indicators of risk should be developed for an assessment tool. These indicators include absolute recidivism rates, percentile ranks, and risk ratios. The framework has already been applied to several tools that have these metrics available, including the Static-99R and Static-2002R (Hanson, Babchishin, et al., 2017), the Violence Risk Scale—Sexual Offense version (VRS-SO; Olver et al., 2018), and the STABLE-2007 (Brankley et al., 2017), and a newly developed tool for juvenile offenders (PrediCT; Tafrate et al., 2018). All but the latter of these tools focus on sexual offenders. As a commonly used actuarial instrument that assesses the risk of future violence, the VRAG-R would be a valuable addition to the existing list of tools to which the framework has been applied. For that to happen, it is first necessary to develop risk ratios for the VRAG-R.
The Current Study
It is important that actuarial risk assessment tools are continually refined and improved (Dawes et al., 1989). Ongoing development may include using larger or more representative normative data, or, as in this case, providing better information for how to communicate results obtained using the instrument. The aim of this study is to develop risk ratios for the VRAG-R (Rice et al., 2013) to broaden the range of risk communication options available when using this tool and to provide information needed for future efforts to apply the Justice Center’s standardized risk framework to the scale. To do so, we used the existing VRAG-R normative data set. This data set has previously been used to develop absolute recidivism estimates and percentile ranks associated with scores on the VRAG-R (Harris, Rice, et al., 2015). Risk ratios developed previously have focused primarily on sexual offenders, making this study one of the first to develop risk ratios for a measure that is focused specifically on risk of future violent offending. The different methods for calculating risk ratios—rate ratios, odds ratios, and hazard ratios—were tested to identify the best approach. Finally, a set of risk ratios associated with the VRAG-R are presented for use by researchers and practitioners.
Method
Measure
VRAG-R
The VRAG-R is a 12-item actuarial risk assessment instrument (Harris, Rice, et al., 2015; Rice et al., 2013). All items on the instrument are historical, with items assessing different aspects of offending history, childhood problems, and sociodemographic variables such as age and marital status at the time of the index offense. Each item has its own weighting with possible scores ranging between −7 and +6 on any one item, developed using the Nuffield (1982) weighting system. Total scores for the full VRAG-R (obtained by addition of scores on the 12 individual items) can range between −34 and +44. Absolute estimated violent recidivism rates over short (5-year) and long (12-year) follow-up periods for each risk bin, and percentile ranks for the individual scores were reported by Harris, Rice, et al. (2015). There are some updates in the VRAG-R category cut-offs and normative data from the original validation study (Rice et al., 2013) in the version and manual published in the book (Harris, Rice, et al., 2015); the book, being more recent, was used as the canonical version.
Sample
In this study, we started with the same sample of 1,261 men that was used to develop and validate the VRAG-R. Full details of the sample group were provided by Rice et al. (2013). Briefly, the sample included 658 men from the Harris et al. (2003) study. Just over half of these men had been found not guilty by reason of insanity for a violent offense, and the other half had been convicted for a violent offense. The remaining 603 men were drawn from the studies by Harris et al. (1993), and Quinsey et al. (1995), which both included men who had been convicted of a sexual offense and, predominantly, had been incarcerated or placed in a secure psychiatric hospital as a result of that offense. Some individuals were included in more than one of those studies but were only included once in our sample. In total, 60% of the 1,261 men had a history of sex offenses, and 34% had an index offense of homicide. On average, the men were 28 years old (SD = 10.5) when they committed their index offense, and 34 years old (SD = 11.6) when they were released. In addition, 8% of the sample had attended any college or university and 16% met the Diagnostic and Statistical Manual of Mental Disorders (3rd ed.; DSM-III; American Psychiatric Association, 1980) criteria for schizophrenia.
From the initial sample, there were five men who did not have a VRAG-R score available in the data set, so these men were excluded from all analyses. There were also 13 men who had missing or inconsistent follow-up data, which was needed for the Cox regression analyses to calculate hazard ratios. These men were also excluded from all analyses, leaving a final sample of 1,238 men.
Recidivism
The men were all released into the community between 1960 and 1995 and follow-up information was collected between 2003 and 2007. The average follow-up length was 256 months (SD = 102). Recidivism was defined as any new charge for a violent offense. Recidivism data for all cases were obtained from the Canadian Police Information Centre (CPIC), maintained by the Royal Canadian Mounted Police (RCMP). In addition, for offenders in the VRAG construction sample, data were occasionally available from other sources, such as INTERPOL reports, parole systems, Review Boards (who manage men found not criminally responsible), and provincial corrections.
The violent recidivism rate over the full variable follow-up period was 51%. Logistic regression analyses require a fixed follow-up period. For these analyses, only men who had sufficient opportunity to recidivate were included; thus, men who had not been released for the required length of time when follow-up data were collected, or who had died before enough time has elapsed, were excluded from the analyses. The recidivism rate over a fixed 5-year follow-up period was 32% (n = 1,191).
Procedure
The VRAG-R was completed using information collected prior to release, and without knowledge of outcome. For participants whose index offense resulted in a community sentence, the VRAG-R was scored based on information prior to the index offense (Rice et al., 2013).
Data Analysis
All analyses were conducted separately by the first and second author to ensure accuracy. For the primary analyses, we examined risk ratios using the nine risk bins of the VRAG-R as the dependent variable, rather than using individual scores (of which there are 79 possible scores). Observed violent recidivism rates are more variable across a larger range of scores because of the lower sample sizes; in this sample, the recidivism rate was zero for several individual scores, making observed and expected rate comparisons impossible for those scores. The risk bins have previously been shown to predict violent recidivism with a very similar level of accuracy to the individual scores (Rice et al., 2013). For the final risk ratio data, however, we have presented results for both risk bins and total scores, so either option is available for risk evaluators. In addition, risk ratio data at the total score level will be needed in future to apply the Justice Center standardized five-level framework to the VRAG-R.
In conducting our analyses, we followed the same approach taken by Hanson et al. (2013). We calculated the rate and odds ratios for a fixed 5-year follow-up, and hazard ratios for the full variable follow-up period. Next, we compared how well the logistic (from the odds ratio) and exponential (from the hazard ratio) functions fitted the observed recidivism rates over the fixed 5-year follow-up. Finally, we tested the stability of the relative risk and base rate estimates over 15 years. All analyses were run using SPSS Statistics version 24 and 25.
Results
Comparing Different Estimation Techniques
The first assumption inherent to a measure of relative risk is that a one-unit increase in scores is associated with a consistent increase in relative risk. To test this assumption, we calculated rate, odds, and hazard ratios for the VRAG-R risk bins, where each bin is compared with the lower bin. Rate and odds ratios for the VRAG-R bins over a 5-year follow-up period are presented in Table 1. The rate ratios ranged between 1.12 and 1.62, whereas the odds ratios ranged between 1.14 and 2.15. The weighted averages indicated that the probability of violent recidivism increased by 31%, and the odds increased by 57%, with each increase in risk bin. As an example in terms of absolute recidivism values, a 31% increase would be going from an absolute probability of recidivism of 30% to roughly 39% (0.31 × 30 = 9, then added to the original 30).
Rate Ratios and Odds Ratios for the VRAG-R Risk Bins Predicting Violent Recidivism Over 5-Year Follow-up (N = 1,191).
Note. VRAG-R = Violence Risk Appraisal Guide—Revised.
For the rate ratios and odds ratios, averages were weighted using the sample size of the higher bin.
To formally test how well the relationship between VRAG-R bins and violent recidivism fit a logistic function, we ran a logistic regression with a 5-year fixed follow-up. The model had a B1 of .414 (SE = .030) and a B0 of −2.968 (SE = .188), which translates to an average odds ratio of 1.51 (95% confidence interval [CI] = [1.43, 1.61]). The Hosmer and Lemeshow test (Hosmer et al., 2013) indicated a good fit between the data and the logistic function (χ2 = 8.27, df = 7, p = .309). A small but significant improvement in the model fit was observed when a curve component (i.e., the square of the VRAG-R bins) was added to the logistic regression model (χ2 change = 5.36, df = 1, p = .021). Similar results were obtained from the VRAG-R total scores. This model had a B1 of .057 (SE = .004) and a B0 of −.935 (SE = .074), translating to an average odds ratio of 1.06 (95% CI = [1.05, 1.07]). The Hosmer and Lemeshow test indicated a good fit between the data and the logistic function (χ2 = 14.82, df = 8, p = .063), but again, the addition of a curve component significantly improved the model fit (χ2 change = 5.41, df = 1, p = .020). The curve component essentially adds a small transformation to the data to improve the fit, if needed.
To test how well the relationship between the VRAG-R and violent recidivism fit an exponential function, we ran a series of Cox regressions. For the VRAG-R bins across the full sample, the model had a β of .304 (SE = .017), which translates to an average hazard ratio of 1.35 (95% CI = [1.31, 1.40]). Allowing a curve component did not significantly improve the model (χ2 change = .02, df = 1, p = .900). Similarly, for the VRAG-R total scores, the model had a β of .042 (SE = .002), and an average hazard ratio of 1.04 (95% CI = [1.04, 1.05]). The model did not demonstrate a significantly better fit when a curve component was added (χ2 change = .05, df = 1, p = .815).
To illustrate how well the different estimation approaches fit the observed violent recidivism rates, in Figures 1 and 2 we present the observed 5-year recidivism rates alongside the estimated recidivism rates derived from logistic and Cox regression. Following the results of the previous section, the logistic regression model included a curve component (but the Cox regression model did not). The Cox regression estimates were calculated by first obtaining the base rate for the median score (Bin 5 in Figure 1 and a score of 0 in Figure 2) derived from logistic regression (which included a curve component), and multiplying that value by the corresponding hazard ratios calculated using the average hazard ratio from the Cox regressions presented above. The logistic and exponential estimates provided an accurate measure of recidivism for low values (whether bins or total scores); for higher values though, while the logistic function continued to provide a close estimation of the observed rates, the exponential function increasingly overestimated the observed recidivism rate. For total scores, this over-estimation also led to predicted recidivism probabilities above 1.00, which are impossible values.
Observed 5-year violent recidivism rates plotted along with the 5-year violent recidivism rates derived from logistic and Cox regression analyses.
Observed 5-year violent recidivism rates for the VRAG-R total scores plotted along with the 5-year violent recidivism rates derived from logistic and Cox regression analyses.
Stability Across Time
The initial analyses indicated that the relationship between the VRAG-R and violent recidivism was more consistent with a logistic rather than an exponential function.
To examine whether that relationship was consistent across time, the odds ratio and estimated recidivism rates derived from logistic regression were calculated for fixed follow-ups ranging from 1 to 15 years. Recidivism estimates (and associated 95% CIs) were calculated using the formulae provided by Hosmer et al. (2013) for deriving estimates from logistic regression. The median values of 5 for the VRAG-R bins and 0 for the VRAG-R scores were again used to calculate these estimates. Figure 3 shows the estimated recidivism rates steadily increased over longer follow-up periods before flattening out over the longest follow-up periods. In contrast, the odds ratios were consistent across all fixed follow-up periods, with ratios ranging between 1.47 and 1.58, representing only small, nonsignificant, deviations from the odds ratio that was calculated for the 5-year follow-up. Figure 4 illustrates that the pattern of results was almost identical for the VRAG-R total scores.
Odds ratios and base rates of violent recidivism for the VRAG-R bins derived from logistic regression across a 15-year follow-up period.
Odds ratios and base rates of violent recidivism for the VRAG-R total scores derived from logistic regression across a 15-year follow-up period.
To examine whether the hazard ratios were consistent over time, we calculated the Schoenfeld residuals for the full follow-up. Results indicated that for both the VRAG-R bins and the total scores, there was a violation of this assumption of Cox regression (i.e., that hazards are proportional over time). A weak but significant negative relationship was observed between the residuals and time for both the VRAG-R bins (r = −.11, n = 629, p = .006) and the total scores (r = −.10, n = 629, p = .010). These results suggested the effect of the VRAG-R on the hazard ratio was not consistent over time, but this relationship was quite small and could be due to relatively minor fluctuations given the high statistical power of this test. Removing the first 6 months of data (which would presumably be the least stable), the relationship was reduced and no longer statistically significant (VRAG-R bins, r = −.081, p = .059, n = 543; total scores, r = −.073, p = .091, n = 543). This result suggested that we need not be particularly concerned about this statistical assumption; after the first 6 months of follow-up, the accuracy of the VRAG-R was fairly consistent across time.
Risk Ratios for the VRAG-R
Although Static-99R and Static-2002R risk ratios were based on hazard ratios from Cox regression, the current analyses suggested the exponential model in Cox regression was a poor fit to the observed violent recidivism data in this sample. Logistic regression was viewed as the preferred model on which to base the development of risk ratios, and analyses suggested that a model with a single curve component added provided the optimal fit. Unfortunately, however, the metric of odds ratios from logistic regression are not intuitive to interpret. For example, an odds ratio of 2 does not mean that this group is twice as likely to reoffend as the reference group; rather, it means that the odds of recidivism are twice as high in that group compared with the odds of recidivism in the reference group. Phrasing for odds ratios is clunkier and liable to be misinterpreted as a rate ratio, which can lead to misleading conclusions.
Consequently, we opted to create risk ratios for the VRAG-R by computing rate ratios (a metric that is easier to interpret) from expected recidivism rates derived from the logistic regression model, which was what provided the best fit for the data. Appendix A presents the final risk ratios for VRAG-R risk bins, and Appendix B presents the final risk ratios for VRAG-R total scores. These ratios were derived by comparing the predicted recidivism rates (from logistic regression) to the rates predicted for the median score (a bin of 5, or a total score of 0). The resulting risk ratios can be interpreted as a rate ratio, consistent with other risk assessment scales (e.g., Static-99R, Static-2002R). Appendix C includes examples of how these risk ratios could be reported in a risk assessment evaluation.
Discussion
The VRAG-R already contains normative data for percentiles and absolute recidivism estimates (Harris, Rice, et al., 2015). Using the same data set, we have developed risk ratios for the VRAG-R to add to the repertoire of risk communication metrics available to evaluators using the scale. In addition to offering an additional risk communication metric, this study also provides necessary information for eventually applying the Justice Center standardized risk levels to the VRAG-R, which would offer empirically grounded risk communication language (Hanson, Bourgon, et al., 2017) consistent with other risk assessment scales (e.g., Brankley et al., 2017; Hanson, Babchishin, et al., 2017; Olver et al., 2018; Tafrate et al., 2018).
Although it is often helpful to follow similar approaches used by other scales or researchers, this study demonstrated that it is necessary to assess whether these approaches are actually applicable. Whereas Static-99R, Static-2002R, and Risk Matrix 2000 risk ratios were based on hazard ratios from Cox regression analyses (Babchishin et al., 2012a; Hanson et al., 2013; Lehmann et al., 2016), this approach proved untenable for the VRAG-R, given poor fit to the data. Following the same methodology for developing risk ratios as the STATIC scales would have led to considerable overestimation of risk, particularly for higher VRAG-R scores.
The reason for this discrepancy is because the ever-increasing exponential function (in Cox regression) and the S-shaped logistic function (in logistic regression) tend to yield similar results when recidivism rates are low, as is often the case for sexual recidivism. However, as recidivism rates increase, results obtained using the different methods start to diverge. For a scale predicting an outcome with a much higher base rate such as violent recidivism, the difference in results can become substantial (with the exponential function overestimating risk), suggesting that Cox regression models are not suitable for developing risk ratios unless the expected base rates are low.
As a result of this key difference, an alternative approach to developing risk ratios was used in this study, where predicted recidivism rates from a logistic regression model were used to calculate rate ratios. Aside from the use of logistic regression over Cox regression (and adding a curve component to increase the fit of the logistic regression model), these analyses demonstrated other similarities with the STATIC scales, including that risk ratios were generally stable across time.
The need to base risk ratios in logistic regression required some additional decisions. Whereas Cox regression analyses would have averaged the results across the full length of follow-up, logistic regression analyses are tied to fixed follow-up periods, and it was necessary to choose a follow-up period for calculations. We selected 5-year follow-up data as the key outcome because it is generally consistent with other research; current VRAG-R data presents 5-year data (in addition to 12-year data) and it allowed us to retain most cases for analyses. Given that the results indicated little variation in odds ratios across follow-up periods ranging from 1 to 15 years, the final risk ratios would not be meaningfully different had other follow-up periods been used.
Justice Center Standardized Risk Framework
Providing risk ratio data is one necessary step in applying the Justice Center’s standardized risk framework for the VRAG-R (Hanson, Bourgon, et al., 2017). Technically, creating the five standardized risk levels requires data on recidivism estimates, risk ratios, and percentiles. However, we do not recommend developing the five risk levels for the VRAG-R yet. In particular, the recidivism estimates should be based on relatively routine (i.e., unselected and broadly generalizable) samples, and we are not yet convinced that the absolute recidivism estimates for the VRAG-R are sufficiently generalizable to routine correctional samples of violent offenders. It is entirely possible that they are, but further empirical validations in both forensic psychiatric and correctional samples is needed to verify this.
Likely due to the myriad factors that influence recidivism (e.g., follow-up length, recidivism definition, variations in jurisdictional practices including quality of criminal records), absolute recidivism estimates for several sex offender risk assessment scales have been found to be unstable across diverse samples (Helmus, 2018). For the original VRAG, two studies using Canadian samples roughly similar to those used in the development of the VRAG have supported the generalizability of the recidivism estimates (Harris et al., 2002, 2003). In contrast, however, exploratory analyses using outcome measures somewhat different than that employed in the VRAG construction studies have found that the VRAG probabilities may overestimate violent recidivism in populations that differ from the one used in construction (Mills et al., 2005; Snowden et al., 2007).
One study of moderately preselected sex offenders in Austria found good calibration between VRAG-R recidivism norms and the Austrian data (Gregório Hertz et al., 2021), whereas another study of treated sex offenders preselected as fairly high risk found recidivism rates per VRAG-R bin roughly one-third lower than what the normative data would predict (these differences were statistically significant for the highest risk bin and for the overall sample [the latter analysis was not reported in the study but can be calculated from the available information]; Olver & Sewall, 2018). It is possible these mixed findings are a result of some of the atypical features of the VRAG-R development and validation sample (e.g., the high prevalence of mental illness). Further validation studies (in both forensic psychiatric and correctional samples) are needed to better explore where there is variability across samples in recidivism rates per VRAG-R risk bin (including studies containing violent non-sexual offenders). In the meantime, we believe that it is wise to hold off on applying the Justice Center risk categories pending further validation of the recidivism estimates for the scale.
Implications for Practice
The VRAG-R risk ratios can be considered an adjunct to the existing normative data for the VRAG-R (contained in Harris, Rice, et al., 2015), and can be included in risk assessment reports. Note that it is just one of several risk communication metrics (e.g., along with the risk bin, percentile, and absolute recidivism estimates). Each risk communication metric has its own strengths and weaknesses (Lehmann et al., 2016). Risk ratios are generally more stable across samples than absolute recidivism estimates (e.g., Helmus et al., 2012), but they have their drawbacks as well. They can be difficult to interpret in the absence of general knowledge of base rates, and consequently, similar to Hanson et al. (2013), we recommend that base rate information be provided when risk ratios are used. In addition, Varela et al. (2014) have demonstrated that laypeople may struggle to interpret risk ratios, so it is important to try and explain them as clearly as possible. The reporting templates in Appendix C may help, but we expect further research and input could lead to better reporting examples. A recent study by Helmus et al. (2018) suggests that optimal risk communication should incorporate all four risk communication metrics (i.e., risk bin, recidivism estimate, percentile, and risk ratio).
Although this article provides risk ratios for both VRAG-R total scores as well as the nine risk bins, we think that it may often be simplest to report the risk ratios for the bins rather than the total score, as this would be consistent with the other normative data currently available for the VRAG-R. The risk ratios per score were largely provided as a necessary step for future empirical efforts in applying the Justice Center’s standardized risk levels to the scale.
The appendices include the recidivism estimates (generated from the logistic regression model) that were used as the basis for calculating the risk ratios. Users will note that there are some minor differences in these logistic-regression-derived estimates and the current recidivism norms for the VRAG-R (found in Harris, Rice, et al., 2015). These are due to small differences in data cleaning decisions, but primarily because they are based on a logistic regression model (with a curve component added), whereas the existing recidivism norms are based on observed data from fixed follow-up periods. Although it is possible that recidivism norms for the VRAG-R may be revised in the future (e.g., based on logistic regression or to incorporate additional samples), the published book (Harris, Rice, et al., 2015) should still be considered as the authoritative recidivism norms, unless and until new recidivism norms are released (if so, they would be posted on www.vrag-r.org).
These implications for practice should be interpreted in light of some limitations of this study. Most notably, as highlighted earlier, the VRAG-R validation sample may have some unique characteristics that raise the question of generalizability. One characteristic that may have particular relevance is the age of the VRAG-R sample. The fact the data were collected a long time ago provides the advantage of allowing long follow-up analyses but also raises the possible issue of cohort effects. Although analyses in the development of the VRAG-R did not find evidence of cohort effects in the study period (1960–1995; Rice et al., 2013), crimes rates in Canada peaked in the early 1990s and have been generally declining since then (this applies to violent, property, and sexual offenses; Mishra & Lalumière, 2009a, 2009b; Public Safety Canada, 2008) so potential cohort effects would likely be more notable in the 1990s and onwards.
Changes over time in crime rates (which would include both first-time and repeat offenders) would not necessarily mirror changes in recidivism rates. However, there are some data for sex offenders suggesting similar declines in recidivism rates in the same time period (Minnesota Department of Corrections, 2007). In addition, other indicators of impulsive behavior (e.g., accidents, suicide, risky sexual behavior, dropping out of school) have shown similar declines (Mishra & Lalumière, 2009b) suggesting that recidivism base rates are likely subject to similar cohort effects as crime rates. Even still though, cohort effects in recidivism rates may be partly attributed to cohort differences in risk factors, which would be accounted for in actuarial risk scales like VRAG-R. In any case, this possible limitation is largely mitigated by the fact any cohort effect of this description will be more relevant to absolute recidivism estimates and is likely to have less impact on relative risk metrics, which are generally more robust across diverse samples and time periods (e.g., Helmus et al., 2012).
Other unique features of this data set may also impact generalizability, such as socioeconomic status, race, ethnicity, language, or sexual orientation. Research generally has not demonstrated these potential differences to be key predictors of recidivism (Bonta & Andrews, 2016). Thus, we do not expect them to meaningfully limit generalizability, but it is nonetheless important to acknowledge, especially as suggestions for future research.
Conclusion
A growing body of literature on commonly used risk communication metrics has supported greater focus on relative risk measures (Harris, Lowenkamp, & Hilton, 2015; Helmus, 2018; Scurich, 2018). This study adds to that body of research by providing risk ratios, a measure of relative risk, for the VRAG-R (and templates for using those risk ratios in practice). As one of the first studies to develop risk ratios for a measure intended to predict violent recidivism, the analyses highlight the importance of the outcome measure being used when developing risk ratios. In this case, the higher base rate of violent recidivism (compared with sexual recidivism, for example) meant logistic regression provided a better method for developing risk ratios than Cox regression. Further research should examine the extent to which a similar approach might be warranted with other risk tools for which relative risk metrics are being developed. These issues will take on increasing importance as more risk tools are applied to the Justice Center’s standardized risk levels, and hopefully assist with the development of a clearer and more universal language of risk communication that can further advance the practice of forensic and correctional psychology.
Footnotes
Appendix A
Risk Ratios for the VRAG-R Bins.
| Bin | Predicted Violent Recidivism Rate as Percentage | Risk Ratio (Relative to Risk Bin 5) |
|---|---|---|
| 1 | 10.2 | 0.40 |
| 2 | 12.1 | 0.47 |
| 3 | 15.0 | 0.58 |
| 4 | 19.3 | 0.75 |
| 5 | 25.7 | 1.00 |
| 6 | 34.6 | 1.35 |
| 7 | 46.3 | 1.80 |
| 8 | 59.8 | 2.33 |
| 9 | 73.1 | 2.85 |
Note. Predicted recidivism rates were obtained from 5-year logistic regression analysis (N = 1,191) with a curve component added. The final model included VRAG-R bin (B1 = .1010, SE = .1358, p = .457), VRAG-R bin squared (B2 = .0296, SE = .0127, p = .020), and the intercept (B0 = −2.3075, SE = .3260, p < .001). VRAG-R = Violence Risk Appraisal Guide—Revised.
Appendix B
Risk Ratios for the VRAG-R Scores.
| Score | Predicted Violent Recidivism Rate as Percentage | Risk Ratio (Relative to Score of 0) |
|---|---|---|
| −34 | 9.1 | 0.36 |
| −33 | 9.3 | 0.36 |
| −32 | 9.4 | 0.37 |
| −31 | 9.6 | 0.38 |
| −30 | 9.8 | 0.39 |
| −29 | 10.0 | 0.39 |
| −28 | 10.2 | 0.40 |
| −27 | 10.5 | 0.41 |
| −26 | 10.7 | 0.42 |
| −25 | 11.0 | 0.43 |
| −24 | 11.3 | 0.44 |
| −23 | 11.6 | 0.46 |
| −22 | 11.9 | 0.47 |
| −21 | 12.2 | 0.48 |
| −20 | 12.6 | 0.49 |
| −19 | 13.0 | 0.51 |
| −18 | 13.4 | 0.52 |
| −17 | 13.8 | 0.54 |
| −16 | 14.2 | 0.56 |
| −15 | 14.7 | 0.58 |
| −14 | 15.2 | 0.60 |
| −13 | 15.7 | 0.62 |
| −12 | 16.2 | 0.64 |
| −11 | 16.8 | 0.66 |
| −10 | 17.4 | 0.68 |
| −9 | 18.0 | 0.71 |
| −8 | 18.7 | 0.74 |
| −7 | 19.4 | 0.76 |
| −6 | 20.2 | 0.79 |
| −5 | 20.9 | 0.82 |
| −4 | 21.7 | 0.86 |
| −3 | 22.6 | 0.89 |
| −2 | 23.5 | 0.92 |
| −1 | 24.4 | 0.96 |
| 0 | 25.4 | 1.00 |
| 1 | 26.5 | 1.04 |
| 2 | 27.5 | 1.08 |
| 3 | 28.7 | 1.13 |
| 4 | 29.8 | 1.17 |
| 5 | 31.1 | 1.22 |
| 6 | 32.3 | 1.27 |
| 7 | 33.7 | 1.32 |
| 8 | 35.0 | 1.38 |
| 9 | 36.4 | 1.43 |
| 10 | 37.9 | 1.49 |
| 11 | 39.4 | 1.55 |
| 12 | 41.0 | 1.61 |
| 13 | 42.6 | 1.67 |
| 14 | 44.2 | 1.74 |
| 15 | 45.9 | 1.80 |
| 16 | 47.6 | 1.87 |
| 17 | 49.4 | 1.94 |
| 18 | 51.1 | 2.01 |
| 19 | 52.9 | 2.08 |
| 20 | 54.7 | 2.15 |
| 21 | 56.5 | 2.22 |
| 22 | 58.4 | 2.30 |
| 23 | 60.2 | 2.37 |
| 24 | 62.0 | 2.44 |
| 25 | 63.8 | 2.51 |
| 26 | 65.6 | 2.58 |
| 27 | 67.4 | 2.65 |
| 28 | 69.2 | 2.72 |
| 29 | 70.9 | 2.79 |
| 30 | 72.6 | 2.85 |
| 31 | 74.2 | 2.92 |
| 32 | 75.8 | 2.98 |
| 33 | 77.3 | 3.04 |
| 34 | 78.8 | 3.10 |
| 35 | 80.3 | 3.16 |
| 36 | 81.6 | 3.21 |
| 37 | 82.9 | 3.26 |
| 38 | 84.2 | 3.31 |
| 39 | 85.4 | 3.36 |
| 40 | 86.5 | 3.40 |
| 41 | 87.6 | 3.44 |
| 42 | 88.6 | 3.48 |
| 43 | 89.5 | 3.52 |
| 44 | 90.4 | 3.55 |
Note. Predicted recidivism rates were obtained from 5-year logistic regression analysis (N = 1,191) with a curve component added. The final model included VRAG-R score (B1 = .05320, SE = .00426, p < .001), VRAG-R score squared (B2 = .00050, SE = .00022, p = .020), and the intercept (B0 = −1.07563, SE = .09617, p < .001). VRAG-R = Violence Risk Appraisal Guide—Revised.
Appendix C
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
