Abstract
Doubt is not a pleasant condition, but certainty is an absurd one.
Many readers and contributors to Criminal Justice and Behavior (CJB) have been trained on and inculcated into a culture that supports the use of actuarial risk assessment instruments (including ourselves). In fact, much of the hallmark research on offender risk assessment has been published in this journal over the last three decades (e.g., Andrews, Bonta, & Hoge, 1990; Barbaree, Seto, Langton, & Peacock, 2001; Bonta, 2002; Gendreau, Goggin, & Law, 1997; Jesness, 1988; Monahan, 1996; Schwalbe, 2008; Simourd, 2004). Still other scholars firmly caution against the use of risk assessment, arguing it produces biased, unfair inequities based on factors such as race, gender, and financial status (Hannah-Moffat, 2013; Harcourt, 2015; Starr, 2015). As scholars who were formerly part of the CJB editorial team, we felt this journal was an important venue for discussing the risk assessment debate, and to encourage ongoing discourse.
Outside the pages of academic journals, and in growing parts of the criminal justice practitioner community, the concept of “risk assessment” has become anathematized. Indeed, a representative of a major grant-awarding foundation recently recommended that one of us (Emily Salisbury) consider removing the area of “risk assessment” from her expertise and instead focus only on “needs assessment” of justice-involved individuals. Although this offhand suggestion was intended as polite advice for self-marketing purposes, Emily wondered if the representative knew the sweeping implications of her statement. Dismissing risk assessment would mean dismissing a criminological fact that people do not pose an equal likelihood of engaging in criminal behavior and that certain social psychological factors are more statistically predictive of offending (i.e., criminogenic vs. noncriminogenic needs).
Much of the controversy over risk assessment is focused on the preconviction stage of the criminal justice system for understandable reasons. Prior to conviction, the liberty interests of individuals accused of crimes is high and significant questions must be answered. What level of risk, for what type of event, over what time period, justifies the use of preventive controls especially when those controls may result in significant social and financial harms and racially disparate outcomes (Sutton, 2013)? Yet, the issues of fairness and equity that have been raised on the front-end of the criminal justice system are also applicable to postconviction stages (e.g., sentencing decisions, probation supervision, prison classification, parole decisions, and revocation decisions). The primary goal of our effort to compile a special section focused on “Risk Assessment and Judicial Decision Making” was to draw out some of the critical nuances that are oftentimes dismissed or overlooked in the discourse. We hoped to bring more clarity (even if only incremental clarity) to the complex issue, though perhaps more questions than answers have been raised.
Various measures of bias exist in relation to risk assessments and decision making, including predictive bias and disparate impact (Berk, 2016; Chouldechova, 2017; Skeem & Lowenkamp, 2016). Overall, a risk assessment is considered well-calibrated when it predicts outcomes equally well across subgroups, yet we must also consider the implications of false positives and false negatives across subgroups (Chouldechova, 2017; Kleinberg, Mullainathan, & Raghavan, 2016; Skeem & Lowenkamp, 2016). As discussed in the first article of the special section (Eckhouse, Lum, Conti-Cook, & Ciccolini, 2019), adhering to all measures of fairness, equal prediction rates across groups, as well as treating members of groups similarly, is impossible. Therefore, it is crucial for judicial decision-makers to carefully consider specific measures of fairness and their contextual implications for sentencing outcomes and, ultimately, public safety.
Eckhouse et al. (2019) provide a critical framework for understanding what they deem as three “layers of bias” that underlie decision making using risk assessments. The top layer challenges fairness within the risk assessment models themselves in regard to varying conceptions of statistical fairness. The second layer reflects the bias of using data from a racially biased criminal justice system. Finally, the authors describe the foundational layer which questions the fairness of using group-level assumptions within risk assessment algorithms to make predictions and decisions about individual defendants. The authors argue that each layer collapses if the layer below is deemed unfair.
The subsequent three articles in the section examine predictive fairness in risk assessments used for judicial decision making across race, gender, or offense-type. First, Lowder, Morrison, Kroner, and Desmarais (2019) investigate potential racial biases using the Level of Service Inventory-Revised (LSI-R) in predicting sentence length and probation outcomes. Their results demonstrate some evidence of racial bias—Black probationers under higher risk classifications received longer sentences and were less likely to receive a new charge in comparison to White probationers classified at the same risk levels. Next, Cohen and Lowenkamp (2019) assess potential racial and gender biases in outcomes predicted by the U.S. federal court’s Pretrial Risk Assessment (PTRA). Overall, the PTRA exhibited moderate to strong predictive validity across racial and gendered subgroups, yet there was some evidence of overprediction for re-arrests among Hispanic and female defendants. Finally, Harbinson, Benson, and Latessa (2019) test the applicability of the Post Conviction Risk Assessment (PCRA) to inform judicial outcomes specifically among those convicted of white-collar offenses, a subpopulation that generally has different social and economic backgrounds than those commonly found among justice-involved persons. Their results indicate that the PCRA has strong predictive validity across risk levels, supporting the application of the risk principle to those convicted of white-collar offenses.
Compiling these articles, and doing our own interdisciplinary research on the subject, has admittedly been somewhat of a dissonant experience. As scholars firmly trained and rooted in evidence-based corrections, which promotes risk assessment as a crucial component, it is uncomfortable to confront the realization that there are inherent inequities embedded in these decision-making tools. However, we understand that regressing back to a time of not having an actuarial risk assessment to inform justice decision making may only exacerbate inequity in the system based on extralegal factors. At the same time, we also recognize that we may unknowingly have entire chasms in knowledge around the issue, in part as a result of risk assessment researchers in the social sciences failing to engage with the criticisms of scholars from other disciplines. These scholars often promote novel questions and approach scientific inquiry using distinct methods compared with criminology and correctional psychology, and we feel dialogue and collaboration is more beneficial than outright dismissal. Our experience with the blind peer review process in developing this section was a telling experience—a reminder to us about the serious need for intellectual humility in our fields. To engage in scientific inquiry means to understand we never know everything about any single phenomenon. Yes, we “know” quite a lot about the importance of risk assessment as it relates to correctional philosophy, but perhaps not as much as it relates to true fairness and equity. It is not as if we should stop asking (or listening to) critical questions about the concept, particularly questions generated by scholars from outside criminology, or individuals who have lived experiences in the criminal justice system.