Abstract
There is ongoing debate and confusion over using actuarial scales to predict individuals’ risk of sexual recidivism. Much of the debate comes from not distinguishing Frequentist from Bayesian definitions of probability. Much of the confusion comes from applying Frequentist probability to individuals’ risk. By definition, only Bayesian probability can be applied to the single case. The Bayesian concept of probability resolves most of the confusion and much of the debate in sex offender risk assessment. Although Bayesian probability is well accepted in risk assessment generally, it has not been widely used to assess the risk of sex offenders. I review the two concepts of probability and show how the Bayesian view alone provides a coherent scheme to conceptualize individuals’ risk of sexual recidivism.
Introduction
There is an ongoing debate in sex offender risk assessment over using actuarial scales like the Static-99/R to predict the risk of individual offenders. Harris and Hanson (2004) claimed the Static-99 assumes that “the probability of recidivism for an individual offender will be the same as the observed recidivism rate for the group to which he most closely belongs” (p. 9). Berlin, Galbreath, Geary, and McGlone (2003) countered this common assumption, arguing that not all individuals with the same score share the same risk. Some writers agree and question whether the Static-99R can accurately predict any individual’s risk (Cooke & Michie, 2010; Donaldson & Abbott, 2011; Donaldson, Abbott, & Michie, 2012). Others contend that the Static-99/R actually does inform individual risk predictions (e.g., Elwood, 2009; Mossman, 2015; Scurich & John, 2011; Singh, 2013; Skeem & Monahan, 2011).
Much of that debate reflects confusion over the meaning of probability. Although evaluators may understand traditional Frequentist probability, many are unfamiliar with the Bayesian concept of probability. Even those who are familiar with calculating Bayesian probabilities may not appreciate that the Bayesian view is a fundamentally different way of defining probability. Bayesian probability is now widely accepted throughout the biomedical, behavioral, and physical sciences but has not been widely adopted in assessing the risk of sex offenders. I review the two definitions of probability and show how the Bayesian concept of probability resolves much of the confusion and debate in sex offender risk assessment.
Probability
It is possible to spend a lifetime analysing data without realising that there are two very different fundamental approaches to statistics: Bayesianism and Frequentism. (Lyons, 2007)
I define risk by the probability of an adverse event or outcome. Probability can be defined in either Frequentist or Bayesian terms. 1 While they may yield the same numerical value, they differ in what probability means. Frequentist probabilities are defined by relative frequencies, how frequently an event or outcome occurs over a series of repeated trials. A relative frequency is the ratio of events to trials. A 50% probability of a coin coming up heads means that heads will come up half the time. “The Frequentist position is not simply that the notions of probability and frequency are intimately connected, but that they are actually identical” (Appleby, 2005, p. 3). However, a single trial has no relative frequency. A single coin toss must result in either heads or tails. There is no series of trials or ratio of heads to tosses. Therefore, Frequentist probability cannot be meaningfully applied to a single event. However, sexual recidivism is a single event. Thus by definition, the concept of Frequentist probability cannot be applied to an offender’s risk of sexual recidivism.
Bayesian probabilities are not defined by relative frequencies but by our confidence in an outcome or event given the information we have. Bayesian probabilities are not the property of events. Rather, they reflect our knowledge about events.
It might be strange to say that we randomly sample from our beliefs, like we randomly sample from a sack of coins. Nevertheless, the mathematical properties of probabilities outside the head and beliefs inside the head are the same in their essentials . . . (Kruschke, 2011, p. 25)
Frequentist probabilities are taken to be objective realities. The probability of heads is considered a property of the coin toss. In the Bayesian scheme, there are no objective, true probabilities. In Bayesian terms, “probability is in the mind” (Yudkowsky, 2008). Some readers may feel uncomfortable with the view that risk reflects an evaluator’s subjective state of mind, rather than a property of the offender being assessed. However, an evaluator’s state of mind includes what they know about the offender. Bayesian probability may be subjective, but it is not arbitrary or capricious. “There is nothing inexact about Bayesian probabilities: they must satisfy precisely the same algebraic rules as frequentist probabilities” (Ambaum, 2012, p. 4). Each evaluator may assign different probabilities because they interpret the extant research differently. Moreover, all common procedures in statistics and probability demand subjective judgments and assumptions. Appleby (2005) contended that the Frequentist approach is just as subjective as the Bayesian, though its subjectivity is less obvious. This brief summary belies the great diversity among Bayesians and Bayesian methods. In practice, Bayesian probability is both subjective and objective. For a broader discussion, see Greenland (1998).
Bayesian Probability in Risk Assessment
In the Bayesian scheme, the risk of an outcome is reflected by prior and posterior probabilities. The prior probability (or simply “prior”) is the probability based on whatever information we have. For example, if all we know is the 10-year sexual recidivism rate of a high-risk reference group, that rate is our prior. The posterior probability reflects additional probability information, such as the recidivism rate of high-risk offenders with a certain Static-99R score. With even more information, the posterior probability may be the recidivism rate of offenders who completed sex offender treatment. It is often said that each posterior probability becomes the next prior probability as we continually add more information. However, Kruschke (2011) pointed out that there is no temporal ordering in the prior and posterior beliefs! Rather, the prior is simply the distribution of beliefs we hold by excluding a particular set of data, and the posterior is the distribution of beliefs we hold by including the set of data. (p. 12)
Bayesian probability is now widely accepted as a proper format for predicting risk in actuarial science (Dudley, 2006; Makov, 2001), epidemiology (Dunson, 2001), and throughout the biomedical and behavioral sciences (e.g., Altman & Royston, 2000; Ashby & Smith, 2000; Aven & Eidesen, 2007; Christian, Croghan, & Maxfield, 2013; Cook, 2008; Elwood, 1993; Gigerenzer, Gaissmaier, Kurz-Milcke, Schwartz, & Woloshin, 2008; McCormick, Rudin, & Madigan, 2012). The historical downside of Bayesian methods has been the difficulty generating the computational algorithms needed to calculate probabilities. However, this problem has largely been overcome by computers and specialized software. Ferson (2005) discussed the advantages and limitations of Bayesian methods in risk assessment.
Some readers may object to my equating risk assessment with risk prediction, agreeing with Doren (2006) that prediction is forecasting whether or not an event will occur. Doren argued that Bayesian probability applies only to prediction, risk assessment is not prediction, and therefore, Bayesian probability does not apply to risk assessment. However, Kruschke (2011) pointed out that statistically “prediction simply means inferring the values of some missing data based on some other included data” (p. 13). In that sense, risk assessment and risk prediction are the same; both are probabilistic. Moreover, the prediction of violence is well established (Monahan et al., 2005; Skeem & Monahan, 2011), and recidivism is routinely predicted in the sex offender literature (Hanson & Morton-Bourgon, 2004; Harris, Phenix, Hanson, & Thornton, 2003; Thornton, 2006; Wollert, 2006). Indeed, Bayesian probability applies to risk assessment precisely because risk is probabilistic.
In their classic paper, Meehl and Rosen (1955) introduced Bayesian probability to clinical psychology. Almost three decades later, Baldessarini, Finklestein, and Arana (1983) introduced it to American psychiatry, coining the terms positive predictive power (PPP) and negative predictive power (NPP) for posterior probabilities. Janus and Meehl (1997) introduced Bayesian probability to sex offender risk assessment. PPP has since been applied to the Static-99 (Beauregard & Mieczkowski, 2009; Bengtson & Långström, 2008; Wollert, 2006) and Static-99R (Campbell & DeClue, 2010; Fazel, Singh, Doll, & Grann, 2012; Neller & Frederick, 2013). Frederick and Bowden (2009) devised the Test Validation Summary, an interactive, graphic computer program to adjust positive predictive values (PPV, equivalent to PPP) for various recidivism base rates.
Sex Offender Risk Assessment
Meehl (1954) contended that “the clinician is always predicting actuarially and from classes whether he knows it or not” (p. 29). In Bayesian terms, an individual’s risk of sexual recidivism is the PPP of recidivism (however defined). Absolute risk probabilities are critical in the United States, where they may be used to commit offenders as sexually violent persons (SVPs). The PPP is derived from recidivism rates found in longitudinal studies of sex offenders. Actuarial scales like the Static-99R provide those rates. The confusion in sex offender risk assessment arises from not recognizing or distinguishing Frequentist from Bayesian probability. Nowhere is that confusion more evident than in the debate over using actuarial data to assess an individual’s risk.
Group Versus Individual Risk
Harris and Hanson’s (2004) assumption that an individual’s risk is the same as the group recidivism rate contradicts their own Static-99 coding rules: “The recidivism estimates provided by the Static-99 are group estimates [that] do not directly correspond to the recidivism risk of an individual offender” (Harris et al., 2003, p. 81). The report format in the Static-99R and Static-2002R Evaluators’ Handbook (Phenix, Helmus, & Hanson, 2012) likewise only cites the recidivism rate of a group, without reference to the risk of an individual.
Given the conflicting advice, many forensic evaluators disavow any individual prediction and offer instead something like “Mr. Smith’s score places him in a group of sexual offenders, 55% of whom were charged with another sex offense within 10 years after release.” At least one evaluator has struggled to answer an attorney’s question, “So doctor, is Mr. Smith in the 55% group or in the 45% group?” The attorney asked whether Mr. Smith is in the recidivist group. Of course, in the Bayesian scheme, Mr. Smith and everyone else has a 55% probability of being in the 55% group that recidivates. Both the question and the difficulty the evaluator had answering it reflect the confusion over “placing” individuals in groups, rather than assigning a probability to the individual.
Some U.S. state courts have accepted experts’ testimony that the Static-99 “is simply a statement about how groups of people perform.” “So you really can’t come up with an individual prediction” (State v. Morales, 2003), and the Static-99 “has no predictive value for an individual” (New York v. Rosado, 2009). However, Bayesian probabilities reflect what we know. If all we know is that everyone in a group shares the same Static-99R score, we know the same thing about each of them and assign the same probability to all of them. Moreover, courts require that evaluators testify to the risk of an individual and not just the recidivism rate in a group of similar individuals. Evaluators cannot coherently disavow any individual risk prediction and then render an opinion on an individual’s risk.
A colleague wrote to me, arguing, An individual’s reoffense probability can’t be 40% or 63% or 1%. That is a group probability. Think about the two segments of that group—the ones who did reoffend and the ones who didn’t. How can you say their individual probability was equal? One was 0% and one was 100%.
However, Bayesian probabilities are not validated by individual outcomes. “Probability statements are not about what is in fact the case, but about what one can reasonably expect to be the case” (Appleby, 2005, p. 448). For example, Alice buys one ticket in a lottery having 106 tickets, and her ticket wins. Even after it is known that Alice did win the lottery, we would still say that Alice was very unlikely to win. And we would be right to say it: because the statement, that Alice is unlikely to win, is not, primarily, a statement about the actual outcome. Rather, it is a statement about what Alice, and [we], could reasonably expect regarding the outcome. The fact, that Alice did win, does not alter the fact that she could not reasonably have expected to win (Appleby, 2005, p. 458).
In a post to an online forum, an evaluator used the example of “people whose true risk was 45%.” A colleague responded, “How can a person’s true risk be 45% [?] I either do or do not reoffend, I don’t do it 45%.” The concept of a true risk reflects the Frequentist view of probability, which, of course, cannot be applied to a single person. In the Bayesian view, there is no true probability. Probability depends on the information we have.
Hanson and Howard (2010) accepted that Bayesian probabilities apply to diagnosing current events but claim they do not apply to predicting future events that are uncertain and may never be known. However, all future outcomes are uncertain, which is precisely why we assign probabilities to them. One need only search “Bayes incidence” on Medline (PubMed, n.d.) to find thousands of studies that use Bayesian probability to predict the incidence of future events. Many studies relate current risk factors like smoking to future outcomes like lung cancer. Examples of prognostic prediction include the well-known Framingham Risk Scale (Framingham Heart Study, 2013) to predict an individual’s risk of heart disease and the Gail Model Calculator (Gail et al., 1989) to predict a woman’s risk of breast cancer, just as the Static-99R is used to predict an offender’s risk of sexual recidivism. Both sexual offenses and heart attacks are discrete events to which we can assign a probability. A Static-99R score does not predict whether a sex offender will reoffend, any more than a Framingham risk score predicts whether an individual will have a heart attack. Both scales are used to predict the probability of individual outcomes.
Because the Static-99R does not account for all recidivism risk factors, evaluators often consider external factors such as the combination of sexual deviance and psychopathy (Hawes, Boccaccini, & Murrie, 2013) and sex offender treatment (e.g., Duwe & Goldman, 2009). In Bayesian terms, these factors comprise additional information that is expressed in the form of probabilities. Because probabilities depend on a reference class, an evaluator incorporates such a factor by making an empirical judgment about a reference class, rather than a clinical judgment about an individual.
To be clear, PPPs derived from a Static-99R score are used to predict the probability that an individual will reoffend, not whether they will reoffend. If I assign a 60% PPP of recidivism, I do not predict that Mr. Smith will recidivate and then say that I am only 60% confident in my prediction. Rather, I predict a 60% probability of recidivism and then specify my confidence in that prediction by a margin of error around the PPP (in Bayesian terms, a credible interval).
Bayesian probabilities >0 and <1 cannot be validated by an individual outcome but, like all probabilities, they can be validated by a series of outcomes. If it rained on about 30% of the days a forecaster had predicted a 30% chance of rain, that finding would validate the forecasts. In fact, Bayesian probability has had enormous success predicting important single-case events (McGrayne, 2011; Silver, 2012). Of course, in predicting sexual recidivism, the outcome may not be known for decades, just as in predicting the risk of cancer (National Cancer Institute, 2013).
Subjective Prior Probability
The prior probability is central to the Bayesian concept of probability. Although priors remain controversial and can be quite complicated, they are relatively straightforward in sex offender risk assessment because sexual recidivism itself is a fairly simple concept. If we have very little information, the prior may be no more than a plausible assumption. “In the absence of genuine prior information, Bayesian methods are inherently subjective” (Efron, 2013, p. 145).
There is less controversy over Bayesian priors in sex offender risk assessment because we usually do have “genuine prior information” from recidivism studies. In that case, the prior is simply the recidivism rate in a reference group. A recent survey found that most forensic evaluators who use the Static-99R choose the high-risk group (Chevalier, Boccaccini, Murrie, & Varela, 2014). Ninety percent of subjects in the high-risk group at the 10-year follow-up came from two samples, both of which defined recidivism by being charged with a new sex offense (Helmus, 2009).
Some critics consider subjective priors, a weakness of Bayesian probability. However, they are also its strength. Being able to quantify subjective judgments can be invaluable in risk assessment, especially when most or all of the available information is subjective (Ferson, 2005). Goldstein (2006) discussed the advantages and disadvantages of subjective priors.
Reference Class
“Probabilities are essentially reference class-dependent” (Hájek, 2007, p. 37). Meehl (1954) stressed that “if nothing is rationally inferable from membership in a class, no empirical prediction is ever possible” (pp. 19-20). Thus, individual risk predictions are inherently derived from group data (Janus & Meehl, 1997; Scurich, Monahan, & John, 2012). To derive the prior probability, evaluators choose a reference group they consider most similar to the individual they assess. This begs the question, “On what attributes should they be similar?” Individuals have attributes of many reference groups (e.g., a typical mature, American, Caucasian, married, male). We often do not know beforehand which attributes are most important or, therefore, which reference group to choose. Reichenbach (1949) coined this the “problem of the reference class”: If we are asked to find the probability holding for an individual future event, we must first incorporate the event into a suitable reference class. An individual thing or event may be incorporated in many reference classes, from which different probabilities will result. (p. 374)
The reference group problem is evident in the debate over selecting a Static-99R risk group (Campbell & DeClue, 2010; DeClue, 2013; DeClue & Zavodny, 2013). Because each risk group has a different recidivism base rate, each prior is specific to a given reference group. As Abbott (2013) pointed out, none of the procedures proposed for selecting a Static-99R risk group has been empirically validated. Various methods of estimating sexual recidivism base rates (and therefore priors) have been proposed (e.g., Harris & Hanson, 2004; Neller & Petris, 2013; Singh, Fazel, Gueorguieva, & Buchanan, 2012). The reference class problem is certainly not unique to Bayesian probability. In various ways, it affects all concepts of probability (Hájek, 2007).
Bayesian Epistemology
In evaluating risk, we cannot avoid uncertainty. Frequentists withhold judgment on individual outcomes. Bayesians assign them credible probabilities. If we have very little information, our prior probability may be no more than a plausible assumption. Some evaluators may insist that we cannot predict outcomes without solid empirical research or settled science. However, few things in science are settled and almost nothing in sex offender risk assessment is solid, let alone settled. Kaplan and Garrick (1981) defined statistics as the science of handling frequency data and probability as the science of handling the lack of data. They observe that one often hears people say that we cannot use probability because we have insufficient data. In light of our current definitions, we see that this is a misunderstanding. When one has insufficient data, there is nothing else one can do but use probability. (Kaplan & Garrick, 1981, p. 18)
Tukey (1962) anticipated Bayesian methods when he suggested, “far better an approximate answer to the right question, which is often vague, than an exact answer to the wrong question, which can always be made precise” (p. 13).
Summary
The Frequentist and Bayesian views present fundamentally different ways of defining probability. Frequentist probabilities are defined by relative frequencies and are considered objective properties of events. Bayesian probabilities are defined by our beliefs about events and are considered subjective assessments of what we know. In their memoriam for the renowned physicist Edwin Jaynes, his colleagues (Clark, Norberg, & Bretthorst, 1998) wrote that Jaynes insisted that some of the thorniest conceptual problems faced in physics, notably in statistical physics and quantum theory, arise from a mistaken identification of probabilities as physical quantities rather than as representations of the available information on a system—a confusion between what is ontological and what is epistemological.
The distinction between Frequentist and Bayesian concepts of probability has important practical consequences in sex offender risk assessment. Foremost, Frequentist probability by definition applies to series, not to single events like sexual recidivism. Much of the confusion and debate in sex offender risk assessment comes from trying to apply Frequentist probability to single events or individuals. The advantage of using the Bayesian scheme is that probability can be coherently applied to a single event. Because sexual recidivism is a single act, that advantage is decisive. Moreover, Bayesian probability resolves the continuing confusion and debate over applying group data to individuals’ risk. Bayesian probabilities reflect what we know, not what any individual eventually does. We assign the same risk to every individual in the group because we know the same thing about every individual.
Individual risk predictions are inherently derived from group data. Actuarial scales like the Static-99R inform our risk predictions by providing data that yield prior and posterior probabilities. Actuarial scales do not classify offenders into recidivists and non-recidivists. They inform dichotomous decisions in forensic SVP cases. However, the dichotomy is not whether or not the individual will reoffend but whether or not their probability to reoffend (PPP) exceeds a legal threshold.
The Bayesian concept of probability enables evaluators to think and communicate clearly and coherently about individual risk. It allows evaluators to avoid such convoluted statements as “Mr. Smith’s score places him in a group of sex offenders, who, on follow-up, showed a 40% rate of being charged with another sex offense within 10 years.” While testifying in a civil commitment trial, an attorney asked me, “Doctor, you’re only saying that 60% of the offenders in those groups reoffended; you’re not saying that Mr. Smith’s risk is 60% are you?” The attorney was taken back when I replied, “Yes, I actually am saying that his risk to reoffend is 60%.” Evaluators cannot coherently render such an opinion from the Frequentist perspective. They can do so only by the Bayesian concept of probability.
Footnotes
Acknowledgements
The author thanks the anonymous reviewers for their helpful comments on earlier drafts of this article.
Author’s Note
All opinions are those of the author and not necessarily those of the Sand Ridge Secure Treatment Center.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Preparation of the article was supported by the Sand Ridge Secure Treatment Center.