Examining Predictive Validity of the Level of Service Inventory–Ontario Revision (LSI-OR) Substance Abuse Subscale for Different Types of Substance Users

Abstract

Substance abuse is a risk factor for recidivism that is commonly assessed by the Level of Service Inventory–Ontario Revision (LSI-OR) via the Substance Abuse subscale. Research has yet to examine the predictive validity of this subscale relative to types of substances abused. To explore this, substance abuse history, LSI-OR information, and recidivism were coded for a sample of 498 individuals convicted of a crime with a current substance abuse problem. These individuals were classified by the types and number of substances abused. Results of this study provide some evidence supporting the predictive validity of the LSI-OR Substance Abuse subscale. Furthermore, we found preliminary evidence supporting the predictive validity of the subscale for substance abusers relative to types of substances abused and for those who abuse a single substance versus multiple substances. These results have implications for research, policy, and correctional practice.

Keywords

substance abuse Level of Service Inventory drug abuse alcohol abuse predictive validity

Substance abuse is a well-known risk factor for criminal behavior. Substance abuse can be defined as “the harmful or hazardous use of psychoactive substances, including alcohol and illicit drugs” (World Health Organization, 2017). In several large meta-analyses, substance abuse has been found to predict general and violent recidivism (e.g., Bonta et al., 1998; Dowden & Brown, 2002; Gendreau et al., 1996; Gutierrez et al., 2013). Consequently, substance abuse is considered to be an established risk factor, or criminogenic need, for criminal behavior and is often included in measures designed to estimate risk of recidivism.

The Relationship between Substance Abuse and Criminal Behavior

Researchers have found that the relationship between substance abuse and crime differs depending on the types of substances abused (e.g., Bennett & Edwards, 2015; Bennett et al., 2008; Dowden & Brown, 2002; Håkansson & Berglund, 2012). For example, Dowden and Brown (2002) conducted a meta-analysis exploring the relationship between substance abuse (alcohol and drugs), drug abuse, and alcohol abuse with both general and violent recidivism. They found a significantly stronger relationship between drug abuse and general recidivism (Mz = .19, 95% confidence interval [CI] = [0.18, 0.20], k = 38, N = 25,409), than between alcohol abuse and general recidivism (Mz = .12, 95% CI = [0.11, 0.13], k = 36, N = 23,922). The same pattern, however, was not observed for violent recidivism.

As well, Bennett and colleagues (2008) conducted a meta-analysis on the relationship between different types of substances and criminal behavior, and found crack cocaine use had the strongest relationship with crime (odds ratio [OR] = 6.09, 95% CI = [2.98, 12.46], k = 6), followed by heroin use (OR = 3.08, 95% CI = [1.70, 5.61], k = 14), cocaine use (OR = 2.56, 95% CI = [1.43, 4.58], k = 11), amphetamine use (OR = 1.93, 95% CI = [1.27, 2.93], k = 6), and cannabis use (OR = 1.46, 95% CI = [1.13, 1.88], k = 10) (the nonoverlapping 95% CI = indicate crack cocaine use significantly differed from amphetamine and cannabis use). These results suggest that the types of substances abused can impact the strength of the relationship between substance abuse and criminal behavior.

Researchers have also found that the relationship between substance abuse and crime differs depending on the number of substances an individual who has engaged in crime abuses (e.g., Bennett & Holloway, 2005; Chaiken & Chaiken, 1990; Håkansson & Berglund, 2012). For example, in a sample of male and females who had been arrested, Bennett and Holloway found that people who abused multiple substances within the last year reported on average twice as many nonviolent offenses as those who only abused a single substance. Furthermore, they found a moderate positive correlation between the number of substances abused within the past year and the number of reported nonviolent offenses (r = .37). In a prospective study, Håkasson and Berglund found that recidivism was positively associated with multiple substance abuse during the 30-day period prior to incarceration. These results indicate that the number of different substances abused impacts the strength of the relationship between substance abuse and criminal behavior. From these results, it can be suggested that when measuring substance abuse in relation to criminal behavior, it may be important to account for the types of substances abused, as well as, the number of different substances abused.

Substance abuse is associated with crime for both males and females who have been convicted of a crime (Dowden & Brown, 2002); however, researchers suggest that females convicted of a crime have different patterns of substance abuse than males convicted of a crime (e.g., Adams et al., 2008; Bennett & Edwards, 2015; Fazel et al., 2006; Haas & Peters, 2000; Henderson, 1998; McClellan et al., 1997). For example, in Fazel and colleagues (2006) review of prevalence rates of drug and alcohol abuse in male and female individuals in prison, they found females had greater drug dependence than males, but similar alcohol dependence. As well, Haas and Peters (2000) found females convicted of a crime (relative to males convicted of a crime) started abusing alcohol and marijuana later in life but started abusing cocaine earlier in their substance abuse trajectory. Furthermore, Bennett and colleagues (2008) found a stronger meta-analytic relationship between drug abuse and crime for females (fixed effects OR = 6.69, 95% CI = [5.59, 8.00]; random effects OR = 2.24, 95% CI = [0.71, 7.06]) relative to males (fixed effects OR = 2.80, 95% CI = [2.48, 3.17]; random effects OR = 2.47, 95% CI = [1.44, 4.22]). This research demonstrates it is important to consider gender when exploring the relationship between substance abuse and criminal behavior.

Measures of Criminogenic Needs and Predictive Validity

Accurately identifying criminogenic needs and estimating risk of recidivism is important, as correctional best practice requires correctional staff to provide services dependent on the risk level and criminogenic needs of their clients (e.g., Bonta, 2002). Theoretically, criminogenic needs are factors that are changeable (dynamic), and when change in these needs occur, this change is associated with changes in criminal behavior (Bonta, 2002). From this, it can be reasoned that if we are measuring criminogenic needs, it is important that the measures used are able to accurately predict criminal behavior.

Predictive validity is defined as “the correlation of a measure at one point in time with performance on another measure or criterion at some point in the future” (Kazdin, 2003, p. 359). For measures of criminogenic needs, evidence that a measure can accurately predict criminal behavior (e.g., recidivism) would support the predictive validity of the tool. As noted above, substance abuse is considered to be a criminogenic need, and research has found that it is positively associated with criminal behavior (e.g., Bennett & Edwards, 2015; Bennett et al., 2008). Thus, we would expect valid measures of substance abuse to predict criminal behavior.

Predictive Validity and the Level of Service Inventory–Ontario Revision (LSI-OR)

One of the most commonly used, and well validated, measures of risk of recidivism is the Level of Service Inventory (LSI, Andrews, 1982; for example, Bonta, 2002; Girard & Wormith, 2004; Olver et al., 2014; Wormith & Bonta, 2018). Generally, the LSI tools are risk and needs assessments that are used to inform correctional interventions, case management, and community supervision. This tool is based on a social learning model of crime and assesses the following domains, commonly known as the Central Eight: Criminal History, Antisocial Personality, Procriminal Thinking, Antisocial Peers, Lack of Education/Employment, Substance Abuse, Problematic Leisure and Recreation Time, and Problematic Relationships with Intimate Partners and Family Members. Researchers have found that the LSI tools are reliable and valid for many populations of people convicted of crime (see Olver et al., 2014). The LSI-OR version of the LSI was the product of a major revision of the earliest versions of the LSI tools. This revision included arranging items assessing criminogenic needs to better align with the Central Eight, adding in specific risk and need factors (i.e., violent and sexual risk factors), and adding in items assessing strengths/protective factors (see Wormith & Bonta, 2018 for a review). Following the LSI-OR, subsequent revisions have included both case management and monitoring plans (i.e., LS/CMI; Wormith & Bonta, 2018) and only monitoring plans (i.e., LS/RNR; Wormith & Bonta, 2018).

The Substance Abuse subscale of the LSI-OR (the version of the LSI tool used in this study) is comprised of eight items that were designed to assess a persons’s problematic use of substances (excluding caffeine and nicotine; Andrews et al., 1995). The LSI-OR Substance Abuse subscale can be used to identify people with substance abuse as a criminogenic need for treatment and case management purposes; the expectation being that addressing substance abuse needs for these individuals will contribute to lower recidivism. Research examining the predictive validity of the Substance Abuse subscale has found that it predicts recidivism (e.g., Andrews et al., 2012; Girard & Wormith, 2004; Olver et al., 2014). For example, Olver and colleagues (2014) meta-analyzed the evidence examining the predictive validity of the LSI tools and subscales; they found the Substance Abuse subscale predicted general and violent recidivism (r = .20, 95% CI = [0.16, 0.23], k = 54, n = 97,511; r = .13, 95% CI = [0.09, 0.18], k = 19, n = 55,447, respectively), and this held true when examining the results by gender, ethnic minority status, and country (the relationship between country and violent recidivism was not examined in Olver et al., 2014). Of note, the Substance Abuse subscale had a stronger relationship with general recidivism for females relative to males (as indicated by the nonoverlapping confidence intervals for the fixed effect models). These results provide convincing evidence supporting the predictive validity of the LSI Substance Abuse subscale with different populations of people convicted of crime.

Predictive Validity of the LSI Substance Abuse Subscale for Substance Abusers

Although research has examined the predictive validity of the LSI Substance Abuse subscale with a variety of populations of people convicted of crime (e.g., females), little research to date has investigated the predictive validity of the LSI Substance Abuse subscale with samples of substance abusers. To date, we are aware of three studies that have examined this research question; generally, researchers have found evidence supporting the predictive validity of the LSI Substance Abuse subscale with substance abusers (Kelly & Welsh, 2008; Pilon et al., 2015, 2016). For example, Kelly and Welsh (2008) examined the predictive validity of the LSI-R for a sample of 276 males convicted of a crime with a documented history of substance abuse, who had completed an intensive substance abuse treatment program. They found that both the LSI-R Total Score and the Substance Abuse subscale significantly predicted reincarceration; this was true even after controlling for age, time at risk, and type of release (parole vs. community corrections center [i.e., a residential facility that assists with transition between an institution and the community]).

Importantly, the LSI Substance Abuse subscale assesses substance abuse as a single construct; there are no distinctions made between the types of drugs abused (e.g., cannabis vs. cocaine), alcohol and drug abuse are weighted equally, and the number of different drugs abused is not accounted for (see Andrews et al., 1995). With regard to predictive validity, it is possible that the type of substance abused and/or the number of different substances abused may impact the ability of the LSI Substance Abuse subscale to predict recidivism. Thus, another consideration is whether the predictive validity of the Substance Abuse subscale differs depending on the types of, and/or the number of different substances a person abuses.

To the best of our knowledge, only one study has explored the predictive validity of the LSI-OR Substance Abuse subscale for different types of substance abusers. Pilon and colleagues (2016) examined the predictive validity of the LSI-OR Substance Abuse subscale with a large sample of males and females convicted of a crime. For males, the Substance Abuse subscale predicted violent recidivism better for alcohol abusers (area under the curve [AUC] = .590, 95% CI = [0.585, 0.607]), relative to drug abusers (AUC = .522, 95% CI = [0.501, 0.542]) and those who abused both alcohol and drugs (AUC = .561, 95% CI = [0.542, 0.581]) as indicated by nonoverlapping confidence intervals, but predicted general recidivism equally well for all three groups. For females, the Substance Abuse subscale predicted violent recidivism for alcohol abusers (AUC = .564, 95% CI = [0.518, 0.611]), but not drug abusers (AUC = .540, 95% CI = [0.487, 0.593]), or both alcohol and drug abusers (AUC = .488, 95% CI = [0.430, 0.546]), but these effects were not significantly different (all of the confidence intervals overlap); the Substance Abuse subscale predicted general recidivism equally well for all three groups. These results support the use of the LSI-OR Substance Abuse subscale for both male and female alcohol and drug abusers and indicate predictive validity may differ depending on the type of substance(s) abused and the gender of the substance abuser. We are unaware of any research that has examined whether the Substance Abuse subscale predicts recidivism for individuals who only abuse a single substance versus those who abuse multiple substances.

The LSI-OR Substance Abuse subscale is used for case management and treatment planning, with the expectation that focusing on individuals with moderate or higher substance abuse needs will contribute to reductions in recidivism. As noted above, it is possible that the predictive validity of the LSI-OR Substance Abuse subscale varies for people dependent on the types of substances abused (e.g., opioid vs. cannabis users); this has yet to be explored in research. Furthermore, it is also possible that predictive validity of the Substance Abuse subscale varies for people who abuse multiple substances, relative to those who only abuse a single substance. Understanding the predictive validity of the LSI-OR Substance Abuse subscale for subtypes of substance abusers may inform how the LSI-OR Substance Abuse subscale is used in correctional supervision decision-making (i.e., treatment and case management decisions).

As such, the purpose of the current study was to (a) examine whether the LSI-OR Substance Abuse subscale predicts recidivism for a sample of male and female substance abusers; (b) investigate whether the LSI-OR Substance Abuse subscale predicts recidivism for subtypes of substance abusers dependent on type and number of substances abused (e.g., alcohol abusers; multiple substance abusers); and (c) explore whether the LSI-OR Substance Abuse subscale predicts recidivism better for some types of substance abusers relative to other types of substance abusers (e.g., opiate abusers vs. stimulant abusers; single vs. multiple substance abusers).

Method

Participants

Participants (N = 498) were randomly selected using SPSS (select cases, random sample) from the cohort of individuals under community supervision in Ontario during fiscal year 2011/2012 (April 1–March 31) who had a complete LSI-OR at that time and were identified as having a current drug problem on the LSI-OR. The LSI-OR assessment coded in the current study was the current LSI-OR completed and may have been completed as the intake assessment for the index offense or as a reassessment during supervision (depending on where the client was during their course of supervision when the data were extracted). The demographic characteristics of the sample are presented in Table 1. The majority of the sample were male (81.7%), single (66.5%), identified as Caucasian (65.9%), and were classified as high risk for recidivism on the LSI-OR (41.0%).

Table 1:

Demographic Characteristics of Sample by Types of Substances Abused (Any Use) Within 1 Year Prior to Assessment

Characteristic	Types of substance abuse within 1 year prior to assessment^a						Total sample
Characteristic	Alcohol	THC	Opioid	Stimulants	Single substance	Multiple substances	Total sample
n	122	196	93	118	238	144	498
	% (n)	% (n)	% (n)	% (n)	% (n)	% (n)	% (n)
Gender
Male	85.2 (104)	88.3 (173)	67.7 (63)	72.9 (86)	81.9 (195)	79.2 (114)	81.7 (407)
Female	14.8 (18)	11.7 (23)	32.3 (30)	27.1 (32)	18.1 (43)	20.8 (30)	18.3 (91)
Age (M [SD])	32.30 (10.09)	28.46 (9.72)	32.45 (10.42)	34.01 (10.02)	30.86 (9.94)	31.89 (10.59)	31.69 (10.20)
Marital status
Single	71.3 (87)	70.9 (139)	58.1 (54)	64.4 (76)	66.8 (159)	69.4 (100)	66.5 (331)
Common law	13.1 (16)	8.2 (16)	16.1 (15)	11.9 (14)	11.8 (28)	11.1 (16)	11.8 (59)
Married	2.5 (3)	6.1 (12)	8.6 (8)	7.6 (9)	5.9 (14)	5.6 (8)	5.8 (29)
Separated	3.3 (4)	2.6 (5)	7.5 (7)	7.6 (9)	4.6 (11)	5.6 (8)	5.0 (25)
Divorced	6.6 (8)	3.6 (7)	5.4 (5)	5.9 (7)	2.5 (6)	5.6 (8)	4.8 (24)
Widowed	0	0	0	0	0	0	0.4 (2)
Missing data	3.3 (4)	8.7 (17)	4.3 (4)	2.5 (3)	8.4 (20)	2.8 (4)	5.6 (28)
Race
Caucasian	59.0 (72)	65.3 (128)	59.1 (55)	72.9 (86)	64.3 (153)	64.6 (93)	65.9 (328)
Black	3.3 (4)	8.7 (17)	3.2 (3)	5.1 (6)	8.4 (20)	3.5 (5)	6.6 (33)
Aboriginal	25.4 (31)	12.2 (24)	24.7 (23)	6.8 (8)	13.0 (31)	18.8 (27)	13.7 (68)
Other	8.2 (10)	8.2 (16)	4.3 (4)	9.3 (11)	8.0 (19)	7.6 (11)	8.0 (40)
Missing data	4.1 (5)	5.6 (11)	8.6 (8)	5.9 (7)	6.3 (15)	5.6 (8)	5.8 (29)
LSI-OR Total Score (M [SD])	24.40 (7.54)	20.49 (7.83)	22.89 (7.68)	23.82 (7.32)	20.97 (8.01)	23.70 (7.29)	21.58 (7.96)
Very low risk	0	0.5 (1)	0	0	0.4 (1)	0	0.4 (2)
Low risk	0	7.7 (15)	3.2 (3)	2.5 (3)	8.0 (19)	0.7 (1)	6.2 (31)
Moderate risk	32.8 (40)	33.2 (65)	33.3 (31)	24.6 (29)	33.2 (79)	28.5 (41)	32.7 (163)
High risk	35.2 (43)	44.4 (87)	41.9 (39)	44.1 (52)	39.5 (94)	46.5 (67)	41.0 (204)
Very high risk	32.0 (39)	14.3 (28)	21.5 (20)	28.8 (34)	18.9 (45)	24.3 (35)	19.7 (98)
LSI-OR Substance Abuse Subscale (M [SD])	6.47 (1.07)	4.83 (1.66)	5.54 (1.41)	5.78 (1.24)	4.92 (1.68)	5.97 (1.28)	5.19 (1.64)

Note. THC = Tetrahydrocannabinol; LSI-OR = Level of Service Inventory–Ontario Revision.

Types of substance abuse not mutually exclusive. Hallucinogens are not included in this table as only eight participants abused hallucinogens within 1 year prior to assessment. Depressants are not included in this table as only five participants abused depressants within 1 year prior to assessment. For N =116 participants, types of substances abused within 1 year of assessment could not be identified.

For the total sample, LSI-OR Substance Abuse subscale scores ranged from 2 to 8 and the average score was 5.19 (SD = 1.64, N = 498). Recidivism was defined as any new conviction incurred post LSI-OR assessment, while on community supervision or following release until February 11, 2017. The average length of follow-up was 5.4 years (M = 1955 days, SD = 107). In our sample, 52.8% (n = 263) had been reconvicted of a nonviolent offense, 31.7% (n = 158) had been reconvicted of a violent (sexual and nonsexual) offense, and 19.1% (n = 95) had been reconvicted of a substance-related offense (categories not mutually exclusive); in total, 57.6% (n = 287) of participants had been reconvicted for any offense.

Participants were divided into different groups based on the information coded from the LSI-OR. More specifically, participants were grouped based on their use of substances (e.g., alcohol, opiates, stimulants) within 1 year prior to the LSI-OR assessment. Within this timeframe, groups were formed in one of the two ways: (a) any abuse groups were formed by including all participants who had abused a substance within 1 year; these groups were not mutually exclusive, and (b) only abuse groups were formed by including participants who had only abused a particular substance within 1 year; these groups were mutually exclusive. Participants were also grouped based on whether they had abused a single substance or multiple different substances in the time period prior to assessment (single vs. multiple substance abuse). The demographic information for the substance abuse types groups based on any use, single substance abusers, multiple substance abusers, and the full sample is presented in Table 1.

Measures

LSI-OR

The LSI-OR (Andrews et al., 1995) is a risk/needs assessment tool that is used to identify an individual who has been convicted of a crime criminogenic needs and estimate risk of recidivism. The LSI-OR consists of 43 items that assess the General Risk/Needs factors, 14 items that assess personal problems with criminogenic potential, and nine items that assess history of perpetration. The General Risk/Needs factors consist of the Central Eight risk factors for criminal behavior (Andrews & Bonta, 2010): prior Criminal History (eight items), problems in Education/Employment (e.g., frequently unemployed; nine items), problematic Family/Marital relationships (e.g., dissatisfaction in martial relationships; four items), problematic use of Leisure/Recreation time (two items), Procriminal Companions (e.g., having criminal friends; four items), Substance Abuse (see below; eight items), Procriminal Attitudes/Orientation (e.g., having attitudes supportive of crime; 4 items), and demonstrating an Antisocial Pattern (e.g., early and diverse engagement in antisocial behavior; 4 items). The General Risk/Needs Factors items are summed and total scores are used to classify people as Very Low (0–4), Low (5–10), Medium (11–19), High (20–29), and Very High (30+) risk of recidivism. As noted above, the LSI tools have demonstrated good validity with a number of populations of people convicted of crime (e.g., Olver et al., 2014; Wormith & Bonta, 2018). To date, the LSI tools have demonstrated moderate levels of interrater reliability; some research has found lower levels of interrater reliability (e.g., LSI-R Total Score intraclass correlation [ICC] = .65, Substance Abuse subscale ICC = .33; Rocque & Plummer-Beale, 2014), whereas other research has found high levels of interrater reliability (LSI-R Total Score ICC = .98 and Substance Abuse subscale ICC > .90; Stewart, 2011 as cited in Wormith & Bonta, 2018). Of note, in the current sample, the LSI-OR Total Score significantly predicted any recidivism (hazards ratio [HR] = 1.06, 95% CI = [1.05, 1.08], p < .001), nonviolent recidivism (HR = 1.06, 95% CI = [1.05, 1.08], p < .001), violent recidivism (HR = 1.07, 95% CI = [1.04, 1.09], p < .001), and substance related recidivism (HR = 1.08, 95% CI = [1.05, 1.11], p < .001) after controlling for age, race, and gender.

LSI-OR Substance Abuse Subscale

This subscale was designed to assess individuals’ who have been convicted of a crime abuse of substances (excluding caffeine and nicotine; Andrews et al., 1995). It is important to note that the focus of this subscale is not on assessing engagement in substance-related offenses, but rather on the problematic use of substances such as alcohol and drugs. The items in this subscale assess (a) whether the person has ever had an alcohol problem, (b) the person has ever had a drug problem, (c) whether the person has had an alcohol problem within the year prior to assessment (currently), (d) whether the person has had a drug problem within the year prior to assessment (currently), (e) whether substance abuse has or will likely contribute to law violation, (f) whether substance abuse has contributed to family and/or martial problems, (g) whether substance abuse has contributed to education and/or employment problems, and (h) whether there are medical or “other” indicators, such as financial problems, that indicate substance abuse. These items are summed to form the subscale score; scores of 0 to 1 indicate very low need, 2 to 5 indicate low need, and scores of 6 to 8 indicate a moderate substance abuse need.

LSI-OR Coding

A number of variables were manually coded from the information recorded in the electronic LSI-OR system. The LSI-OR assessment was completed by the individual’s supervising Probation and Parole Officer using information obtained via interview with the individual and information provided by collateral sources (e.g., police reports, past assessments, interviews with family members). The second and third authors coded these variables from the information recorded in the LSI-OR electronic system. The first 10 participants were used for training purposes. Following training, both the second and third authors coded 50 files for the variables representing “substance abuse types used within 1 year of assessment” and 91 files for the use of professional override. Overall, the inter-rater reliability was good, the G statistic (see description below) ranged from 0.68 to 1.00 (84%–100% agreement) for the types of substances abused within 1 year of assessment and was 0.78 (83.5%) for the use of professional override.

Substance Types

Substances were coded as belonging to one of the six categories: alcohol, (tetrahydrocannabinol (THC), e.g., cannabis, hash, oil), opiates (e.g., heroin, morphine, oxycodone), stimulants (e.g., crack cocaine, cocaine, methamphetamine), hallucinogens (e.g., lysergic acid diethylamide [LSD], mushrooms, phencyclidine [PCP]), and depressants (e.g., benzodiazepines, valium, barbiturates). The types of substances participants abused were coded for within 1 year prior to assessment. The types of substances participants abused were also coded for within 10 years prior to assessment; these results are available in Davis et al. (2019).

Single Versus Multiple Substance Use

Participants were categorized into the single or multiple substance abuser categories depending on the number of different substances abused within 1 year prior to assessment (e.g., opiate user [single substance abusers] vs. alcohol, THC, and opiate user [multiple substance abuser]).

Recidivism

As noted above, recidivism was defined as any new conviction incurred post LSI-OR assessment, while on community supervision or following release until February 11, 2017. The average length of follow-up was 5.4 years (M = 1,955 days, SD = 107). Recidivism was further divided into violent recidivism, nonviolent recidivism, substance-related recidivism, and any recidivism. Violent recidivism was defined as any new conviction for a violent offense (including sexual violence), nonviolent recidivism was defined as any new conviction for a nonviolent offense, substance-related recidivism was defined as any new conviction for a drug or alcohol related offense (e.g., impaired driving, possession of narcotics), and any recidivism was defined as any new conviction and encompassed nonviolent, violent, and substance-related recidivism.

Demographic Information

Demographic information such as age, gender, race, and marital status were extracted from client’s electronic files.

Procedure

The data used in this study were archival and the analyses were exploratory in nature. A random sample of 498 participants was drawn from individuals who were on community supervision, had a complete LSI-OR assessment, and had a current drug problem as indicated on the LSI-OR Substance Abuse subscale (from a possible N = 11,943) in the 2011–2012 fiscal year. The random sample was determined using SPSS random selection of cases. Sample size was determined based on feasibility for the research team. Data were coded from the electronic client files and coded by the second and third authors.

Analyses

Inter-Rater Reliability

The data were coded by the second and third authors. Inter-rater reliability was computed using percent agreement and Holley and Guilford’s G statistic (1964). Percent agreement is an intuitive measure of inter-rater reliability but is limited in that it does not account for chance agreement between raters. Cohen’s (1960) Kappa statistic is typically used to assess inter-rater reliability for categorical data; but, this statistic is limited by its sensitivity to distributional skew and low base rates (e.g., Xu & Lorber, 2014). Research exploring alternative statistics for inter-rater reliability analyses has found that Holley and Guilford’s G statistic accounts for chance agreement and is not sensitive to distributional skew and low base rates (Xu & Lorber, 2014). Thus, this statistic, along with percent agreement was computed for the variables coded in this study.

AUC

AUC analyses can be interpreted as the probability that a randomly selected recidivist will have a higher LSI-OR Substance Abuse subscale score, than a randomly selected nonrecidivist (for an overview see Helmus & Babchishin, 2016). An AUC is statistically significant if the 95% CI does not include 0.50. For the variables examined in this study, AUCs greater than .50 represent higher LSI-OR Substance Abuse subscales scores being associated with higher likelihood of recidivism. AUCs less than .50 represent higher LSI-OR Substance Abuse subscale scores being associated with lower likelihood of recidivism. As a rough heuristic, an AUC of .56 corresponds to a small effect size, while .64 reflects a moderate effect, and .71 reflects a large effect size, as these values correspond to Cohen’s ds of .20, .50, and .80, when certain assumptions are satisfied (see Rice & Harris, 2005). Conversely, AUC values of .44, .36, and .29 would represent small, moderate, and large effects in the other direction. Differences between AUC statistics were computed using the method reported by Hanley and McNeil (1982).

Cox Regression Survival Analysis

Cox regression survival analysis is used to model the relationship between a set of predictors (covariates) and the time to a specified outcome (i.e., survival time) and is expressed as a hazard function (see Bradburn et al., 2003; Tabachnick & Fidell, 2007). A hazard function represents the probability of the outcome at a given time based on a set of covariates. When multiple covariates are entered into a model simultaneously, the unique contribution of each covariate is assessed after controlling for the variance explained in the outcome by the other covariates. For each covariate, a HR is computed; HRs greater than 1 indicate that as the covariate increases the outcome hazard increases and survival time decreases (i.e., covariate is positively associated with the outcome), whereas HRs less than 1 indicate that as the covariate increases, the outcome hazard decreases and survival time increases (i.e., covariate is negatively associated with the outcome) (Bradburn et al., 2003).

Cox regression survival analysis is semi-parametric, with the primary assumption being the Proportionality of Hazards. Proportionality of Hazards assumes the relationship between survival rate and time is the same for all levels of a covariate. For all models in this study, Proportionality of Hazards was examined by testing the interaction between each covariate and time. Covariates that violated this assumption were either included as a stratification variable or the interaction term was included in the model (Tabachnick & Fidell, 2007).

Effect Sizes

In the analyses below, sample size varies for the different groups of substance abusers. Some groups of substance abusers have small samples and limited power; this may lead to not finding significant effects even though they exist (i.e., Type II errors). As well, we ran a large number of significance tests and this may lead to finding false significant effects (i.e., Type I errors). To mitigate both the possibility of both Type I and II errors, effect sizes were considered in conjunction with significance; moderate or larger effects were interpreted as meaningful.

Results

Exploring Predictive Validity for the LSI-OR Substance Abuse Subscale for all Substance Abusers

AUC analyses were used to explore the predictive validity of the LSI-OR Substance Abuse subscale. In Table 2, the AUCs and associated 95% CIs are presented. In the full sample of substance abusers, the LSI-OR Substance Abuse subscale significantly predicted nonviolent recidivism, violent recidivism, substance-related recidivism, and any recidivism. For female substance abusers, the LSI-OR Substance Abuse subscale significantly predicted nonviolent recidivism, substance-related recidivism, and any recidivism, but not violent recidivism. For male substance abusers, the LSI-OR Substance Abuse subscale significantly predicted nonviolent recidivism, violent recidivism, and any recidivism, but not substance-related recidivism. All of the significant effects were small to moderate in size.

Table 2:

Predictive Validity of the LSI-OR Substance Abuse Subscale

Sample	n	AUC (95% CI)
Sample	n	Nonviolent recidivism	Substance abuse recidivism	Violent recidivism	Any recidivism
Males and females
Current substance abuse problem (LSI-OR)^a	498	.58** [0.53, 0.63]	.58* [0.51, 0.64]	.60*** [0.55, 0.66]	.58** [0.53, 0.63]
Only current drug problem (LSI-OR)^b	321	.54 [0.47, 0.60]	.64** [0.56, 0.71]	.54 [0.47, 0.61]	.56 [0.50, 0.62]
Current drug and current alcohol problem (LSI-OR)	177	.61* [0.52, 0.69]	.57 [0.46, 0.68]	.61* [0.53, 0.70]	.61* [0.52, 0.69]
Females
Current substance abuse problem (LSI-OR)^a	91	.61* [0.52, 0.69]	.69* [0.61, 0.78]	.59 [0.49, 0.68]	.64* [0.55, 0.72]
Only current drug problem (LSI-OR)^b	66	.55 [0.40, 0.69]	.79* [0.66, 0.91]	.59 [0.44, 0.75]	.59 [0.45, 0.73]
Current drug and current alcohol problem (LSI-OR)	25	.67 [0.50, 0.83]	.57 [0.39, 0.76]	.54 [0.36, 0.73]	.69* [0.52, 0.87]
Males
Current substance abuse problem (LSI-OR)^a	407	.57* [0.53, 0.61]	.55 [0.50, 0.60]	.60* [0.56, 0.65]	.57* [0.53, 0.61]
Only current drug problem (LSI-OR)^b	255	.54 [0.46, 0.61]	.60* [0.51, 0.68]	.53 [0.44, 0.61]	.55 [0.48, 0.62]
Current drug and current alcohol problem (LSI-OR)	152	.60* [0.53, 0.66]	.57 [0.48, 0.65]	.63* [0.57, 0.69]	.60* [0.53, 0.66]

Note. LSI-OR = Level of Service Inventory–Ontario Revision; AUC = area under the curve; CI = confidence interval.

All participants were scored as having a current drug problem, but not all participants were scored as having a current alcohol problem. ^b Excluding alcohol abuse.

p < .05. **p < .01. ***p < .001.

Restricting the sample to only participants who had a drug problem within the year prior to assessment (i.e., current drug problem, on LSI-OR Substance Abuse subscale), but no alcohol problem within the year prior to assessment (i.e., current alcohol problem on LSI-OR Substance Abuse subscale), the LSI-OR Substance Abuse subscale significantly predicted substance-related recidivism (total sample, moderate effect; males, small effect; and females, large effect). For participants who had a current drug problem and a current alcohol problem, the LSI-OR Substance Abuse subscale significantly predicted nonviolent (total sample and males, small effects), violent (total sample and males, small effects), and any recidivism (total sample, small effect; males, small effects; and females, moderate effects) (see Table 2). Due to the way the sample was selected, it was not possible to run the results for participants who only had a current alcohol problem (i.e., as all participants had a “current drug problem” scored as yes on LSI-OR Substance Abuse subscale).

Cox regression survival analysis was also used to examine the predictive validity of the LSI-OR Substance Abuse subscale while controlling for age, race, and gender (see Table 3). When age, race, and gender were controlled for, the LSI-OR Substance Abuse subscale significantly predicted violent recidivism (HR = 1.18, 95% CI = [1.07, 1.30]), nonviolent recidivism (HR = 1.13, 95% CI = [1.05, 1.22]), substance-related recidivism (HR = 1.14, 95% CI = [1.01, 1.29]), and any recidivism (HR = 1.13, 95% CI = [1.05, 1.21]).

Table 3:

Cox Regression Survival Analyses Exploring the Predictive Validity of the LSI-OR Substance Abuse Subscale While Controlling for Age, Race, and Gender

Predictor	N	χ² (df), p	B	SE	p	HR	95% CI
Any recidivism^a
	469	41.31 (3), p < .001
Age			−.03	.01	<.001	0.97	[0.96, 0.99]
Race			−.41	.13	.001	0.66	[0.52, 0.85]
LSI-OR SA subscale			.12	.04	.001	1.13	[1.05, 1.21]
Nonviolent recidivism^a
	469	35.71 (3), p < .001
Age			−.03	.01	< .001	0.97	[0.96, 0.99]
Race			−.32	.13	.014	0.72	[0.56, 0.94]
LSI-OR SA subscale			.12	.04	.001	1.13	[1.05, 1.22]
Violent recidivism
	469	41.60 (4), p < .001
Age			−.03	.01	< .001	0.97	[0.95, 0.99]
Race			−.59	.17	< .001	0.55	[0.40, 0.77]
Gender			−.28	.23	.219	0.75	[0.49, 1.19]
LSI-OR SA subscale			.17	.05	.001	1.18	[1.07, 1.30]
Substance-related recidivism
	469	17.77 (4), p = .001
Age			−.03	.01	.015	0.97	[0.95, 1.00]
Race			−.50	.21	.018	0.60	[0.40, 0.92]
Gender			−.02	.28	.954	0.98	[0.57, 1.69]
LSI-OR SA subscale			.13	.06	.034	1.14	[1.01, 1.29]

Note. Sample size varies for each model depending on missing data on control variables (age, race, and gender). LSI-OR = Level of Service Inventory–Ontario Revision; HR = hazards ratio; CI = confidence interval; SA = substance abuse.

Gender is used as a stratification variable in these analyses as this variable violated the proportionality of hazards assumption.

We also investigated whether the LSI-OR Substance Abuse subscale predicted recidivism after controlling for age, race, gender, and each of the other LSI-OR criminogenic need subscales. We found the LSI-OR Substance Abuse subscale did not significantly predict any type of recidivism once all of the LSI-OR subscales were included in the model (see Table 4).

Table 4:

Cox Regression Survival Analyses Exploring the Predictive Validity of the LSI-OR Substance Abuse Subscale While Controlling for Age, Race, Gender, and the LSI-OR Subscales

Predictor	N	χ² (df), p	B	SE	p	HR	95% CI
Any recidivism^a,b
	469	117.95 (11), p < .001
Age			−.04	.01	< .001	0.96	[0.95, 0.98]
Race			−.30	.13	.020	0.74	[0.57, 0.95]
LSI-OR: Criminal History			.22	.04	< .001	1.25	[1.16, 1.33]
LSI-OR: Education/Employment			.01	.03	.772	1.01	[0.95, 1.07]
LSI-OR: Family/Marital			.07	.06	.231	1.07	[0.96, 1.20]
LSI-OR: Leisure/Recreation			.26	.16	.087	1.30	[0.96, 1.76]
LSI-OR: Companions			.03	.07	.657	1.03	[0.90, 1.19]
LSI-OR: Procriminal Attitudes/Orientation			.10	.06	.113	1.10	[0.98, 1.25]
LSI-OR: Substance Abuse			.01	.04	.828	1.01	[0.93, 1.09]
LSI-OR: Antisocial Pattern			−.13	.10	.189	0.88	[0.72, 1.07]
Nonviolent recidivism^a,b
	469	108.66 (11), p < .001
Age			−.04	.01	<.001	0.96	[0.95, 0.98]
Race			−.20	.14	.145	0.82	[0.63, 1.07]
LSI-OR: Criminal History			.22	.04	< .001	1.25	[1.16, 1.34]
LSI-OR: Education/Employment			.002	.03	.956	1.00	[0.94, 1.07]
LSI-OR: Family/Marital			.08	.06	.182	1.08	[0.96, 1.22]
LSI-OR: Leisure/Recreation			.34	.16	.036	1.41	[1.02, 1.93]
LSI-OR: Companions			.06	.08	.474	1.06	[0.91, 1.23]
LSI-OR: Procriminal Attitudes/Orientation			.06	.06	.350	1.06	[0.94, 1.20]
LSI-OR: Substance Abuse			.01	.04	.866	1.01	[0.93, 1.10]
LSI-OR: Antisocial Pattern			−.12	.11	.257	0.89	[0.72, 1.09]
Violent recidivism^c
	469	88.85 (12), p < .001
Age			−.04	.01	< .001	0.96	[0.94, 0.98]
Race			−.52	.17	.002	0.59	[0.43, 0.83]
Gender			−.18	.24	.466	0.84	[0.52, 1.35]
LSI-OR: Criminal History			.18	.05	<.001	1.20	[1.10, 1.31]
LSI-OR: Education/Employment			.04	.04	.299	1.04	[0.96, 1.13]
LSI-OR: Family/Marital			.08	.08	.338	1.08	[0.92, 1.26]
LSI-OR: Leisure/Recreation			.23	.15	.125	1.26	[0.94, 1.69]
LSI-OR: Companions			−.16	.10	.099	0.85	[0.70, 1.03]
LSI-OR: Procriminal Attitudes/Orientation			.23	.08	.005	1.25	[1.07, 1.47]
LSI-OR: Substance Abuse			.08	.06	.148	1.08	[0.97, 1.21]
LSI-OR: Antisocial Pattern			−.48	.17	.006	0.62	[0.44, 0.87]
Substance-related recidivism
	469	59.98 (11), p < .001
Age			−.03	.01	.043	0.98	[0.95, 1.00]
Race			−.43	.22	.050	0.65	[0.43, 1.00]
Gender			.08	.29	.768	1.09	[0.62, 1.91]
LSI-OR: Criminal History			.22	.06	< .001	1.24	[1.10, 1.40]
LSI-OR: Education/Employment			.08	.06	.161	1.08	[0.97, 1.20]
LSI-OR: Family/Marital			−.08	.10	.391	0.92	[0.77, 1.11]
LSI-OR: Leisure/Recreation			.09	.20	.640	1.10	[0.74, 1.63]
LSI-OR: Companions			−.10	.12	.425	0.91	[0.72, 1.15]
LSI-OR: Procriminal Attitudes/Orientation			.15	.10	.144	1.16	[0.95, 1.41]
LSI-OR: Substance Abuse			.01	.07	.894	1.01	[0.88, 1.16]
LSI-OR: Antisocial Pattern			.02	.17	.928	1.02	[0.73, 1.42]

Note. LSI-OR = Level of Service Inventory–Ontario Revision; HR = hazard ratio; CI = confidence interval.

Gender is used as a stratification variable in these analyses as this variable violated the proportionality of hazards assumption. ^bThe interaction between Leisure Recreation and Time was included as a covariate because this variable violated the proportionality of hazards assumption. ^cThe interaction between Antisocial Pattern and Time was included as a covariate because this variable violated the proportionality of hazards assumption.

Exploring the Predictive Validity of the LSI-OR Substance Abuse Subscale for Different Types of Substance Abusers

AUC analyses were used to explore the predictive validity of the LSI-OR Substance Abuse subscale for substance abusers based on types of substances used. The results of the AUC analyses for each group of substance abuser are presented in Table 5. For participants who abused alcohol the LSI-OR Substance Abuse subscale predicted nonviolent recidivism for any alcohol abusers (small effect), and, although not significantly so, moderately predicted nonviolent and violent recidivism for only alcohol abusers. The LSI-OR Substance Abuse subscale had a moderate nonsignificant negative relationship with substance-related recidivism for only alcohol abusers. Of note, all of the “only alcohol abusers” had a positive score for “current drug problem” on the LSI-OR Substance Abuse subscale but did not have any substances noted besides alcohol in the description of their substance abuse.

Table 5:

AUC Analyses Exploring the Predictive Validity for the LSI-OR Substance Abuse Subscale for Different Types of Substance Abusers and With the Use of Professional Override

Sample	n	AUC (95% CI)
Sample	n	Nonviolent recidivism	Substance abuse recidivism	Violent recidivism	Any recidivism
Any abuse within 1 year of assessment
Alcohol	122	.61 * [0.50, 0.71]	.51 [0.38, 0.64]	.60 [0.50, 0.70]	.59 [0.49, 0.70]
THC	196	.59 * [0.51, 0.67]	.65 ** [0.54, 0.75]	.58 [0.49, 0.67]	.57 [0.49, 0.65]
Opiates	93	.62 * [0.51, 0.74]	.62 [0.50, 0.74]	.64 * [0.51, 0.77]	.64 * [0.53, 0.76]
Stimulants	118	.57 [0.47, 0.68]	.54 [0.43, 0.66]	.56 [0.45, 0.67]	.54 [0.43, 0.65]
Only abuse within 1 year of assessment
Alcohol	47	.64 [0.48, 0.80]	.33 ^a [0.16, 0.51]	.66 [0.50, 0.82]	.61 [0.45, 0.78]
THC	110	.57 [0.46, 0.67]	.64 ^a [0.49, 0.80]	.58 [0.46, 0.70]	.54 [0.43, 0.64]
Opiates	38	.52 [0.33, 0.70]	.68 ^a [0.46, 0.90]	.66 [0.44, 0.87]	.59 [0.40, 0.77]
Stimulants	41	.60 [0.42, 0.79]	.67 ^a [0.49, 0.85]	.63 [0.44, 0.81]	.59 [0.39, 0.79]
Single vs. multiple substance abuse within 1 year of assessment
Single substance	238	.59* [0.52, 0.66]	.61* [0.52, 0.70]	.63** [0.55, 0.71]	.59* [0.52, 0.66]
Multiple substance	144	.62* [0.52, 0.71]	.54 [0.44, 0.65]	.58 [0.49, 0.68]	.61* [0.52, 0.71]
Use of professional override
Original scores^b	498	.56* [0.51, 0.61]	.58* [0.51, 0.64]	.58** [0.53, 0.64]	.56* [0.51, 0.61]
Professional override	496	.53 [0.48, 0.58]	.52 [0.46, 0.59]	.54 [0.48, 0.59]	.54 [0.48, 0.58]

Note. Depressants are not included as only five participants abused depressants within 1 year prior to assessment. Hallucinogens are not included 1 year prior to assessment as only eight participants abused hallucinogens. For N = 116 participants, types of substances abused within 1 year of assessment could not be identified. AUC = area under the curve; CI = confidence interval; THC = Tetrahydrocannabinol.

Superscript letter denotes significant differences between types of substance abusers. Moderate or larger and/or significant AUCs were bolded. ^bn = 2 participants had a score of “unknown” for professional override and were not included in the analyses. Same pattern of results when sample restricted to N = 496 (i.e., only those with scores for professional override).

p < .05. **p < .01. ***p < .001.

For participants who abused THC, the LSI-OR Substance Abuse subscale predicted nonviolent (small effect) and substance-related recidivism (moderate effect) for any THC abusers, but only moderately predicted substance-related recidivism for only THC abusers (nonsignificant). For participants who abused opiates, the LSI-OR Substance Abuse subscale predicted nonviolent (small effect), violent (moderate effect), and any recidivism (moderate effect) for any opiate abusers, and moderately predicted violent and substance-related recidivism for only opiate abusers (nonsignificant). For participants who abused stimulants, the LSI-OR Substance Abuse subscale did not predict recidivism for any stimulant abusers but did moderately predict substance-related recidivism for only stimulant abusers (nonsignificant).

Does the LSI-OR Substance Abuse Subscale Predict Recidivism for Some Types of Substance Abusers Better than Other Types of Substance Abusers?

In addition to examining the predictive validity for substance abusers based on types of substances used, we were also interested in whether the evidence of predictive validity would significantly differ based on types of substances abused. To investigate this, the method reported by Hanley and McNeil (1982) was used to test whether the AUC statistics significantly differed. These analyses were only computed for the only abusers groups as these groups were independent from one another. The only significant differences found were for substance-related recidivism. For only stimulant, opiate, and THC abusers, the LSI-OR Substance Abuse subscale predicted substance-related recidivism significantly better than for only alcohol abusers.

Exploring the Predictive Validity of the LSI-OR Substance Abuse Subscale for Single and Multiple Substance Abusers

AUC analyses were also used to examine the predictive validity of the LSI-OR Substance Abuse subscale for participants who abused only a single substance and those that abused multiple substances (see Table 5). The LSI-OR Substance Abuse subscale predicted nonviolent, violent, substance-related, and any recidivism for single substance abusers (small effects). For multiple substance abusers, the LSI-OR Substance Abuse subscale predicted nonviolent and any recidivism (small effects). No significant differences between single and multiple substance abusers were observed.

Does the use of Professional Override Impact the Predictive Validity of the LSI-OR Substance Abuse Subscale?

The impact of professional override on the predictive validity of the LSI-OR Substance Abuse subscale was also investigated. Participants were classified as having a substance abuse need (or not) in two ways: (a) using the original LSI-OR substance abuse scores—“original scores” method; (b) using original scores and, if used, professional override—“professional override” method. AUC analyses were used to explore the relationship between having a substance abuse need identified and recidivism for each method (see Table 5). For participants where the original Substance Abuse subscale scoring was used to identify substance abuse as a criminogenic need (i.e., original scores method), the LSI-OR Substance Abuse subscale significantly predicted nonviolent, violent, and any recidivism (small effects). For participants where professional override was used to identify substance abuse as a criminogenic need, the LSI-OR Substance Abuse subscale did not significantly predict recidivism and none of the effect sizes surpassed the threshold for a small effect.

Discussion

Exploring Predictive Validity for the LSI-OR Substance Abuse Subscale in a Sample of Substance Abusers

The purpose of this study was to examine the predictive validity of LSI-OR Substance Abuse subscale in a sample of male and female substance abusers; for substance abusers based on types and number of substances used; and for subtypes of substance abusers relative to other subtypes of substance abusers (e.g., comparing predictive validity for people who only abuse opiates to those who only abuse stimulants). In the full sample of substance abusers, the LSI-OR Substance Abuse subscale predicted nonviolent, violent, substance-related, and any recidivism. This also held true in the multivariate analyses that controlled for age, race, and gender. The LSI-OR Substance Abuse subscale also predicted at least one type of recidivism when restricted to males, females, and for certain types of substance abusers (e.g., single and multiple substance abusers). These results provide some evidence that supports the predictive validity of the LSI-OR Substance Abuse subscale with a sample of substance abusers. These results are consistent with past research that has found the LSI-OR Substance Abuse subscale predicts recidivism in samples that have not been restricted to substance abusers (e.g., Girard & Wormith, 2004; Olver et al., 2014), and samples that have been restricted to substance abusers (e.g., Kelly & Welsh, 2008; Pilon et al., 2015, 2016).

We also found that the LSI-OR Substance Abuse subscale did not incrementally predict recidivism when each of the other criminogenic need scales of the LSI-OR were controlled for in the analyses. In these models, recidivism was primarily predicted by age, race, and the Criminal History subscale of the LSI-OR. Of note, in the sample used for these models (N = 469), just under half (45%) had a moderate substance abuse need as identified by the Substance Abuse subscale (i.e., a score of 6 or higher), suggesting the lack of incremental prediction is not a result of a lack of variability in the subscale scale scores. These results are not surprising as the LSI-OR Substance Abuse subscale is correlated with the other criminogenic need subscales on the LSI-OR; this subscale assesses the impact of substance abuse on these other criminogenic need areas (e.g., family/martial relationships), and we would expect other problematic criminogenic need areas (e.g., problems with education/employment) to negatively impact substance use. Thus, these results demonstrate the LSI-OR Substance Abuse subscale is impacted by, and influences, the other criminogenic need scales on the LSI-OR.

Does the LSI-OR Substance Abuse Subscale Predict Recidivism for Some Types of Substance Abusers Better than Other Types of Substance Abusers?

An important research question was whether the predictive validity of the LSI-OR Substance Abuse subscale would significantly differ for some types of substance abusers relative to others. As noted above, the LSI-OR Substance Abuse subscale assesses substance abuse as a single construct; there are no distinctions made between the types of drugs abused (e.g., cannabis vs. cocaine), alcohol and drug abuse are weighted equally, and the number of drugs abused is not formally accounted for (see Andrews et al., 1995). Researchers have found that the strength of the relationship between substance abuse and crime differs dependent on the types of substances abused (e.g., Bennett & Edwards, 2015; Bennett et al., 2008; Dowden & Brown, 2002; Håkansson & Berglund, 2012). Thus, we proposed that predictive validity may vary for substance abusers based on types of substances used. There was some evidence supporting this notion; however, the majority of the effect sizes did not differ. The exception to this being differences observed in predicting substance-related recidivism; the LSI-OR Substance Abuse subscale significantly predicted substance-related recidivism better for only stimulant, opiate, and THC abusers relative to only alcohol abusers within 1 year of assessment. These results indicate more research is needed exploring the predictive validity of the LSI-OR Substance Abuse subscale with larger samples of different types of substance abusers.

We also investigated whether the predictive validity of the LSI-OR Substance Abuse subscale significantly differed for participants who abused multiple substances relative to those that abused a single substance. As noted above, researchers have found a positive relationship between number of different substances abused and amount of criminal behavior engaged in (e.g., Bennett & Holloway, 2005; Chaiken & Chaiken, 1990; Håkansson & Berglund, 2012). Thus, we suggested that the number of substances abused by a person may impact the predictive validity of the LSI-OR Substance Abuse subscale. The LSI-OR Substance Abuse subscale predicted nonviolent, violent, substance-related, and any recidivism for single substance abusers. For multiple substance abusers, the LSI-OR Substance Abuse subscale predicted nonviolent and any recidivism. There were no significant differences found between single and multiple substance abusers for use within 1 year of assessment. Thus, these results provide evidence for the predictive validity of the LSI-OR Substance Abuse subscale for people convicted of a crime who abuse a single or multiple substances.

One possible explanation for the overall lack of differences between different types of substance abusers is that half (four of the eight) of the items on the LSI-OR Substance Abuse subscale assess whether substance abuse is impairing and/or interfering with the substance abuser’s daily life. More specifically, the LSI-OR Substance Abuse subscale assesses whether substance abuse results in law violations, family and/or marital problems, education and/or employment problems, and/or other life areas such as finances. Having suboptimal functioning in these areas likely acts as a proxy for the severity of the substance abuse problem. It may not matter what types of substances or the number of substances abused; but instead, what matters in predicting criminal behavior is how severe the substance abuse problem is. Thus, when examining the predictive validity of the LSI-OR Substance Abuse subscale, number and types of substances abused may not be as important as the LSI-OR Substance Abuse subscale captures severity of substance abuse.

Does the use of Professional Override Impact the Predictive Validity of the LSI-OR Substance Abuse Subscale?

LSI-OR assessors have the option of using professional override to identify substance abuse as a criminogenic need for clients. In this study, we examined the predictive validity of the judgments made regarding substance abuse as a criminogenic need. Specifically, we were interested in whether there was evidence to support the predictive validity of the judgments made about substance abuse as a criminogenic need when professional override was used. We found evidence supporting the predictive validity of judgments made based on the original scoring of the LSI-OR Substance Abuse subscale; however, this was not the case when professional override was used to identify substance abuse as a criminogenic need. These results are preliminary but suggest that the use of professional override should be carefully considered when identifying substance abuse as a criminogenic need. Again, as with the other results in this study, it is important to replicate these results prior to drawing firm conclusions.

Limitations in the Current Study

There are several noteworthy limitations in this study. First, the types of substances abused by individuals were coded from the assessor notes included in the LSI-OR assessment. It is possible that not all substances abused by an individual were recorded accurately, and/or it is possible that some individuals did not disclose all of the substances they abused. If types of substances abused were missing and/or were inaccurate for an individual, this would have negatively impacted the substance abuse groups that were formed. Of note, this was problematic when coding the “only alcohol abusers” group. All of these participants had a “current drug problem” marked as yes indicating drug abuse within 1 year of assessment, but there was no information on drug use within 1 year of assessment (only alcohol abuse). To complete the LSI-OR assessment, Probation and Parole Officers use both self-reported and collateral information; future research should use additional methods for identifying the types of substances abused by participants.

Although the overall sample size was adequate (N = 498), when participants were divided into groups based on the types of substances that they abused, sample sizes became quite small. The sample size was determined based on the resources available to the research team; it was not feasible for us to increase the sample size for this study. Small samples are problematic as they result in less powerful analyses and increase the risk of Type II errors. As well, we ran a large number of significance tests and it is possible that this may lead to finding false significant effects (i.e., Type I errors). To mitigate both the possibility of both Type I and II errors, we examined effect sizes in conjunction with significance testing, to identify meaningful results. Furthermore, this resulted in the confidence intervals for our findings to be quite large. Future research should replicate and expand on our findings using larger samples.

A third limitation of this study was that when examining substances abused within 1 year of assessment, for a large number of participants (n = 116) the types of substances abused could not be identified. Often this was because a timeline for the abuse of a particular substance was not identified by the assessor and/or the timeline for abused substances was greater than 1 year. Understanding the types of substances currently being abused by people is important for case management; as a result, this finding has important implications for policy on completing the LSI-OR. As well, as noted above, future research should explore these research questions using different methods for identifying types of substances abused.

There were several methodological limitations that are noteworthy. First, by nature of the LSI-OR Substance Abuse subscale, the lowest score someone with a current substance abuse problem could receive is a score of 2 (out of 8); this resulted in restricted variability for the LSI-OR Substance Abuse subscale. Second, one of the items on the LSI-OR Substance Abuse subscale assesses whether the person’s substance abuse has, is, or might contribute to law violation. It is possible that an assessor had knowledge of violations while completing the assessment that resulted in convictions counted as recidivism; this would bias the results in favor of finding evidence of predictive validity. Third, the LSI-OR assessments used were completed at different times during participants’ supervision period, and we were unable to determine if this impacted the results. The results of this study should be interpreted with these limitations in mind.

Implications

Our results suggest the LSI-OR Substance Abuse subscale can predict recidivism for male, female, certain types of, single substance, and multiple substance abusers. These results have important implications for the use of this subscale within a forensic context. Practically, our results demonstrate that the LSI-OR Substance Abuse subscale can distinguish between substance abusers who are more likely to engage in crime versus those who are less likely to engage in crime (i.e., can accurately identify substance abuse as a criminogenic need among different types of substance abusers). Theoretically, criminogenic needs are factors that are changeable (dynamic), and when change in these needs occur, this change is associated with changes in criminal behavior (Bonta, 2002). Thus, accurately identifying criminogenic needs is critical in correctional intervention, and as a result, correctional best practice requires correctional staff to provide services dependent on the risk level and criminogenic needs of their clients (e.g., Bonta, 2002). Our results indicate that among substance abusers, the LSI-OR Substance Abuse subscale can be used to determine who should be prioritized for substance abuse related interventions and services, resulting in greater likelihood of reducing recidivism.

Conclusion

The purpose of this study was to examine the predictive validity of the LSI-OR Substance Abuse subscale with a sample of male and female substance abusers, and in particular, within types of substances abused. The results of this study provide some evidence supporting the predictive validity of the LSI-OR Substance Abuse subscale with a sample of substance abusers. Furthermore, we found preliminary evidence supporting the predictive validity of the LSI-OR Substance Abuse subscale for substance abusers who abuse alcohol, THC, opiates, and or stimulants. As well, we found preliminary evidence supporting the predictive validity of the LSI-OR Substance Abuse subscale for both single and multiple substance abusers. Although the LSI-OR Substance Abuse subscale did not significantly predict any type of recidivism when the other criminogenic need scales of the LSI-OR were included in the model, this was not surprising due to the interrelatedness of these subscales. Finally, although further research is required, we found that the LSI-OR Substance Abuse subscale did not significantly predict recidivism when professional override was used, suggesting the use of professional override should be carefully considered when identifying substance abuse as a criminogenic need. The results of this study support the continued use of the LSI-OR Substance Abuse subscale for case management and treatment planning for clients who abuse substances.

Footnotes

Authors’ Note:

The authors thank Melinda Vasily for the recidivism data extraction. The authors also thank Stephanie Fernane and Dr. Chris Koegl for providing feedback on an earlier draft of this manuscript. Of note, the Ministry of the Solicitor General was formerly the Ministry of Community Safety and Correctional Services.

ORCID iD

Paula Davis

Paula Davis is the manager of the Statistical Analysis Unit for the Ontario Ministry of the Solicitor General. Paula has worked at the Ministry of the Solicitor General for 26 years in the areas of research, evaluation, and statistics with a primary focus on risk assessment and rehabilitation.

Chantal A. Hermann is a program effectiveness analyst in the Program Evaluation Unit (PEU) for the Ontario Ministry of the Solicitor General. She completed her undergraduate degree in Forensic Science (specialization Psychology) at Laurentian University and her master’s and doctorate in psychology at Carleton University. Her areas of research primarily focus on the causes of sexual and violent offending.

Stuart Freedman is a police officer with the Ontario Provincial Police. He completed his undergraduate degree in Clinical Psychology at Bishop’s University and his master’s in psychology at Ryerson University. His research primarily focuses on youth interrogation rights.

Leslie Lucenti is a program effectiveness analyst in the Program Evaluation Unit (PEU) for the Ontario Ministry of the Solicitor General. She completed her undergraduate degree in criminology and sociology at the University of Western Ontario and her master’s in criminology at Bond University, in Australia.

References

Adams

Leukefeld

C. G.

Peden

A. R.

(2008). Substance abuse treatment for women offenders: A research review. Journal of Addictions Nursing, 19(2), 61–75. https://doi.org/10.1080/10884600802111648

Andrews

D. A.

(1982). The Level of Supervision Inventory (LSI). Ontario Ministry of Correctional Services.

Andrews

D. A.

Bonta

(2010). The psychology of criminal conduct (5th ed.). Anderson.

Andrews

D. A.

Bonta

Wormith

J. S.

(1995). The LSI-OR: Interview and scoring guide. Ontario Ministry of Solicitor General and Correctional Services.

Andrews

D. A.

Guzzo

Raynor

Rowe

R. C.

Rettinger

L. J.

Brews

Wormith

J. S.

(2012). Are the major risk/need factors predictive of both female and male reoffending? A test with the eight domains of the Level of Service/Case Management Inventory. International Journal of Offender Therapy and Comparative Criminology, 56(1), 113–133. https://doi.org/10.1177/0306624X10395716

Bennett

Edwards

(2015). What has been learned from research on the drugs–crime connection? In Brownstein

H. H.

(Ed.), The handbook of drugs and society (pp. 285–299). John Wiley.

Bennett

Holloway

(2005). The association between multiple drug misuse and crime. International Journal of Offender Therapy and Comparative Criminology, 49(1), 63–81. https://doi.org/10.1177/0306624X04269003

Bennett

Holloway

Farrington

(2008). The statistical association between drug misuse and crime: A meta-analysis. Aggression and Violent Behavior, 13(2), 107–118. https://doi.org/10.1016/j.avb.2008.02.001

Bonta

(2002). Offender risk assessment: Guidelines for selection and use. Criminal Justice and Behavior, 29(4), 355–379. https://doi.org/10.1177/0093854802029004002

10.

Bonta

Law

Hanson

R. K.

(1998). The prediction of criminal and violent recidivism among mentally disordered offenders. Psychological Bulletin, 123(2), 123–142. https://doi.org/10.1037/0033-2909.123.2.123

11.

Bradburn

M. J.

Clark

T. G.

Love

S. B.

Altman

D. G.

(2003). Survival analysis part II: Multivariate data analysis—An introduction to concepts and methods. British Journal of Cancer, 89(3), 431–436. https://doi.org/10.1038/sj.bjc.6601119

12.

Chaiken

J. M.

Chaiken

M. R.

(1990). Drugs and predatory crime. Crime and Justice, 13, 203–239. https://doi.org/10.1086/449176

13.

Cohen

(1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46. https://doi.org/10.1177/001316446002000104

14.

Davis

Hermann

C. A.

Freedman

Lucenti

(2019). Examining the predictive validity of the Level of Service Inventory Ontario Revision (LSI-OR) substance abuse subscale for different types of substance users. Ministry of the Solicitor General.

15.

Dowden

Brown

S. L.

(2002). The role of substance abuse factors in predicting recidivism: A meta-analysis. Psychology, Crime and Law, 8(3), 243–264. https://doi.org/10.1080/10683160208401818

16.

Fazel

Bains

Doll

(2006). Substance abuse and dependence in prisoners: A systematic review. Addiction, 101(2), 181–191. https://doi.org/10.1111/j.1360-0443.2006.01316.x

17.

Gendreau

Little

Goggin

(1996). A meta-analysis of the predictors of adult recidivism: What works! Criminology, 34(4), 401–433. https://doi.org/10.1111/j.1745-9125.1996.tb01220.x

18.

Girard

Wormith

J. S.

(2004). The predictive validity of the Level of Service Inventory-Ontario Revision on general and violent recidivism among various offender groups. Criminal Justice and Behavior, 31(2), 150–181. https://doi.org/10.1177/0093854803261335

19.

Gutierrez

Wilson

H. A.

Rugge

Bonta

(2013). The prediction of recidivism with Aboriginal offenders: A theoretically informed meta-analysis. Canadian Journal of Criminology and Criminal Justice, 55(1), 55–99. https://doi.org/10.3138/cjccj.2011.E.51

20.

Haas

A. L.

Peters

R. H.

(2000). Development of substance abuse problems among drug-involved offenders: Evidence for the telescoping effect. Journal of Substance Abuse, 12(3), 241–253. https://doi.org/10.1016/S0899-3289(00)00053-5

21.

Håkansson

Berglund

(2012). Risk factors for criminal recidivism–a prospective follow-up study in prisoners with substance abuse. BMC Psychiatry, 12(1), 111–119. https://doi.org/10.1186/1471-244X-12-111

22.

Hanley

J. A.

McNeil

B. J.

(1982). The meaning and use of the Area under a Receiver Operating Characteristic (ROC) curve. Radiology, 148, 29–36.

23.

Helmus

L. M.

Babchishin

K. M.

(2016). Primer on risk assessment and the statistics used to evaluate its accuracy. Criminal Justice and Behavior, 44(1), 8–25. https://doi.org/10.1177/0093854816678898

24.

Henderson

D. J.

(1998). Drug abuse and incarcerated women: A research review. Journal of Substance Abuse Treatment, 15(6), 579–587. https://doi.org/10.1016/S0740-5472(97)00319-X

25.

Kazdin

A. E.

(2003). Research design in clinical psychology (4th ed.). Allyn & Bacon.

26.

Kelly

C. E.

Welsh

W. N.

(2008). The predictive validity of the Level of Service Inventory—Revised for drug-involved offenders. Criminal Justice and Behavior, 35(7), 819–831. https://doi.org/10.1177/0093854808316642

27.

McClellan

D. S.

Farabee

Crouch

B. M.

(1997). Early victimization, drug use, and criminality: A comparison of male and female prisoners. Criminal Justice and Behavior, 24(4), 455–476. https://doi.org/10.1177/0093854897024004004

28.

Olver

M. E.

Stockdale

K. C.

Wormith

J. S.

(2014). Thirty years of research on the Level of Service Scales: A meta-analytic examination of predictive accuracy and sources of variability. Psychological Assessment, 26(1), 156–176. https://doi.org/10.1037/a0035080

29.

Pilon

A. J. M.

Jewell

L. M.

Wormith

J. S.

(2015). Impaired drivers and their risk of reoffending. Centre of Forensic Behavioural Science and Justice Studies.

30.

Pilon

A. J. M.

Jewell

L. M.

Wormith

J. S.

(2016). The predictive validity of the LS/CMI on substance abuse offenders risk of reoffending. Centre of Forensic Behavioural Science and Justice Studies.

31.

Rice

M. E.

Harris

G. T.

(2005). Comparing effect sizes in follow-up studies: ROC Area, Cohen’s d, and r. Law and Human Behavior, 29(5), 615–620. https://doi.org/10.1007/s10979-005-6832-7

32.

Rocque

Plummer-Beale

(2014). In the eye of the beholder? An examination of the inter-rater reliability of the LSI-R and YLS/CMI in a correctional agency. Journal of Criminal Justice, 42(6), 568–578. https://doi.org/10.1016/j.jcrimjus.2014.09.011

33.

Tabachnick

B. G.

Fidell

L. S.

(2007). Using multivariate statistics (5th ed). Pearson Education.

34.

World Health Organization. (2017). Health topics: Substance abuse. http://www.who.int/topics/substance_abuse/en/

35.

Wormith

J. S.

Bonta

(2018). The Level of Service (LS) Instruments. In Singh

J. P.

Kroner

D. G.

Wormith

J. S.

Desmarais

S. L.

Hamilton

(Eds.), Handbook of recidivism risk/need assessment tools (pp. 117–145). John Wiley.

36.

Lorber

M. F.

(2014). Interrater agreement statistics with skewed data: Evaluation of alternatives to Cohen’s kappa. Journal of Consulting and Clinical Psychology, 82(6), 1219–1227. https://doi.org/10.1037/a0037489