Abstract
Pre- and postcharge diversion programs have been used as a formal intervention strategy for youth offenders since the 1970s. This meta-analysis was conducted to shed some light on whether diversion reduces recidivism at a greater rate than traditional justice system processing and to explore aspects of diversion programs associated with greater reductions in recidivism. Forty-five diversion evaluation studies reporting on 73 programs were included in the meta-analysis. The results indicated that diversion is more effective in reducing recidivism than conventional judicial interventions. Moderator analysis revealed that both study- and program-level variables influenced program effectiveness. Of particular note was the relationship between program-level variables (e.g., referral level) and the risk level targeted by programs (e.g., low or medium/high). Further research is required implementing strong research designs and exploring the role of risk level on youth diversion effectiveness.
A variety of approaches have been used to address the problem of youth crime. However, most jurisdictions within Western societies reflect what is often termed a modified justice model (Corrado, 1992). In this case, youth are processed within a formal judicial system, and sanctions vary between punitive (e.g., incarceration, fines) and rehabilitative actions. Only a very few exceptions, such as Scotland, utilize a corporatist model, where the treatment of youth in conflict with the law is integrated into a larger social service system.
Those employing the modified justice model exhibit variability in the nature of the judicial processing, although all ensure some legal protections for the youth. Various jurisdictions employing this model also display variability in the extent to which they reflect a punitive versus rehabilitative orientation. Many jurisdictions, particularly in the United States, depend heavily on punitive sanctions, such as incarceration, use of fines, or community service orders. In other cases, while punitive sanctions may be available for youth who commit serious crimes, the emphasis is on the provision of treatment interventions designed to address the criminogenic needs of the youth. The Canadian system, governed by the Youth Criminal Justice Act, reflects the latter approach. While provision is made for punitive sanctions, such as incarceration, the goals of the Act emphasize a rehabilitative strategy whereby the factors placing the youth at risk for criminal activity are addressed.
Many jurisdictions embracing at least some measure of a rehabilitative approach employ diversion programs. One form of diversion, often reserved for lower-risk youth, is precharge diversion, where, following initial contact with the police, the youth is diverted from the system with no further police or judicial processing. Another form involves diverting youth postcharge, where a charge has been laid by the police or prosecution but the youth is diverted into an alternative system and no further judicial processing takes place. We will see that while diversion programs show variability in format, all are designed to reduce the youth’s involvement in the police and judicial systems. The next section presents an overview of differing formats displayed by diversion programs, followed by a discussion of relevant theoretical and empirical developments.
Formats of Diversion Programs
As indicated above, a basic distinction among diversion programs can be made according to program type and referral level. There are two types of diversion programs that involve differing levels of intervention. Caution or warning programs are the least invasive and serve to divert the youth out of the system with no further action, aside from a warning or formal caution. Formal diversion programs, however, generally involve some conditions, including an admission of guilt and an agreement to participate in programming if available within the program and if deemed suitable. Programming may be provided within the diversion program or through referral to a contracted external agency. However, not all formal diversion programs involve interventions and may simply be based on some sort of surveillance. Successful completion of the conditions of the formal diversion program will generally result in no further actions.
Youth can be referred to both caution and intervention programs at two levels: prior to or after the laying of a charge. Precharge referrals oftentimes involve the youth being apprehended by police and diverted immediately, either by caution and release or by referral to an intervention program. The term true diversion is reserved for precharge caution programs, as their involvement in the traditional justice system is at its most limited (Binder & Geis, 1984; Polk, 1984). Postcharge referrals apply to youth formally charged with a criminal offense. In this case, the youth who accepts responsibility for his or her actions and agrees to participate in recommended programming will undergo no further judicial processing. Successful completion of the diversion program generally results in dismissal of charges. Decisions to refer to diversion at a postcharge level may rest with the prosecutor and, under some circumstances, the judge.
Relevant Theoretical and Empirical Developments
Theoretical support for the use of diversion, whether involving therapeutic interventions or not, is provided by labeling theory (Becker, 1963) and differential association theory (Cressey, 1952; Sutherland, 1974). The former emphasizes the negative consequences of labeling a youth as delinquent. This creates an expectation of continued antisocial behavior, which may in turn limit access to conventional roles and opportunities. Differential association theory generally argues that antisocial attitudes and behaviors are learned through the social learning process. Association with others (particularly peers) exhibiting such attitudes and behaviors encourages their adoption in the youth. Both pre- and postcharge diversion can help to reduce the impact of labeling and association with antisocial peers by reducing the youth’s exposure to the traditional justice system.
A growing body of results from empirical research is also providing at least indirect support for the use of diversion. This research demonstrates clearly that involvement in the juvenile justice system, holding all other factors constant, is associated with an increased likelihood of offending behavior (Farrington, 1977; Huizinga, Schumann, Ehret, & Elliott, 2003; Klein, 1986; McAra & McVie, 2007; Smith, Goggin, & Gendreau, 2002; Tracy & Kempf, 1996). For example, McAra and McVie (2007) compared two samples of youth matched on criminal history and other variables. Recidivism rates were significantly higher over the following year for the sample of youth drawn furthest into the justice system. They concluded: “Taken to its extremes, this research would suggest (in a manner akin to labeling theory) that contact with the youth justice system is inherently criminogenic” (p. 318).
An important question arising in the context of diversion concerns the necessity for providing therapeutic interventions within the diversion process. As we will see, considerable variability is observed in existing diversion programs, with some simply cautioning the youth, others depending on general services, such as community service, and still others providing more or less intensive therapeutic interventions to address the needs of the youth.
The risk/need/responsivity model of offender intervention suggests that, under some circumstances, focused therapeutic interventions are required for the diversion program to effectively address the youth crime issue (Andrews & Bonta, 2010; Andrews, Bonta, & Hoge, 1990). Both theoretical and empirical support for this model has been provided (Dowden & Andrews, 1999; Dowden, & Andrews, 2003; Hanson, Bourgon, Helmus, & Hodgson, 2009). Three core principles underlie the model. The risk principle states that the intensity of interventions should reflect the level of criminogenic risk exhibited by the youth; intensive services should be reserved for high-risk youth, with less-intensive services reserved for lower risk youth. This principle has important implications for diversion programs. Most of these are directed toward youth with low and moderate levels of risk, and it is necessary to ensure that the level of intervention is adjusted to the youth’s level of risk. Of particular importance is ensuring that youth presenting low levels of risk are provided minimal levels of intervention or none at all. That point is discussed further below.
The two other principles of the risk/need/responsivity model have implications for the type of programming provided within the diversion program. The need principle states that interventions should be directed toward the specific criminogenic needs of the youth. If the antisocial behaviors of the youth seem primarily related to parenting problems and substance abuse, then these should be the primary targets of intervention. The responsivity principle states that decisions about programming should take account of noncriminogenic needs of the youth (e.g., academic skills, emotional problems) and strengths exhibited by the youth.
One other body of empirical research supportive of diversion programs derives from comparisons of service delivery in community versus institutional settings. Several meta-analyses (e.g., Andrews, Zinger, Hoge, Bonta, Gendreau, & Cullen, 1990; Lipsey, 2009) have demonstrated that, holding all other factors constant, therapeutic interventions delivered in the community setting are more efficacious than those delivered in institutional settings. This research supports the practice of diversion programs which offer interventions in community rather than institutional settings.
There is, then, considerable indirect theoretical and empirical support for the potential efficacy of diversion programs. However, empirical investigations involving direct comparisons of diversion and traditional processing approaches have yielded inconclusive results. The previous meta-analyses conducted by Gensheimer, Mayer, Gottschalk, and Davidson (1986) and Lipsey (2009) did not find consistent results favoring either diversion or traditional processing strategies. However, neither meta-analysis included potentially relevant moderator variables (e.g., type of diversion program, referral level), and the Lipsey analysis investigated the effects of treatment within the diversion and traditional contexts, rather than the contexts themselves.
The current meta-analysis was designed to further compare the impact of participation in either diversion or traditional processing on reoffending rates. The analysis also included a comparison of diversion programs employing only cautions or warnings with programs using treatment interventions. As well, a number of moderator variables were included in an effort to explore their potential impact on diversion and traditional processing. The latter included characteristics of participating youth, aspects of program delivery, the nature of the interventions provided, and quality of research design.
Method
Selection of Studies
Computer searches of PsycINFO, Web of Science, Criminal Justice Abstracts, Journal of Criminology, Crime & Justice, the National Criminal Justice Reference System, Journal of Research in Crime & Delinquency, Justice Quarterly, Dissertation Abstracts, and the DART-Europe E-theses portal were conducted using search terms that were a variation of diversion, alternative programs, and extrajudicial measures. These search terms were crossed with terms restricting the search to youth offenders and studies reporting on some form of recidivism. Additional articles were obtained through an examination of reference lists of the collected articles and previous meta-analyses (e.g., Gensheimer et al., 1986). Studies were eligible for coding if produced before January 1, 2011.
To be included in the meta-analysis, a study had to examine the recidivism rate of youth offenders referred to a diversion program compared to those subject to traditional processing. For the purpose of this study, diversion was defined broadly as any program that allows the youth to avoid (a) official processing through a screening process prior to the laying of a charge, (b) full prosecution after the laying of a charge, or (c) a traditional sentence (e.g., imprisonment) after conviction. This could include, but is not limited to, victim–offender mediation, community service work, restitution, and/or treatment/educational programs. However, a study was excluded if any of these interventions were a condition of a traditional disposition (e.g., probation). Studies that evaluated a diversion program that accepted referrals from educational institutions for noncriminal behavior (e.g., repeatedly missing class) were rejected. Teen court and drug treatment court evaluations were also excluded. To be accepted, the comparison group must have been processed by conventional means (e.g., probation, incarceration) and could not have participated in any alternative programming.
Studies had to include sufficient statistical information to calculate an effect size (i.e., odds ratio) and the recidivism rate. Only one effect size was calculated for each diversion program per sample and per outcome measure (e.g., general, violent recidivism). If a study provided more than one comparison group, the comparison group that most closely matched the diversion group (on risk-related information and demographics) was chosen. The inclusion criteria included studies reporting general, violent, or sexual recidivism. Only two studies reported violent recidivism in addition to general recidivism; therefore, the analysis was limited to general recidivism only.
Measures and Coding Procedure
Each study was coded using a coding manual and forms developed by (and available from) the first author. Fifty-seven variables were coded consisting of study descriptors (e.g., country of origin, research design), sample descriptors (e.g., gender, race), and program descriptors (e.g., source of referral, hours of diversion services).
In line with previous intervention meta-analyses demonstrating the impact of research design on outcome variables (Shadish & Ragsdale, 1996; Weisburd, Lum, & Petrosino, 2001), studies were coded for quality of study design. Study quality was coded as successful, somewhat successful, and nonsuccessful. An example of a successful study is a well-executed random assignment design (e.g., no differences between groups found post hoc, follow-up greater than 12 months, diversion sample size greater than 100, loss of follow-up data less than 10%). A study utilizing a matched design could also be considered successful if youth were matched on demographic and risk-related variables (e.g., criminal history) and no significant differences were found post hoc, in keeping with the aforementioned requirements. Quality of research design has primarily been linked to the internal validity of a study, and while this has typically been associated with randomized designs as the gold standard, well-designed and well-implemented matched designs have demonstrated high internal validity (Heinsman & Shadish, 1996; Shadish & Ragsdale, 1996). An example of a somewhat successful design is a matched design with no verification of group equivalency or the use of a convenience sample with the controlling of demographic and risk-related variables at the analysis stage. An example of a nonsuccessful study is if there were clear differences in risk levels between the two groups and little to no attempt was made to control for these differences.
For programs that provided some form of treatment, the degree to which the treatment adhered to the rehabilitative principles of risk, need, and responsivity was also assessed (Andrews & Bonta, 2010; Andrews, Bonta, et al., 1990). A program was coded as adhering to the principle of risk if treatment was provided to youth who were deemed high risk to reoffend and little to no services were provided to low-risk offenders. As no studies reported providing treatment to youth according to their level of risk, a program could also have been coded as adhering to this principle if the overall diversion sample was medium/high risk.
The need principle suggests that treatment must address an offender’s criminogenic needs to reduce recidivism (Andrews, Bonta, et al., 1990). Treatment was coded as adhering to the need principle if more than 50% of treatment targets have been empirically demonstrated to predict recidivism with general offenders (e.g., the Central eight; Andrews & Bonta, 2010). It was not enough for a study to merely indicate that it assessed the needs of the youth and provided the appropriate treatment; specific targets must be listed for this variable to be coded. Only two treatment programs were coded as adhering to the need principle; therefore, this variable was not included in the moderator analysis.
Adherence to the responsivity principle was met if the treatment was tailored to the learning style of the offender. In accordance with Andrews, Bonta, et al. (1990), treatment providing cognitive-behavioral therapy was coded as adhering. Similarly, a program that provided other treatment philosophies (e.g., functional family therapy) was coded as failing to adhere to this principle. A program could also be coded as adhering if the treatment accommodated an offender’s particular learning style or characteristics (e.g., maturity level).
The risk level of the youth referred to the diversion programs was also of interest. Unfortunately, few studies reported the risk level of their sample or even mentioned the target risk level (e.g., medium risk). Therefore, a proxy variable for risk was coded using available information (e.g., accepts only first-time offenders, majority of sample with previous record). The target sample was coded as low or medium/high risk, using only two groups due to the infrequent use of diversion for high-risk offenders.
To assess the interrater reliability of descriptive variables and effect sizes, eight studies were coded by a second rater. Intraclass correlation coefficients were used to calculate effect sizes (k = 10) and continuous descriptive variables (k = 11), and kappa values were employed for categorical descriptive variables (k = 41). The intraclass correlation coefficient for the effect sizes was .999 for a single rater and 1.000 for the average of the two raters, with both raters identifying 10 effect sizes. The high reliability is not surprising given that all effect sizes were calculated using a 2 × 2 table and a standard effect size workbook. The intraclass correlation coefficients for the continuous descriptive variables ranged from .87 to 1.0, with a median value of .98. The kappa values for the categorical variables ranged from .20 to 1.0, with a median value of .87. The only value below .54 was the variable identifying whether treatment other than counseling or skill building was present (kappa = .20), as one rater did not identify crisis intervention as a form of “other treatment.” This variable was used only for descriptive purposes. Last, the percentage agreement was 77.8% for one variable (referral agent) where a kappa value could not be calculated.
Index of Program Effectiveness
The odds ratio, defined as a comparative measure of risk for a particular outcome, was chosen as the most appropriate measure of program effectiveness, as both variables of interest are dichotomous and it can be estimated from a variety of study designs (Fleiss & Berlin, 2009; Haddock, Rindskopf, & Shadish, 1998). In this case, it represents (a) the probability of reoffending given participation in a diversion program divided by the probability of not reoffending given participation in the same diversion program, divided by (b) the probability of reoffending given the comparison processing divided by the probability of not reoffending given the comparison processing. An odds ratio of 1.00 indicates no difference in recidivism between the diversion group and the comparison group. Values from 0 to 0.999 suggest that the diversion program is more effective than the comparison, whereas values from 1.00 to infinity indicate that the comparison group is more effective in preventing recidivism.
As recommended by Hanson and Broom (2005), analysis was performed on the natural log of the odds ratio, thereby normalizing the distribution. Effect sizes were also weighted by the inverse of the variance, allowing studies with larger sample sizes to contribute more to the overall effect size than studies with smaller sample sizes. Effect sizes were converted back into odds ratios, which were reported.
To assess the homogeneity of variance, Cochran’s Q statistic was utilized (Hedges & Olkin, 1985). The Q statistic is commonly used to test homogeneity of variance in meta-analyses and follows a chi-square distribution with kp – 1 degrees of freedom (kp = number of programs; Huedo-Medina, Sánchez-Meca, Marín-Martínez, & Botella, 2006). Q values higher than the predetermined statistical level (e.g., p = .05) indicate that there are significant differences among studies (Huedo-Medina et al., 2006). The Q-change statistic, also known as Q-between and represented by Q∆, tests whether the magnitude of the effect sizes are significantly associated with certain variables. The I2 statistic was used to quantify the degree of heterogeneity and represents the ratio of excess dispersion to total dispersion (Higgins, Thompson, Deeks, & Altman, 2003). It was estimated from Huedo-Medina et al. (2006) using Formula 10. According to Huedo-Medina et al., percentages of 25, 40, and 75 indicate small, medium, and large proportions of heterogeneity, respectively. A negative value of I2 was interpreted as 0.
The results were presented for both fixed and random effects models. Fixed effect models tend to restrict conclusions to the set of studies observed, or studies with the exact same parameters, by failing to consider between-study variability (Hedges & Vevea, 1998). This leads to narrow confidence intervals and typically underestimates the uncertainty of the results (Overton, 1998). Random effects models permit the generalization of results to studies outside the observed set of studies by including a measure of between-study variability in its computation of estimates (Hedges & Vevea, 1998). In turn, this conservative approach gives less importance to sample size (resembling unweighted averages) and has broader confidence intervals compared to fixed models. When the variability between studies is less than would be expected by chance (Q < degrees of freedom), both fixed and random effects models provide the same results. The formulas presented in Hedges (1994) were used to calculate the fixed effect means, standard errors, and moderator analysis. The random effects estimates were calculated using Formula 10, 12, and 14 in Hedges and Vevea (1998). A minimum of three unique effect sizes per moderator variable was required to be included in the analysis to produce reliable findings.
Summary of Accepted Studies, Programs, and Youth
Forty-five studies (denoted as ks), reporting on 73 diversion programs (denoted as kp), met the criteria for inclusion. Studies were coded as published (ks = 19) if they were reported in a peer-reviewed journal and unpublished (ks = 26) if found in any other form of dissemination (e.g., book chapter, dissertation, government report). The majority of studies came from the United States (ks = 34), with only 6 from Australia, 3 from Canada, and 2 from other countries. The year the studies were produced ranged from 1972 to 2010, with a median year of 1992.
Of the 73 diversion programs, 13 were caution programs, and 60 provided some form of intervention (intervention programs). All caution programs were delivered prior to the laying of a charge, with 11 being referred and run by police. Youth referred to these programs had no further contact with the criminal justice system as part of the program and were therefore provided no services.
Youth were referred to intervention programs prior to the laying of a charge (kp = 17) as well as after the laying of a charge (kp = 41). The referral level could not be coded from one study (two programs). Youth were referred to intervention programs by police (kp = 24), judges (kp = 12), probation officers (kp = 6), court intake officers (kp = 5), crown/lawyers (kp = 4), and others (e.g., researchers; kp = 7). Of the studies reporting on the voluntary nature of the program (kp = 33), only 1 reported being mandatory. Program services were delivered by non–mental health professionals (e.g., mediators; kp = 14), criminal justice professionals (kp = 14), laypeople (e.g., volunteers; kp = 11), and mental health professionals (kp =7). This variable was missing for 14 programs.
Diversion programs provided a variety of services to youth. Services included community service referrals (kp = 20), restitution (kp = 17), restorative justice (kp = 16), and justice conferences (kp = 4). The amount of diversion services, in hours (excluding treatment), ranged from 0.33 to 42 hours, with a median of 1.22 hours (kp = 8). Treatment services were provided in 50 programs, with 40 providing some form of counseling, 23 offering skill-building programming (e.g., cognitive-behavioral techniques, employment training), and 24 offering some other form of treatment (e.g., child advocacy, crisis intervention). The diversion program served as the treatment provider in 21 of the programs, with youth being referred to an outside agency in 20 programs and both in the case of 5. Due to the number of programs offering treatment by outside community agencies and limited information on the use of manuals, training of administrators, and so on, the nature and integrity of the treatment provided could not be consistently assessed.
Sufficient information to code adherence to any of the risk, need, responsivity principles of treatment was available in 27 of the 50 studies reporting the presence of treatment. No programs adhered to all three principles. Five programs adhered to the risk principle, with 9 treatment programs failing to adhere. Of the 18 programs providing enough information to code the need principle, only 2 were in adherence. Eleven programs were coded for the responsivity principle, and only 3 adhered. Overall adherence to risk/need/responsivity could not be assessed, as only one study provided enough information to code all three principles.
The average age of the diversion samples for all programs was 14.72 (ranging from 12 to 18). The samples were more likely to be male (kp = 54) and Caucasian (kp = 26) and to have committed a property-related index offense (kp = 29). Of the studies that reported the completion status of their total sample (kp = 61), 38 reported that they included all youth referred to the diversion program, rather than just successful completers.
Results
The studies included in the analysis involved 73 diversion programs assessing 14,573 diverted youth and 18,840 youth processed by the traditional justice system. The recidivism rates for all diverted youth ranged from 2% to 81%, with an unweighted average base rate of 31.5%. The recidivism rate for the traditionally processed youth ranged from 8% to 81%, with an average of 41.3%, which was significantly different from that of the diverted youth, t(144) = −3.264, p = .001, two-tailed. In 60 of the 73 programs, the recidivism rate of diverted youth was lower than that of youth processed by the traditional justice system.
The analysis was first conducted for all diversion programs (kp = 73), followed by intervention programs (kp = 60) and caution programs (kp = 13; see Table 1). The odds ratios for all diversion programs ranged from 0.07 to 8.91, with an unweighted mean of 0.79. For fixed effect, the weighted mean was 0.61, 95% CI [.59, .63]. For random effects, this mean was slightly higher at 0.57, 95% CI [.51, .64]. Both weighted odds ratios do not exceed 1.0, indicating that diversion is more effective in reducing recidivism than the traditional justice system; however, there was more variability among effect sizes than expected by chance (Q = 365.74, df = 72, p < .001).
Meta-Analysis on the Effects of Diversion on Recidivism: Average Odds Ratios and 95% Confidence Intervals (CI)
p < .001.
When the diversion programs were broken down by program type, there was no significant difference found between intervention and cautions programs in their effectiveness in reducing recidivism compared to the traditional justice system (see Table 1). For intervention programs, the average recidivism rate of diverted youth was significantly lower than that of the comparison group (33.1% and 41.0%, respectively). The effect sizes ranged from 0.65 to 8.91, with a fixed weighted mean of 0.63, 95% CI [.59, .67]. For random effects, the weighted mean was 0.58, 95% CI [.50, .68], and there was more variability among individual effect sizes than expected by chance (Q = 271.27, df = 59, p < .001).
The cautioned youth had an average base rate of 26.8%, and the respective comparison group had an average recidivism rate of 39.5%. The effect sizes ranged from 0.12 to 1.92, with a fixed mean of 0.60, 95% CI [.57, .62], and a random effects mean of 0.52, 95% CI [.40, .66]. Similar to the intervention programs, there was more variability than expected by chance (Q = 92.82, df = 12, p < .001). For both intervention and caution programs, the weighted means do not exceed 1.0, indicating that both programs are more effective in reducing recidivism than the traditional justice system. The degree of effectiveness, however, appeared to be influenced by a number of variables.
Impact of Variables on Recidivism
All diversion programs
As shown in Table 2, a number of moderator variables were analyzed to determine their degree of influence on the effectiveness of all diversion programs. It should be noted that the majority of moderator variables assessed for all types of diversion programs (i.e., all programs combined, intervention, and caution) had a significant degree of variability (Q greater than df) and, therefore, results should be considered with this in mind.
Influence of Moderator Variables on Effectiveness of All Diversion Programs: Average Odds Ratios and 95% Confidence Intervals (CI)
Note. CJS = criminal justice system.
p < .05. **p < .01. ***p < .001.
Study characteristics were first analyzed for their moderating effect on diversion effectiveness. The quality of the study design was significantly associated with differences in recidivism, with those implementing a successful research design demonstrating less diversion effectiveness than study designs assessed as somewhat or nonsuccessful. In fact, studies employing a successful research design were more likely to conclude that diversion programs were no more effective in reducing recidivism than the traditional justice system. Significant differences in effect sizes were also found according to published status, as unpublished studies appeared to report greater diversion effectiveness. The role of the evaluator was also examined for its effect on diversion effectiveness. Evaluation studies conducted by individuals or agencies associated with the program in question found diversion reduced recidivism at a greater rate compared to conventional processing than studies completed by an independent researcher.
Two program-specific characteristics were also examined as potential moderating variables. Programs that targeted youth prior to the laying of a charge were found to be more effective in reducing recidivism than programs accepting charged youth, although the difference was small (1.69 and 1.49 times less likely to reoffend, respectively). As it has been argued that low-risk youth are more likely to be diverted prior to being charged than medium/high-risk youth (Bala, 2003), this variable was further broken down to examine the interaction of risk level and referral level on effectiveness (bottom of Table 1). Programs that targeted low-risk youth prior to the laying of a charge demonstrated significantly greater effectiveness than programs targeting low-risk youth who had been charged. For programs targeting medium/high-risk youth, there was no evidence of differential effectiveness according to referral level.
The agency that sponsored the program also influenced its reported effectiveness. Programs provided by the criminal justice system appeared to reduce recidivism at a greater rate than those provided by either non–criminal justice public agencies (e.g., social services) or researchers. Programs run by private agencies were found to be the least effective, with no statistical difference in effectiveness between diversion and traditional processing reported.
Finally, youth characteristics were examined. There was no statistical difference found in the effectiveness of diversion programs serving low- or medium/high-risk youth. However, all but one caution program reported serving low-risk youth; therefore, analysis was conducted to examine whether there were differences in effectiveness for low-risk youth according to the type of diversion program (i.e., intervention or caution). For low-risk youth, caution programs appeared to be more effective in reducing recidivism than programs providing some form of intervention. In fact, low-risk youth referred to caution programs were 2.44 times less likely to reoffend than the comparison group, whereas low-risk youth referred to intervention programs were only 1.49 times less likely to reoffend. There was insufficient data to investigate whether caution programs targeting medium/high-risk youth were more effective than intervention programs targeting the same risk level, as only one caution program reported accepting medium/high-risk youth.
Demographic variables for the sample were also examined. There were no statistical differences in diversion effectiveness of programs working with majority male or female offenders or according to age. Programs with a high prevalence of Caucasian offenders showed a greater reduction in recidivism than programs with a high prevalence of African American youth. Other ethnicities were considered (e.g., Hispanic, Aboriginal); however, there were too few effect sizes in each to be included in the analysis.
Intervention programs
Similar to the analysis of all diversion programs combined, a number of study and program characteristics influenced the degree of effectiveness of intervention programs (see Table 3). Due to the overlap in findings with all diversion programs, only differing results for the analysis specific to intervention programs will be described here.
Influence of Moderator Variables on Effectiveness of Intervention Programs: Average Odds Ratios and 95% Confidence Intervals (CI)
Note. CJS = criminal justice system.
p < .05. **p < .01. ***p < .001.
Most study-level variables (i.e., design quality and role of investigator) predicted effectiveness for intervention programs similarly to all diversion programs except published status, as there were no differences found according to whether a study was published. There was also no statistical difference found in the effectiveness of programs that received referrals at the pre- or postcharge level, indicating that intervention programs were just as effective regardless of whether they accepted charged or noncharged youth.
Treatment was offered in 50 of the intervention programs; however, there was no statistical difference in the effectiveness of programs that offered treatment compared to those that did not. Further analysis broken down by treatment type (e.g., counseling, skill building) could not be conducted, as these variables were not mutually exclusive and a number of programs offered more than one treatment philosophy. Within the offered treatment, 14 programs provided enough information to code adherence to the risk principle. Of these, programs that offered treatment targeting medium- to high-risk offenders were more effective in reducing recidivism than those that did not. Programs that offered treatment that adhered to the responsivity principle also demonstrated greater effectiveness in reducing recidivism than those providing treatment other than cognitive-behavioral therapy or those tailored to the offender’s learning style. Only two treatment programs adhered to the need principle; therefore, this variable was excluded from analysis.
Unlike the analysis with all diversion programs combined, the risk level of participants accounted for significant differences in degree of effectiveness. In line with the risk principle of rehabilitation (Andrews, Bonta, et al., 1990), intervention programs targeting medium/high-risk youth were more effective in reducing recidivism than those working with low-risk offenders (1.96 and 1.69 times less likely to reoffend, respectively).
Caution programs
As three effect sizes were required to conduct moderator analysis, the number of evaluated caution programs limited the number of moderator analyses performed (see Table 4). For example, of all caution programs reporting the risk level of participating youth (kp = 11), only one reported working with medium/high-risk offenders. Also, only two caution programs were evaluated by those involved in the development of the program, which is not surprising, as most caution programs included in the meta-analysis were run by police agencies (kp = 10).
Influence of Moderator Variables on Effectiveness of Caution Programs: Average Odds Ratios and 95% Confidence Intervals (CI)
p < .05. ***p < .001.
Unlike the intervention programs, there was a significant difference found according to published status, with unpublished studies demonstrating greater effectiveness. There appeared to be no difference in design quality if studies were coded as somewhat or nonsuccessful. Only two studies implemented a successful research design and, therefore, this variable was excluded from the analysis.
Discussion
This meta-analysis was conducted to help shed some light on the widely debated effectiveness of diversion programs. The summary results indicate that diversion programs, both caution and intervention, are significantly more effective in reducing recidivism than the traditional justice system. This conclusion is consistent with hypotheses derived from labeling and differential association theories reviewed in the introduction. These results are also consistent with research reviewed earlier indicating that simple contact with the judicial system can increase the likelihood of reoffending. This conclusion differs, however, from that of the previous diversion meta-analysis, which concluded that diversion produces no significant effects on youth (Gensheimer et al., 1986). The differing results could be due, in part, to differential inclusion criteria and the considerable difference in number of studies and subjects (this review included nearly 4 times the number of programs and 8 times the sample size of the previous study). This meta-analysis also differs from the 1986 study by distinguishing between referral levels and intervention and caution programs as well as by conducting moderator analyses to help identify which aspects of the programs are associated with reductions in recidivism. A number of these variables will be discussed here.
Study-level variables were significantly influential in contributing to the effect sizes reported by individual studies. Studies coded as implementing a successful research design found that there is no significant difference in reoffense rates of diverted youth and those processed by the conventional justice system. This is in line with a number of other intervention outcome meta-analyses demonstrating that better research designs are typically associated with smaller effects (Latimer, 2001; Lipsey, 2003; Weisburd et al., 2001). As this variable is typically considered a proxy variable for the level of internal validity, successful designs are assumed to yield more accurate results (Weisburd et al., 2001). If conclusions regarding diversion were limited to those studies that had successful research designs, it would be justifiable to conclude that diversion is no more effective than the traditional justice system in reducing recidivism. However, the programs within these studies (kp = 6) are diverse and consider both intervention and caution approaches, target different groups of youth, and provide differing services, which significantly influence the degree of effectiveness of intervention programs (Lipsey, 2009).
The role of the investigator was also significantly associated with degree of effectiveness. Evaluations that were conducted by investigators involved in the implementation of the diversion program (e.g., program directors) demonstrated significantly lower effect sizes (indicating greater effectiveness) than those found by independent researchers. This finding has been explained by scholars by the more rigorous implementation of the intervention on the part of the researcher if he or she is involved in the development of the program (Gensheimer et al., 1986; Lipsey, 2003). Interestingly, Lipsey (2003) used data from 342 studies in Lipsey and Wilson’s (1998) meta-analysis on youth intervention strategies to demonstrate that the type of research design (e.g., random or nonrandom assignment), which typically demonstrates less effectiveness for better-quality studies, and the role of the investigator, which typically demonstrates greater effectiveness for program-based researchers, are almost equally associated with effect sizes. Both these findings were replicated here; unfortunately, the limited number of successful research designs (kp = 6) prevented further analysis of the confounding influence of the role of the investigator on the research design variable, particularly since five of the six successful research designs were conducted by independent researchers.
Interestingly, all diversion programs demonstrated a difference in effectiveness according to published status; however, the results were in the opposite direction expected by a publication bias. This type of bias commonly represents the greater likelihood of studies with statistically significant findings being published, thereby biasing the results of a meta-analysis primarily making use of published articles (Gillett, 2001). In this case, however, unpublished studies demonstrated greater effectiveness and, therefore, fewer nonsignificant results. This difference does not likely have a meaningful effect on the overall results, as the difference was just barely statistically significant and both sets of studies found diversion to be more effective than conventional means.
While a number of program- and youth-level variables were examined independently, the relationship between program variables and the risk level targeted by the programs requires further discussion. According to a large body of literature commenting on the negative impact of traditional processing (Bernburg & Krohn, 2003; Huizinga & Henry, 2008; Huizinga et al., 2003; McAra & McVie, 2007), significant differences in the recidivism rates of youth referred with little police/system contact (e.g., precharge) and those referred postcharge should indicate that the further the youth is processed, the greater the likelihood that he or she will reoffend. The results are supportive of this theory when examining all diversion programs; however, an interesting trend develops when grouping the programs by targeted risk level. Though the effectiveness of diversion programs targeting medium/high-risk offenders did not differ according to referral level (i.e., they were equally effective whether accepting youth pre- or postcharge), programs accepting low-risk youth demonstrated significantly greater effectiveness when accepting them precharge than postcharge. It may be that the detrimental effect of traditional processing is more salient for low-risk youth who generally have less experience in the justice system. Perhaps diversion at any stage is equally beneficial for medium- or high-risk offenders, as it is common for these youth to have already had experience with the justice system and could therefore be less affected by official charges.
The impact of referral to a caution or intervention diversion program has been considered a way to indirectly examine the validity of labeling theory, as youth are exposed to greater processing and official involvement in intervention programs. As theorized by differential association theory, the additional processing of intervention programs should also increase the offender’s exposure to negative peers and values, which in turn increases the risk of reoffending. However, as explained in the Results section, a difference in effectiveness between programs was observed only when the risk level of the participants was considered. Of programs targeting low-risk youth, programs providing the minimum amount of services and maximum diversion (i.e., caution programs) were most effective. This is congruent with greater effectiveness found for programs serving low-risk youth prior to charging, as the amount of justice system interaction is limited. Unfortunately, because only one caution program reported targeting medium/high-risk youth, there was insufficient data to examine whether caution programs serving medium/high-risk youth were also more effective than intervention programs.
Regarding the target population of intervention programs, programs targeting medium/high-risk youth offenders achieved greater reductions in recidivism than programs targeting low-risk offenders. This is consistent with the risk principle of offender rehabilitation demonstrating that medium- and high-risk offenders are at a greater risk of reoffending and have greater needs that require services (Andrews, Bonta, et al., 1990). These results seem to suggest that detrimental labeling caused by the criminal justice system and increased association with negative peers cannot entirely account for the effectiveness of programs targeting youth with higher needs. It seems likely that providing these youth with some form of services contributes to their reduction in recidivism above and beyond the negative labeling and peer exposure; otherwise, intervention programs targeting low-risk youth would have been equally as effective.
Interestingly, however, there was no evidence of differential effectiveness according to the use of treatment, as programs providing treatment did not demonstrate greater effectiveness than programs that did not. As demonstrated by previous literature (Andrews, Zinger, et al., 1990b; Dowden & Andrews, 1999; Lipsey, 2009), certain treatment philosophies are associated with greater reductions in recidivism than others. It is therefore not surprising that a variable encompassing all forms of treatment did not distinguish between effective and less effective programs. Other meta-analyses specifically investigating adherence to the risk principle have also found that when controlling for treatment type and duration, providing services to high-risk offenders still produced reductions in recidivism (Lipsey, 2009; Lowenkamp, Latessa, & Holsinger, 2006). This could explain why intervention programs appear more effective for medium/high-risk offenders despite the nonsignificant treatment variable.
Unfortunately, the lack of information provided in each study regarding the quality, type, and dose of treatment provided was not sufficient to empirically examine treatment delivered within these diversion programs. However, despite this missing information and the varying forms of treatment, programs that adhered to the risk and responsivity principles still demonstrated greater effectiveness than treatment programs that did not.
Lipsey (2009) focused his meta-analysis on youth treatment philosophies and concluded that when controlling for methodological and sample characteristics, the context in which treatment programs are delivered (e.g., diversion, community supervision, or incarceration) did not influence effectiveness. He did, however, find that skill-building interventions (e.g., cognitive-behavioral therapy, academic training) were significantly more effective in a diversion setting than when offered through probation/parole or in custody. Though the lack of treatment information in this meta-analysis precludes any conclusions regarding treatment offered within diversion programs, it could be speculated that the existing effectiveness of diversion programs could be enhanced with the use of evidence-based treatment programs. For example, referring medium/high-risk youth at any level to an intervention program providing skill-building treatment adhering to the risk/need/responsivity principles is likely to optimize the benefits offered by diversion.
A number of moderator variables were found to significantly influence the degree of effectiveness of diversion programs (see Table 5 for summary). In fact, all variables, except the use of a successful research design and sponsorship by a private agency, suggested that diversion is significantly more effective than the criminal justice system in reducing recidivism. However, the relationship between diversion and recidivism is complex. There was considerable variability found among studies, and, therefore, these observations need to be interpreted with caution. There is also a clear indication that there are confounding variables between and within studies not considered and too few studies evaluating critical factors (e.g., successful research design) for diversion to be considered more effective under all circumstances. This meta-analysis does, however, provide some evidence to suggest that certain factors increase the degree of effectiveness of diversion and should therefore be explored further.
Summary of Moderator Analysis by Program Type: Moderator Variables Demonstrating Greater Diversion Effectiveness
Note. As there were few differences between fixed and random effects results, these results represent fixed effect findings. A dash (—) denotes too few studies to analyze or not applicable.
Implications and Future Directions for Researchers
Although this meta-analysis has provided some important trends for the use of diversion with youth, it has highlighted a number of areas that require further investigation. One of the most prominent observations is the need for stronger research designs when evaluating diversion programs. The influence of methodological variables in meta-analyses, particularly the quality of the research design, is well established (Latimer, 2001; Lipsey, 2003; Weisburd et al., 2001); therefore, the continued use of lower-quality designs makes it difficult to confidently assess the effectiveness of these programs.
Conducting the meta-analysis was also complicated by missing information about the characteristics of the clients and the nature, quality, and amount of services that youth were receiving. While this issue has been experienced within other meta-analyses (e.g., Gensheimer et al., 1986; Hanson et al., 2009), it appears to be of particular concern when evaluating diversion programs, as many provide brokerage services, referring youth to independent, external agencies. It is important in determining aspects of diversion that work that this information is recorded and used to help evaluate the effectiveness of the overall program. Also, despite the growing popularity of risk assessments and their use in working with offenders (Andrews & Bonta, 2010), few studies reported the risk level of the targeted youth. Evaluators should place more emphasis on determining the level of risk of the sample they are utilizing, as it does appear to play a role in the effectiveness of diversion, though the full scope of this role is not yet determined.
The benefits of meta-analysis are to summarize and aggregate an entire body of literature; however, it is recommended that the conclusions, particularly those drawn from post hoc moderator analysis, serve to generate, rather than test, hypotheses (Thompson & Higgins, 2002). Moderator analysis within intervention meta-analyses are used to assess specific attributes of a program or treatment philosophy; therefore, any conclusions drawn regarding the participants is considered an ecological association and cannot be generalized to other samples (Lau, Ioannidis, & Schmid, 1998; Thompson & Higgins, 2002). It is important that primary evaluation studies continue to be conducted to further explore the relationships between diversion programs and participants, particularly those involving risk level, as well as additional moderator variables not considered.
One final recommendation is that researchers should expand the range of outcome variables assessed. In almost all cases, reoffending rates constitute the sole basis for evaluating the impact of diversion. However, other outcomes relating to attitudes and values, school performance and adjustment, and mental health functioning should also be assessed.
Implications and Future Directions for Program Developers
Despite the unanswered questions emphasized by this study regarding the impact of juvenile diversion programs, a number of policy recommendations can be offered with some confidence. The meta-analysis provided strong support for the efficacy of diversion programs, whether these involved cautioning or direct interventions with the youth. In nearly all cases, these programs led to lower levels of reoffending than traditional processing through the juvenile justice system. As indicated, this conclusion is consistent with a growing body of research demonstrating that, under many circumstances, involvement in the judicial system provides negative outcomes. An additional potential benefit of using diversion as an alternative to traditional processing is the growing evidence that the former strategy is more cost-effective than the latter. There seems little reason to abstain from adopting a strategy that is more effective than traditional processing and considerably cheaper.
The conclusions of the meta-analysis also reinforce the recommendation that agencies pay particular attention to assessing the risk and needs level of youth entering the system (Hoge, 2008; Hoge & Andrews, 2010). It is clear that diversion efforts involving minimal intervention (e.g., cautioning) are appropriate for youth presenting lower levels of risk and needs. However, offenders at moderate and higher levels will benefit from more active interventions.
As we saw, reaching conclusions of the efficacy of different types of treatment was not possible, as studies provided such limited information about the quality, type, and dose of treatment. However, research from other sources supports the importance of employing evidence-based interventions, particularly structured behavioral and cognitive techniques (Guerra, Kim, & Boxer, 2008; Lipsey, 2009), within a diversion setting.
Footnotes
Authors’ Note:
We would like to thank Leticia Gutierrez for her assistance with interrater reliability.