Examining What Works for Youth With Moderate Risk Involved With the Juvenile Justice System: Comparing the Effect of the Community Connections Program and a Vocational Support Program Over a 9-Year Time Period

Abstract

Using administrative data from an urban juvenile probation department between January 2007 and August 2016, the study included youth who were placed on court-ordered postadjudication community supervision and who were deemed to have a moderate risk of reoffending by the department’s risk and needs assessment. The two programs evaluated include a vocational support program (VSP) and the Community Connections program (CC). Youth across both groups were matched using propensity score matching, creating a final sample of 301 individual youth per program. When examining the program effect of CC versus VSP across six time-to-event variables (i.e., time to second program, detention, out-of-home placement, another offense, violation of court order, and days in program), the findings were mixed. However, across both programs, analyses revealed youth with a successful discharge and longer time spent in their program had better outcomes.

Keywords

juvenile justice community programing diversion

Introduction

Adolescents generally lack the capacity for self-regulation in emotionally charged situations (Somerville et al., 2011), have a heightened sensitivity to external influences (Gardner & Steinberg, 2005) and show reduced ability in making judgments and decisions requiring consideration for future implications of those decisions (Steinberg, 2009). The combination of these factors can be causally related to adolescents engaging in risky behaviors, especially those with a high likelihood of immediate satisfaction or reward (Steinberg & Scott, 2003). Steinberg (2009) found the tendency to act on this risky behavior increases by one third of a standard deviation between ages 10 and 16 and then decreases by one half of a standard deviation by age 26.

Subsequently, adolescence is often defined as a time when youths learn to pause or curb their impulsive behavior, consider the future impact of their decisions, and function autonomously in the world (Steinberg & Caufman, 1996). Involvement with the juvenile justice system negatively impacts positive psychosocial development, and youth who are removed from their homes have been shown to face increased risks for violence, destructive behavior, and engaging in delinquent activity (Duke et al., 2010). Recidivism studies have shown youth returning from juvenile justice placements have re-arrest rates ranging from 40% (Taylor et al., 2009) and 65% (Benda et al., 2001) to as high as 85% (Trulson et al., 2005).

Community Success and Juvenile Justice-Involved Youth

Youth involved with the juvenile justice system who are supported by community-based services have lower re-offense rates than youth who are not supported by community-based services (Cuellar et al., 2006). These youth have lower rates of pretrial detention (Cuellar et al., 2006), fewer follow-up arrests (Cuellar et al., 2004), and are less likely to be adjudicated or placed deeper within the juvenile justice system (Colwell et al., 2012).

System reformers have targeted deinstitutionalization, alternatives to detention, and deeper system involvement through community-based diversion for justice-involved youth (Bishop & Decker, 2006; Loeb et al., 2015; McAra & McVie, 2007). Focusing on the balance between accountability for delinquent acts and rehabilitation (Beck et al., 2006; Mackin et al., 2010), community-based programs serve as an alternative to traditional disposition or system processing options (Harris et al., 2011; Leve & Chamberlain, 2005). These programs vary in their design and approach (Hamilton et al., 2007) depending on the type of program (Hoge, 2016; Mears et al., 2016), the risk level of the youth (Vincent et al., 2012), and the intercept point in the juvenile justice system process where the program is used or introduced as a delinquency intervention (Cocozza et al., 2005).

Community Connections Program

The Community Connections program (CC) was established in 2005. Developed in partnership with County Departments of Social Services, Mental Health, and Juvenile Probation, the program evolved out of the Southwest Key Programs (SWK) Family Keys program. It was formed when the advisory team from the departments noticed youth and families needed more comprehensive services and supports than what was offered in the 30-day Family Keys program. CC is a 6-month program targeting female and male youth between the ages of 10 and 17 on postadjudication supervision and who are at risk of out-of-home placement. The program model includes six service levels ranging from the highest to the lowest level of services and supports, and as youth move through the service levels, families gain self-sufficiency in accessing community-based services and resources. The CC model includes the following: a monitoring component to ensure probation compliance, a counseling component for youth and families to provide short-term supportive counseling or to supplement the need for long-term therapy, intensive case management to link youth and families to needed services and resources, and group counseling sessions to develop youth coping skills. While each youth and family progress through the different program levels, the CC is designed to support youth and families based on their individualized areas of need. The program uses the New Freedoms/Phoenix Core Curriculum, which incorporates evidence-based practices including elements of cognitive-behavioral therapy and motivational interviewing. The New Freedom curriculum is provided bi-monthly in a group setting for all youth enrolled in CC.

Current Study

This study examined outcomes for juveniles who participated in the CC program compared to similar youth in a vocational support program (VSP). Administrative data were collected on all youth processed by an urban juvenile probation department (JPD) between January 2007 and August 2016. Programs like VSP have operated in juvenile justice systems across the country in various formats for many years (Development Services Group, Inc., 2017). While there is a general lack of research on the impact of vocational training on juvenile offenders, some studies have begun to examine the effectiveness of supported employment and vocational support for adults and youth with mental health needs in various systems (Bond et al., 2008). Burke-Miller et al. (2012) found that, compared to adults, youth who sought out employment at a younger age had better work attainment. Geenan and colleagues (2015) found that youth who received vocational support had significantly more change over time on measures of career decision making, barriers to education, and postsecondary education preparation in foster care without vocational support. While Lipsey (2009) found that vocational programs were generally effective, they were the least effective when compared to other more therapeutic services and supports. VSP is an independent program outside of probation supervision offered by the JPD targeting youth who may benefit from vocational training or job placement support. All youth who are referred to the JPD receive an assessment of risk on the Positive Achievement Change Tool (PACT) by a trained juvenile probation officer. Youth were included in the study if they had been identified as at moderate risk of reoffending, received a postadjudication disposition decision of formal probation, and if they were enrolled in their first program (either VSP or CC) after January 1, 2007.

Based on the tenets of the risk-needs-responsivity (RNR) model, which has shown to be effective for reducing recidivism for adult offenders (Andrews & Bonta, 2017), juvenile jurisdictions have begun to implement risk assessments as a part of their process for case planning or determining supervision strategies and approaches. The RNR framework suggests the youth with the highest risk should receive the most intensive services and supports to reduce risk of reoffending (risk principle), and that programming should target a youth’s criminogenic needs while also addressing factors that can impact the treatment response (responsivity principle). These factors can include both general (the strategy used) and individual characteristics (e.g., motivation; Nelson & Vincent, 2018).

The primary study aim was to explore the program characteristics, risk factors, and demographic variables that translated to a successful discharge status for youth receiving the CC and VSP programs. The primary research question was: Do youth who participate in CC have longer times before receiving a violation of court order, detention, out-of-home placement, another offense, or another program as compared to youth in VSP? The secondary research question was: Do outcomes vary by age, ethnicity, felony history, history of violence, gender, substance use history, or mental health needs when CC is compared to VSP?

Participants

Administrative data collected included 676 CC and 354 VSP youth, though these groups were not comparable, given their differing levels of risk and background characteristics (see Appendices A, B, and C¹). These group differences were a result of judges not randomly assigning youth to programs, thus precluding a valid group comparison. Given this was the primary goal of this study, this study employed propensity score matching (PSM) analyses to create statistically comparable groups and increase statistical and internal validity. This methodology also ensured any group differences on the outcome variables were a result of the program participation and not the youth’s risk and background characteristics. After conducting PSM (see section “PSM,” see Appendix D), there were 301 youth in each of the CC and VSP groups, with both groups nearly identical as shown in Table 1. These risk level variables are described in more detail in the “Measures” section.

Table 1.

Categorical Demographic Variable Information for CC and VSP Youth Post-Matching.

Variables	CC (n = 301)n (%)	VSP (n = 301)n (%)	Test statistic (df)	p value
Gender
Male	233 (77)	235 (78)	χ² (1) = 0.04	.8446
Female	68 (23)	85 (22)
Race/ethnicity
White	33 (11)	29 (10)	χ² (2) = 0.29	.8660
Black	63 (21)	64 (21)
Hispanic	205 (68)	208 (69)
Mental health needs
Yes	204 (68)	205 (68)	χ² (2) = 0.36	.8338
No	68 (22)	71 (24)
Unknown	29 (10)	25 (8)
Substance use
Yes	206 (68)	206 (68)	χ² (1) = 0.00	.0000
No	95 (32)	95 (32)
Previous felony
Yes	190 (63)	194 (64)	χ² (1) = 0.12	.7345
No	111 (37)	107 (36)
Previous violence
Yes	225 (75)	235 (78)	χ² (1) = 0.92	.3371
No	76 (25)	66 (22)
Discharge status
Unsuccessful	77 (26)	74 (25)	χ² (1) = 0.65	.7779
Successful	224 (74)	227 (75)

CC = Community Connections program; VSP = vocational support program.

The statistics in Table 1 indicated most of the sample consisted of male individuals of Hispanic descent. The average youth age was 15 years old, with no statistically significant difference between CC (mean [M] = 15.56, standard deviation [SD] = 1.15) and VSP (M = 15.61, SD = 1.14), F (1, 600) = 0.24, p = .6440. Regardless of the program, the majority of all youth in the study had previous mental health needs, substance use, prior felonies, and previous violent histories prior to placement in their respective programs (see Table 1).

Although the amount of time youth were in their programs differed prematching, youth remained in their respective programs for an equal number of days for CC (M = 114.23, SD = 50.25) and VSP (M = 114.30, SD = 60.34) postmatching, F (1, 600) ≈ 0.00, p = .9889. Despite the estimated SDs being comparable across groups postmatching, these statistics were large in magnitude, suggesting significant variation in the time youth spent in their program. Despite there being significant differences between youth in the two groups before matching (see Appendix C), the number of previous programs received by each youth was not significantly different between groups after matching. The postmatching means and estimated SDs were comparable for CC (M = 4.88, SD = 2.93) and VSP (M = 4.97, SD = 2.92), F (1, 600) = 0.15, p = .6965.

Measures

Based on the data collected by the JPD, variables were classified into the following categories: risk factors, discharge status, demographic variables, and outcome variables. The first three categories were predictor variables, with the outcome variables being the time-to-event variables. These variables are described below, and there was no missing data present. It is important to note that data were first collected when youth entered the system and their data, in particular their outcome variables, were available into adulthood. In cases were youth did not enter the system or receive a violation again after their program ended, these outcome variables were considered censored (see “Statistical Analyses for Survival Analyses” section for more details).

Predictor Variables

Risk factors

Four risk factors were evaluated in this study. Mental health needs, which is identified by the youth’s juvenile probation officer, consists of three categories: “No, mental health needs,” “Yes, mental health needs,” and “Unknown.” For these models, the “No, mental health needs” classification was used as the reference group. Previous felony, defined as whether youths had a previous felony on their record resulting in their receiving a program, was classified as either “Yes” or “No.” For analyses, “No” was used as the reference group. Previous violence, which is defined as whether or not the youth had a previous violent offense, was classified as either “Yes” or “No,” with “No” being used as the reference group in statistical models. Substance use is defined as whether the youth had reported substance use such as marijuana or alcohol use. This variable was classified as either “Yes” or “No,” with “No” being used as the reference group.

The number of previous programs within the timeframe of January 2007 until they started CC or VSP was also used as a covariate, as it is possible that less-intensive programs may have provided added benefits to their enrollment in the current program. Having been involved with previous less-intensive programs also provides an indication of previous problems within the juvenile justice system, suggesting these youths may have a higher risk. With that said, one limitation of this variable is the dose and type of previous program is not included in the model and could differ across groups.

Program characteristics

The discharge status variable indicated whether or not the youth had a successful or unsuccessful discharge from the program, with an “Unsuccessful discharge” being used as the reference group. One limitation of this variable is that program success was defined by juvenile probation staff rather than by staff from either program; therefore, the reason and justification for discharge is unknown by the programs under study. Moreover, subjective perceptions are likely to vary across probation officers, thus influencing the variable’s reliability.

The days in program variable was utilized as both a predictor and an outcome variable; when employed as a predictor, it was used to determine whether the number of days in the program resulted in better, or possibly worse, program outcomes (see Outcome variables below) and, thus, reflects a dose effect variable. As an outcome variable, the other predictor variables were used to predict the number of days the youth stayed in the program.

Demographic variables

To understand whether the outcome variables differed based on youth demographics, the following demographic variables were included in the model as predictor variables or interaction terms: race/ethnicity (White, Black, and Hispanic), gender (Male and Female), and age at the beginning of the program. For these categorical variables, Whites (ethnicity) and males (gender) were used as the reference groups.

Time-to-Event Variables/Outcome Variables

Six time-to-event variables (measured in days) were examined to evaluate the long-term impact of each program on youth, while also using the aforementioned predictor variables to explain differences in survival rates. The first outcome variable, Time to second program measures the number of days until a youth is enrolled in another community program after CC or VSP. As with all outcome variables examined, the data were censored if the youth had not received an event, in this case a detention, as of August 31, 2016. Depending on the youth’s age when they entered the program, most of the time-to-event variables were collected when they were young adults (i.e., early twenties).

Time to detention assessed the number of days until the youth had received a detention, or in other words been arrested in a police cell or placed in a detention center before trial or sentencing, after their program ended. Time to placement was used to determine the number of days until the youth was removed from their home and received an out-of-home placement. These could be either secure or nonsecure out-of-home placements and could include treatment programs as well as correctional models. The Time to next offense outcome variable measured the number of days until the youth committed another new or repeat offense.

Time to a violation of court order is the number of days until youths were charged with violating their court orders. Although a violation of a court order is typically handled as a new offense throughout the court process, it was treated as an independent outcome to help evaluate the interaction of probation with CC and VSP. Finally, days in program measures the number of days a youth spent in the program before being released. Again, this variable was also used as a predictor variable when predicting the other time-to-event variables.

PSM

To create equal CC and VSP groups at pretreatment, this study selected a set of variables that were required to be equivalent across groups for valid inferences. The following variables were included: age, gender, race, mental health needs, previous violence, previous substance use, previous felony, number of previous programs/services, discharge status, and days in program.

Nearest neighbor matching was used to create statistically equivalent groups via PSM. Given exact matching (i.e., youth being identical on all variables) would have reduced the sample size significantly, this study elected to use a caliper to assess the degree of difference between the groups with an acceptable range. While several calipers were explored, a caliper of .11 best maximized the sample size and minimized group differences on the covariates of interest and ultimately produced no significant groups differences postmatching. As seen in Appendix D, there were no group differences postmatching on these variables.

Statistical Analyses for Survival Analyses

Model Building Process

Nonparametric Cox survival analysis models were estimated to predict the time-to-event on the predictor variables. All models were graphically examined using PROC LIFEREG and built using PROC PHREG within SAS, version 9.4, while following a 5-step process. Step 1, all predictor variables, along with their two-way interactions, were included in the model to determine those variables that significantly predicted the time-to-event outcome variable. This was done using a stepwise method with an entry and stay value of p = .20 and p = .05, respectfully. After the “best” stepwise model was selected, predictor variables associated with any statistically significant interaction term were brought back into the model in Step 2, if not already included.

In Step 3, the model fit (i.e., R² statistic, C-statistics/area of the curve, AUC) was assessed to evaluate the overall model quality and determine whether complex interaction terms could be removed from the model without significantly harming model quality, prediction, or interpretability. This was done to reduce model complexity. Next for Step 4, we tested whether the model assumptions (such as proportional hazards assumption) were met and made model modifications that included time-varying covariates, if model assumptions were violated. Finally for Step 5, the final overall model quality was reassessed, and interactions were explored in detail to better understand the results.

Terminology

Time-to-event variables refer to the number of days until the event (e.g., next offense) of interest occurs. An observation is said to be censored if the event (e.g., next offense) did not occur before August 31, 2016, and, therefore, the time until that event occurs, if ever, is unknown. Given censoring depended on when the youth entered the program, for example, a youth entering the program in June of 2007 is less likely to be censored than a youth entering the program in June of 2016, the days since program ended variable was included in the model to adjust the parameter estimates, although this variable was not of practical interest.

The hazard function, f(t), describes the risk of the event occurring at time t, conditional on the youth’s survival up to that time t, S(t). The hazard ratio (HR), which is an effect size measure, assesses how often a hazard event occurs in one group (e.g., CC) compared to how often it occurs in another group (e.g., VSP) over time. With categorical variables, it compares the hazard rate of the comparison group to the hazard rate of the reference group, $HR = Comparison hazard rate / Reference hazard rate$ . For continuous predictor variables, the HR measures the change in hazards for a one-unit change in the predictor variable. For example, it would evaluate the change in hazards for being in the program 20 versus 21 days. Therefore, hazards were compared at different time intervals (i.e., 30, 60, . . . , 150 days) to make the results easier to interpret.

It is important to note HRs can range from 0 to ∞, with a value of 1 indicating the hazard rate is the same for both groups (i.e., it is conceptually similar to an odds ratio). It is also important to recognize an HR of .80 does not have the same impact as an HR of 1.20. For comparability and interpretability purposes, one can take the reciprocal (1/HR) to change the direction of the HR. For example, an HR of .50 (female vs. male) is equal to an HR of two (male vs. female, 2 = 1/.50). It is also worth mentioning any 95% confidence interval (CI) that does not include one is significant at the .05 level.

To assess how well the model predicted the outcome variables, the AUC or C-statistic was used to provide the predictive accuracy at specific times—in this case, days. An AUC or C-statistic above .80 provides evidence of a good predictive model; however, it is important to recall prediction accuracy is time dependent. For this study, the model’s prediction accuracy tended to be poorer immediately after the program’s end (this is likely due to the large number of changes in the youth’s lives and the adjustment of going back to their homes and neighborhoods), but improved over time. The generalized R² (see Allison, 2010, pp. 282−283) was also provided to allow an alternative perception of model prediction accuracy, with values closer to one considered preferable.

Results Roadmap

For each outcome variable, the results are presented in three major sections. The section “Model performance” focuses on the performance or predictive accuracy of the overall model while also discussing the level of censoring. More specifically, this section focuses on whether those variables in the model adequately predict survival rates and whether model prediction varied over the course of the study.

Second “Program-related effects” evaluated the program effect variables of program, discharge status, and days in the program while also including potential interactions with other background variables. The program variable of CC versus VSP is of greatest interest, as the study’s primary goal was to assess differences in survival rates between CC and the VSP comparison group. Discharge status was also of interest, given it evaluated the impact of youth successfully completing the program, which should serve as a marker of later success. The number of days in the program was also examined here to determine whether youth who stayed in the program longer (and thus received a larger dose effect) displayed better outcomes.

Section “Background variable effects,” which was more of secondary interest, focused on those background variables that significantly predicted the time-to-event variables. While these background variables cannot be manipulated to alter program outcomes of time-to-event variables, it is extremely useful to know which specific youth background characteristics predict later success in a program. In turn, knowing these background characteristics could help target program referrals for youth who are a better fit for program success or modifying the program to be more successful for broader types of youth.

Results

Time to Second Program

Model performance

After following the 5-step process described above, the variables and terms in Table 2 served as significant predictors of time to their second program. The overall model’s likelihood ratio (LR) χ² was statistically significant, $χ_{L R}^{2}$ (df = 13) = 144.48, p < .0001, R²= .21. The overall model’s AUC (AUC = .69) was less than desired (AUC = .80); however, the model’s AUC improved over time (see Appendix E) and provided good model predictions after about 300 days. This finding implies that while the prediction/classification of youth who receive a second program is less accurate within about a year of their release, the prediction accuracy increases the longer they are removed from the program.

Table 2.

Parameter Estimates and Test Statistics for Time to Second Program Model.

Predictor	NRG1	NRG2	χ²	p value	HR
Model 1: Time to second program
Program	CC		1.18	.2764	1.13
Discharge status	Successful		9.12	.0025	0.69
Days in program	114.26		12.09	.0005	1.00
Race	Black		6.14	.0132
Race	Hispanic		4.74	.0295
Gender	Female		11.78	.0006	0.60
Substance use	Yes		11.48	.0007
Mental health needs	Unknown		4.32	.0377	0.54
Mental health needs	Yes		12.57	.0004	1.73
Race*Substance use	Black	Yes	8.36	.0038
Race*Substance use	Hispanic	Yes	7.29	.0069
Age	15.58		58.12	<.0001	0.73
Days since program ended	1862.72		5.88	.0153	1.00

Note. Mental health needs, discharge status, days in program, days since program ended, and nonreference group are denoted by MHN, PO, DIP, DSPE, and NRG, respectfully. NRG2 is the additional reference group when an interaction is modeled. CC = Community Connections program; HR = hazard ratio.

Program-related effects

When examining the program effect (i.e., CC vs. VSP), the HR = 1.13 (p = .2764) was nonsignificant and indicated that there was no difference in their time until receiving a second program. Yet, discharge status was a significant predictor (HR = .69 or HR = 1/0.69 = 1.45, p = .0025), with those who had a successful discharge status having a longer time until their next program (or a lower hazard rate) compared to those with an unsuccessful discharge status.

The significant effect (i.e., HR ≈ 1) of days in the program is less intuitive due to the continuous nature of the predictor variable. When calculating 30-day intervals, the analyses revealed being in the program for 30 days resulted in a 10.6% (1 − HR or 1 − 0.89) reduction in their hazard of receiving a second program. This reduction in hazard increased as the number of days in the program increased to 60 (20.0% reduction), 90 (28.4% reduction), 120 (36.0% reduction), and 150 (42.8% reduction) days. Stated more simply, the longer a youth remained in their program (regardless of whether it was CC or VSP), the less likely they were to receive another program in the future. This is critical, as it indicates an increased dose effect resulted in better youth outcomes.

Background variable effects

Exploring the significant (Table 2) race by substance use interaction first, the results revealed that there was no significant difference in hazard rates between those with and without substance use for either Black (HR = 0.92, 95% CI = 0.54–1.52) or Hispanic (HR = 1.18, 95% CI = 0.89–1.56) youth. However, there was a significant difference in the hazard rates between those with and without substance use for White youth (HR = 3.29, 95% CI = 1.62–6.51), with the hazard rate being 3.29 times larger for White youth with a substance use problem than for White youth without a substance use problem.

Gender was also a significant predictor of time to their next program, with females having a much smaller hazard rate than males (HR = 0.60 or 1/0.60 = 1.67). This indicates that the hazard rate for males was 1.67 times the hazard rate for females, and males tended to receive another program before females. The mental health needs variable was also a significant predictor of time to next program; youth with mental health needs were 73% more likely (HR = 1.73) receive a second program when compared to youth with no mental health needs. Interestingly, those with an unknown mental health status were at a lower risk of being enrolled in a second program when compared to the no mental health needs group. The last finding relates to the youth’s age and its effect on time to a second program. Out results indicated younger youth were less likely to receive a second program than older youth based on the HR below one (see Table 2).

Time to Detention

Model performance

The overall model significantly predicted youth’s time to detention, $χ_{L R}^{2}$ (df = 13) = 170.76, p < .0001, R² = .29, and AUC = .69. Once again, the model’s AUC improved over time and provided better model predictions with longer times after the program ended.

Program-related effects

When considering the program direct effect, there was a marginally significant effect (p = .0642, Table 3) on time to detention. However, this direct effect should be interpreted with caution, given the program-by-felony interaction was statistically significant (p = .0259). Exploring this interaction in more detail, follow-up analyses indicated the CC program impact did not significantly change (HR = 0.89, 95% CI = 0.94–1.24) whether or not the youth had a previous felony. However, the VSP youth without a felony had a 50% increase (HR = 1.50, 95% CI = 1.08–2.05) in their risk of experiencing a second detention when compared to VSP youth with a felony. Examining a different pair of HRs related to this interaction, the results revealed no difference between CC and VSP when the youth did (HR = 1.19, 95% CI = 0.90–1.57) and did not (HR = 0.71, 95% CI = 0.49–1.02) have a previous felony.

Table 3.

Parameter Estimates and Test Statistics for Time to Detention Model.

Predictor	NRG1	NRG2	χ²	p-value	HR
	Model 2: Time to detention
Program	CC		3.43	.0642
Discharge status	Successful		15.00	.0001
Days in program	114.26		10.70	.0011	1.00
Gender	Female		1.05	.3050
Felony	Yes		6.17	.0130
Substance use	Yes		3.52	.0607	0.80
Mental health needs	Unknown		1.07	.3009	0.74
Mental health needs	Yes		22.11	<.0001	2.09
Discharge status*Gender	Female	Successful	8.66	.0033
Program*Felony	CC	Yes	4.96	.0259
Age	15.58		4.82	.0282	0.87
Age*Time			15.97	<.0001	1.00
Days since program ended	1862.72		6.43	.0112	1.00

Another interesting program effect was program discharge status (p = .0001); however, this effect differed by gender based on the significant discharge status by gender interaction (p = .0033). The survival plots indicated the following order for the best survival outcomes (longer time to detention): females with a successful discharge status, males with a successful discharge status, males with an unsuccessful discharge status, and finally females with an unsuccessful discharge status. Statistically, this interaction emerged because there was no difference between males and females (HR = 1.28, 95% CI = 0.78–2.00) when the discharge status was unsuccessful, but there was a significant gender difference (HR = 0.53, 95% CI = 0.36–0.75) when the discharge status was successful.

For the number of days in the program effect, the results revealed that regardless of the program, the more time youths spent in a program, the better their long-term outcome, or greater time until next detention. Specifically, this reduction in hazard rates increased as the number of days in the program increased to 30 (10.5% reduction), 60 (19.9% reduction), 90 (28.3% reduction), 120 (35.8% reduction), and 150 (42.5% reduction) days.

Background variable effects

While many of the background variables interacted with program effect variables, an unrelated significant effect was that youth with a previous substance use history had a greater hazard rate compared to those without substance use issues (HR = 0.80 or 1/.80 = 1.25). In fact, those youths without a history had a 25% decrease in hazard rates when compared to youth with a history of substance use. Finally, while there was no difference in hazards between youth without mental health needs and unknown mental health needs (HR = 0.74, p = .3009), youth with mental health needs were at a greater risk of receiving a detention sooner compared to those with no mental health needs (HR = 2.09, p < .0001).

Time to Placement

Model performance

The variables included in the model (Table 4) were significant predictors of time to placement, $χ_{L R}^{2}$ (df = 15) = 187.60, p < .0001, R² = .27, and AUC = .71. These results again suggest a good prediction of the outcome variable, with prediction accuracy improving the more days from when the program ended.

Table 4.

Parameter Estimates and Test Statistics for Time to Placement Model.

Predictor	NRG1	NRG2	χ²	p-value	HR
	Model 3: Time to placement
Program	CC		21.11	<.0001
Discharge status	Successful		3.06	.0804
Days in program	114.26		24.24	<.0001	0.99
Gender	Female		3.57	.0589	0.73
Violence	Yes		7.40	.0065
Mental health needs	Unknown		6.10	.0135
Mental health needs	Yes		3.88	.0489
Discharge status*Violence	Yes	Successful	5.65	.0175
Program*Mental health needs	CC	Unknown	4.69	.0304
Program*Mental health needs	CC	Yes	8.69	.0032
Days in program*Time			12.91	.0003	1.00
Program*Time			15.61	<.0001	1.00
Age	15.58		0.19	.6613	1.04
Age*Time			27.62	<.0001	1.00
Days since program ended	1862.72		9.36	.0022	1.00

Program-related effects

Once again, the direct effect suggested a significant program effect (p < .0001); however, this effect should be interpreted with caution given that the program by mental health needs interaction was also statistically significant. An evaluation of this interaction revealed the difference between CC and VSP was nonsignificant when the mental health status was unknown (HR = 0.86, 95% CI = 0.16–6.39). However, the program effect increased, and was statistically significant, for those with mental health needs (HR = 0.42, 95% CI = 0.27–0.64) and no mental health needs (HR = 0.10, 95% CI = 0.04–0.25). Collectively, this interaction suggested CC youth consistently had a smaller hazard rate than VSP youth with and without mental health needs. Note that the significant program by time interaction as seen in Table 3 implies the hazard rates were not proportional over time, which indicates the difference in hazard rates was not equal over the course of the study.

Although discharge status was not a significant predictor of time to placement (p = .0804), the interaction between discharge status and previous violence was significant (p = .0175). Despite the large HR = 2.31 (95% CI = 0.99–6.76) estimated from follow-up analyses, there was no significant difference between those with a successful and an unsuccessful discharge status when the youth did not have a history of violence (due to the large estimated standard error). Conversely, the hazard rate was significantly lower for youth with a successful discharge status compared to those with an unsuccessful discharge status (HR = 0.70, 95% CI = 0.52–0.97). This implies that nonviolent youth with a successful discharge status exhibited a longer time until receiving another placement, or a smaller hazard rate, when compared to youth with a history of violence.

The number of days in program also had a significant direct effect on time to placement (HR = 0.99, p < .0001), with those in the program longer displaying greater time until their next placement. Once again, the reduction in hazards increased as the number of days in the program increased to 30 (25.5% reduction), 60 (44.4% reduction), 90 (58.6% reduction), 120 (69.1% reduction), and 150 (77.0% reduction) days.

Background variable effects

The only notably significant background variable was gender, with the results indicating males had a greater decrease in hazard rate than females (HR = 0.73, Table 4). Stated more simply, males had a shorter amount of time before their next placement than females.

Time to Next Offense

Model performance

A larger number of predictors were used to predict time to next offense, with these variables resulting in a significant model prediction, $χ_{L R}^{2}$ (df = 19) = 217.49, p < .0001, R² = .30, and AUC = .66. These results revealed an adequate prediction of the outcome variable; however, prediction accuracy improved the more days from when the program ended.

Program-related effects

Results presented in Table 5 indicate seven significant interaction terms; however, only one of the program effects (i.e., dismissal status by gender interaction), was significant. The test statistics associated with the program variable effect revealed no difference between CC and VSP (p = .1398). In addition, the number of days in the program was also not a significant predictor of time until the youth’s next offense (p = .2649).

Table 5.

Parameter Estimates and Test Statistics for Time to Next Offense Model.

Predictor	NRG1	χ²	p-value	HR
	Model 4: Time to next offense
Program	CC	2.18	.1398	1.17
Discharge status	Successful	2.98	.0844
Days in program	114.26	1.24	.2649	1.00
Race	Black	0.45	.5031
Race	Hispanic	1.74	.1868
Gender	Female	11.80	.0006
Felony	Yes	0.01	.9277
Substance use	Yes	5.54	.0186
Mental health needs	Unknown	0.30	.5843	0.86
Mental health needs	Yes	9.75	.0018	1.56
Gender*Felony	Female	3.91	.048
Discharge status*Gender	Female	12.51	.0004
Race*Substance use	Black	3.72	.0539
Race*Substance use	Hispanic	6.81	.0091
Race*Gender	Black	1.93	.1646
Race*Gender	Hispanic	7.75	.0054
Days in program*Time		12.91	.0003	1.00
Age	15.58	18.68	<.0001	0.83
Age*Time		73.25	<.0001	1.00
Days since program ended	1862.72	44.67	<.0001	1.00

Although dismissal status was only a marginally significant predictor of the amount of time until their next offense (p = .0844), this effect should be interpreted with caution due to the significant (p = .0004) dismissal status by gender interaction. A probe of this interaction revealed that there was no difference between those with a successful and an unsuccessful dismissal status for males (HR = 0.80, 95% CI = 0.63–1.03); however, there was a significant difference in the hazard rates for females (HR = 0.28, 95% CI = 0.17–0.49). In this case, the hazard rates for youth with an unsuccessful dismissal was 3.71 times (HR = 1/.28 = 3.71) greater than it was for youth with a successful dismissal.

Background variable effects

As seen in Table 5, both gender and race interacted with one another and with two other variables in the model—previous felony and substance use. A closer analysis of the race-by-gender interaction revealed no significant difference in hazard rates between White males and females (HR = 1.52, 95% CI = 0.70–3.07) or Black males and females (HR = 0.80, 95% CI = 0.43–1.40); however, a significant gender difference emerged for Hispanics (HR = 0.58, 95% CI = 0.41–0.80). Hispanic males had a greater hazard rate than females and were more likely to commit another offense sooner during the study timeframe.

A probe of the race-by-substance use interaction revealed no significant difference in hazard rates between those with and without substance use issues for Black (HR = 1.03, 95% CI = 0.62–1.65) and Hispanic (HR = 0.89, 95% CI = 0.68–1.17) youth. However, White youth with substance use issues had a significantly higher hazard rate compared to those without a substance use history (HR = 2.27, 95% CI = 1.15–4.36). Follow-up analyses of the gender-by-felony interaction determined there was no difference between male youth with or without a felony (HR = 1.01, 95% CI = 0.80–1.28), but the hazard rate was significantly greater for females with a felony (HR = 1.79, 95% CI = 1.05–2.97) than females with no felony history. Consequently, females with a felony were more likely to commit a new offense sooner than females without a felony, regardless of which program they were in.

With the exception of age—which suggested hazard rates decrease as youth increased in age and was nonproportional over time—the only other direct effects not included with an interaction term was mental health status. Table 5 results revealed no significant difference in hazard rates between those with unknown and no mental health needs (p = .5843); however, those youths with mental health needs had a significantly greater hazard rate compared to those without mental health needs (HR = 1.56, p = .0018).

Time to Violation of Court Order

Model performance

Despite being a simpler model, the model variables in Table 6 were significant predictors of time to violation of court order and resulted in a large AUC regardless of study time point, $χ_{L R}^{2}$ . (df = 9) = 275.39, p < .0001, R² = .37, and AUC = .81.

Table 6.

Parameter Estimates and Test Statistics for Time to Violation of Court Order Model.

Predictor	NRG1	NRG2	χ²	p-value	HR
	Model 5: Time to violation of court order
Program	CC		0.69	.4064
Discharge status	Successful		1.43	.2316
Days in program	114.26		8.18	.0042	1.00
Mental health needs	Unknown		3.18	.0746	0.60
Mental health needs	Yes		4.81	.0283	1.54
Substance use	Yes		5.43	.0198	0.70
Program*Discharge status	CC	Successful	7.08	.0078
Program*Time			6.50	.0108	1.00
Days since program ended	1862.72		168.42	<.0001	1.00

Program-related effects

Despite the model being a strong predictor of time to violation of court orders, neither the program (p = .4064) nor the discharge status (p = .2316) variables were significant predictors. However, the program-by-discharge status interaction was significant (p = .0078) and suggests these two variables interact to influence the time to violation of court orders. Follow-up analyses of the interaction revealed there was a significant difference between CC and VSP when the program discharge status was successful (HR = 0.33, 95% CI = 0.20–0.54), but not when the program discharge status was unsuccessful (HR = 0.79, 95% CI = 0.45–1.40). In fact, CC youth with a successful discharge status had the best (or smallest) hazard rate, meaning longer times until their violation of court orders, whereas CC youth with an unsuccessful discharge status had the worst (or largest) hazard rate. The program effects were not proportional over time based on the significant program by time interaction (p = .0108, Table 6), thus implying distance between hazard rates varied over time.

The number of days the youth were in the program was once again a significant predictor (p = .0042) of receiving a violation of a court order. However, unlike the other outcome variables, the hazards increased as the number of days in the program increased to 30 (11.5% increase), 60 (24.4% increase), 90 (38.7% increase), 120 (54.7% increase), and 150 (72.6% increase) days. Suggesting the more days’ youth spent in the program increased hazard of violating their court order.

Background variable effects

Only two background variables, mental health needs, and substance use, were significant predictors of time to violation of court orders. In terms of mental health needs, there was no significant difference between youth with no mental health needs and those with unknown mental health needs (p = .0746, Table 6); however, youth with mental health needs had a hazard rate 1.54 times greater than youth with no mental health needs (HR = 1.54, p = .0283). Youth with a substance use history had a greater hazard rate than those without a substance use history (HR = 0.70, p = .0198).

Days in Program

Model performance

This model had several significant predictors (Table 7) of days in program; however, the model overall did not allow accurate predictions based on the lower AUC, $χ_{L R}^{2}$ (df = 13) = 102.67, p < .0001, R² = .17, and AUC = .56. No data were censored for this model, given all youth had an official program end date.

Table 7.

Parameter Estimates and Test Statistics for Times Today in Program Model.

Predictor	NRG1	NRG2	χ²	p-value	HR
	Model 6: Time to Days in program
Program	CC		7.57	.0059	0.56
Discharge status	Successful		40.11	<.0001
Race	Black		2.64	.1044
Race	Hispanic		11.78	.0006
Gender	Female		2.16	.1420
Mental health needs	Unknown		2.98	.0845	1.33
Mental health needs	Yes		3.15	.0758	1.20
Race*Gender	Black	Female	3.12	.0775
Race*Gender	Hispanic	Female	0.29	.5934
Discharge status*Gender	Female	Successful	7.35	.0067
Program*Time			9.09	.0026	1.00
Discharge status*Time			37.68	<.0001	1.01
Age	15.58		8.20	.0042	1.12

Program-related effects

The significant program variable effect (HR = 0.56, p = .0059) suggested the hazard rates for the programs differed, with CC performing better; however, this finding needs to be interpreted with some caution due to the significant program by time interaction (p = .0026).

The discharge status variable had mixed findings, as the direct effect of discharge status (p < .0001), interaction between discharge status and gender (p = .0067), and the interaction between discharge status and time (p < .0001) were significant. For the gender by discharge status interaction, the survival rates were not significantly different between males and females for a successful discharge (HR = 0.83, 95% CI = 0.66–1.04), but the gender survival rates differed for those with an unsuccessful discharge (HR = 1.65, 95% CI = 1.04–2.40).

Background variable effects

Predictors of importance here include mental health needs and the interaction between gender and race (Table 7). Results suggested that compared to those with no mental health needs, youth with unknown (HR = 1.33, p = .0845) and known (HR = 1.20, p = .0758) mental health needs had greater hazard rates. There was no difference between youth with unknown and known mental health needs (HR = 1.11, p = .4856).

Gender-by-race interaction follow-up analyses revealed no difference between males and females among White (HR = 0.97, 95% CI = 0.54–1.67) and Hispanic (HR = 1.14, 95% CI = 0.90–1.44) youth; however, there was a difference in hazard rates between genders for Black youth (HR = 0.50, 95% CI = 0.31–0.79) youth. Here, Black males tended to have a larger hazard rate than Black females.

Discussion

The study results provide several beneficial findings that could be used to aid researchers, practitioners, and the juvenile justice system as a whole. First, discharge status always had a significant effect, either directly or moderating, on the time to event variables. From these findings, it is clear practitioners and juvenile justice organizations should make every effort to ensure youth are on progress to have a successful discharge, and this includes continually communicating with probation offices to identify those youth in danger of not having a successful discharge.

A second finding is that for all but one model (Model 4), the number of days in the program was a significant predictor of later success. Essentially, these results imply that, with the exception of receiving a violation of probation, the longer the youth remained in the program, the better the likelihood of long-term success. Clearly, there exists a fine line between keeping youth in the program longer (which increases the cost to treat them) and ending the program before they receive the necessary services to be successful after leaving.

When examining the program effect (i.e., CC vs. VSP), conclusions were outcome variable dependent. There were no significant differences in program effect between CC and VSP on time to second program or time to next offense. However, there was a significant program effect when CC performed better for time to placement. There was a marginally significant program effect on the time to detention variable, with CC performing better. However, this program effect was moderated by whether or not the youth had a previous felony. Whether or not the youth had a previous felony did not significantly change the outcome for CC youth; however, VSP youth with a felony were at increased risk of experiencing a detention when compared to VSP youths without a felony.

Although of secondary interest, this study did provide several interesting and useful model predictions using background variables of gender, race, age, and previous history of a felony, violence, mental health needs, and substance use. From Models 1 to 5, the following variables or their interactions were included as significant predictors in 80% or more cases: gender (80%), age (80%), mental health needs (100%), and substance use (80%). However, race (40%), history of a felony (40%), and history of violence (20%) were less often included in the models. From these findings, it is evident that of the risk variables, substance use, and mental health needs were more often significant predictors of negative outcomes for both programs, and maybe more generally for youth involved in the juvenile justice system more generally, than violence or history of a felony. For both programs, youth with known mental health needs were more likely than youth without mental health needs to receive a violation of a court order, in addition to an increased likelihood of being detained or sent to an out-of-home placement. For girls, an unsuccessful discharge greatly increased the likelihood of committing an additional offense.

From a general model performance perspective, the AUC statistics indicate the models, with the exception of the days in program model, frequently performed worse immediately following the completion of the program and then continued to improve thereafter. In other words, predicting the model’s success is more difficult immediately after the youth leaves the program. Consequently, other variables should be added to future models to improve model predictions immediately following program completion. For example, it is certainly possible other variables such as social support of friends and family, dedication to turning their lives around, and environmental conditions play a critical role after their program is completed.

Limitations

A few limitations are worth mentioning related to the program effects (i.e., program received, discharge status, and program length). This study does not consider previous programs or services the youth might have received. CC or VSP were the first program the youth received after January 1, 2007. Therefore, this study is really a test of the program effect independent of previous youth programs. Any programs the youth might have received after CC or VSP is largely irrelevant, as the outcome variables were time-to-event variables. Consequently, if a second program was received after CC or VSP, it would be captured in the time to a second program variable, and it would be an indication that a failure had already occurred, and their time-to-event was already measured in this study.

This study created statistically equivalent groups via PSM to ensure the groups were nearly identical in terms of risk and background characteristics. Moreover, this study assumed any previous programs the youths might have received were inadequate in addressing their needs since the youth eventually ended up in another intensive program—either CC or VSP. Another significant limitation of this study is that there were no program characteristic variables included in the model, nor was there any accurate measure of program dose. Specifically, the study models did not consider the actual number of contact hours, the quality of the relationship between program staff, probation officer and youth, or the length, quality, and amount of interaction between the youth and their caseworkers. Moreover, this study did not include measures of treatment/program integrity; thus, the degree of variation between youth program experiences is unknown and not included in the models.

Conclusion

Appropriate and effective services and supports are critical toward the rehabilitation of youth involved with the juvenile justice system. Subsequently, policymakers have targeted diversion strategies from deeper system involvement through community-based diversion for justice-involved youth. However, community-based programs vary in their design and approach. With the establishment of RNR in the juvenile justice system as a guiding framework for supervision and case management strategies, the results of this study indicate a need for further evaluation on the matching of services and supports based on youth needs.

Footnotes

Appendix A

Categorical Demographic Variable Information for CC and VSP Youth Prematching.

	CC (n = 676)	VSP (n = 354)
	n (%)	n (%)
Gender
Male	502 (74)	283 (80)
Female	174 (26)	71 (20)
Race/ethnicity
White	53 (8)	34 (10)
Black	167 (25)	71 (20)
Hispanic	456 (67)	249 (70)
Mental health needs
Yes	439 (65)	242 (63)
No	168 (25)	83 (23)
Unknown	69 (10)	29 (8)
Substance abuse
Yes	531 (79)	220 (62)
No	145 (21)	134 (38)
Previous felony
Yes	432 (64)	225 (64)
No	244 (36)	129 (36)
Previous violence
Yes	503 (74)	283 (80)
No	173 (26)	71 (20)
Discharge status
Unsuccessful	254 (38)	76 (21)
Successful	422 (62)	278 (79)

CC = Community Connections program; VSP = vocational support program.

Appendix B

Continuous Demographic Variable Information for CC and VSP Youth Pre-Matching.

Variables	CC (n = 676)	VSP (n = 354)
Variables	M (SD)	M (SD)
Age	15.20 (1.26)	15.69 (1.12)
Days in current program	96.10 (52.84)	124.75 (71.49)
Number of previous programs	3.87 (2.76)	5.30 (3.13)

CC = Community Connections program; VSP = vocational support program.

Appendix C

Pre-Matching Comparison of the CC and VSP Demographic and Risk Variables.

	CC vs. VSP
	Test statistic (df)	p-value
Gender	χ² (1) = 4.14	.0419
Race/ethnicity	χ² (2) = 3.31	.1911
Previous violence	χ² (1) = 3.94	.0472
Previous substance abuse	χ² (1) = 31.65	<.0001
Previous felony	χ² (1) = 0.01	.9126
Discharge status	χ² (1) = 27.68	<.0001
Age	F (1) = 29.21	<.0001
Number of previous programs	χ² (1) = 108.72	<.0001
Days in current program	χ² (1) = 1790.32	<.0001

Note. The CC program had a significantly lower percentage of females, previous nonviolent acts, previous substance abuse convictions, and positive discharge status compared to VSP. The average age, number of previous programs, and days in current program was lower also for the CC program. Sample sizes were CC = 676, VSP = 354, and total = 1,030. CC = Community Connections program; VSP = vocational support program.

Appendix D

Appendix E

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Erin M. Espinosa

Notes

Author Biographies

Erin M. Espinosa is a senior researcher with the National Council on Crime and Delinquency. She has published numerous articles on the intersection between juvenile delinquency, mental health, and service delivery systems.

Dan Sass is the director of the Statistical Consulting Center and associate professor in the Department of Management Science and Statistics at The University of Texas at San Antonio. In addition to his methodological/statistical research, he is also dedicated to applied collaborative research projects.

Johanna Creswell Báez is the manager of course development and an adjunct assistant professor at Columbia University School of Social Work with the Online Campus. She earned her masters from Columbia University School of Social Work in 2006. In 2015, she earned her PhD in clinical social work from Smith College in Northampton, Massachusetts.

Cassandra Harper was the data integration and research specialist at Southwest Key Programs.

References

Allison

P. D.

(2010). Survival analysis using SAS: A practical guide (2nd ed.). SAS Publishing, Cary.

Andrews

D. A.

Bonta

(2017). The psychology of criminal conduct (6th ed.). Taylor & Francis.

Beck

Ramsey

Lipps

Travis

(2006). Juvenile diversion: An outcome study of the Hamilton County, Ohio, Unofficial Juvenile Community Courts. Juvenile and Family Court Journal, 57(1), 1–10.

Benda

B. B.

Corwyn

R. F.

Toombs

N. J.

(2001). Recidivism among adolescent serious offenders: Prediction of entry into the correctional system for adults. Criminal Justice and Behavior, 28(5), 588–613.

Bishop

D. M.

Decker

(2006). Punishment and control: Juvenile justice reform in the USA. In Junger-Tas

Decker

S. H.

(Eds.), International handbook of juvenile justice (pp. 3–35). Springer.

Bond

G. R.

Drake

R. E.

Becker

D. R.

(2008). An update on randomized controlled trials of evidence-based supported employment. Psychiatric Rehabilitation Journal, 31(4), 280–290.

Burke-Miller

Razzano

L. A.

Grey

D. D.

Blyler

C. R.

Cook

J. A.

(2012). Supported employment outcomes for transition age youth and young adults. Psychiatric Rehabilitation Journal, 35(3), 171–179.

Cocozza

Veysey

Chapin

Dembo

Walters

Farina

(2005). Diversion from the juvenile justice system: The Miami-Dade Juvenile Assessment Center Post-Arrest Program. Substance Use and Misuse, 40(7), 935–951.

Colwell

Villarreal

Espinosa

(2012). Preliminary outcomes of a pre-adjudication diversion initiative for juvenile justice involved youth with mental health needs in Texas. Criminal Justice and Behavior, 39(4), 444–460.

10.

Cuellar

Markowitz

Libby

(2004). Mental health and substance abuse treatment and juvenile crime. Journal of Mental Health Policy and Economics, 7(2), 59–68.

11.

Cuellar

McReynolds

Wasserman

(2006). A cure for crime: Can mental health treatment diversion reduce crime among youth? Journal of Policy Analysis and Management, 25(1), 197–214. https://doi.org/10.1002/pam.20162

12.

Development Services Group, Inc. (2017). Model programs guide literature review: Juvenile reentry. Office of Juvenile Justice and Delinquency Prevention. https://www.ncjrs.gov/App/Publications/abstract.aspx?ID=271832

13.

Duke

Pettingell

McMorris

Borowsky

(2010). Adolescent violence perpetration: Associations with multiple types of adverse childhood experiences. Pediatrics, 125, 778–787.

14.

Gardner

Steinberg

(2005). Peer influence on risk taking, risk preference, and risky decision making in adolescence and adulthood: An experimental study. Developmental Psychology, 41(4), 625–635.

15.

Geenan

Powers

Phillips

(2015). Better futures: A randomized field test of a model supporting young people in foster care with mental health challenges to participate in higher education. The Journal of Behavioral Health Services & Research, 42(2), 150–171.

16.

Hamilton

Sullivan

Veysey

Grillo

(2007). Diverting multi-problem youth from juvenile justice: Investigating the importance of community influence on placement and recidivism. Behavioral Sciences and the Law, 25, 137–158.

17.

Harris

Lockwood

Mengers

Stoodley

(2011). Measuring recidivism in juvenile corrections. OJJDP Journal of Juvenile Justice, 1(1), 1–16.

18.

Hoge

(2016). Application of precharge diversion programs. Criminology & Public Policy, 15(3), 991–999.

19.

Leve

Chamberlain

(2005). Association with delinquent peers: Intervention effects for youth in the juvenile justice system. Journal of Abnormal Child Psychology, 33(3), 339–347.

20.

Lipsey

(2009). The primary factors that characterize effective interventions with juvenile offenders: A meta-analytic overview. Victims & Offenders: An International Journal of Evidence-based Research, Policy, and Practice, 4(2), 124–147.

21.

Loeb

R. C.

Waung

Sheeran

(2015). Individual and familial variables for predicting successful completion of a juvenile justice diversion program. Journal of Offender Rehabilitation, 54(3), 212–237.

22.

Mackin

Lucas

Lambarth

Herrera

Walter

Carey

Finigan

(2010). Baltimore county juvenile drug court outcome and cost evaluation. NPC Research.

23.

McAra

McVie

(2007). Youth justice? The impact of system contact on patterns of desistance from offending. European Journal of Criminology, 4, 315–345.

24.

Mears

Kuch

Lindsey

Siennick

Pesta

Greenwald

Blomberg

(2016). Juvenile court and contemporary diversion: Helpful, harmful, or both? Criminology & Public Policy, 15(3), 1–29.

25.

Nelson

Vincent

(2018). Matching services to crimineiogenic needs following comprehensive risk assessment implementation in juvenile probation. Criminal Justice and Behavior, 45(8), 1136–1153.

26.

Somerville

L. H.

Fani

McClure-Tone

E. B.

(2011). Behavioral and neural representation of emotional facial expressions across lifespan. Journal of Cognitive Neuroscience, 23, 2123–2134.

27.

Steinberg

(2009). Should the science of adolescent brain development inform public policy? American Psychologist, 43(6), 1531–1543.

28.

Steinberg

Caufman

(1996). Maturity of judgement in adolescence: Psychosocial factors in adolescent decision making. Law and Human Behavior, 20, 249–272.

29.

Steinberg

Scott

(2003). Less guilty by reason of adolescence: Developmental immaturity, diminished responsibility, and the juvenile death penalty. American Psychologist, 58(12), 1009–1018.

30.

Taylor

Kemper

T. S.

Loney

B. R.

Kistner

J. A.

(2009). Recidivism in subgroups of severe male juvenile offenders. Psychology, Crime and Law, 15(5), 395–408.

31.

Trulson

C. R.

Marquart

J. W.

Mullings

J. L.

Caeti

T. J.

(2005). In between adolescence and adulthood: Recidivism outcomes of a cohort of state delinquents. Youth Violence and Juvenile Justice, 3(4), 355–387.

32.

Vincent

Guy

Grisso

(2012). Risk Assessment in Juvenile Justice: A Guidebook for Implementation. John D. & Catherine T. MacArthur Foundation. http://www.nysap.us/Risk%20Guidebook.pdf