Resistance to Antisocial Peers in Adolescents Found Not Criminally Responsible on Account of Mental Disorder: Predictive and Incremental Validity With the VRAG-R

Abstract

There has been a recent theoretical shift toward the inclusion of protective factors within risk assessment. However, there is a lack of empirical evidence surrounding this practice in unique forensic populations. Using a long-term retrospective design, we examined the predictive and incremental validity of the protective factor resistance to antisocial peers and the Violence Risk Appraisal Guide—Revised in 119 individuals who were found Not Criminally Responsible on Account of Mental Disorder (NCRMD) as adolescents. The results indicated that resistance to antisocial peers significantly predicted general nonrecidivism (area under the curve [AUC] = .647) and violent nonrecidivism (AUC = .654) in the long term (maximum 35-year follow-up). Incorporation of resistance to antisocial peers into the Violence Risk Appraisal Guide—Revised did not significantly increase the incremental validity for general or violent recidivism. Using logistic regression, adolescents’ age at their NCRMD start date had no significant relationship with recidivism and was unrelated to the protective effect of resistance to antisocial peers.

Keywords

violence risk assessment protective factors juvenile offenders adolescence desistance predictive validity recidivism risk assessment

Introduction

Risk Assessment

Within the field of forensic psychology and forensic psychiatry, evaluating individuals who have previously offended for their likelihood of possible recidivism is referred to as risk assessment (Kocsis, 2011). The field of risk assessment has made notable progress in terms of clinicians’ ability to accurately predict individuals’ risk of violence and offending. First-generation risk assessment approaches, which involved unstructured professional judgment (Andrews et al., 2006) and reliance on subjective decision-making, had poor predictive accuracy (Bonta & Andrews, 2017). Second-generation actuarial prediction, which is based on static risk factors empirically related to risk (Andrews et al., 2006), outperforms clinical judgment (Hanson & Morton-Bourgon, 2009; Harris et al., 2015). Third-generation assessments measure offender criminogenic needs that can be targeted through treatment (Bonta & Andrews, 2017). Structured professional judgment tools, while not originally considered third generation (see Bonta, 1996), have also been defined as such by recent scholars, given their inclusion of both static and dynamic risk factors and their ability to inform management decisions (Abidin et al., 2013; De Bortoli et al., 2017). Through the evolution of risk assessment practice, researchers have helped to establish risk assessment approaches and tools with empirical validity and clinical utility.

Research pursuits of alternative strategies and theories to performing risk assessment must always occur in an ethically informed rational and empirical manner (Andrews et al., 2006). Although there is much evidence supporting the accuracy of actuarial prediction over unstructured clinical judgment (Ægisdóttir et al., 2006; Grove et al., 2000; Harris et al., 2015), the quality of this evidence has some limitations due to research biases (Viljoen et al., 2021). There is also ongoing debate around the advantages and limitations of second versus third-generation approaches (Coid et al., 2009; Nicholls et al., 2016). Metanalytic studies have found that actuarial measures and structured professional judgment tools perform similarly (Yang et al., 2010). More recently, fourth-generation risk assessment has emerged. This approach emphasizes the link between assessment and case management across time from intake through closure (Andrews et al., 2006). It also acknowledges the role of the assessed individual’s personal strengths and considers factors that play a role in maximizing an individual’s response to treatment (Bonta & Andrews, 2017). In line with fourth-generation approaches, the incorporation of strength-based factors that may proactively reduce risk allows for a fairer, balanced, and comprehensive evaluation of individuals (Rogers, 2000). The need remains for researchers to critically examine and evaluate elements of fourth-generation approaches, such as the inclusion of protective factors within risk assessment. In particular, knowledge on protective factors that are theoretically dynamic in nature may have clinical utility for both assessment and treatment.

Protective Factors

Strength-based variables that buffer individuals from problems in the face of risk and promote positive outcomes are called protective factors (Brumley & Jaffee, 2016). While direct protective factors (also referred to as promotive factors) are characteristics of individuals or their environments that are associated with a decreased likelihood of problem behaviors, buffering protective factors function to moderate the impact or influence of risk factors (Durrant, 2017). For example, in a study that used the LS/CMI (Olver et al., 2014) to examine the interface between risk and protection in the prediction of recidivism among convicted males, some protective factors (e.g., relationships with prosocial people) had direct promotive effects—that is, a direct inverse effect on the likelihood of recidivism, irrespective of risk factors—whereas others (e.g., negative attitudes toward crime) had buffering effects—that is, an indirect effect on the likelihood of recidivism by mitigating existing risk factors (Guay et al., 2020). Depending on whether protective factors are conceptualized as correlates, causal factors, or social facilitators, they may function as predictors of risk, potential treatment targets, or guiding factors for reintegration, respectively (Fortune & Ward, 2017). Due to definitional inconsistencies around protective factors, there is a lack of empirical evidence supporting the identification of specific protective factors (Klepfisz et al., 2017). Factors should only be considered protective if they are empirically related to reduced reoffending (Klepfisz et al., 2017). In practice, this has not always been the case. For example, the SAPROF (de Vogel et al., 2015) protective factor scale was developed using a bottom-up approach whereby forensic professionals suggested the inclusion of factors that they simply believed were protective against violence risk (Klepfisz et al., 2017).

Research that has empirically examined the relationship between protective factors and future risk is limited but emerging (Guay et al., 2020; Kivisto, 2016). In a literature review on protective factors against adolescent offending (Patricny, 2021), we summarized 21 protective factors that were supported by longitudinal studies. Examined in isolation, most protective factors had weak associations with later offending or did not significantly reduce the odds of offending across the entire adolescent developmental period (Hartman et al., 2009; Kim et al., 2016). Within and across studies, most protective factors varied in significance and strength across different subgroups of adolescents, but no consistent trends were apparent (Patricny, 2021). There was more robust evidence for cumulative protective effects (i.e., the combined effect of multiple protective factors) occurring across the adolescent developmental period (Jennings et al., 2016; Kim et al., 2016) and generalizing to different populations, including general population adolescents (Andershed et al., 2016; van der Laan et al., 2010), those with high-risk for offending (Dubow et al., 2016; Hartman et al., 2009; Herrenkohl et al., 2003), and those with low socioeconomic status (Fontaine et al., 2016; Herrenkohl et al., 2005).

Protective factors and predictive validity

Some researchers (e.g., de Vries Robbe et al., 2011) have asserted that protective factors may add incrementally to risk factors in the prediction of risk. Such research has only begun to emerge and is not yet well-established. For example, Neil et al. (2020) examined the predictive validity of the SAPROF and found evidence that protective factors predicted the absence of inpatient violence for males within forensic hospitals. The study, however, did not establish incremental validity for the SAPROF over and above the HCR-20V3 (Douglas et al., 2013) alone (Neil et al., 2020). Another study (Yesberg & Polaschek, 2015) found that protective factors contained within the DRAOR (Serin, 2007) predicted reconvictions, reimprisonment, and breaches of parole for males considered high-risk for offending but did not add significant incremental predictive accuracy over static or dynamic risk measures. Similarly, Viljoen et al. (2020) found that protective factors contained within the SAVRY and the YLS/CMI (Hoge et al., 2002) inversely predicted recidivism for youth on probation but did not add incremental validity to risk factors. A more promising study found evidence of incremental validity for dynamic strengths—that is, positive behaviors, attitudes, skills, and supports as measured by the YASI (Orbis, 2000) and the SPIn (Orbis, 2003)—in increasing the predictive accuracy over and above dynamic criminogenic needs for five different samples of justice-involved youth in Canada and the United States (S. L. Brown et al., 2020). The strengths of individuals with low- and moderate-need levels buffered recidivism risk, highlighting the potential utility of protective factors within the context of risk assessment (S. L. Brown et al., 2020). A further empirical exploration of the incorporation of strengths (i.e., protective factors) into criminal justice assessment tools is warranted (S. L. Brown et al., 2020).

Developmental considerations

Adolescence and emerging adulthood is a time of significant cognitive, emotional, social, and behavioral development for individuals (Sroufe, 1997), wherein identity is being formed (Kaltiala-Heino & Eronen, 2015), priorities may shift, and individuals begin to explore and understand how they fit within the world (Steinberg & Morris, 2001). Many problems experienced by adolescents, such as substance use, unemployment, and delinquency, are transient in nature and resolve by the beginning of adulthood (Steinberg & Morris, 2001). Given this period of vast change, protective factors against problem behaviors tend to change in their strength across time for adolescents. They may also vary depending on the age at which predictors are examined and the time at which outcomes are measured. For example, a longitudinal study of adolescents who were deemed high-risk found that perceived recognition for school involvement during early adolescence was protective against serious violence during middle and late adolescence, yet the same factor during middle adolescence was not protective against serious violence during late adolescence (Kim et al., 2016). At the same time, school bonding during middle adolescence was protective against serious violence during late adolescence (Kim et al., 2016). In another longitudinal study of adolescents with high externalizing problems, having supportive friendships during early adolescence was a protective factor, reducing adolescents’ risk of fighting at late adolescence (Vassallo et al., 2016). However, supportive friendships during middle adolescence was a risk factor, increasing adolescents’ risk of fighting at late adolescence and emerging adulthood (Vassallo et al., 2016). Similarly, in a comprehensive review of predictors of offending among juveniles who had committed serious or violent offenses, Loeber and Farrington (1998) concluded that the best predictors of serious delinquency varied by different developmental periods. In other words, influential factors in one developmental period may not be as relevant in another developmental period, with some predictors being predictive only within a single developmental period, and others remaining predictive across different developmental periods (Kim et al., 2016). These studies support the notion that age is an important variable to consider in the quest to identify and understand protective factors for adolescents against recidivism.

Not Criminally Responsible

Within Canada, judges can offer a verdict of Not Criminally Responsible on Account of Mental Disorder (NCRMD) when individuals who committed an offense meet specific criteria for this designation. Under Section 16 of the Criminal Code of Canada,

No person is criminally responsible for an act committed or an omission made while suffering from a mental disorder that rendered the person incapable of appreciating the nature and quality of the act or omission or of knowing that it was wrong. (Criminal Code, 1985)

Rather than being criminally charged, individuals found NCRMD fall under the jurisdiction of a provincial or territorial review board for an indeterminate amount of time until they are determined to no longer present a significant threat to public safety and can receive an absolute discharge (see Section 672.54 of the Criminal Code, 1985). Only a small group of accused individuals have the issue of mental illness brought forth to court and meet the criteria to receive an NCRMD verdict within Canada, with most estimates falling below 1% (Department of Justice, 2020). Based on information from Statistics Canada (2020), the percentage of individuals found NCRMD in Alberta, Canada, relative to the number of persons convicted has ranged from only .02% to .06% from 2013 to 2017.

Protective Factors Within the NCRMD Adolescent Population

Individuals who are found NCRMD during adolescence represent a unique and vulnerable population on two accounts: (a) being NCRMD due to a severe mental disorder and (b) being in an ongoing stage of important social, emotional, and cognitive development. An exploration of protective factors in this population is warranted to gain an understanding of strength-based factors that may potentially promote healthy developmental trajectories and positive mental health outcomes while also reducing the risk of recidivism. From an evolutionary-developmental perspective, protective factors teach developing adolescents that the future is likely to be a relatively stable and risk-free environment, thereby promoting the development of thought patterns and behaviors that involve less risk-taking, commitment to social relationships, and involvement with pro-social institutions (Durrant, 2017).

Resistance to antisocial peers

During adolescence, individuals increase how much time they spend with their friends and decrease how much time they spend with their family (Larson & Richards, 1991). When among classmates or in romantic relationships, adolescents often act in ways that are incongruent with their true selves (Steinberg & Morris, 2001). Peer influence can be either positive (e.g., prosocial behavior) or negative (e.g., substance use and delinquency; Steinberg & Morris, 2001). In our recent retrospective study (Patricny, 2021) that explored theoretical protective factors for long-term nonrecidivism in adolescents found NCRMD, resistance to antisocial peers—defined as resisting negative peer influences and having mostly prosocial peers or keeping to oneself—was found to be the only significant protective factor for adolescents during their NCRMD warrant. This protective factor uniquely predicted nonrecidivism, increasing individuals’ rate of general nonrecidivism from 71.4% to 90.4% and violent nonrecidivism from 79.4% to 94.2% (Patricny, 2021).

The Present Study

Building upon the study of protective factors for nonrecidivism in adolescents found NCRMD (Patricny, 2021) we examine the predictive validity of the protective factor resistance to antisocial peers for long-term nonrecidivism and its incremental validity with the Violence Risk Appraisal Guide—Revised (VRAG-R). Although the VRAG-R is intended to measure violent recidivism, it has shown moderate levels of predictive validity for general recidivism (Glover et al., 2017). We therefore considered both violent and general nonrecidivism. In addition, we consider the developmental influence of age on nonrecidivism. We investigate the following research questions:

Research Question 1: What is the predictive validity of resistance to antisocial peers for general nonrecidivism and violent nonrecidivism with adolescents found NCRMD?

Research Question 2: Does the inclusion of resistance to antisocial peers with the VRAG-R add incremental validity in terms of predicting general recidivism and violent recidivism?

Research Question 3: From a developmental perspective, when considered together with the protective effect of resistance to antisocial peers, is age at the start of the NCRMD warrant related to general and violent nonrecidivism? Furthermore, does the protective effect of resistance to antisocial peers vary depending on individuals’ age at their NCMRD start date?

Method

Data Collection

All data were pulled from the Alberta NCR Project, a large-scale long-term retrospective study of all individuals ever found NCRMD in Alberta’s history. Ethics approval for the overarching Alberta NCR Project, of which the current study is a part, was obtained in June 2014. Researchers involved in the Alberta NCR Project have been examining population-wide sociodemographic, mental health, and criminological profiles (Haag et al., 2016) and long-term reoffending rates for general, violent, and sexual offenses (Richer et al., 2018) through extensive file review of all patient charts. All files were located at Alberta Hospital Edmonton in Alberta, Canada, and include copies of all reports submitted to the Alberta Review Board.

Participants

We included 119 individuals (89.1% male and 10.9% female) who received an NCRMD verdict at the age of 25 years or younger within Alberta, Canada. Our rationale for this age bracket was to include all individuals with ongoing brain development. Given that the prefrontal cortex—which is responsible for higher order cognitive processes and executive functions—continues to develop into the mid-20s (Johnson et al., 2009), we opted to include individuals within emerging adulthood. Our sample represented 81% of the entire adolescent population found NCRMD since 1941. All individuals had one or more mental health diagnoses, with the vast majority (85.7%) having a psychotic disorder. All received their NCRMD verdict between 1972 and 2018. In terms of the geographic location of the NCR index offense, 94.1% were committed within Alberta and 5.9% were out of province. Nearly half (49.6%) of the index offenses involved direct violence, when excluding sexual offenses, homicide, and attempted homicide. The remainder of the offenses included weapons (18.5%), homicide (14.3%), arson (9.2%), attempted homicide (8.4%), robbery (6.7%), sexual offenses (5.9%), criminal harassment (1.7%), and other offense types (31.9%). Some individuals had multiple offenses.

Measures

Resistance to Antisocial Peers

The protective factor, as scored, was static in nature, based on an estimate of an individual’s overall display of resistance to antisocial peers across time throughout their NCRMD warrant. It captures active, as well as more passive, or incidental, forms of resistance to antisocial peers. The following criteria were used: Individuals with the protective factor had been described by forensic professionals (e.g., psychiatrists, psychologists, or nurses who had interacted with the individuals) as keeping to themselves (may be due to lack of social skills), having mostly prosocial peers, or showing evidence of resisting negative peer influences (e.g., declining substances offered by peers) during their NCRMD warrant as documented by forensic professionals in patients’ hospital files and/or in their review board disposition reports. Conversely, individuals without this protective factor were described as interacting selectively with antisocial peers (i.e., those who frequently disregarded unit rules / misused substances) or showing evidence of being easily influenced by negative peers (e.g., involvement with bringing contraband onto the unit for peers). The first author, a psychologist, collected data for this variable during March and April 2020, from all available forensic files stored securely at Alberta Hospital Edmonton and was able to identify the absence versus presence of this protective factor for all but four individuals in the sample (i.e., 115 individuals or 96.6%). The psychologist who performed the scoring was blind to the outcomes of general recidivism and violent recidivism.

VRAG-R

The VRAG-R is a well-validated actuarial tool for the assessment of the likelihood of violent or sexual reoffending in males (Harris et al., 2016). The tool contains 12 risk factors—antisociality, admission to corrections, failure on conditional release, elementary school maladjustment, conduct disorder score, nonviolent criminal history, age at index offense, violent criminal history, substance abuse, sex offending, lived with both parents until age 16 years, and marital status—each empirically associated with reoffending (Harris et al., 2015). Based on Nuffield’s (1982) system, raters score items from a minimum of −6 to a maximum of +6, with a 1-point difference reflecting a 5% increment above (or below) the base rate of reoffending within the sample that the tool was constructed on (Harris et al., 2015). The sum of all items yields the total VRAG-R score, with higher scores reflecting a greater likelihood of violent recidivism. Raters use the total VRAG-R score to classify individuals into one of nine VRAG-R risk categories, each that has a unique proportion of violent recidivism (Harris et al., 2015). For example, an individual with a VRAG-R score of 15 falls into VRAG-R Category 7 (VRAG-R score of 12 to 17), and this categorization is associated with a 41% likelihood of reoffending after 5 years, or a 66% chance of reoffending after 12 years (Harris et al., 2015). The VRAG-R has strong predictive validity, with a receiver operating characteristic (ROC) area of .760, 95% confidence interval (CI) = [.734, .787] for dichotomous violent recidivism and .758, [.731, .784] for VRAG-R categories (Harris et al., 2015). Depending on the length of follow-up period, the accuracy of the VRAG-R ranges from a ROC area of .752 at 36 years follow-up to .784 at 20 years follow-up (Harris et al., 2015). The predictive accuracy is also strong for individuals below the age of 18 at the time of their index offense (AUC = .74; Serin & Lowenkamp, 2015) and forensic patients (AUC = .80; Harris et al., 2002).

We scored the VRAG-R retrospectively from comprehensive forensic file information for all but two individuals (N = 117 or 98.3%) in our sample (whose files were not scorable because they lacked information on violence risk factors). The scoring was completed by a student who was trained in scoring the VRAG-R, and a forensic psychologist completed regular and random data checks of scoring to ensure accuracy. With respect to missing VRAG-R items for the 117 scoreable files, one file lacked information for “marital status at time of index offense,” two files lacked information for “lived with both biological parents to age 16,” three files lacked information for “conduct disorder” and “antisociality,” and four files lacked information for “elementary school maladjustment” and “history of alcohol or drug problems.” For files with four or fewer missing items, scores were prorated according to Harris et al.’s (2015) criteria. The mean VRAG-R score was .32 (SD = 16.67; SE = 1.54) and ranged from −25 to 43. The skewness was .51 (SE = .22) and kurtosis −.69 (SE = .44). As per Levene’s test, equal variances in VRAG-R total score between recidivists and nonrecidivists were assumed, F(1, 115) = .534, p = .466).

Age

The age variable was a continuous variable based on individuals’ age in years at the time that they received an NCRMD verdict (i.e., their NCRMD start date) and began their warrant (i.e., were under the supervision of the Review Board). The average age was 21.7 years (SD = 2.67) at the start of their NCRMD warrant. We conducted the Box-Tidwell Test to determine whether the relationship between the age variable and the logit (log odds) of the recidivism outcomes is linear. The log interaction term was not significant for either general or violent recidivism, indicating that the assumption of linearity between the age variable and logit of the dependent variable was met for the purpose of binary logistic regression analysis.

Recidivism

For analyses that examined the impact of the protective factor on the VRAG-R, the dependent variables were general recidivism and violent recidivism (coded dichotomously as yes/no) at any point during the follow-up period. This was measured by recording official convictions from Criminal Police Information Centre criminal records from the Royal Canadian Mounted Police. The follow-up for recidivism started at the time of the earliest unsupervised privilege, or post-NCRMD verdict for those with convictions prior to the first unsupervised privilege (Richer et al., 2018). Follow-up continued throughout the NCRMD warrant (i.e., while individuals remain under the jurisdiction of their review board) and following absolute discharge, where applicable. As of April 2018, the average follow-up time for recidivism was 13.6 years (SD = 10.0, N = 111); this excludes individuals who did not receive any form of unsupervised privilege (n = 7), unless they were a recidivist. For individuals known to be deceased (n = 7), the follow-up times were determined by calculating the difference between their date of death and the date of first unsupervised privilege. Recidivism data were available for 100% of the sample. The base rate of recidivism was 21.0% (n = 21) for general offenses and 14.3% (n = 17) for violent offenses. For analyses that examined the predictive accuracy of the protective factor alone, or with consideration of age, the dependent variables were general nonrecidivism and violent nonrecidivism throughout the follow-up period.

Analyses

The researchers used IBM SPSS Statistics (Version 26.0.0.0) to conduct all statistical analyses. We used ROC curve analyses to explore the predictive validity of the protective factor resistance to antisocial peers and the VRAG-R total score. This procedure provides AUC values, which range from 0 to 1, where a value of 1 reflects perfect discrimination between two groups and .5 is equivalent to a chance prediction (Neil et al., 2020). For outcomes with a base rate of 50%, an AUC value of .556 is considered a small effect, .639 is medium, and .714 is large (Rice & Harris, 2005). When base rates are lower than 50%, as was the case with our NCR adolescent sample, the associated point-biserial correlations (r_pb) that typically correspond to small, medium, and large effects decrease (Salgado, 2018). We used Salgado’s (2018) conversion charts and guidelines to determine the effect sizes of our results.

Creation of Modified VRAG-R Total Score

To create our modified VRAG-R Total score (mod-VRAG-R Total) that incorporates the protective effect of resistance to antisocial peers, we used Harris et al.’s (2015) method of assigning a one-point value to a predictor for each 5% increment change in the base rate of reoffending. The difference in the base rate of violent recidivism for individuals who gravitated toward select antisocial peers (19.6%) and the base rate of violent recidivism for the NCRMD youth sample (13.4%) was 6.2%, which was rounded down to 5%. This 5% difference corresponded to one increment of 5% change in recidivism or a +1 value for individuals without the protective factor. The difference in the base rate of violent recidivism for individuals who displayed resistance to antisocial peers (5.8%) and the base rate of violent recidivism for the NCRMD youth sample (13.4%) was 7.6%, which was rounded up to 10%. This 10% difference corresponded to two increments of 5% change against recidivism, or a −2 value for individuals with this protective factor. We added the protective factor on to the VRAG-R as an additional item, effectively decreasing, or increasing, the VRAG-R Total score for individuals who displayed resistance to antisocial peers, or did not, respectively. For example, for an individual who had a VRAG-R Total score of −11, who displayed resistance to antisocial peers (“−2”), we calculated a mod-VRAG-R Total score of −13. If the same individual did not have this protective factor (i.e., “+1,” displaying gravitation toward select antisocial peers), we calculated a mod-VRAG-R Total score of −10.

Results

Discriminant Validity of Resistance to Antisocial Peers

First, as a test of discriminant validity, we analyzed the relationship between the protective factor of resistance to antisocial peers and the VRAG-R total risk score. Using crosstabulation analysis, there was a negative relationship between resistance to antisocial peers and the VRAG-R total risk score (Pearson’s r = −.378, SE = .077, p < .001). As the VRAG-R total risk score is a strong predictor of violent recidivism (Harris et al., 2015), this negative association provides evidence of discriminant validity for resistance to antisocial peers as a protective factor against recidivism. Using Harris et al.’s (2015) VRAG-R Total score categories, we also examined the percentage of individuals with the protective factor, resistance to antisocial peers, within each category (see Table 1). As shown in Table 1, there is general trend for individuals within lower VRAG-R categories (i.e., those with lower VRAG-R total scores) to display the protective factor resistance to antisocial peers, and for individuals within higher VRAG-R categories (i.e., those with higher VRAG-R total scores) to not display this protective factor.

Table 1:

Individuals With and Without Protective Factor Resistance to Antisocial Peers Within Harris et al.’s (2015) Nine Violence Risk Appraisal Guide—Revised (VRAG-R) Score Categories (N = 114)

VRAG-R category	VRAG-R total score	% (n) with protective factor	% (n) without protective factor	Total (N)
1	≤–24	100.0 (1)	0.0 (0)	1
2	–23 to –17	66.7 (14)	33.3 (7)	21
3	–16 to –11	35.7 (5)	64.3 (9)	14
4	–10 to –4	72.2 (13)	27.8 (5)	18
5	–3 to +3	50.0 (9)	50.0 (9)	18
6	4 to 11	22.2 (2)	77.8 (7)	9
7	12 to 17	41.7 (5)	58.3 (7)	12
8	18 to 26	8.3 (1)	91.7 (11)	12
9	≥27	11.1 (1)	88.9 (8)	9

Note. VRAGR = Violence Risk Appraisal Guide—Revised.

Predictive Accuracy of Resistance to Antisocial Peers and the VRAG-R

Table 2 displays the results of the ROC curve analyses. With respect to risk, the VRAG-R total score did not significantly predict general recidivism, AUC = .602, 95% CI [.47, .73] or violent recidivism, AUC = .632, [.50, .77] for the full sample. When excluding females (n = 13), the results remained nonsignificant for general recidivism, AUC = .591, [.46, .72] and violent recidivism, AUC = .634, [.49, .78]. Typically, the VRAG-R has strong predictive ability for violent recidivism with a large AUC value of .76 (Harris et al., 2015). The nonsignificant results may have been accounted for by the unique characteristics of the sample, including the adolescent age and NCRMD population, as well as reduced power due to the low base rate of violent recidivism (13.4%) and general recidivism (19.6%). On the protective end, resistance to antisocial peers showed some predictive ability for general nonrecidivism, AUC = .647, [.53, .77] and violent nonrecidivism, AUC = .654, [.52, .79], a moderate effect size. When excluding females, the results remained significant, with a moderate effect for general nonrecidivism, AUC = .667, [.54, .79] and violent recidivism, AUC = .681, [.55, .82]. The modified VRAG-R, that combined the VRAG-R score with the protective effect of resistance to antisocial peers (or lack of), had moderate predictive ability for violent recidivism, AUC = .643, [.51, .78], but it did not predict general recidivism, AUC = .615, [.49, .74]. When excluding females, the modified VRAG-R was nonsignificant for violent recidivism, AUC = .616, [.47, .76] and general recidivism, AUC = .590, [.46, .72].

Table 2:

Predictive Accuracy of Resistance to Antisocial Peers, the VRAG-R, and the Modified VRAG-R for General and Violent Recidivism and Nonrecidivism (N = 117)

Outcome
Predictors	General recidivism		General nonrecidivism		Violent recidivism		Violent nonrecidivism
Predictors	AUC	95% CI	AUC	95% CI	AUC	95% CI	AUC	95% CI
Resistance to antisocial peers			.647	[.53, .77]			.654	[.52, .79]
VRAG-R	.602	[.47, .73]			.632	[.50, .77]
Mod-VRAG-R	.615	[.49, .74]			.643	[.51, .78]

Note. VRAG-R = Violence Risk Appraisal Guide—Revised total score; Mod-VRAG-R = VRAG-R total score combined with presence/absence of protective factor resistance to antisocial peers (−2 or +1); AUC = area under the curve (from receiver operating characteristic (ROC) curve analysis); CI = confidence interval.

The significance is expressed by the corresponding 95% confidence intervals.

Using the mod-VRAG-R Total score and Harris et al.’s (2015) VRAG-R Total score categories, we also examined the percentage of individuals who demonstrated general recidivism and violent recidivism within each VRAG-R category (Table 3). Consistent with the AUC value of .643, 95% CI = [.51, .78], there was a general trend for individuals within higher mod-VRAG-R categories to have a higher proportion of violent recidivism, as compared with individuals within lower VRAG-R categories. A notable exception is for the 10 individuals whose mod-VRAG-R Total scores were 27 or higher. Upon further examination, only three of these individuals had received absolute discharge (33.3%) and one was transferred out of province. The other six remained under the supervision of the Review Board with conditions in place that reduced their likelihood of reoffending (e.g., restrictions around substance use) or limited their opportunity to reoffend (e.g., constant supervision during community visits). In contrast, of the 104 individuals with a Mod-VRAG-R Total score of 26 or lower, 40 (38.5%) had received an absolute discharge, free of any supervision or conditions, providing them with increased opportunity to reoffend. Thus, the unexpected lack of general and violent recidivism found within the highest VRAG-R category is likely a reflection of risk being appropriately managed by forensic professions through ongoing supervision and restrictions. As reported by Harris et al. (2015), 76% of individuals in category 9 of the VRAG-R are expected to demonstrate violent recidivism after 5 years, and 87% are expected to demonstrate violent recidivism after 12 years.

Table 3:

Percentage of Cases Demonstrating General and Violent Recidivism as a Function of Harris et al.’s (2015) Nine VRAG-R Score Categories

VRAG-R category	Mod-VRAG-R Total score	Type of recidivism		Total (N)
VRAG-R category	Mod-VRAG-R Total score	% (n) with general recidivism	% (n) with violent recidivism	Total (N)
1	≤−24	20.0 (1)	0.0 (0)	5
2	–23 to –17	5.6 (1)	5.6 (1)	18
3	–16 to –11	12.5 (2)	12.5 (2)	16
4	–10 to –4	22.2 (4)	5.6 (1)	18
5	–3 to +3	28.6 (4)	21.4 (3)	14
6	4 to 11	7.1 (1)	7.1 (1)	14
7	12 to 17	33.2 (2)	33.3 (2)	6
8	18 to 26	38.5 (5)	30.8 (4)	13
9	≥27	20.0 (2)	10.0 (1)	10

Note. N = 113. Base rate of general recidivism was 19.6% with an average time to general recidivism of 79 months (SD = 57). Base rate of violent recidivism was 13.4% with an average time to violent recidivism of 90 months (SD = 62). VRAG-R = Violence Risk Appraisal Guide—Revised total score; Mod-VRAG-R = VRAG-R total score combined with absence/presence of protective factor resistance to antisocial peers (+1 or −2).

Incremental Validity of Resistance to Antisocial Peers and the VRAG-R

We used Delong’s test to statistically compare the AUC values of the VRAG-R and modified VRAG-R for prediction of general and violent recidivism. There was no significant difference in the predictive performance of the VRAG-R compared with the modified VRAG-R for either general recidivism (p = .12) or violent recidivism (p = .28). There was a marginal (but nonsignificant) increase of .013 in AUC value for general recidivism and .010 for violent recidivism when using the modified VRAG-R over the original VRAG-R. Although the incremental validity of resistance to antisocial peers to the VRAG-R is not statistically significant, the change is in the expected direction for both general and violent recidivism.

Age Analyses

To investigate the possible effect of age (at the start of the NCRMD warrant) on general and violent nonrecidivism, we ran bivariate logistic regression analyses. We used the enter method, with age (at time of NCRMD start date) and resistance to antisocial peers as predictor variables, and the general and violent recidivism outcomes as the dependent variable in each model. Table 4 displays the results. The model predicting general nonrecidivism was significant, χ²(2, n = 52) = 108.111, p = .030, and explained between 5.9% (Cox & Snell R²) and 9.3% (Nagelkerke R²) of the variance in general nonrecidivism. The Hosmer–Lemeshow test suggested that the model adequately fit the data (p = .359). As expected, resistance to antisocial peers was a significant protective factor for general nonrecidivism (p = .015), with individuals demonstrating this factor being around 73% less likely to reoffend (for general offenses) than individuals who gravitated toward select antisocial peers (odds ratio [OR] = .266, 95% CI = [.09, .78]). Age at time of NCRMD verdict (i.e., start date) was not a significant predictor (p = .663) within the model. The model predicting violent nonrecidivism was significant, χ²(2, n = 52) = 86.062, p = .035, and explained between 5.7% (Cox & Snell R square) and 10.2% (Nagelkerke R²) of the variance in violent nonrecidivism. The Hosmer–Lemeshow test suggested that the model adequately fit the data (p = .072). As expected, resistance to antisocial peers was a significant protective factor for general nonrecidivism (p = .031), with individuals demonstrating this factor being around 77% less likely to reoffend (for violent offenses) than individuals who gravitated toward select antisocial peers (OR = .234, 95% CI = [.06, .88]). Age at time of NCRMD start date was not a significant predictor (p = .325) within the model.

Table 4:

Bivariate Logistic Regression of Predictors Age at Time of NCRMD Verdict and Resistance to Antisocial Peers on General and Violent Nonrecidivism (n = 52)

General nonrecidivism
Variable	Regression coefficient			Odds ratio		Model
Variable	β	SE β	Wald	Exp(β)	95% CI	−2 Log likelihood	Model χ²(df)	R²_CS—R²_N
Constant	3.090	2.014	2.354	21.987
Age	–.039	.089	.190	.962	[.81, 1.15]
Resistance to antisocial peers	–1.326	.547	5.869^*	.266	[.09, .78]	108.111	6.982^* (2)	.059–.093
Violent nonrecidivism
Constant	5.130	2.478	4.285^*	168.966
Age	–.106	.108	.968	.899	[.73, 1.11]
Resistance to antisocial peers	–1.452	.673	4.649^*	.234	[.06, .88]	86.062	6.712^* (2)	.057–.102

Note. NCRMD = Not Criminally Responsible on Account of Mental Disorder; CI = confidence interval; R²_CS = Cox & Snell; R²_N = Nagelkerke.

p < .05.

To examine whether the protective effect of resistance to antisocial peers against recidivism may differ depending on individuals’ age at the time of their NCRMD verdict, we also examined the interaction term of age and resistance to antisocial peers within each bivariate logistic regression model containing the predictors of resistance to antisocial peers and age at the time of NCRMD verdict. The interaction term between these two variables fell short of significance for prediction of general nonrecidivism (β = .066, SE = .222, p = .142, 95% CI = [.69, 1.65]) and violent nonrecidivism (β = .328, SE = .248, p = .185, 95% CI = [.86, 2.26]), suggesting that the protective effect of resistance to antisocial peers against recidivism is unrelated to individuals’ age at the time of their NCRMD start date. Taken together, these results suggest that when considering the protective effect of resistance to antisocial peers, individual’s age at the time of their NCRMD start date is not predictive of later general or violent recidivism. Furthermore, the protective effect of resistance to antisocial peers appears to be unrelated to individuals’ age at the time that they were found NCRMD given that there was no significant interaction between these two variables.

Discussion

Predictive Validity of Resistance to Antisocial Peers for Recidivism

In this study, we examined the predictive validity of the protective factor, resistance to antisocial peers, in predicting general nonrecidivism and violent nonrecidivism among the population of individuals in Alberta who were found NCRMD as adolescents or emerging adults. The results provided evidence for this protective factor as being predictive of later general nonrecidivism and violent nonrecidivism, with AUC values of .647, 95% CI = [.53, .77] and .654, 95% CI = [.52, .79], a moderate effect size. This finding is consistent with research on the central eight risk/need factors of criminal behavior, with criminal history, procriminal attitudes, procriminal associates, and antisocial personality pattern being most predictive of recidivism (Bonta & Andrews, 2017). With respect to the risk/need factor of procriminal associates (Bonta & Andrews, 2017), our research provides evidence that prosocial association and resistance to the influence of antisocial peers within one’s environment, is a protective strength for NCRMD adolescents. As discussed by Klepfisz et al. (2017), conceptually, some protective factors exist as the reverse of, or inverted, risk factors, whereas others are qualitatively distinct. Although it may be tempting to conceptualize resistance to antisocial peers as representing the positive end of a single continuum of peer influence (with gravitation toward antisocial peers representing the negative end of the continuum), this would be an oversimplification. As some researchers have asserted (de Vries Robbé, 2014; Klepfisz et al., 2017), risk and protective factors can coexist within a single domain (e.g., an individual may have both prosocial influences and negative social influences concurrently), and this complexity is unlikely captured without independently considering the protective value of a variable. Nearly all individuals in our study had the risk factor “procriminal associates” during their NCRMD warrant, owing to their close proximity to other detained individuals involved with crime, yet only 44% demonstrated resistance to antisocial peers. Although we would expect there to be a negative relationship between procriminal associates and the resistance to antisocial peers, the two are not mutually exclusive.

The results align with research findings on justice-involved Canadian youth which suggest that adolescents’ peer associations are the most robust predictor of a life-course offending trajectory, rather than an adolescent-limited offending trajectory, with individuals displaying a chronic pattern of offending into adulthood having more negative and unconstructive ties with their peers (Yessine & Bonta, 2012). As our study was the first to examine resistance to antisocial peers specifically among individuals who found NCRMD, we encourage the replication of this research in other provincial and territorial jurisdictions within Canada as well as internationally. Exploration of whether this factor is generalizable to adolescents within correctional settings would also be warranted.

Incremental Validity of Resistance to Antisocial Peers for Recidivism

We explored the incremental validity of resistance to antisocial peers in combination with the VRAG-R, by creating a mod-VRAG-R total score. Although the mod-VRAG-R approached significance for predicting violent recidivism, AUC = .643, 95% CI = [.51, .78], it did not add significantly to the incremental validity of the VRAG-R for violent or general recidivism, possibly due to shared variance between VRAG-R items related to antisocial peers. Adding the protective factor to the VRAG-R did, however, slightly improve the AUC value in the expected direction, but the change was not statistically significant.

Although not the main focus of our study, our results did not provide evidence of predictive validity for the VRAG-R in predicting general recidivism. This was not unexpected, given that the VRAG-R is not designed to predict general recidivism. We were, however, surprised to find a nonsignificant AUC value for the VRAG-R for violent recidivism, which the VRAG-R is designed to predict. Of note, the 13.4% base rate of violent recidivism (and 19.6% base rate of general recidivism) within the sample of adolescents found NCRMD is much lower than the base rate of violent recidivism of the sample on which the VRAG-R was developed (i.e., the 5-year base rate of 32% and 12-year base rate of 51%; Harris et al., 2015). As recommended by Rossegger et al. (2014), following an investigation of individuals who offended who was released in Switzerland, clinicians should use caution in practice around the probabilistic risk estimates of the VRAG, as differences in base rates of violent recidivism can lead to poor calibration validity and an overestimation of the risk of recidivism. This is problematic as individuals may receive unnecessary supervision or be mandated to costly treatment that is unwarranted (Rossegger et al., 2014). From a legal perspective, a review board should consider “the need to protect the public from dangerous persons, the mental condition of the accused, the reintegration of the accused into society and the other needs of the accused” (Section 672.52 of the Canadian Criminal Code, 1985). Moreover, dangerousness should not be presumed, rather, there must be evidence of significant risk for a review board to order continued detention or conditional supervision (Section 672.54 of the Canadian Criminal Code, 1985). In other words, there is a mandate for individuals found NCRMD to be absolutely discharged once there is no longer significant evidence of ongoing risk. Forensic professionals are responsible for ensuring that they utilize risk assessment measures and make probabilistic risk estimates with consideration of calibration validity for unique populations involved in the forensic system.

Developmental Considerations

Finally, we explored whether age at the start of the NCRMD, when considered together with the protective factor of resistance to antisocial peers, was related to general and violent recidivism. The results suggested that age at NCRMD start date is not a significant predictor of general or violent nonrecidivism and is unrelated to the protective effect of resistance to antisocial peers. These results were somewhat unexpected. Research has demonstrated that adolescents are most easily influenced by their peers during early to middle adolescence (age 13–16 years) as compared with those in older age groups (17 years and older; B. B. Brown, 1990). We therefore hypothesized that there may be a significant interaction between age at NCRMD start date and resistance to antisocial peers, with the protective effect of resistance to antisocial peers being stronger for younger individuals, but this was not the case. Perhaps our sample did not include a wide enough age range to identify possible age differences. Another possibly pertains to the variation in risk level of individuals found NCRMD, which we were unable to control for. Given that age of first offense is a risk factor for recidivism of young individuals—with those whose first offense occurs at a younger age being more likely to reoffend that those who first offense occurs at an older age—(Putniņš, 2005), individuals found NCRMD at younger ages may have had higher preexisting risk levels than those found NCRMD when older. Moreover, because of judicial leniency toward juveniles who offend, judicial processing biases may also exist wherein those who are convicted at younger ages tend to have more serious offenses than older individuals who offend (Putniņš, 2005). Perhaps younger individuals were more likely to be those characterized by Moffitt’s (1993) life course–persistent offending trajectory rather than an adolescent-limited offending trajectory, as compared with older individuals. Future research that examines the protective effect of resistance to antisocial peers in different age groups could explore possible interactions with existing risk levels to better understand how this protective factor operates across time. Of note, we measured resistance to antisocial peers as an overall estimate across time, from the time of individuals’ NCRMD verdict until their last contact with the review board. We were unable to assess this variable across time or measure any change.

Strengths and Limitations

This research has several notable strengths. It builds on a previous exploration of protective factors in adolescents found NCRMD (Patricny, 2021) by considering the combined utility of an empirically supported protective factor, resistance to antisocial peers, and the VRAG-R, a well-validated actuarial risk assessment tool. It was also based on population-level data, with results that reflect 81% of the entire population of adolescents ever found NCRMD in Alberta’s history. In addition, the presence versus absence of the protective factor resistance to antisocial peers was made by an independent researcher, which reduced possible biases that can occur with multiple raters.

There were also several limitations that are inherent to the retrospective file review nature of the study. The researchers scored all independent and dependent variables from available file information. Due to insufficient file information and inaccessibility of files, not all individuals who were found NCRMD could be included. The outcome measures, general and violent recidivism, were also based on official charges, therefore excluding possible criminal behavior that may have gone undetected or unreported. As noted earlier, the VRAG-R is not designed to measure general recidivism and we did not find evidence of predictive validity for the VRAG-R in predicting general recidivism. Utilization of a risk assessment measure designed specifically to measure general recidivism would have allowed for a more targeted exploration of the incremental validity of resistance to antisocial peers and general recidivism. There was also no attempt to measure the interrater reliability of resistance to antisocial peers, given that this variable was scored by a single psychologist, although the rater was blind to the recidivism outcomes to minimize researcher bias. The small size of the population of adolescents and emerging adults found NCRMD in Alberta also limited the statistical power for analyses, possibly lending itself toward a Type II error. Researchers who explore the protective effect of resistance to antisocial peers with larger samples of adolescents involved in the general criminal justice system may have greater opportunity to conduct analyses with more statistical power to better detect possible age-related variations in protective factors. Finally, we encourage researchers to conduct prospective research for the purpose of capturing a greater range of theoretical protective factors and measuring any changes in them across time.

Summary

The results of our study provided additional evidence to the findings (Patricny, 2021) that resistance to antisocial peers is an important protective factor, with a medium effect size for later general and violent nonrecidivism. This suggests that resistance to antisocial peers may have clinical utility for forensic professionals; given that it is a predictor of nonrecidivism, it may help professionals make informed decisions about the community reintegration of individuals found NCRMD. Based on its negative association with the VRAG-R, we also provided evidence of discriminant validity for this protective factor. Despite these promising results, we did not find statistically significant evidence for the incremental validity of resistance to antisocial peers in combination with the VRAG-R. Our investigation of individuals’ age at their NCRMD start date also suggested that age was not predictive of general or violent nonrecidivism and was unrelated to the protective effect of resistance to antisocial peers against recidivism. Although it goes beyond the scope of our study, we suggest that future researchers consider possible interactions between risk and protective factors when examining possible age effects. Researchers could also build on our research by exploring whether resistance to antisocial peers and choice of friends is a dynamic variable and if changes in this variable across time may lead to a reduction in later recidivism. Such research would help to determine whether resistance to antisocial peers may be a useful treatment target for clinicians working with adolescents who found NCRMD. For example, researchers could prospectively explore whether adolescents who cut ties with antisocial peers and develop prosocial peer relationships reduce their likelihood of reoffending. Any attempt to measure the change in this protective factor would require careful operational definition. Such research with consideration of change scores could help to inform clinicians on how to address the criminogenic needs of justice-involved adolescents through the development of protective factors that reduce risk over time.

Footnotes

Authors’ Note:

This WCHRI graduate studentship has been funded through the Stollery Children’s Hospital Foundation through the Women and Children’s Health Research Institute.

ORCID iD

Nicol Patricny

Nicol Patricny is a practicing clinical and counseling psychologist in Alberta, Canada. She has a PhD in school and clinical child psychology (University of Alberta), a MEd in counseling psychology (University of Alberta), and a BSc (Hons) in psychology with a Forensic Specialization (University of British Columbia). Her research has focused on exploring strength-based approaches in working with forensic populations, FASD and homelessness, and eating disorders.

Andrew M. Haag is a forensic psychologist at Alberta Hospital Edmonton and on faculty with the Psychiatry Department at the University of Alberta and teaches sessional classes for the Psychology Department at the University of Alberta.

Jacqueline R. Pei (R. Psych., PhD) is a professor in the Department of Educational Psychology an assistant clinical professor in the Department of Pediatrics at the University of Alberta, and a practicing registered psychologist. Her research work includes gaining a greater understanding of developmental trajectories and strengths-based approaches to supporting success for children, youth, and young adults. She values collaborative, community-oriented approaches to both her practice and her research.

References

Abidin

Davoren

Naughton

Gibbons

Nulty

Kennedy

H. G.

(2013). Susceptibility (risk and protective) factors for in-patient violence and self-harm: Prospective study of structured professional judgement instruments START and SAPROF, DUNDRUM-3 and DUNDRUM-4 in forensic mental health services. BMC Psychiatry, 13(1), 1–18.

Ægisdóttir

White

M. J.

Spengler

P. M.

Maugherman

A. S.

Anderson

L. A.

Cook

R. S.

Nichols

C. N.

Lampropoulos

G. K.

Walker

B. S.

Cohen

Rush

J. D.

(2006). The meta-analysis of clinical judgment project: Fifty-six years of accumulated research on clinical versus statistical prediction. The Counseling Psychologist, 34(3), 341–382.

Andershed

A. K.

Gibson

C. L.

Andershed

(2016). The role of cumulative risk and protection for violent offending. Journal of Criminal Justice, 45, 78–84.

Andrews

D. A.

Bonta

Wormith

J. S.

(2006). The recent past and near future of risk and/or need assessment. Crime & Delinquency, 52(1), 7–27.

Bonta

(1996). Risk-needs assessment and treatment. In Harland

A. T.

(Ed.), Choosing correctional options that work: Defining the demand and evaluating the supply (pp. 18–32). SAGE.

Bonta

Andrews

A. D.

(2017). The psychology of criminal conduct (6th ed.). Taylor & Francis.

Brown

B. B.

(1990). Peer groups and peer cultures. In Feldman

S. S.

Elliot

G. R.

(Eds.), At threshold: The developing adolescent (pp. 171–196). Harvard University Press.

Brown

S. L.

Robinson

Wanamaker

K. A.

Wagstaff

(2020). Strengths matter: Evidence from five separate cohorts of justice-involved youth and adults across North America. Criminal Justice and Behavior, 47(11), 1428–1447.

Brumley

L. D.

Jaffee

S. R.

(2016). Defining and distinguishing promotive and protective effects for childhood externalizing psychopathology: A systematic review. Social Psychiatry and Psychiatric Epidemiology, 51(6), 803–815.

10.

Coid

Yang

Ullrich

Zhang

Sizmur

Roberts

Farrington

Rogers

R. D.

(2009). Gender differences in structured risk assessment: Comparison of the accuracy of five instruments. Journal of Consulting and Clinical Psychology, 77(2), 337–348.

11.

Criminal Code. (1985). C-46. PART II: Offences against public order treason and other offences against the Queen’s authority and person. Statues and Regulations, Government of Canada.

12.

De Bortoli

Ogloff

Coles

Dolan

(2017). Towards best practice: Combining evidence-based research, structured assessment and professional judgement. Child & Family Social Work, 22(2), 660–669.

13.

Department of Justice. (2020, August). The review board systems in Canada: An overview of results from the mentally disordered accused data collection study. https://www.justice.gc.ca/eng/rp-pr/csj-sjc/jsp-sjp/rr06_1/p1.html

14.

de Vogel

de Ruiter

Bouman

de Vries Robbe

(2015). SAPROF: Structured assessment of protective factors for violence risk (2nd ed.). Van der Hoeven Kliniek.

15.

de Vries Robbé

(2014). Protective factors: Validation of the structured assessment of protective factors for violence risk in forensic psychiatry [Doctoral thesis, Radbound University Nijmegen]. Radboud Repository. https://repository.ubn.ru.nl/bitstream/handle/2066/126440/126440.pdf.

16.

de Vries Robbe

de Vogel

de Spa

(2011). Protective factors for violence risk in forensic psychiatric patients: A retrospective validation study for the SAPROF. International Journal of Forensic Mental Health, 10, 178–186.

17.

Douglas

K. S.

Hart

S. D.

Webster

C. D.

Belfrage

(2013). HCR-20V3: Assessing risk of violence—User guide. Mental Health, Law, and Policy Institute, Simon Fraser University.

18.

Dubow

E. F.

Huesmann

L. R.

Boxer

Smith

(2016). Childhood and adolescent risk and protective factors for violence in adulthood. Journal of Criminal Justice, 45, 26–31.

19.

Durrant

(2017). Why do protective factors protect? An evolutionary developmental perspective. Aggression and Violent Behavior, 32, 4–10.

20.

Fontaine

N. M.

Brendgen

Vitaro

Tremblay

R. E.

(2016). Compensatory and protective factors against violent delinquency in late adolescence: Results from the Montreal longitudinal and experimental study. Journal of Criminal Justice, 45, 54–62.

21.

Fortune

C. A.

Ward

(2017). Problems in protective factor research and practice. Aggression and Violent Behavior, 32, 1–3.

22.

Glover

A. J.

Churcher

F. P.

Gray

A. L.

Mills

J. F.

Nicholson

D. E.

(2017). A cross-validation of the Violence Risk Appraisal Guide—Revised (VRAG–R) within a correctional sample. Law and Human Behavior, 41(6), 507–518.

23.

Grove

W. M.

Zald

D. H.

Lebow

B. S.

Snitz

B. E.

Nelson

(2000). Clinical versus mechanical prediction: A meta-analysis. Psychological Assessment, 12(1), 19–30.

24.

Guay

J. P.

Parent

Benbouriche

(2020). Disentangling promotive and buffering protection: Exploring the interface between risk and protective factors in recidivism of adult convicted males. Criminal Justice and Behavior, 47(11), 1468–1486.

25.

Haag

A. M.

Cheng

Wirove

(2016). Describing the not criminally responsible population in Alberta’s history: Socioeconomic, mental health, and criminological profiles. Journal of Safety & Well-Being, 1(3), 68–74.

26.

Hanson

R. K.

Morton-Bourgon

K. E.

(2009). The accuracy of recidivism risk assessments for sexual offenders: A meta-analysis of 118 prediction studies. Psychological Assessment, 21(1), 1–21.

27.

Harris

G. T.

Rice

M. E.

Quinsey

V. L.

(2016). Violence risk appraisal guide—Revised, 2013. http://hdl.handle.net/10864/12053

28.

Harris

G. T.

Rice

M. E.

Cormier

C. A.

(2002). Prospective replication of the violence risk appraisal guide in predicting violent recidivism among forensic patients. Law and Human Behavior, 26(4), 377–394.

29.

Harris

G. T.

Rice

M. E.

Quinsey

V. L.

Cormier

C. A.

(2015). Violent offenders: Appraising and managing risk (3rd ed.). https://dx-doi-org.web.bisu.edu.cn/10.1037/14572-002

30.

Hartman

J. L.

Turner

M. G.

Daigle

L. E.

Exum

M. L.

Cullen

F. T.

(2009). Exploring the gender differences in protective factors: Implications for understanding resiliency. International Journal of Offender Therapy and Comparative Criminology, 53(3), 249–277.

31.

Herrenkohl

T. I.

Hill

K. G.

Chung

I. J.

Guo

Abbott

R. D.

Hawkins

J. D.

(2003). Protective factors against serious violent behavior in adolescence: A prospective study of aggressive children. Social Work Research, 27(3), 179–191.

32.

Herrenkohl

T. I.

Tajima

E. A.

Whitney

S. D.

Huang

(2005). Protection against antisocial behavior in children exposed to physically abusive discipline. Journal of Adolescent Health, 36(6), 457–465.

33.

Hoge

R. D.

Andrews

D. A.

Leschied

(2002). Youth Level of Service/Case Management Inventory: YLS/CMI manual. Multi-Health Systems.

34.

Jennings

W. G.

Gonzalez

J. R.

Piquero

A. R.

Bird

Canino

Maldonado-Molina

(2016). The nature and relevance of risk and protective factors for violence among Hispanic children and adolescents: Results from the Boricua Youth Study. Journal of Criminal Justice, 45, 41–47.

35.

Johnson

S. B.

Blum

R. W.

Giedd

J. N.

(2009). Adolescent maturity and the brain: The promise and pitfalls of neuroscience research in adolescent health policy. Journal of Adolescent Health, 45(3), 216–221.

36.

Kaltiala-Heino

Eronen

(2015). Ethical issues in child and adolescent forensic psychiatry: A review. The Journal of Forensic Psychiatry & Psychology, 26(6), 759–780.

37.

Kim

B. E.

Gilman

A. B.

Hill

K. G.

Hawkins

J. D.

(2016). Examining protective factors against violence among high-risk youth: Findings from the Seattle Social Development Project. Journal of Criminal Justice, 45, 19–25.

38.

Kivisto

A. J.

(2016). Violence risk assessment and management in outpatient clinical practice. Journal of Clinical Psychology, 72(4), 329–349.

39.

Klepfisz

Daffern

Day

(2017). Understanding protective factors for violent reoffending in adults. Aggression and Violent Behavior, 32, 80–87.

40.

Kocsis

R. N.

(2011). Book review: Mills, J., Kroner, D., & Morgan, R. (2010). Clinician’s guide to violence risk assessment. Guilford. International Journal of Offender Therapy and Comparative Criminology, 55(4), 665–666.

41.

Larson

Richards

M. H.

(1991). Daily companionship in late childhood and early adolescence: Changing developmental contexts. Child Development, 62(2), 284–300.

42.

Loeber

Farrington

D. P.

(1998). Never too early, never too late: Risk factors and successful interventions for serious and violent juvenile offenders. Studies on Crime & Crime Prevention, 7(1), 7–30.

43.

Moffitt

(1993). Adolescence-limited and life-course persistent antisocial behavior: A developmental taxonomy. Psychological Review, 100, 674–701.

44.

Neil

O’Rourke

Ferreira

Flynn

(2020). Protective factors in violence risk assessment: Predictive validity of the SAPROF and HCR-20V3. International Journal of Forensic Mental Health, 19(1), 84–102.

45.

Nicholls

T. L.

Petersen

K. L.

Pritchard

M. M.

(2016). Comparing preferences for actuarial versus structured professional judgment violence risk assessment measures across five continents: To what extent is practice keeping pace with science. In Singh

J. P.

Bjorkly

Fazel

(Eds.), International perspectives on violence risk assessment (pp. 127–149). Oxford University Press.

46.

Nuffield

(1982). Parole decision-making in Canada: Research toward decision guidelines. Research Division Canada. https://www.ncjrs.gov/App/Publications/abstract.aspx?ID=126179

47.

Olver

M. E.

Stockdale

K. C.

Wormith

J. S.

(2014). Thirty years of research on the level of service scales: A meta-analytic examination of predictive accuracy and sources of variability. Psychological Assessment, 26(1), 156–176.

48.

Orbis. (2000). The Youth Assessment and Screening Instrument (YASI). https://www.orbispartners.com/juvenile-risk-assessment

49.

Orbis. (2003). The Service Planning Instrument (SPIn). https://www.orbispartners.com/adult-risk-assessment

50.

Patricny

(2021). The search for protective factors against recidivism in adolescents found not criminally responsible on account of mental disorder [Doctoral dissertation, University of Alberta]. ERA. https://doi.org/10.7939/r3-w3pr-fm86

51.

Putniņš

A. L.

(2005). Assessing recidivism risk among young offenders. Australian & New Zealand Journal of Criminology, 38(3), 324–339.

52.

Rice

M. E.

Harris

G. T.

(2005). Comparing effect sizes in follow-up studies: ROC area, Cohen’s. D, and R. Law and Human Behavior, 29(5), 615–620.

53.

Richer

Cheng

Haag

A. M.

(2018). Historical recidivism rates of Alberta’s not criminally responsible population. Journal of Community Safety & Well-Being, 3(2), 59–64.

54.

Rogers

(2000). The uncritical acceptance of risk assessment in forensic practice. Law and Human Behavior, 24, 595–605.

55.

Rossegger

Endrass

Gerth

Singh

J. P.

(2014). Replicating the violence risk appraisal guide: A total forensic cohort study. PLOS ONE, 9(3), Article e91845.

56.

Salgado

J. F.

(2018). Transforming the area under the normal curve (AUC) into Cohen’s d, Pearson’s r_pb, odds-ratio, and natural log odds-ratio: Two conversion tables. European Journal of Psychology Applied to Legal Context, 10(1), 35–47.

57.

Serin

R. C.

(2007). The dynamic risk assessment for offender re-entry (DRAOR): Guidelines for case planning and risk management using structured assessment of dynamic risk and protective factors. [Unpublished user manual]. Carleton University.

58.

Serin

R. C.

Lowenkamp

C. T.

(2015). Selecting and using risk and need assessments. National Drug Court Institute, 10(1), 1–23.

59.

Sroufe

L. A.

(1997). Psychopathology as an outcome of development. Development and Psychopathology, 9(2), 251–268.

60.

Statistics Canada. (2020). Verdicts of not criminally responsible on account of mental disorder in adult criminal courts. https://www150.statcan.gc.ca/n1/pub/85-002-x/2014001/article/14085-eng.htm

61.

Steinberg

Morris

A. S.

(2001). Adolescent development. Annual Review of Psychology, 52(1), 83–110.

62.

van der Laan

A. M.

Veenstra

Bogaerts

Verhulst

F. C.

Ormel

(2010). Serious, minor, and non-delinquents in early adolescence: The impact of cumulative risk and promotive factors. The TRAILS study. Journal of Abnormal Child Psychology, 38(3), 339–351.

63.

Vassallo

Edwards

Forrest

(2016). Childhood behavior problems and fighting in early adulthood: What factors are protective? Journal of Criminal Justice, 45, 85–93.

64.

Viljoen

J. L.

Bhanwer

A. K.

Shaffer

C. S.

Douglas

K. S.

(2020). Assessing protective factors for adolescent offending: A conceptually informed examination of the SAVRY and YLS/CMI. Assessment, 27(5), 959–975.

65.

Viljoen

J. L.

Vargen

L. M.

Cochrane

D. M.

Jonnson

M. R.

Goossens

Monjazeb

(2021). Do structured risk assessments predict violent, any, and sexual offending better than unstructured judgment? An umbrella review. Psychology, Public Policy, and Law, 27(1), 79–97.

66.

Yang

Wong

S. C.

Coid

(2010). The efficacy of violence prediction: A meta-analytic comparison of nine risk assessment tools. Psychological Bulletin, 136(5), 740–767.

67.

Yesberg

J. A.

Polaschek

D. L.

(2015). Assessing dynamic risk and protective factors in the community: Examining the validity of the Dynamic Risk Assessment for Offender Re-entry. Psychology, Crime & Law, 21(1), 80–99.

68.

Yessine

A. K.

Bonta

(2012). The offending trajectories of youth probationers from early adolescence to middle adulthood: Relation to dual taxonomies (Research report: 2012-4). National Crime Prevention Centre, Public Safety Canada.