The Effects of the Growing Pro-Social Program on Early Maladaptive Schemas and Schema-Related Emotions in Male Young Offenders: A Nonrandomized Trial

Abstract

This study aimed to test the effects of a 25-session version of the Growing Pro-Social (GPS-25) program over schemas and schema-related emotions in male young offenders. Participants included 123 youth aged between 14 and 19 years, placed in eight Portuguese detention facilities. Youth were allocated to receive GPS (n = 63) or treatment as usual (n = 60), and answered a self-report measure assessing schemas and schema-related emotions at baseline and posttreatment. Two-factor mixed multivariate analysis of variance (MANOVA; group change) and the Reliable Change Index (individual change) revealed nonsignificant differences between groups for the schema’s endorsement. Significant differences between groups were found for the schema-related emotions: Treatment participants presented lower scores and/or higher clinical improvements after GPS, when compared with controls. GPS-25 produced change at an emotional level but not in schema’s endorsement, suggesting that longer interventions should be tested in their capability to promote cognitive and emotional change in young offenders.

Keywords

clinical trials early maladaptive schemas disruptive emotions Growing Pro-Social Program male young offenders

Introduction

The efficacy of cognitive–behavioral group interventions in the rehabilitation of male young offenders has been well established (e.g., Koehler et al., 2013; Raynor et al., 2014; Trotter, 2013). Cognitive–behavioral programs conceptualize antisocial behavior as the result of maladaptive (criminal) thinking styles (e.g., Walters, 2007). It is well known that offenders use cognitive distortions when processing social information (Chabrol et al., 2011; Helmond et al., 2015), and these program’s ultimate goal is to promote a more adaptive social information processing, by reducing underlying cognitive distortions (Antonio & Crosset, 2017; Brazão et al., 2017). Nevertheless, research has also shown that early maladaptive schemas (EMSs) play a major role in the onset and maintenance of antisocial behavior (e.g., Chakhssi et al., 2012; Gilbert & Daffern, 2013).

EMSs have been defined as core dysfunctional beliefs that develop from early negative experiences with significant others, being elaborated throughout development. Maladaptiveness comes both from the rigidity of this type of beliefs, which, when triggered, led to misinterpretation and misattribution of relevant events (i.e., cognitive distortions), and from the intense experience of schema-related negative emotional states whenever a particular EMS dominates the individual’s cognitive set (Rafaeli et al., 2011). For instance, if someone has been repeatedly physically and emotionally abused by one or both parents/caregivers in early stages of life, she or he may develop an EMS of mistrust/abuse. Later in life, the prominence of such EMS leads to an increased selective attention to cues of possible experiences of abuse when in interpersonal contexts, thus facilitating the interpretation of neutral interpersonal behavior as abusive (Brazão et al., 2017). Consequently, the person tends to experience schema-congruent emotional arousal (e.g., humiliation and/or anger feelings) and tends to behave accordingly (e.g., counterattacking).

A considerable amount of research (Calvete, 2008; Calvete & Orue, 2010; Chakhssi et al., 2012; Gilbert & Daffern, 2013; Specht et al., 2009) found a positive association of mistrust/abuse, insufficient self-control, and entitlement EMSs with antisocial behavior in samples of young and adult offenders. A more recent study (Shorey et al., 2014) showed that EMSs belonging to the disconnection/rejection domain, which includes mistrust/abuse, abandonment, emotional deprivation, defectiveness/shame, and social isolation EMSs, were positively associated with increased antisocial behavior. These findings suggest that specific EMSs play a major role on the cognitive correlates of antisocial behavior. Thus, EMSs should be identified as targets of change in rehabilitation programs for offenders (Brazão et al., 2017). Schema-focused therapy is currently being offered to offenders in forensic settings (Farrell et al., 2009; Giesen-Bloo et al., 2006; Nadort et al., 2009; van Asselt et al., 2008), and has proven to be effective in reducing schema’s endorsement, cognitive distortions, emotion regulation difficulties, and aggressive behavior (Brazão et al., 2017, 2018a, 2018b; Keulen-de Vos et al., 2013).

As previously specified, for an EMS to be maladaptive, it should elicit the experience of schema-related negative emotional states whenever triggered (Rafaeli et al., 2011). Nevertheless, research has scarcely investigated the relationship between EMSs and negative emotional arousal, namely in young offenders. Schmidt et al. (1995) studied the relationship between the endorsement of EMSs and a measure of positive (feeling enthusiastic, active, and alert) and negative (feeling anger, disgust, guilty, or fear) affect, in a sample of undergraduate students. Findings showed that EMSs were associated with the measures of positive and negative affect; individuals scoring higher in EMSs also scored higher in negative affect and lower in positive affect. The issue of schema-related negative emotions was also studied by Rijo (2017), when developing a new questionnaire for the assessment of EMSs using schema-related life scenarios as a strategy to trigger EMSs. Results in community and clinical adult samples showed that participants reporting greater similarity between their life experiences and the proposed scenarios scored higher not only on EMSs but also on the intensity of schema-related emotions. Capinha et al. (2017) used a similar procedure to develop a schema questionnaire for adolescents, aiming the assessment of the EMSs underlying antisocial behavior. Results in community and forensic adolescent samples showed that youth reporting greater similarity between previous real-life experiences and the presented scenarios scored higher on EMSs, as well as in emotional arousal during the scenarios imagery exercises.

To sum up, research has shown that EMSs and schema-related emotions are associated with offending behavior. Nevertheless, few intervention programs for young offenders take into account the need to promote change at a deeper level, such as EMSs, to modify antisocial behavior (Brazão et al., 2017). To overcome this shortcoming, Rijo and colleagues (2007) developed the Growing Pro-Social (GPS) program, a 40-session cognitive–behavioral group program, which is strongly based on schema theory (e.g., Rafaeli et al., 2011). GPS was specifically designed to be used in the rehabilitation of young and adult offenders, conceptualizing antisocial behavior as a result of a distorted view of the self and the others. The program aims to achieve behavioral and emotional change by changing the cognitive correlates of antisocial behavior, namely, EMSs, cognitive distortions, and cognitive products. The ultimate goal is to reach some degree of change in specific EMSs, underlying the social information processing of offenders, such as emotional deprivation, abandonment, mistrust/abuse, defectiveness/shame, social isolation/alienation, failure, entitlement, and insufficient self-control (Brazão et al., 2013, 2017; Rijo et al., 2007). It is important to add that GPS aims to change not only the cognitive component of EMSs but also schema-related disruptive emotions.

As previously specified, the GPS program was designed to be delivered with both young and adult offenders. Nevertheless, so far, the efficacy of GPS was tested exclusively with adult samples. A randomized controlled trial testing the GPS’s effects over time in a sample of male prison inmates was accomplished (Brazão et al., 2017, 2018a, 2018b, 2019). Results showed that the GPS program had positive effects on the offenders’ cognitive functioning, by reducing cognitive distortions and the prominence of EMSs. The program was also effective in reducing feelings of anger and shame, paranoid ideation, emotion regulation difficulties, and disciplinary infractions (prison misconduct). Results also showed that severely disturbed inmates (with antisocial personality disorder and two or more comorbid diagnoses) were responsive to treatment, presenting improvements in all outcome measures.

The current study consisted of a nonrandomized trial aimed to test the effects of a short 25-session version of the GPS (GPS-25) in EMSs and schema-related emotions in male young offenders placed in juvenile detention facilities. Therefore, the study’s main goal was to assess whether male young offenders who participated in the GPS-25 showed changes in the EMSs targeted by the program, as well as in schema-related emotions, when compared with a control group. Participants receiving the GPS-25 were expected to present a significant decrease in the prominence of EMSs, as well as a decrease in schema-related emotions, when compared with controls.

Method

This study was designed in accordance with Transparent Reporting of Evaluations With Nonrandomized Designs (TREND) 2004 Statement guidelines for reporting nonrandomized trials/designs (Des Jarlais et al., 2004).

Trial Design and Participants

This study was a nonrandomized trial carried out in eight juvenile detention facilities in three city areas in mainland Portugal (Lisbon, Oporto, and Coimbra) and in the Madeira Island. Participants were male young offenders aged between 14 and 19 years.

The selection of participants had the following exclusion criteria: (a) presence of cognitive disabilities (because this kind of intervention is not suitable for the cognitively impaired), (b) psychotic symptoms (experiential strategies used in GPS are contraindicated for psychotic patients), (c) being sentenced exclusively for sexual offenses (sex offenders would benefit from more specific intervention programs), and (d) remaining in juvenile detention facilities for less than 3 months (GPS’s length), since the beginning of the program. Exclusion criteria (a) and (b) were assessed by staff psychologists and/or collected from the justice report files.

Sample size

A power analysis was conducted with the G*Power v3.1 software (Faul et al., 2009). Results showed that a sample of 98 participants was necessary to detect medium effects with a significance level of .05 and a power of .90.

Interventions

GPS

A reduced version of the GPS (condensed into 25 sessions) has been designed to be delivered with adolescents in Portuguese juvenile correctional facilities, due to the sentence length (which is significantly shorter when compared with adults’ sentence), and due to time restrictions of facilitators to deliver the original program (Rijo et al., 2007). As presented in Table 1, the GPS-25 was composed of an initial presentation session, 23 sessions that follow the five specific modules, and a final session. Each module is composed of a different number of sessions: (a) human communication (three sessions addressing the ambiguity of human interactions), (b) interpersonal relationships (two sessions addressing maladaptive interpersonal behavior patterns), (c) cognitive distortions (four sessions addressing the ability to recognize and counteract cognitive distortions), (d) function and meaning of emotions (five sessions addressing how emotions affect our lives, their adaptive value, and how to cope with them), and (e) EMSs (nine sessions addressing EMSs and their influence in the information processing).

Table 1.

GPS Modules and Contents.

Modules	Number of sessions	Content summary
Initial session	1	Presentation of the facilitators and the participants, as well as the structure and the methodology of the program
1. Human communication	3	The communication process as a complex phenomenon, its obstacles, and verbal and nonverbal communication skills
2. Interpersonal relationships	2	Behavioral styles in interpersonal relationships, self-concept and interpersonal behavior, ideas about the others, and their influence in interpersonal behavior
3. Cognitive distortions	4	Identifying and changing cognitive distortions: selective abstraction, overgeneralization, mind reading, crystal ball, minimization, disqualifying the positive experiences, dichotomous thinking, labeling, and personalization
4. Function and meaning of emotions	5	The diversity of the emotional experience; the nature and function of emotions: sadness, shame, fear, anger, guilt, and happiness
5. EMSs	9	EMSs and their influence in giving meaning to reality; identifying and changing EMSs: failure, social isolation/alienation, mistrust/abuse, defectiveness/shame, emotional deprivation, abandonment/instability, grandiosity/entitlement; fighting EMS’ influences in thoughts, emotions, and behavior
Final session	1	Reflection and consolidation of learning, and generalization of gains made during the program

Note. GPS = Growing Pro-Social; EMS = early maladaptive schema.

Treatment as usual (TAU)

In Portuguese juvenile detention facilities, TAU consisted of supervision of school frequency, academic, occupational/job-related tasks and sentence planning supervision over time, and counseling (once a week). The control group received TAU and did not attend the GPS or any other intervention program during the research period.

Outcome Measure

Both groups reported on a self-report measure of EMSs and schema-related emotions. Sociodemographic and legal data on participants were collected from juvenile facilities staff members and individual files.

Schema Assessment Inventory Through Activating Scenarios for Antisocial Behavior (SAIAS-AB)

SAIAS-AB is a Portuguese self-report measure composed of 40 items assessing the eight EMSs (mistrust/abuse, defectiveness/shame, failure, social isolation/alienation, abandonment, emotional deprivation, entitlement, and insufficient self-control) underlying antisocial behavior (Capinha et al., 2017).¹

When answering the SAIAS-AB, adolescents are asked to read a life scenario, describing an early life experience that is expected to be related to the origins of a particular EMS:

You got home by the end of the day and there is nobody there. You call your father . . . you call your mother but nobody answers you. You are completely alone. You have a thought that they may not come back anymore . . . Whenever this happens, you never know if they will come back home or not. You remember that time they went out without telling you and you were left alone for several days. You fear they would have abandoned you and you would stay all alone forever.

Participants are asked to imagine that the situation described in each scenario had just happened, and then rate its similarity with their own experiences using a scale from 1 = no similarity to 6 = exactly the same.

After reading, imagining, and rating the scenario, participants are asked to answer a group of five items that assess the specific EMS. Finally, participants are also asked to identify which emotions they have may have felt during the imagination task. A list of 11 emotions is presented (sadness, anger, shame, loneliness, anxiety, fear, emptiness, guilty, humiliation, disgust, and envy/jealousy) and participants are asked to rate the level of intensity for each emotion, using an 8-point scale (0 = did not feel at all to 7 = felt a lot).

The original version of the SAIAS-AB presented good internal consistency values for the eight EMSs ranging from α = .75 to α = .91 (Capinha et al., 2017). In the current study, a total score (resulting from the sum of the eight EMSs) was computed, and an alpha of .98 was found. For the specific EMSs, internal consistency was .88 for abandonment; .89 for failure, emotional deprivation, and defectiveness/shame; .90 for social isolation/alienation and mistrust/abuse; and .91 for insufficient self-control and entitlement.

Procedures

The current study was approved by the Ethics Committee of the Faculty of Psychology and Educational Sciences of the University of Coimbra where the research center is based.

In Portugal, juvenile detention facilities altogether have about 150 youth (on average, 30 youth per each facility), facing 6 to 36 months of detention. About 10 youth enter and leave juvenile detention facilities each month, which makes it difficult to randomly assign participants to conditions. To minimize this shortcoming and to maximize time and human resources, researchers decided to assign the first 65 youth entering in the juvenile detention facilities during the research period to the treatment group, and the following 65 youth to the control group (130 participants in total, although seven youth have refused to participate in the study; please see “Recruitment and Retention” section).

At each juvenile detention facility, adolescents who did not meet the exclusion criteria were invited to participate voluntarily in this study. After the study was approved by detention institutions’ boards, informed consent was obtained from participants’ legal guardian. A first meeting between the research team and the selected participants occurred, in which researchers explained the goals of the study and presented a brief overview of the intervention program. It was also explained that their participation in the study would not affect their sentencing in any way. Participants who agreed to participate, completed the baseline (1 week before the GPS-25 sessions) and posttreatment (1 week after the program/3 months after baseline) assessments. Participants in the control group were informed that they would be offered the GPS-25 treatment after the study’s completion. Assessors did not serve as therapists and vice versa. Respondent-specific codes were used to link the data from one time point to the other.

GPS-25 facilitators were chosen among the psychologists who already had training and experience in delivering the program with young offenders. To assure program consistency, facilitators received regular supervision by the research team (including the program’s main author) during the time GPS was delivered in detention facilities. To increase treatment integrity, all groups/sessions were carried out by two therapists. While one therapist was leading the session, the other one observed the implementation and helped in keeping it close to the program handbook. This second therapist only intervened when the first one deviated from the protocol, and an established codebook for helping this therapist determine what counted as a deviation (e.g., discussing topics not related or irrelevant to the session’s goals) was provided in the GPS’s handbook. Quality control procedures, such as recording sessions and/or the presence of external assessors in the GPS sessions, were not allowed in juvenile detention facilities.

Data Analysis

Data were analyzed with IBM SPSS Statistics v22.0, following an intention-to-treat approach. Independent samples t tests with Bonferroni multiple significance-test correction (taking into account the multiple comparisons) were performed to analyze differences between the two groups at baseline. To investigate intervention effects on EMSs and schema-related emotions, a two-factor (i.e., between subjects—groups—and within subjects—time) mixed multivariate analysis of variance (MANOVA) was performed. Effect sizes for the time and Time × Group effects were computed using partial eta square $(η_{p}^{2})$ , with $η_{p}^{2} = . 01$ referring to a small effect size, .06 to a medium effect size, and .14 to a large effect size (Tabachnick & Fidell, 2013).

In addition, individual clinical change was assessed by computing the Reliable Change Index (RCI; Jacobson & Truax, 1991). Instead of focusing on the differences on group mean scores, RCI provides information about treatment effects for each individual, allowing to test whether an individual improves or deteriorates after intervention. To ascertain whether the observed change is not due to measurement unreliability, and whether change places the individual inside the norms of functional groups, RCI allows the testing of the null hypothesis of no clinically meaningful change, taking into account the normal distribution and the measurement error of the instruments (Jacobson & Truax, 1991). RCI is computed using the following formula: $RCI = (X p o s t - X p r e) / \sqrt 2 (S D 0 \times \sqrt 1 - α) 2$ , where X post represents the score of the individual in the posttreatment, X pre represents the score of the individual in the pretreatment, SD represents the standard deviation of the variable in a community sample, and α represents the internal consistency of the scale in the current study sample.

According to Wise (2004), if RCI scores are greater than 0.84, we can assert, with a confidence interval of 80%, that real, reliable, and significant change has been verified; however, if the result exceeds 1.28 or 1.96, that confidence interval increases to 90% and 95%, respectively. On the contrary, if the result is less than −0.84, we can say that deterioration occurred. All values between 0.84 and −0.84 indicate that no change was observed. For the interpretation of the RCI in the current study, three broad categories were defined: “Global Improvement” (GI), “Global Deterioration” (GD), and “No Change” (NC). To compare both groups in the distributions by clinical change categories, chi-square statistics with Fisher’s exact tests were performed. Effect sizes of the differences found in the distributions by clinical change category between groups were calculated using Cramer’s V. Following the guidelines by Cohen (1988), .1 was considered a small effect, .3 a medium effect, and .5 a large effect.

Results

Recruitment and Retention

A sample of 130 male young offenders, who did not meet the exclusion criteria, were invited to participate in this study (see Figure 1). Seven (3.8%) youths refused to participate, and 123 (96.2%) youths completed the baseline assessment. Of these, 63 (51.2%) were assigned to the GPS treatment and 60 (48.8%) were assigned to the control group. From the initial 63 treatment participants, four (6.3%) dropped out the program and three (4.7%) were transferred to another detention facility. Fifty-six (89%) participants completed the intervention and the posttreatment assessments. All these 56 participants attended the full GPS treatment (i.e., 25 sessions). Of the 60 youths assigned to the control group, two (3.3%) refused to complete the posttreatment assessment and 58 (96.7%) completed the protocol.

Figure 1.

Flowchart of youth’s participation.

Baseline Differences

Demographic and legal features by group are reported in Table 2, and no significant differences between groups were found.

Table 2.

Demographic and Legal Features by Group.

	Treatment group		Control group		t	p	Cohen’s d
Demographic and legal features	M	SD	M	SD	t	p	Cohen’s d
Age	16.21	0.93	16.40	1.31	0.936	.352	0.16
Years of education	5.62	1.54	5.47	1.25	0.598	.551	0.10
Sentence length (in months)	20.81	6.12	19.17	5.24	1.594	.113	0.28
	n	%	n	%	Fisher’s	p	Cramer’s V
SES
Low	62	98.4	57	95.0	1.408	.422	0.109
Medium	1	1.6	2	3.3
High	—	—	1	1.7
Type of crime
Against people	27	42.9	21	35.0	1.594	.750	0.105
Against property	33	52.4	34	56.7
Against life in society	2	3.2	4	6.7
Drug trafficking	1	1.6	1	1.7

Note. SES was measured by parents’ profession, considering the Portuguese professions classification (Instituto Nacional de Estatística, 2011). Examples of professions in the high SES group are judges, higher education professors, or MDs; in the medium SES group are nurses, psychologists, or school teachers; and in the low SES group are farmers, cleaning staff, or undifferentiated workers. Crimes against people include simple and aggravated assault, and intimidation; crimes against property include robbery, theft, and qualified theft; crimes against life in society include counterfeiting, forgery of documents, and fire setting. SES = socioeconomic status.

Treatment and control groups were also compared at baseline for the outcome measures (EMSs and schema-related emotions). As presented in Table 3, the Bonferroni adjustment for multiple comparisons revealed nonsignificant differences between groups at the onset of the study.

Table 3.

Baseline Differences on the Outcome Measures.

	Treatment group		Control group		t	p	Cohen’s d
Outcome measures	M	SD	M	SD	t	p	Cohen’s d
EMSs prominence
Abandonment/instability	3.32	1.26	3.47	1.34	0.634	.527	0.11
Defectiveness/shame	1.81	1.02	2.13	1.08	1.667	.098	0.30
Failure	1.95	0.95	2.35	1.22	2.009	.047	0.36
Mistrust/abuse	2.57	1.17	2.96	1.21	1.776	.078	0.32
Social isolation/alienation	1.64	0.80	1.76	0.89	0.749	.429	0.14
Emotional deprivation	1.98	1.09	2.08	1.11	0.501	.618	0.09
Grandiosity/entitlement	2.19	0.88	2.47	0.91	1.770	.079	0.31
Insufficient self-control	2.15	1.08	2.62	1.23	2.208	.029	0.40
EMSs total score	88.25	27.11	99.33	26.51	2.290	.024	0.41
EMS-related emotions
Abandonment/instability	39.53	19.60	40.73	15.89	0.372	.711	0.06
Defectiveness/shame	38.01	20.81	38.06	17.32	0.015	.988	0.00
Failure	37.28	21.33	32.80	15.28	1.345	.181	0.24
Mistrust/abuse	39.52	20.82	43.58	18.91	1.130	.261	0.20
Social isolation/alienation	36.76	20.14	38.55	19.03	0.506	.614	0.09
Emotional deprivation	37.26	19.64	40.76	19.51	0.990	.324	0.17
Grandiosity/entitlement	35.77	19.14	32.31	18.73	1.013	.313	0.18
Insufficient self-control	33.42	20.22	31.10	19.98	0.673	.502	0.11

Note. Bonferroni p < .002.

Intervention Effects in the Prominence of EMSs and Schema-Related Emotions

Two-factor mixed MANOVA

Concerning EMSs, Multivariate tests showed there was no difference between groups, Wilks’ λ = .947, F = 0.707, p = .702, $η_{p}^{2} = . 053$ . Univariate tests also indicated there were no intervention effects in the prominence of EMSs (see Table 4). Regarding schema-related emotions, multivariate tests revealed a significant Time × Group interaction effect, Wilks’ λ = .690, F = 6.392, p < .001, $η_{p}^{2} = . 310$ . Accordingly, univariate tests showed that the treatment group presented significantly lower scores at the end of GPS in all schema-related emotions, when compared with the control group. These differences corresponded to medium or strong effects (please see Table 4).

Table 4.

Mean Scores and Standard Deviations for Both Groups at Pre- and Posttreatment, and Univariate Tests.

	TG		CG		Time	Time × Group
	T1	T2	T1	T2
Outcome measures	M (SD)	M (SD)	M (SD)	M (SD)
EMSs prominence
Abandonment/instability	3.32 (1.26)	3.12 (1.14)	3.47 (1.47)	3.24 (1.18)	F = 3.914; p = .050; $η_{p}^{2} = . 031$	F = 0.019; p = .891; $η_{p}^{2} = . 000$
Defectiveness/shame	1.81 (1.02)	1.63 (0.86)	2.13 (1.08)	1.89 (0.95)	F = 5.474; p = .021; $η_{p}^{2} = . 043$	F = 0.097; p = .755; $η_{p}^{2} = . 001$
Failure	1.95 (0.95)	2.05 (1.17)	2.35 (1.23)	2.48 (0.108)	F = 1.169; p = .282; $η_{p}^{2} = . 010$	F = 0.023; p = .879; $η_{p}^{2} = . 000$
Mistrust/abuse	2.57 (1.17)	2.42 (1.22)	2.96 (1.16)	2.78 (1.10)	F = 2.206; p = .140; $η_{p}^{2} = . 018$	F = 0.016; p = .901; $η_{p}^{2} = . 000$
Social isolation/alienation	1.64 (0.80)	1.70 (0.98)	1.76 (0.89)	1.82 (0.82)	F = 0.377; p = .540; $η_{p}^{2} = . 003$	F = 0.000; p = .983; $η_{p}^{2} = . 000$
Emotional deprivation	1.98 (1.09)	1.76 (0.96)	2.08 (1.11)	1.96 (1.04)	F = 2.623; p = .108; $η_{p}^{2} = . 021$	F = 0.244; p = .622; $η_{p}^{2} = . 002$
Grandiosity/entitlement	2.19 (0.88)	2.00 (0.92)	2.47 (0.91)	2.54 (1.00)	F = 0.421; p = .518; $η_{p}^{2} = . 003$	F = 1.996; p = .163; $η_{p}^{2} = . 016$
Insufficient self-control/self-discipline	2.15 (1.08)	1.99 (1.08)	2.62 (1.23)	2.51 (1.05)	F = 1.828; p = .179; $η_{p}^{2} = . 015$	F = 0.084; p = .772; $η_{p}^{2} = . 001$
Total score	88.25 (27.11)	83.50 (29.86)	99.33 (26.51)	96.15 (25.66)	F = 2.777; p = .098; $η_{p}^{2} = . 022$	F = 0.108; p = .743; $η_{p}^{2} = . 001$
EMS-related emotions
Abandonment/instability	39.53 (19.60)	26.90 (14.15)	40.73 (15.89)	39.41 (14.56)	F = 16.905; p < .001; $η_{p}^{2} = . 123$	F = 11.126; p = .001; $η_{p}^{2} = . 084$
Defectiveness/shame	38.01 (20.81)	22.82 (12.47)	38.06 (17.32)	42.60 (17.97)	F = 7.888; p = .006; $η_{p}^{2} = . 061$	F = 27.018; p < .001; $η_{p}^{2} = . 183$
Failure	37.28 (21.33)	25.25 (14.94)	32.80 (15.28)	38.83 (17.35)	F = 2.965; p = .088; $η_{p}^{2} = . 024$	F = 26.896; p < .001; $η_{p}^{2} = . 182$
Mistrust/abuse	39.52 (20.82)	25.84 (14.72)	43.58 (18.91)	42.58 (15.91)	F = 17.062; p < .001; $η_{p}^{2} = . 124$	F = 12.730; p = .001; $η_{p}^{2} = . 095$
Social isolation/alienation	36.76 (20.14)	22.23 (11.74)	38.55 (19.03)	40.86 (17.72)	F = 11.787; p = .001; $η_{p}^{2} = . 089$	F = 22.432; p < .001; $η_{p}^{2} = . 156$
Emotional deprivation	37.26 (19.64)	23.76 (13.83)	40.76 (19.51)	43.60 (17.92)	F = 9.310; p = .003; $η_{p}^{2} = . 071$	F = 21.817; p < .001; $η_{p}^{2} = . 153$
Grandiosity/entitlement	35.77 (19.14)	19.55 (10.52)	32.31 (18.73)	38.63 (17.34)	F = 8.625; p = .004; $η_{p}^{2} = . 067$	F = 44.652; p < .001; $η_{p}^{2} = . 270$
Insufficient self-control/self-discipline	33.42 (20.22)	20.14 (10.06)	31.10 (19.98)	34.58 (16.50)	F = 9.050; p = .003; $η_{p}^{2} = . 070$	F = 26.484; p < .001; $η_{p}^{2} = . 180$

Note. EMS = early maladaptive schema; TG = treatment group; CG = control group; T1 = pretreatment; T2 = posttreatment; $η_{p}^{2}$ = partial eta square.

Clinical change

Data relating to clinical change in EMSs and schema-related emotions on both groups are presented in Table 5. On one hand, results revealed nonsignificant differences between groups in the distribution by clinical change categories concerning the prominence of EMSs. Most participants in both groups presented no clinical change after GPS completion. On the other hand, significant differences between groups were found in the distribution by clinical change categories for all schema-related emotions: Whereas a high percentage of the treatment group presented clinical improvement, an equally high percentage of the control group presented clinical deterioration after GPS completion. The observed effect size for the abandonment/instability-related emotions was small and for the grandiosity-related emotions, it was strong. For the remaining variables, effect sizes were medium.

Table 5.

Reliable Change Index for EMSs Prominence, and for EMSs Disruptive Emotions by Group.

	Categories	TG		CG		Fisher’s	p	Cramer’s V
Outcome measures	Categories	n	%	n	%	Fisher’s	p	Cramer’s V
EMSs prominence
Abandonment/instability	GI	27	42.9	27	45.0	0.475	.803	.061
	NC	18	28.6	19	31.7
	GD	18	28.6	14	23.3
Defectiveness/shame	GI	21	33.3	24	40.0	1.599	.451	.114
	NC	28	44.4	20	33.3
	GD	14	22.2	16	26.7
Failure	GI	22	34.9	13	21.7	3.375	.198	.166
	NC	23	36.5	22	36.7
	GD	18	28.6	25	41.7
Mistrust/abuse	GI	25	39.7	20	33.3	0.809	.647	.080
	NC	24	38.1	23	38.3
	GD	14	22.2	17	28.3
Social isolation/alienation	GI	13	20.6	13	23.3	0.422	.814	.056
	NC	34	54.0	29	48.3
	GD	16	25.4	17	28.3
Emotional deprivation	GI	13	20.6	17	28.3	5.199	.072	.207
	NC	40	63.5	26	43.3
	GD	10	15.9	17	28.3
Grandiosity/entitlement	GI	18	28.6	11	18.3	2.598	.295	.156
	NC	35	55.6	34	56.7
	GD	10	15.9	15	25.0
Insufficient self-control/self-discipline	GI	21	33.3	18	30.0	1.023	.619	.091
	NC	30	47.6	26	43.3
	GD	12	19.0	16	26.7
Total score	GI	37	58.7	30	50.0	1.401	.482	.107
	NC	2	3.2	4	6.7
	GD	24	38.1	26	43.3
EMSs disruptive emotions
Abandonment/instability	GI	44	69.8	25	41.7	9.865	.006	.284
	NC	4	6.3	7	11.7
	GD	15	23.8	28	46.7
Defectiveness/shame	GI	41	65.1	18	30.0	19.493	<.001	.395
	NC	9	14.3	7	11.7
	GD	13	20.6	35	58.3
Failure	GI	44	69.8	18	30.0	19.843	<.001	.400
	NC	5	7.9	9	15.0
	GD	14	22.2	33	55.0
Mistrust/abuse	GI	45	71.4	27	45.0	11.923	.003	.311
	NC	6	9.5	4	6.7
	GD	12	19.0	29	48.3
Social isolation/alienation	GI	46	73.0	22	36.7	20.094	<.001	.400
	NC	8	12.7	8	13.3
	GD	9	14.3	30	50.0
Emotional deprivation	GI	46	73.0	22	36.7	19.237	<.001	.392
	NC	9	14.3	11	18.3
	GD	8	12.7	27	45.0
Grandiosity/entitlement	GI	47	74.6	16	26.7	35.439	<.001	.525
	NC	9	14.3	9	15.0
	GD	7	11.1	35	58.3
Insufficient self-control/self-discipline	GI	39	61.9	16	26.7	18.849	<.001	.389
	NC	10	15.9	9	15.0
	GD	14	22.2	35	58.3

Note. EMS = early maladaptive schema; TG = treatment group; CG = control group; GI = Global Improvement; NC = No Change; GD = Global Deterioration.

Discussion

The current study tested the effects of the GPS-25 (a 25-session reduced version of the original 40-session program) in reducing the prominence of EMSs underlying antisocial behavior, as well as schema-related emotions in male young offenders placed in juvenile detention facilities. Data on recruitment and retention rates showed that most participants (89%) assigned to GPS attended the full-treatment program (i.e., 25 sessions). It is noteworthy that only a small number of participants (6.3%) dropped out the program. These data suggest that GPS’s methodology may account for the favorable program retention rate. It also asserts that facilitators were successful in engaging participants with the full treatment. This issue is especially relevant, taking into account that dropouts typically reoffend at a higher rate than treatment completers (Neumann et al., 2010; Olver et al., 2011). Comparisons between the treatment and the control group on demographic and criminal features revealed nonsignificant differences between conditions. The same result was obtained when comparing groups on the outcome measures at baseline. Equivalent groups at baseline (despite the fact that participants were not randomized) allowed for reliable conclusions on the GPS’s effects over EMSs and schema-related emotions.

Concerning change in the endorsement of the EMSs, nonsignificant differences between groups were found. When looking at reliable clinical change in EMSs, results showed that there were no significant differences in the distributions by change categories between groups. Taken together, these findings suggest that GPS-25 was not able to produce any significant (statistical or clinical) change at this level. Different explanations may account for the understanding of these findings. First, even if Module 5 (encompassing EMS identification and fighting against one’s own EMSs) was kept equal to the full GPS version, the 25-session version may have not been strong enough to produce change at this level. Another factor that may have contributed to these outcomes may relate to the option of delivering GPS-25 twice a week, which reduced the period of delivery but may have been weakening the program’s impact in changing EMSs. To promote change in cognitive structures, as is the case of EMSs, participants should have the opportunity to fight against their own EMSs, trying to change the way EMSs affect the processing of schema-relevant information in everyday life situations (Brazão et al., 2013, 2017; Rafaeli et al., 2011; Rijo et al., 2007). If there was not enough time between sessions to apply the acquired knowledge and strategies to real-life situations, EMS change may had become impaired. In addition, the shortening of the initial modules of the GPS may have also contributed to decrease its impact over cognitive change, because the initial focus on changing cognitive distortions was reduced, and the first modules could have the role of facilitating further structural cognitive change, by reducing cognitive rigidity. Recommendations about the length of cognitive–behavioral interventions aiming the rehabilitation of offenders are well known (e.g., Lipsey, 2009) and larger formats are usually preferred. Even if there is no study testing the impact of the full 40-session GPS with young offenders, available findings from its delivery with male prison inmates demonstrated its capability to promote change at this level (Brazão et al., 2017). Further research is needed to ascertain whether the full program would produce similar change in EMSs in young offenders as it was achieved in adult prison inmates.

Outcomes regarding change in schema-related emotions revealed that the treatment group had significantly lower scores at the end of GPS in all schema-related emotions, when compared with the control group. In line with these findings, significant differences between groups were found in the distribution by clinical change categories for disruptive emotions associated with each one of the eight EMSs, being that a large proportion of treated participants showed clinical improvement (a decrease in the intensity of disruptive emotions), and a similar large proportion of controls showed clinical deterioration at posttreatment (with a significant increase in the intensity of schema-related emotions). Taken together, these findings showed that the GPS-25 was capable of reducing the intensity of schema-related emotions, whereas nontreated participants deteriorated from baseline to posttreatment. These findings are in agreement both with schema theory (Rafaeli et al., 2011) and the GPS’s theoretical framework (Brazão et al., 2013; Rijo et al., 2007). A core feature of EMSs has to do with schema-related emotions, which are experienced each time a particular EMS is triggered by a relevant event (Rafaeli et al., 2011). Through increasing knowledge about one’s own EMSs, GPS participants are encouraged to get in touch with (and fight against) their EMSs by connecting with EMS-driven thoughts and emotions (e.g., remembering early life situations where emotional needs where not met, current EMS-triggering situations, and associated emotional and behavioral reactions; Brazão et al., 2013; Rijo et al., 2007). This type of intervention may decrease participant’s emotional reactivity when a particular EMS is triggered, thus helping to gain control over one’s own emotional experiences. Once GPS (and GPS-25) pays great attention to the emotional experience and expression (increasing knowledge about the diversity of the emotional experience and the adaptive function of different emotions), it may contribute to increase emotion regulation when disruptive schema-related emotions arise. Using different types of measures, previous research showed that the full version of the GPS produced significant change over time in emotion and behavior regulation in male prison inmates (Brazão et al., 2018a). Nevertheless, the present study was the first one to look at schema-related emotions in such a specific way. These findings suggest that a relevant feature of EMSs, that is, schema-related emotions, can be decreased by GPS-25, and this effect can be associated with an increase in emotion and behavior regulation, once EMSs maladaptiveness directly relates to the lack of control derived from the experience of intense disruptive emotions (Capinha et al., 2017; Rafaeli et al., 2011; Rijo, 2017). These findings should be further investigated, namely in what relates with the maintenance of change over time. If future longitudinal research corroborates these findings, then GPS-25 can be effective in decreasing EMSs maladaptiveness, not because it changes self-representation per se, but because it decreases one of the major features of that same maladaptiveness (i.e., the intensity of negative emotional arousal).

It is important to add that the GPS positive outcomes on schema-related emotions may be related to nonspecific factors, namely the fact that youth were included in a regular group activity, which per se might have been helpful, considering that youth participating in the current study did not attend any other intervention program or treatment.

One major limitation of the current study has to do with the lack of randomization of participants, as well as the absence of a follow-up assessment, thus impeding to test the maintenance of change over time. The fact that EMSs are usually assessed through self-report measures encompasses another limitation, because this kind of measure is not free of response bias. Generalization should also be made carefully, because all the participants included in this study were male young offenders. Female young offenders were excluded from the current study due to their small numbers in Portuguese juvenile detention facilities. Nonetheless, and to establish the generalizability of the program, future GPS efficacy studies should be carried out with female offenders. Future studies are also needed in other settings (e.g., community-based interventions, residential care). Another shortcoming has to do with the fact that no systematic quality control procedures of program’s delivery were carried out. As previously stated, recording sessions or the presence of external assessors in sessions was not permitted in detention facilities. Researchers tried to overcome this issue by training and supervising the GPS’s facilitators during the program’s length. Moreover, the GPS structured and manualized design, as well as the simultaneous presence of two experienced therapists in all groups/sessions ensured, at least partially, treatment integrity.

The effects of the GPS in the reduction of criminal recidivism rates were not analyzed in this study. The positive effects of a rehabilitation program over recidivism rates are usually presented as a major requirement for the selection of effective intervention practices (e.g., McGuire, 2013). However, a recent trend in research (e.g., Antonio & Crosset, 2017; Brazão et al., 2017, 2018a, 2018b, 2019; Skeem et al., 2009) has begun to identify and to assess other relevant variables as treatment outcome measures, namely cognitive and emotional correlates of antisocial behavior. The current study added to this new wave of research and proposed to test the effects of a cognitive–behavioral structured group intervention program in cognitive and emotional correlates of antisocial behavior. Future studies should assess other relevant variables associated with antisocial behavior, as well as variables that do not rely exclusively on self-report measures (e.g., behavioral measures, disciplinary incidents). Mental health disorders, which are well known to be high prevalent among male young offenders (e.g., Rijo et al., 2016) must be assessed, and should also be tested as predictors/moderators of treatment effects in clinical outcomes. Finally, follow-up studies should also focus on possible delayed effects of intervention and the stability of change over time.

Supplemental Material

Supplementary_Material – Supplemental material for The Effects of the Growing Pro-Social Program on Early Maladaptive Schemas and Schema-Related Emotions in Male Young Offenders: A Nonrandomized Trial

Supplemental material, Supplementary_Material for The Effects of the Growing Pro-Social Program on Early Maladaptive Schemas and Schema-Related Emotions in Male Young Offenders: A Nonrandomized Trial by Daniel Rijo, Rita Ramos Miguel, Marlene Paulo and Nélio Brazão in International Journal of Offender Therapy and Comparative Criminology

Footnotes

Acknowledgements

The authors would like to thank Carolina da Motta for the great help in research procedures and data collection, and the psychologists from the juvenile justice system who delivered the Growing Pro-Social (GPS) program. The authors would also like to thank Ana Ramires of the General Directorate of Reintegration and Prison Services of the Portuguese Ministry of Justice for facilitating the access to juvenile facilities and data collection.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was financed by Portuguese funds through FCT—Fundação para a Ciência e a Tecnologia in the framework of the project “GPS—Growing Pro-Social, a Prevention and Rehabilitation Program for Individuals With Antisocial Behavior: Efficacy Studies in Forensic Samples” (PTDC/PSI-PCL/102165/2008).

ORCID iD

Nélio Brazão

Supplemental Material

Supplemental material is available online with this article.

Notes

References

Antonio

M. E.

Crosset

(2017). Evaluating the effectiveness of the national curriculum and training institute’s “cognitive life skills” program among parolees supervised by Pennsylvania’s board of probation & parole. American Journal of Criminal Justice, 42, 514–532.

Brazão

da Motta

Rijo

(2013). From multimodal programs to a new cognitive–interpersonal approach in the rehabilitation of offenders. Aggression and Violent Behavior, 18, 636–643.

Brazão

Rijo

Ribeiro da Silva

Salvador

M. C.

Pinto-Gouveia

(2019). Personality pathology profiles as moderators of the Growing Pro-Social Program outcomes on cognitive, emotion and behavior regulation in male prison inmates. Journal of Personality Disorders. Advance online publication. https://doi.org/10.1521/pedi_2019_33_424

Brazão

Rijo

Salvador

M. C.

Pinto-Gouveia

(2017). The effects of the Growing Pro-Social Program on cognitive distortions and early maladaptive schemas over time in male prison inmates: A randomized controlled trial. Journal of Consulting and Clinical Psychology, 11, 1064–1079.

Brazão

Rijo

Salvador

M. C.

Pinto-Gouveia

(2018a). Promoting emotion and behavior regulation in male prison inmates: A secondary data analysis from a randomized controlled trial testing the efficacy of the Growing Pro-Social Program. Law and Human Behavior, 42, 57–70.

Brazão

Rijo

Salvador

M. C.

Pinto-Gouveia

(2018b). The efficacy of the Growing Pro-Social Program in reducing anger, shame and paranoia over time in male prison inmates: A randomized controlled trial. Journal of Research in Crime and Delinquency, 55, 649–686.

Calvete

(2008). Justification of violence and grandiosity schemas as predictors of antisocial behavior in adolescents. Journal of Abnormal Child Psychology, 36, 1083–1095.

Calvete

Orue

(2010). Cognitive schemas and aggressive behavior in adolescents: The mediating role of social information processing. The Spanish Journal of Psychology, 13, 190–201.

Capinha

da Motta

C. D.

Rijo

(2017). Inventário de Avaliação de Esquemas por Cenários Activadores-Comportamento Anti-Social-IAECA-CA. [Schema assessment inventory through activating scenarios for antisocial behavior–SAIAS-AB]. In Simões

M. R.

Almeida

L. S.

Gonçalves

M. M.

(Eds.), Psicologia Forense: Instrumentos de Avaliação [Forensic psychology: Assessment instruments] (pp. 287–304). PACTOR.

10.

Chabrol

Leeuwen

Rodgers

Gibbs

(2011). Relations between self-serving cognitive distortions, psychopathic traits, and antisocial behavior in a non-clinical sample of adolescents. Personality and Individual Differences, 51, 887–892.

11.

Chakhssi

Bernstein

Ruiter

(2012). Early maladaptive schemas in relation to facets of psychopathy and institutional violence in offenders with personality disorders. Legal and Criminological Psychology, 19, 356–372.

12.

Cohen

(1988). Statistical power analysis for the behavioral sciences. Lawrence Erlbaum.

13.

Des Jarlais

D. C.

Lyles

Crepaz

, & the TREND Group. (2004). Improving the reporting quality of nonrandomized evaluations of behavioral and public health interventions: The TREND statement. American Journal of Public Health, 94, 361–366.

14.

Farrell

J. M.

Shaw

I. A.

Webber

M. A.

(2009). A schema-focused approach to group psychotherapy for outpatients with borderline personality disorder: A randomized clinical trial. Journal of Behavior Therapy and Experimental Psychiatry, 40, 317–328.

15.

Faul

Erdfelder

Buchner

Lang

A. G.

(2009). Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41, 1149–1160.

16.

Giesen-Bloo

van Dyck

Spinhoven

van Tilburg

Dirksen

van Asselt

Arntz

(2006). Outpatient psychotherapy for borderline personality disorder: Randomized trial of schema-focused therapy vs transference-focused psychotherapy. Archives of General Psychiatric, 63, 649–658.

17.

Gilbert

Daffern

(2013). The association between early maladaptive schema and personality disorder traits in an offender population. Psychology, Crime and Law, 19, 933–946.

18.

Helmond

Overbeek

Brugman

Gibbs

J. C.

(2015). A meta-analysis on cognitive distortions and externalizing problem behavior: Associations, moderators, and treatment effectiveness. Criminal Justice and Behavior, 42, 245–262.

19.

Instituto Nacional de Estatística. (2011). Classificação Portuguesa das profissões 2010 [2010 Portuguese classification of professions].

20.

Jacobson

N. S.

Truax

(1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Contemporary Psychotherapy, 39, 12–19.

21.

Keulen-de Vos

Bernstein

D. P.

Arntz

. (2013). Schema therapy for aggressive offenders with personality disorders. In Trafate

R. C.

Mitchell

(Eds.), Forensic CBT: A handbook for clinical practice (pp. 66–85). Wiley-Blackwell.

22.

Koehler

J. A.

Lösel

Akoensi

T. D.

Humphreys

D. K.

(2013). A systematic review and meta-analysis on the effects of young offender treatment programs in Europe. Journal of Experimental Criminology, 9, 19–43.

23.

Lipsey

M. W.

(2009). The primary factors that characterize effective interventions with juvenile offenders: A meta-analytic overview. Victims and Offenders, 4, 124–147.

24.

McGuire

(2013). “What works” to reduce re-offending: 18 years on. In Craig

L. A.

Dixon

Gannon

T. A.

(Eds.), What works in offender rehabilitation: An evidence-based approach to assessment and treatment (pp. 20–49). Wiley-Blackwell.

25.

Nadort

Arntz

Smit

J. H.

Giesen-Bloo

Eikelenboom

Spinhoven

. . . van Dyck

(2009). Implementation of outpatient schema therapy for borderline personality disorder with versus without crisis support by the therapist outside office hours: A randomized trial. Behavior Research and Therapy, 47, 961–973.

26.

Neumann

Ojong

T. N.

Yanes

P. K.

Tumiel-Berhalter

Daigler

G. E.

Blondell

R. D.

(2010). Differences between adolescents who complete and fail to complete residential substance abuse treatment. Journal of Addictive Diseases, 29, 427–435.

27.

Olver

M. E.

Stockdale

K. C.

Wormith

J. S.

(2011). A meta-analysis of predictors of offender treatment attrition and its relationship to recidivism. Journal of Consulting and Clinical Psychology, 79, 6–21.

28.

Rafaeli

Bernstein

D. P.

Young

(2011). Schema therapy: Distinctive features. Routledge.

29.

Raynor

Ugwudike

Vanstone

(2014). The impact of skills in probation work: A reconviction study. Criminology and Criminal Justice, 14, 235–249.

30.

Rijo

(2017). O Questionário de Esquemas de Young (YSQ-S3). In Gonçalves

M. M.

Simões

M. R.

Almeida

L. S.

(Coord.), Psicologia clínica e da saúde: Instrumentos de avaliação [Clinical and health psychology: Assessment instruments] (pp. 159–174). PACTOR.

31.

Rijo

Brazão

Barroso

Ribeiro

Silva

Vagos

Vieira

. . .Macedo

(2016). Mental health problems in male young offenders in custodial versus community based-programs: Implications for juvenile justice interventions. Child and Adolescent Mental Health and Psychiatry, 10, 40–52.

32.

Rijo

Sousa

M. N.

Lopes

Pereira

Vasconcelos

Mendonça

M. C.

Massa

(2007). Gerar Percursos Sociais: Programa de prevenção e reabilitação para jovens com comportamento social desviante [Growing Pro-Social: Prevention and rehabilitation program for youths with deviant social behavior]. EQUAL.

33.

Schmidt

N. B.

Joiner

T. E.

Young

J. E.

Telch

M. J.

(1995). The Schema Questionnaire: Investigation of psychometric properties and the hierarchical structure of a measure of maladaptive schemas. Cognitive Therapy and Research, 19, 295–321.

34.

Shorey

Anderson

Stuart

(2014). The relation between antisocial and borderline personality symptoms and early maladaptive schemas in a treatment seeking sample of male substance users. Clinical Psychology and Psychotherapy, 21, 341–351.

35.

Skeem

J. L.

Polaschek

D. L.

Manchak

(2009). Appropriate treatment works but how? Rehabilitating general psychopathic, and high-risk offenders. In Skeem

J. L.

Douglas

K. S.

Lillenfeld

S. O.

(Eds.), Psychological science in the courtroom: Consensus and controversy (pp. 358–384). Guilford.

36.

Specht

M. W.

Chapman

Cellucci

(2009). Schemas and borderline personality symptoms in incarcerated woman. Journal of Behavior Therapy and Experimental Psychiatry, 40, 256–264.

37.

Tabachnick

Fidell

(2013). Using multivariate statistics. Pearson.

38.

Trotter

(2013). Reducing recidivism through probation services: What we know and don’t know from four decades of research. Federal Probation, 77, 43–88.

39.

van Asselt

A. D.

Dirksen

C. D.

Arntz

Giesen-Bloo

J. H.

van Dijk

Spinhoven

. . . Severens

J. L

. (2008). Outpatient psychotherapy for borderline personality disorder: Cost-effectiveness of schema-focused therapy versus transference-focused therapy. British Journal of Psychiatry, 192, 450–457.

40.

Walters

G. D.

(2007). The latent structured of the criminal lifestyle: A taxometric analysis of the lifestyle criminality screening form and Psychological Inventory of Criminal Thinking Styles. Criminal Justice and Behavior, 34, 1623–1637.

41.

Wise

E. A.

(2004). Methods for analyzing psychotherapy outcomes: A review of clinical significance, reliable change, and recommendations for future directions. Journal of Personality Assessment, 82, 50–59.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.14 MB