Abstract
Group climate in residential youth care is considered to be essential for treatment of youth and young adults. Various instruments exist to measure quality of living group climate, but some are lengthy, use complicated wording, which make them difficult to fill out by youth and individuals with a mild intellectual disability. The present study describes the development and rationale for the Group Climate Instrument—Revised (GCI-R). Construct validity and reliability of the GCI-R were examined by means of Confirmatory Factor Analysis (CFA) in a two-step validation process using a construction sample (n = 190 youth, representing 41 groups) and a validation sample (n = 207 youth, representing 42 groups). Results indicated a good fit of a five-factor model (Support, Growth, Physical Environment, Peer interactions, and Repression). Reliability of the scales was good. These findings indicate that the GCI-R can be used as a parsimonious, valid, and reliable instrument to assess perceptions of group climate in youth. Recommendations for future research and practice are suggested.
The social climate or group climate in residential facilities has been the subject of research since “The Other 23 Hours” by Trieschmann et al. (1969), and is widely recognized as an important factor in treatment and rehabilitation of youth in residential care. Over the past two decades, there has been increasing interest in research and practice, aiming to better understand social climate and develop good practice regarding process monitoring of the social climate in various populations and treatment settings. There are several conceptualizations of social climate in residential facilities—also referred to as ward atmosphere, prison social climate, institutional climate, and (living) group climate—depending on theoretical orientation, the type of institution, as well as the unit of measurement (Eltink, 2020; Van Ginneken & Nieuwbeerta, 2020). Currently, there is relative consensus on what elements constitute the social climate, and several instruments that assess social climate have been developed (Eltink, 2020; Tonkin, 2016). The present study describes the theoretical background and validation process of a self-report measure to assess perception of group climate in residential care, the Group Climate Instrument—Revised (GCI-R).
In the Netherlands, residential youth care may be seen as a continuum of 24/7 out-of-home care varying from open residential to secure residential care and youth prisons (Grietens et al., 2015; Leloux-Opmeer et al., 2017). Youth are referred to residential care services for complex and persisting psychosocial and behavioral problems for whom home-based care has not been sufficient. In secure care settings, placement of youth is mandated by a judge (civil measure), either because youth may be a danger to their environment or themselves, or youth are convicted for delinquent behavior. In all settings, youth receive pedagogical care, education, and treatment. Notably, even in (youth) prisons, one of the main goals is rehabilitation in order reduce the risk for criminal recidivism. Therefore, residential youth care is also referred to as therapeutic residential care (TRC) as defined by J. W. Whittaker et al. (2014): “Therapeutic residential care involves the planful use of a purposefully constructed, multi-dimensional living environment designed to enhance or provide treatment, education, socialization, support, and protection to children and youth with identified mental health or behavioral needs in partnership with their families and in collaboration with a full spectrum of community based formal and informal helping resources” (p. 24 in J. K. Whittaker et al., 2016).
In a recent meta-analytic review, Eltink (2020) identified several dimensions of (residential) group climate: (1) supportive and responsive behavior by staff, (2) opportunities for growth and learning, (3) a structured environment with clear rules and regulations, (4) safety of clients against physical and psychological harm, (5) justice and fairness, (6) social interactions between clients, and (7) Repression exercised by staff members. Stams and Van der Helm (2017) define living group climate as “the quality of the social and physical environment with regard to sufficient and necessary conditions for physical and mental health, recovery and personal growth of clients, with due regard for their human dignity, human rights, but also (insofar as not limited through legal measures) with due regard for their personal autonomy, with the aim of successful participation in society” (p. 4). This operationalization of living group climate is in line with basic psychological needs of self-determination (relatedness, autonomy, and competence) as outlined by Ryan and Deci (2017) and assumed to be necessary conditions for treatment motivation and recovery. This definition is particularly relevant in (residential and/or correctional) settings where youth or young adults actively receive treatment aiming at rehabilitation into society.
Living group climate can vary from closed and repressive to open and therapeutic. An open and therapeutic living group climate is characterized by a safe, structured, and rehabilitating environment in which there is support from staff, clear opportunities for growth, minimal repression, and in which a good balance between flexibility and control ensures positive interactions among group members and staff members. Research has shown that an open living group climate contributes to higher levels of treatment motivation, increased empathy, greater internal locus of control, more emotional stability, and better coping strategies in clients as well as fewer aggressive incidents (De Decker et al., 2018; Heynen et al., 2017; Neimeijer et al., 2021; Ros et al., 2013; Van den Tillaart et al., 2018; Van der Helm et al., 2009; Van der Helm, Stams, & Van der Laan, 2011, for a complete overview see: Leipoldt et al., 2019).
Research on group climate provides insight in the perception of the living group climate by youth and young adults who stay in a (forensic) residential institution (Stams & Van der Helm, 2017). Administering questionnaires, discussing the results with clients and staff appear to work as an intervention to improve group climate (Levrouw et al., 2018; Stams & Van der Helm, 2017). Consequently, specific goals and actions are jointly formulated and evaluated. In doing so, a positive living group climate can be established through practice-oriented research (Levrouw et al., 2018; Neimeijer et al., 2019; Strijbosch et al., 2019).
In the Netherlands, the Group Climate Instrument (Neimeijer et al., 2019, derived from the Prison Group Climate Instrument [PGCI], Van der Helm, Stams, & Van der Laan, 2011) is widely used in various (secure) youth care facilities, youth prisons, and also in adult forensic treatment facilities. Over de past decade, the GCI has also been used in other European countries and Australia (Heynen et al., 2014; Levrouw et al., 2018; Neimeijer et al., 2019; Strijbosch et al., 2013, 2019; Van Miert et al., 2021). Various other group climate measures have been developed too, which are mostly lengthy, use complicated wording, making them less suitable for youth and young adults with limited cognitive abilities, and may underrepresent the construct of living group climate. For example, the Ward Atmosphere Scale (WAS: Moos, 1968) and the EssenCES (Schalast et al., 2008) are frequently used social climate measures in adult health care (Dickens et al., 2022; Tonkin, 2016). The WAS comprises 100 items measuring three higher-order factors “Relationship,” “Personal growth,” and “System maintenance.” The EssenCES consists of 15 items measuring three factors “Therapeutic hold,” “Patients” “cohesion and mutual support,” and “Experienced safety.”
The original PGCI consisted of 50 items, measuring four aspects of group climate (support, growth, repression, and atmosphere). A version consisting of 36 items with highest factor loadings has been widely used in practice. The GCI (Neimeijer et al., 2019) has been adapted with the aim of simpler wording of items, resulting in a 29-item version for use in adult populations in forensic care. Several studies have suggested that the (P)GCI would benefit from further development, specifically with regard to the Repression and Atmosphere scale (De Vries et al., 2018). In the light of recent theoretical and empirical studies as well as based on feedback from clinical practice, the GCI has been revised (hence, GCI-R), distinguishing five aspects of living group climate, namely support, growth, physical environment, peer interactions, and repression.
Responsivity and Support
Responsiveness and supportive behavior of staff are characteristic of a therapeutic attitude toward the client in which attending to the clients’ needs is central (Van der Helm et al., 2009; Van der Helm, Stams, & Van der Stel, et al., 2011). The need for connection with others arises from a biological, psychological, and social human need and is a key component of self-determination theory (Ryan & Deci, 2017). In a therapeutic setting, a good working alliance (i.e., collaborative relationship consisting of an affective bond and task collaboration) between care provider and client is an important condition for promoting recovery in both adults and youth (Norcross & Lambert, 2019). Further, the working alliance concept in residential youth care can be conceived as a collaborative affective relationship of mutual trust, understanding, empathy, and genuineness, with a focus on providing a safe environment, and on the other hand as a collaboration to achieve therapeutic or behavioral change, focusing on strengths, resilience, and self-efficacy by working together with youth and parents on therapeutic or developmental tasks and goals that are meaningful for youth.
An affective bond may be established when professionals are genuinely interested in clients, their developmental history, developmental needs, and cultural values. Professionals should invest in building this alliance with clients, which may often take time to develop and maintain. An important condition for support and responsiveness is that professionals are physically present at the living group, and that professionals make time for clients and are accessible. Providing support in residential care also entails providing structure. This requires professionals to strike a balance between flexibility and control. Flexibility can be described in terms of responsiveness and providing opportunities for growth through autonomy granting. Control refers to structure, predictability, security, and effective use of rules (Clark Craig, 2004). Keeping the balance between flexibility and control is one of the most important tasks for professionals. In the context of residential care this means, among other things: Professionals are available and physically and mentally present (Baart, 2004; Schaftenaar 2018), instead of spending too much time in the office, busy with other clients or reporting activities. Punishment or threat of punishment can lead to mistrust and avoidance (punitive capability, De Dreu et al., 1998), possibly resulting in clients’ perception of staff as the “enemy.” Systems that focus on “shaping” behavior instead of building trust hinder a working alliance and support (Hachtel et al., 2019; Van der Helm, 2019). Cultural factors can also influence the working alliance (Sevilir et al., 2020). Responsiveness and understanding of clients’ cultural origins are important to build trust. This requires a cultural sensitive approach by staff.
Growth
Growth refers to competence (Ryan & Deci, 2017) and opportunities to learn. In addition, it refers to the conditions of the stay that make it possible for clients to learn (Van der Helm et al., 2009; Van der Helm, Stams, & Van der Laan, 2011). Shared decision-making is an important aspect of providing clients with opportunities for growth. This means that professionals make decisions about the treatment of the client together with clients (Ten Brummelaar et al., 2018). Every client, regardless of age, has the right to express his or her opinion. The premise is to follow the preferences of the client, although this is not always possible—for example when safety and security are at stake. In addition, treatment will only be effective when professionals create conditions in which the client has confidence in the effectiveness of the treatment and also in their own ability to move forward (King et al., 2014). To achieve this, it is desirable that professionals explain all aspects of residential treatment to clients in a way that is understandable to them. When choices are made in treatment, all parties must be aware of the options. When the professional makes a different choice, it must be clearly explained why and what the consequences are. In doing so, professionals should be cognizant of the risk of overestimating youth’s abilities to fully comprehend all treatment options and their consequences given age-limited cognitive abilities and possible overestimation of their own competences (Owens et al., 2007).
Further, promoting growth opportunities requires a supportive stance by professionals, communicating openly and honestly with clients, showing genuine interest in clients, and initiating conversation with attention to the internal state of mind and emotions of the client. During this process, stimulating the youth’s autonomy is important, by giving the youth the opportunities to make mistakes, to solve problems, and to learn prosocial behavior. Professionals are expected to support this by striking a balance between protection of the client on the one hand, and stimulating autonomy on the other (De Wit et al., 2011). By stimulating the problem-solving capacity of clients, they acquire more autonomy and competence in the future.
Institutional Repression
Article 5 (1) (d) of the European Convention on Human Rights (ECHR) and Article 37 (b) of the International Convention on the Rights of the Child (CRC) stipulate that minors can be placed in secure care, as long as they are not deprived of their liberty in an illegal or arbitrary manner. If there is a risk that the client will withdraw from care, a stay in a residential institution may be considered necessary (Article 6.1.2 Youth Act). Aspects inherent in staying in a residential setting are power, structure, and coercion. Unlawful or arbitrary use of power, structure, and coercion are seen as repressive.
The most recent and comprehensive definition of institutional repression is given by De Valk et al. (2016): “Repression is an authority figure who deliberately harms a young person or an authority figure who is illegal or arbitrary. limits the freedom / autonomy of the young person” (p. 14). Repressive behavior of staff negatively impacts the quality of the social climate, frustrating clients’ psychological needs. Professionals can exercise repression consciously—for example out of countertransference, revenge, or pleasure—or unconsciously, through, for example, professional helplessness, fear, or ignorance. Repression can take on “hard” forms, such as prolonged seclusion, afflicting pain, unnecessary restraint, group punishment, inappropriate sexual behavior and sexist jokes, social exclusion and ignoring clients, and confiscating personal belongings as means of control or punishment. Also, more subtle (“softer”) forms of repression may occur, such as too many, unfair or too intrusive rules as perceived by youth, relating youth’s rights such as the right to go to school to good behavior, unfair or unequal treatment, threatening (if. . ., then. . .), and humiliation (De Valk et al., 2016, 2019).
Recent studies on coercion and the quality of group climate show that on the one hand coerced (mandatory) treatment can work, if there is an authoritative positive climate marked by trust and cooperation in which basic psychological needs of clients are met (De Valk et al., 2015; Hachtel et al., 2019; Meijers, 2018; Van der Helm et al., 2018). Hachtel et al. (2019) state that mandatory treatment not necessarily results in perceived coercion, and that a caring and authoritative treatment style contributes to a positive working alliance, more motivation for treatment, and better therapy outcomes. If conditions for a positive group climate are not met, higher levels of coercion may result in higher levels of aggressive incidents and higher rates of recidivism (Chieze et al, 2019; De Valk et al., 2015, 2019; Hachtel et al., 2019; Ros et al., 2013; Roy et al., 2020; Schaftenaar, 2018; Van der Helm et al., 2018).
Peer Interactions
The extent to which clients feel comfortable and safe at the group depends on the interactions among clients. Positive interactions can make an important contribution to connectedness and competence and may thus lead to motivation for treatment (Van der Helm et al., 2018). Recent studies by Sonderman et al. (2020) have shown that two aspects of peer interaction can be distinguished, namely Support and Acceptance and Relational Aggression. Accepting each other is an important condition for positive interactions. Negative interactions such as threatening each other, teaching each other rule-breaking behavior, and provoking each other may result in chaos and anarchy at the group, and is detrimental to the recovery and rehabilitation of clients. Institutions are faced with the challenge of understanding defense and coping mechanisms of clients (fleeing, fighting, and freezing), to initiate and restore positive interactions, and provide a safe and supporting peer culture.
Physical Environment
Several studies show that a stimulating and enriched environment contributes to cognitive development, less psychopathology, and recovery (Kinnaficka & Thøgersen-Ntoumanib, 2014; Klitzman & Stellman, 1989; Raine, 2013). At the living group, an environment that is pleasantly furnished and clean with sufficient outdoor space, fresh air and a place where someone can withdraw is assumed to positively affect clients’ well-being. Deprivation of a pleasant physical environment due to broken furniture and dirty rooms signal loss of control and cause stress (Van der Helm, 2019).
Present Study
This study describes the development and two-step validation process of the GCI-R. A specific aim of the study was to increase conceptual clarity of the GCI-R. The “Atmosphere” scale now distinguishes between social interactions (peer interactions) and the perception of the physical environment in separate scales. Also, the “Repression” scale contains items referring to unfair rule enforcement, perceived power imbalance between youth and staff, and coercive behavior of staff. Further, practice-oriented research on assessment and improvement of group climate often uses aggregated scores at the group level (Levrouw et al., 2018; Strijbosch et al., 2019). However, as self-report measures of group climate assess individual perception of group members, it is unclear whether aggregated scores at the group level represent a valid and reliable score of group climate. Therefore, in the present study, interrater reliability and within-group agreement of the GCI-R scales were also examined.
Method
Participants
The construction sample consisted of N = 190 youth (67.4% male); mean age of participants was 16.78 years (SD = 2.22, Min = 12, Max = 24), and 78.4% of the youth had the Dutch nationality. The sample included youth from four (semi-)secure youth care institutions and three youth prisons in the Netherlands. Youth represented 41 living groups with an average cluster size of 5 (range = 1–9 youth per group).
The validation sample consisted of N = 207 youth (70.5% male); mean age of respondents was 17 years (SD = 2.35, Min = 12, Max = 26). The sample included youth from five (semi-) secure youth care institutions and three youth prisons in the Netherlands. Youth represented 42 living groups with an average cluster size of 5 (range = 1–9 youth per group). The design of the present study was cross-sectional, and the samples were independent.
Procedure
Data were obtained from September 2017 until February 2018 for the construction sample, and from September 2018 until October 2019 for the validation sample. All youth filled out the newly developed version of the GCI-R as well as the original version of the PGCI. All youth participated voluntarily and signed an informed consent form. Questionnaires were assigned a number in order to guarantee anonymity of the participants. Participants were told their answers would only be accessed by the researchers, and were ensured that teachers and other staff members would not have access to their answers. Participants received a small token of gratitude (showering gel) for their participation in the study, after they had completed the questionnaire. Permission to conduct this study was obtained from the Ethical Commission of University of Applied Sciences Leiden.
Measures
Development of the Group Climate Instrument—Revised
Various steps were taken to develop the GCI-R in an iterative process. The research team critically reviewed item content, based on feedback from practitioners and clients. Also, several self-report measures institutional repression (De Valk, 2019) and peer interactions in residential youth care (Sonderman et al., 2020) that were under development at that time were studied, resulting in a preliminary item pool for further discussion in focus groups with practitioners, experts, and youth.
Several focus groups were organized with youth and professionals in three different facilities. Next, an expert meeting was held in which professionals associated with different facilities provided feedback on the content of existing scales and items. The feedback pertained to the extent to which the item content adequately captured the concepts “atmosphere” and “repression.” In the original version of the GCI, the items of the Atmosphere scale referred to feelings of mutual trust and safety among youth as the perception of the environment (i.e., “We trust each other here,” “I get some peace of mind at the group,” and “We have enough fresh air and daylight”). Also, items in the repression scale differed widely in content referring to perception of strictness and haphazard rules, lack of trust, lack of stimulation, or deprivation (i.e., “They don’t understand me here” and “We have nothing to do here”). Therefore, based on this feedback, an important aim of the revision process of the GCI was to increase conceptual clarity of the scales, such that social interactions among youth and the perception of the physical environment would be captured in the different scales. The Repression scale was improved by adding items referring to unfair rule enforcement, perceived power imbalance between youth and staff, and coercive behavior of staff. Further, feedback was provided on the complexity and length of the items. Therefore, special attention was paid to the wording of the items to ensure that the questionnaire would be suitable for clients with a mild intellectual disability or autism spectrum disorder.
This resulted in an item pool that was used as a pilot version of the GCI-R consisting of 101 items. The items were formulated from the perspective of the youth. Using a 5-point Likert type scale, youth were asked to indicate to what extent the statement applied to him or her. For the response options, the following wording had been chosen: Strongly disagree, disagree, neither agree nor disagree, agree, and strongly agree. This version of the questionnaire was once again presented to members of the expert group, who gave feedback on the wording of the items.
In 2017, the pilot version of the GCI-R was administered during a first round of data collection in seven institutions; four (semi-)secure youth care facilities and three youth prisons. Youth completed the GCI-R in addition to the original PGCI (Van der Helm, Stams, & Van der Laan, 2011) and received a token of gratitude in the form of a shower gel for participating. A total of 190 youth representing 41 different living groups participated in the study. Based on a series of confirmatory factor analyses, we first looked at which items were suitable for use in the individual scales. The content of the items, the factor loadings, and item-total correlations were examined. Subsequently, model fit of a multifactor model was examined, consisting of five factors: Support, Growth, Peer interactions, Physical environment, and Repression. Of the 101 items, 45 were retained for use in a second round of data collection.
In addition to filling out the questionnaire, youth were asked to evaluate all aspects of group climate using a single item rating (report mark between 1 [very poor] and 10 [excellent]) for all aspects of group climate, for example: “How do you rate the support from staff at the group?” or “How do you rate the possibilities for learning at the group?”. With regard to repression, the single item rating focused on the quality of the rules: “How do you rate the rules at the group?”. Also, youth were asked to rate the safety at the group using a single item rating: “How do you rate the safety at the group?”.
Group Climate Instrument
The Group Climate Instrument (GCI) is based on the Prison Group Climate Instrument (PGCI; Van der Helm, Stams, & Van der Laan, 2011). The GCI measures various aspects of living group climate and consists of 36 items rated on a 5-point Likert type scale, ranging from 1 (I do not agree) to 5 (I totally agree). The GCI consists of four subscales: Support, Growth, Atmosphere, and Repression. The scale Support (12 items) measures the extent to which the youth feel supported by the group workers in terms of trust, respect, and taking complaints seriously. An example of a support item is “Group workers treat me with respect.” The Growth scale (eight items) measures the extent to which the youth find their stay in the institution meaningful, feel they are able to learn things during their stay, and have hope for the future. An example of a growth item is: “I learn the right things here.” The Repression scale (nine items) measures the youth’s experience of strictness and control by staff, and unfair and haphazard rules. The Atmosphere scale (seven items) measures how youth perceive the interactions with peers at the group, whether they can trust each other, being able to get some peace of mind at the group, but also whether they get enough daylight and fresh air. An example of an atmosphere item is “We trust each other here.” Higher scale scores are indicative of more support, growth, a positive atmosphere, and more repression.
Statistical Analyses
Using the construction sample, construct validity was examined by means of confirmatory factor analysis (CFA), using the lavaan package (Rosseel, 2012) in the R environment (version 4.2; R Core Team, 2022). A CFA was conducted to examine a five-factor model. To account for non-normally distributed ordinal variables, the mean and variance-adjusted weighted least squares (WLSMV) estimation procedure was chosen. Pairwise deletion was used to handle missing data. Modification indices, giving the expected drop in chi-square if the parameter in question is freely estimated, were inspected to improve model fit if needed. In doing so, only residual variances of items that were similarly worded were allowed to correlate. Due to the nested nature of the data (youth within groups), multilevel confirmatory factor analysis (MCFA) was considered. MCFA allows the specification of factor models at the within-group level (i.e., variability in perception of the individual youth) and at the between-level (i.e., the group in which the youth resides). However, the sample size was limited, particularly with respect to the number of groups at the between-level. Specification of MCFA models resulted in convergence problems.
Model fit was evaluated by various model fit indices; comparative fit index (CFI), Tucker-Lewis index (TLI), and the root mean square residual (RMSEA). Cut off values of CFI > 0.95, TLI > 0.95, RMSEA < 0.05, and SRMR < 0.08 are required for good model fit, and values of CFI > 0.90, TLI > 0.90, RMSEA < 0.08, and SRMR < 0.10 are indicative of acceptable model fit (Hu & Bentler, 1999; Kline, 2015).
Intraclass correlations (ICC) of the scales were computed to examine the amount of variability between groups as well as the interrater reliability of scores between group members (Biemann et al., 2012). The ICC-1 value refers to the proportion of variance explained by the group in which youth reside (i.e., heterogeneity between groups). ICC-2 (interrater reliability) refers to the extent to which an individual score can be considered a reliable assessment of a group-level construct. Also, within-group rater agreement for multi-item measures (rWG(J); James et al., 1984) was computed to examine the agreement in scores across group members on all items of a scale. ICC-2 and rWG(J) ≥ .70 are generally accepted as good (Bliese, 2000). ICC-1, ICC-2, and rWG(J) were calculated using the bruceR package (Bao, 2023).
Convergent validity was examined by calculating Pearson’s r correlations between the scales of the GCI-R and the original GCI scales, as well as between the GCI-R scales and single item ratings (between 1 and 10). A positive moderate to strong correlation between the scales Support, Growth, Peer interactions, Physical Environment and the corresponding report marks, as well as a negative moderate to strong correlation between Repression and the report mark regarding the fairness of the rules would be indicative of convergent validity. Pearson’s correlations of r = .10 to .30 are seen as small, r = .30 to .50 are seen as a moderate, and r > .50 are seen as a large (Cohen, 1992).
Results
Descriptive Statistics
In the construction sample, missing values for the individual items ranged between 1 (0.5%) and 18 (9.5%). Little’s MCAR test (χ22010 = 2,206.81, p < .001) indicated that missing data were not completely at random. In the validation sample, missing values for the individual items ranged between 2 (1.0%) and 20 (9.7%). Little’s MCAR test (χ22290 = 2,408.44, p = .042) was significant, indicating that missings were not missing completely at random. All skewness and kurtosis values of the items in both samples were within an acceptable range. Descriptive statistics of all items for both samples are depicted in the Supplemental Appendix.
Confirmatory Factor Analysis
First, a single level CFA was conducted using the construction sample. A five-factor model was specified, consisting of Support, Growth, Peer interactions, Physical environment, and Repression. Initial fit of the model was acceptable: χ2 454 = 1,055.54; p < .001, CFI = 0.92, TLI = 0.91, RMSEA = 0.084 (90% CI [0.077, 0.090]). Factor loadings of the items were generally higher for the Support and Growth factors (range between 0.78 and 0.92) compared to the Peer interactions, Physical environment, and Repression factors (range between 0.48 and 0.81 for positively worded items and −0.53 to −0.89 for negatively worded items). Reliability of the factors was good, with Cronbach’s alpha for Support α = .93, Growth α = .92, Peer interactions, α = .86, Physical environment α = .72, and Repression α = .71. The factors Support, Growth, Peer interactions, and Physical environment were moderately to strongly correlated with each other. The factor Repression was moderately and inversely correlated with other factors, and not significantly correlated with Physical environment (Table 1).
Descriptive Statistics, ICCs, and Correlations for the GCI-R Scales (Construction Sample).
Note. ICC = intraclass correlation coefficient; rWG(J) = within-group rater agreement for multi-item measures.
p < .01.
The 5-factor model found in the construction sample was replicated using the validation sample. A five-factor model at the within-group level showed a good fit to the data: χ2454 = 895.66; p < .001, CFI = 0.94, TLI = 0.93, RMSEA = 0.069 (90% CI [0.062, 0.075]). Factor loadings of the items were consistent with findings from the construction sample. Reliability of the factors was good and also consistent with findings from the construction sample, with Cronbach’s alpha for Support α = .92, Growth α = .92, Peer interactions, α = .86, Physical environment α = .74, and Repression α = .82. The factors Support, Growth, Peer interactions, and Physical environment were moderately to strongly correlated with each other. The factor Repression was moderately and inversely correlated with other factors (Table 2).
Descriptive Statistics, ICCs, and Correlations for the GCI-R Scales (Validation Sample).
Note. ICC = intraclass correlation coefficient; rWG(J) = within-group rater agreement for multi-item measure.
p < .01.
Convergent Validity
Convergent validity was examined in both samples by calculating Pearson’s r correlations between the GCI-R scales and the single item ratings for different aspects of group climate. Results indicated positive and strong correlations between the scales and the corresponding single item rating in the construction sample and validation sample, respectively: Support (r = .76 and .74), Growth (r = .79 and .81), Peer interactions (r = .73 and .70), and Physical environment (r = .45 and .65). Also, Repression (r = −.47 and −.62) was strongly and inversely correlated with the single item rating “Quality of the rules.”
Also, Pearson’s r correlations were examined between the GCI-R scales and the single item rating for perceived safety at the group. Correlations were weak to moderate in the expected direction in the construction sample and validation sample, respectively: Support (r = .37 and .34), Growth (r = .31 and .30), Peer interactions (r = .54 and .56), Physical environment (r = .30 and .30), and Repression (r = −.21 and −.19).
Convergent validity was examined by calculating Pearson’s r correlations between the scales of the GCI-R and the corresponding scale of the previous version of the GCI. Of note, the scales Peer interactions and Physical environment were correlated with the Atmosphere scale of the PGCI. Results indicated positive and strong correlations between the scales in the construction sample and validation sample, respectively: Support (r = .70 and .82), Growth (r = .74 and .82), Peer interactions (r = .51 and .72), Physical environment (r = .28 and .45), and Repression (r = .35 and .69).
ICC(1)s were calculated for the GCI-R scales to examine the amount of variability in scores between groups, that is, to what extent variation in scores could be attributed to individual perception and to the group level. The Support scale demonstrated relatively small ICCs, indicating that 3% of variance in scores could be attributed to the group level. The Growth scale also demonstrated a small ICC in the construction sample (.02), but larger in the validation sample (.14). ICCs of the other scales (Peer interactions, Physical environment, and Repression) were larger, suggesting that between 8% and 19% of variation of scores could be attributed to the group level. Further, ICC-2 and rWG(J) values were calculated. ICC-2, referring to the reliability of scores as a group construct were somewhat larger in the validation sample compared to the construction sample, indicating poor reliability for all scales (values <.50) in both samples. rWG(J) values were also somewhat larger in the validation sample compared to the construction sample, indicating fair to good within-group agreement on items of the scales in both samples, except for the Growth scale. ICC-1, ICC-2, and rWG(J) values for all scales are depicted in Tables 1 and 2 for the construction and validation sample, respectively.
Discussion
The present study examined the psychometric properties of the Group Climate Instrument—Revised (GCI-R). Building on previous work on the PGCI and GCI, theoretical underpinnings of the instrument were reviewed increasing conceptual clarity of the scales, thus content validity. Confirmatory factor analyses were used to examine the theory-derived factors in a multifactor model (factorial validity), convergent validity was examined, and reliability of the scales including within-group agreement and stability of aggregated group scores were investigated.
In a two-step validation process in two samples of adolescents and young adults in (forensic) residential youth care facilities, a 5-factor model consisting of Support, Growth, Physical Environment, Peer interactions, and Repression demonstrated good fit to the data. Factor loadings were generally higher for the Support and Growth factors compared to other factors, but consistent across samples. Higher factor loadings for the Support and Growth scales are also consistent with findings from previous studies on the GCI for both youth (Van der Helm, Stams, & Van der Laan, 2011) and adult samples (Neimeijer et al., 2019). Convergent validity was demonstrated in strong correlations in the expected direction with single item ratings of the constructs as well as with the previous versions of the scales. Of note, correlations between the Physical environment scale and Atmosphere scale were moderate, which was desirable given the conceptual differences between the newly constructed scales and previous scales. Reliability of the factors was good.
The GCI-R scales Support, Growth, and Physical environment were weakly to moderately and positively correlated with perceived safety at the group. This finding demonstrates that perception of group climate is associated with youth’s perceived safety at the group. It has been argued that safety at the group is an important feature of social climate, and therefore should be included in self-report measures of social climate (Boone et al., 2016; Bosma et al., 2020; De Vries et al., 2018; Schalast et al., 2008). We of course agree that providing a safe environment for youth in residential care is imperative. Also, there is a paucity of research on youth’s perception of safety in residential care, and measures to routinely assess safety from the youth’s perspective seem to be lacking. From a therapeutical perspective, we argue that assessment of perception of safety in youth should be conducted individually along with an interview by a trained professional.
Our findings further indicated that for Support and Growth, a relatively small portion of the variance could be attributed to the group, whereas for Peer interactions, Physical environment, and Repression, the amount of variance attributed to the group was larger. This could be explained by the fact that the items of the Support and Growth scale are formulated from a first person perspective, whereas items of the other scales ask youth to account for the perspective or behavior of other youth (peer interactions), ask about the surroundings (Physical environment), or interactions between group workers and youth at the group (Repression). This finding illustrates that in capturing group climate or social climate using self-report measures, it is relevant to account for the perspective from which items are worded. Future studies on group climate measures may therefore examine whether items formulated from a first-person perspective and from the perspective of group members result in different findings, which may also be relevant for how group scores are interpreted.
An important limitation of the present study pertains to the conceptualization of group climate. It should be noted that a distinction should be made between group climate as a characteristic of the group versus youth’s perception of group climate. Because of the nested nature of the data, perception of group climate varies across youth, groups, and facilities may vary in group climate scores. Thus, perceptions of group climate of youth from the same group are expected to be more similar than the perception of youth from different groups. Multilevel factor analysis (MCFA) can be used to examine the factor structure at the within-group level (i.e., individual level) and between-group level (i.e., group level). The sample size in the present study, particularly the number of groups in relation to the complexity of the factor model, proved to be insufficient to employ MCFA, resulting in convergence issues.
In practice, individual scores of group climate are often aggregated into a group score, whereas factorial validity of group climate measures are often based on regular factor analytic techniques that do not take into account the nested nature of the data. Our findings suggest that within-group rater agreement on the items of the scales was good, except for the Growth scale. Although the GCI-R can be used to measure perception of group climate, it is important to further study the factorial structure of the GCI-R using multilevel factor analysis to investigate to what extent aggregated group climate scores are indeed a valid assessment of group climate as a group construct. Future studies should be conducted using larger sample sizes to examine multilevel factor models in which variation between groups as well as variation between facilities are modeled. Also, measurement invariance should be examined regarding potential differences in perceived group climate between groups based on characteristics of youth (e.g., gender and age) as well as characteristics of types of facilities (e.g., open vs. secure settings).
Another limitation is that we used a single item measure to assess convergent validity, because these items might not capture all relevant aspects of the different group climate factors. In the present study we chose to examine correlations with the original version of the GCI, given that this measure has been validated and measures the same constructs. Future studies should assess convergent validity of the GCI-R with other validated group climate measure, such as the Dutch version of the EssenCES (De Vries et al., 2018).
All in all, we conclude that the GCI-R is a comprehensive measure focusing on both social and physical elements of group climate. We conclude that, based on the findings of the present study, the GCI-R can be used as a valid and reliable measure of group climate in adolescent and young adult populations. Future studies are needed to examine whether the GCI-R can be used in adult populations. The GCI-R can be used in practice for process monitoring of group climate in residential care, enabling client feedback. Given that the GCI-R has been developed from a development-oriented perspective related to fulfillment of psychological needs, makes the measure particularly relevant for youth and young adults in a therapeutic residential care or rehabilitative treatment setting.
Supplemental Material
sj-docx-1-ijo-10.1177_0306624X231219984 – Supplemental material for Group Climate in Residential Youth Care: Development and Validation of the Group Climate Instrument—Revised
Supplemental material, sj-docx-1-ijo-10.1177_0306624X231219984 for Group Climate in Residential Youth Care: Development and Validation of the Group Climate Instrument—Revised by G. H. P.(Peer) van der Helm, Jesse J. Roest, Anna Leonora Dekker, Veronique Suzanne Lisette van Miert, Chris H. Z. Kuiper and Geert Jan J. M. Stams in International Journal of Offender Therapy and Comparative Criminology
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
