Abstract
Behavioral and emotional problems among children and adolescents can lead to numerous negative outcomes without intervention. From a prevention standpoint, screening for behavioral and emotional risk is an important step toward identifying such problems before the point of diagnosis or referral. The present study conducted a k-means cluster analysis to determine the subtypes of risk captured by one such screening instrument, the Behavioral and Emotional Screening System (BESS). The final solution produced four clusters: Well-Adapted, Internalizing/Adjustment Problems, Mild Externalizing Problems, and General Problems-Severe; these results were similar to those found with the full Behavioral Assessment System for Children, Second Edition (BASC-2), suggesting that the BESS assesses similar constructs. Predictive validity evidence suggested that cluster membership was associated with standard achievement scores and in-school disciplinary incidents.
Keywords
Behavioral and emotional problems account for more than 50% of referrals to school mental health professionals (Brestan & Eyberg, 1998; Lahey, Miller, Gordon, & Riley, 1999). An important step in the identification of such difficulties is classification, which is often done in the school setting to determine eligibility for services such as special education (Dever & Kamphaus, 2013). Although classification typically involves diagnosis of presence or absence of disorder, many researchers and clinicians argue that the dichotomous nature of a categorical model fails to adequately capture differences in the degree, severity, and manifestations of behavioral and emotional problems among youth (Achenbach & McConaughy, 1992; Kamphaus, Rowe, Dowdy, & Henry, 2006). In addition, high levels of comorbidity of mental health problems indicate the need to consider a more dimensional approach of classification that reflects the presence of a continuum of behavioral and emotional difficulties, as opposed to a binary system (Clark, Watson, & Reynolds, 1995; Wittchen, 1996).
Person-centered approaches to analysis of behavioral and emotional assessment instruments allow for the exploration of patterns of mental health strengths and difficulties along a continuum. As opposed to variable-centered analyses that focus on relationships among variables, person-centered analyses focus on how constructs tend to group together within individuals to form clusters or profiles. For example, the Behavioral Assessment System for Children, Second Edition (BASC-2; Reynolds & Kamphaus, 2004) is a commonly-used instrument that has been examined using cluster analytic techniques (DiStefano & Kamphaus, 2006; DiStefano, Kamphaus, & Mindrila, 2010). DiStefano and Kamphaus found six distinct clusters of students (well-adapted, average, disruptive behavior problems, learning problems, general problems-severe, and mildly disruptive) using the first edition of the BASC teacher rating form. DiStefano et al. replicated this analysis with the BASC-2 teacher rating form, and identified seven clusters of students, some overlapping with the previous findings (well-adapted, average, disruptive behavior problems, academic problems, worry/physical complaints, clinical internalizing, and clinical externalizing).
Although omnibus assessments of behavioral and emotional problems such as the BASC-2 are commonly administered in schools, the staggering number of students experiencing mental health issues has resulted in a push toward early identification efforts. As a result, universal mental health screenings in schools to identify students who are at-risk of behavioral and emotional problems have increased in popularity (Bruhn, Woods-Groves, & Huddle, 2014; Stiffler & Dever, 2015). Broadband screening tools can usually be administered in less than 15 min, and provide an indication of a student’s risk of general behavioral and emotional problems, rather than for a specific disorder. Screening outcomes both identify previously undetected cases of behavioral and emotional problems (Vander Stoep et al., 2005) and predict future diagnosis of mental health disorders with adequate accuracy (Najman et al., 2008). Early identification of behavioral and emotional risk can also mitigate or prevent negative outcomes such as school dropout and lower academic achievement (Brestan & Eyberg, 1998; Chin, Dowdy, & Quirk, 2013).
Mental health screening assessments are well-aligned with the dimensional approach to classification, as the goal is to detect subsyndromal psychopathology that is functionally impairing but below diagnostic thresholds (Hudziak, Wadsworth, Heath, & Achenbach, 1999). However, such screening instruments are still typically used in a categorical fashion, with students being classified as either “at-risk” or “not at-risk.” The Behavioral and Emotional Screening System (BESS; Kamphaus & Reynolds, 2007) is a brief screening tool that was developed from the BASC-2 standardization item pool to identify behavioral and emotional risk among school-aged youth. Despite being from the same family of assessments, the BESS has not been analyzed using person-centered analyses to determine the typology of the subsyndromal behavioral and emotional risk it assesses. The purpose of the present study was to determine whether the BESS screener identifies similar clusters of students to the full BASC-2. It was hypothesized that the BESS would demonstrate a similar cluster solution to the BASC-2; however, one or more clusters with milder problems were expected, with less differentiation in terms of problem type. Predictive validity was examined by using the clusters as predictors of behavioral and academic outcomes.
Method
Participants and Procedure
A total of 1,935 students across Grades 3 to 8 in six schools (four elementary, two middle) in the Northeast completed the Behavioral and Emotional Screening System Student self-report form (BESS Student). Passive parental consent was used by the six schools, and less than 1% chose not to participate. Forty-nine percent of participating students were female; 61% were Hispanic, 24% Caucasian non-Hispanic, 12% Black/African American, and 3% Other races/ethnicities. Data were collected as part of a universal screening program for behavioral and emotional risk; screening instruments were completed during the students’ homeroom periods using scannable forms.
Measures
The BASC-2 BESS Student consists of 30 items to assess for behavioral and emotional risk among students in Grades 3 through 12 (Kamphaus & Reynolds, 2007). Each item on the BESS Student uses a four-point response scale (never, sometimes, often, almost always). Previous research (Dowdy et al., 2011) indicated that the BESS Student consists of four factors: inattention/hyperactivity, internalizing problems, school problems, and personal adjustment. The psychometric properties of the BESS Student version are generally acceptable, including adequate split-half reliability (.90-.93) and test–retest reliability (.80). Student demographic characteristics, office disciplinary referrals (ODRs), suspensions, and Pennsylvania System of School Assessment (PSSA) scores were obtained from the district.
Results and Discussion
Due to the large sample size, a k-means cluster analysis procedure was used (Aldenderfer & Blashfield, 1984). The four subscales were standardized prior to clustering, and were used as the clustering variables. Cluster solutions from 2 to 7 clusters were considered based on work with the full BASC and BASC-2 (e.g., DiStefano & Kamphaus, 2006). The final solution resulted in four clusters: Well-Adapted, Internalizing/Adjustment Problems, Mild Externalizing Problems, and General Problems-Severe (Table 1). Chi-square analyses revealed significant differences in the composition of the clusters by race/ethnicity, χ2(18) = 42.52, p < .001; English language learner status, χ2(3) = 10.12, p < .05; gender, χ2(3) = 22.26, p < .001; special education status, χ2(3) = 10.23, p < .05; and risk classification on the BESS, χ2(6) = 1451.39, p < .001. Hispanic students were overrepresented in the Internalizing/Adjustment Problems and General Problems-Severe clusters, whereas African American students were overrepresented in the Mild Externalizing Problems cluster. Females and English language learners were more likely to be classified in the Internalizing/Adjustment Problems cluster, whereas students in special education were more likely to be in the General Problems-Severe cluster. Almost all (99%) of those in the General Problems-Severe cluster were classified as at-risk on the BESS, and none of those in the Well-Adapted cluster were considered at-risk. The BESS classified 39.5% of students in the Internalizing/Adjustment Problems cluster and 2.3% of those in the Mild Externalizing Problems cluster as at-risk of behavioral and emotional problems.
Descriptive Information by Cluster on Demographics, Subscales, and Outcomes.
Note. BESS = Behavioral and Emotional Screening System.
Tests with same superscript for each outcome measure were not statistically significant.
All other tests above were statistically significant, p < .05.
Predictive validity evidence suggested that cluster membership was associated with standardized achievement scores (Math and English Language Arts) and ODRs (Table 1). Those in the Well-Adapted cluster had higher achievement scores than the other three clusters. Students in the Mild Externalizing Problems and General Problems-Severe clusters had higher disciplinary incidents than the other two groups; however, those in the General Problems-Severe cluster had more suspensions than those in the Mild Externalizing cluster.
Results of this study identified four clusters of students using the BESS Student, each characterized by distinctive patterns of behavioral/emotional risk. Differences in risk classification and behavioral and academic outcome measures provided validity evidence for these subgroups. Consistent with previous studies (DiStefano & Kamphaus, 2006; DiStefano et al., 2010), the clusters that emerged from the brief BESS form (i.e., Well-Adapted, Internalizing/Adjustment Problems, and General Problems-Severe) showed a high degree of overlap with those identified by the full BASC-2, suggesting that the less comprehensive BESS may effectively assess similar constructs. However, a milder externalizing category was found in our study, with few students in this cluster being identified as at-risk on the BESS; however, these students were involved in more disciplinary incidents and had lower achievement test scores than the Well-Adapted group. This finding suggests that the BESS Student form may be better at identifying behavioral and emotional risk in the internalizing domain. Future research is needed to replicate these clusters, as well as examine their predictive and applied utility. In addition, cluster analyses of the new BASC-3 (Reynolds & Kamphaus, 2015) and BASC-3 BESS (Kamphaus & Reynolds, 2015) must be conducted to determine whether the same clusters emerge for the updated system. However, the correlation between the scale scores of the new and previous version of the Student BESS are .95 (Kamphaus & Reynolds, 2015), suggesting that these results are likely to extend to the new BASC-3 BESS system.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported in part by a Faculty Innovation Grant from Lehigh University to Drs. Bridget Dever, Craig Hochbein, and George White.