Examining Subtypes of Behavioral/Emotional Risk Using Cluster Analysis

Abstract

Behavioral and emotional problems among children and adolescents can lead to numerous negative outcomes without intervention. From a prevention standpoint, screening for behavioral and emotional risk is an important step toward identifying such problems before the point of diagnosis or referral. The present study conducted a k-means cluster analysis to determine the subtypes of risk captured by one such screening instrument, the Behavioral and Emotional Screening System (BESS). The final solution produced four clusters: Well-Adapted, Internalizing/Adjustment Problems, Mild Externalizing Problems, and General Problems-Severe; these results were similar to those found with the full Behavioral Assessment System for Children, Second Edition (BASC-2), suggesting that the BESS assesses similar constructs. Predictive validity evidence suggested that cluster membership was associated with standard achievement scores and in-school disciplinary incidents.

Keywords

externalizing/internalizing disorders psychopathologies and symptomologies behavioral and emotional risk universal screening person-centered analyses

Behavioral and emotional problems account for more than 50% of referrals to school mental health professionals (Brestan & Eyberg, 1998; Lahey, Miller, Gordon, & Riley, 1999). An important step in the identification of such difficulties is classification, which is often done in the school setting to determine eligibility for services such as special education (Dever & Kamphaus, 2013). Although classification typically involves diagnosis of presence or absence of disorder, many researchers and clinicians argue that the dichotomous nature of a categorical model fails to adequately capture differences in the degree, severity, and manifestations of behavioral and emotional problems among youth (Achenbach & McConaughy, 1992; Kamphaus, Rowe, Dowdy, & Henry, 2006). In addition, high levels of comorbidity of mental health problems indicate the need to consider a more dimensional approach of classification that reflects the presence of a continuum of behavioral and emotional difficulties, as opposed to a binary system (Clark, Watson, & Reynolds, 1995; Wittchen, 1996).

Person-centered approaches to analysis of behavioral and emotional assessment instruments allow for the exploration of patterns of mental health strengths and difficulties along a continuum. As opposed to variable-centered analyses that focus on relationships among variables, person-centered analyses focus on how constructs tend to group together within individuals to form clusters or profiles. For example, the Behavioral Assessment System for Children, Second Edition (BASC-2; Reynolds & Kamphaus, 2004) is a commonly-used instrument that has been examined using cluster analytic techniques (DiStefano & Kamphaus, 2006; DiStefano, Kamphaus, & Mindrila, 2010). DiStefano and Kamphaus found six distinct clusters of students (well-adapted, average, disruptive behavior problems, learning problems, general problems-severe, and mildly disruptive) using the first edition of the BASC teacher rating form. DiStefano et al. replicated this analysis with the BASC-2 teacher rating form, and identified seven clusters of students, some overlapping with the previous findings (well-adapted, average, disruptive behavior problems, academic problems, worry/physical complaints, clinical internalizing, and clinical externalizing).

Although omnibus assessments of behavioral and emotional problems such as the BASC-2 are commonly administered in schools, the staggering number of students experiencing mental health issues has resulted in a push toward early identification efforts. As a result, universal mental health screenings in schools to identify students who are at-risk of behavioral and emotional problems have increased in popularity (Bruhn, Woods-Groves, & Huddle, 2014; Stiffler & Dever, 2015). Broadband screening tools can usually be administered in less than 15 min, and provide an indication of a student’s risk of general behavioral and emotional problems, rather than for a specific disorder. Screening outcomes both identify previously undetected cases of behavioral and emotional problems (Vander Stoep et al., 2005) and predict future diagnosis of mental health disorders with adequate accuracy (Najman et al., 2008). Early identification of behavioral and emotional risk can also mitigate or prevent negative outcomes such as school dropout and lower academic achievement (Brestan & Eyberg, 1998; Chin, Dowdy, & Quirk, 2013).

Mental health screening assessments are well-aligned with the dimensional approach to classification, as the goal is to detect subsyndromal psychopathology that is functionally impairing but below diagnostic thresholds (Hudziak, Wadsworth, Heath, & Achenbach, 1999). However, such screening instruments are still typically used in a categorical fashion, with students being classified as either “at-risk” or “not at-risk.” The Behavioral and Emotional Screening System (BESS; Kamphaus & Reynolds, 2007) is a brief screening tool that was developed from the BASC-2 standardization item pool to identify behavioral and emotional risk among school-aged youth. Despite being from the same family of assessments, the BESS has not been analyzed using person-centered analyses to determine the typology of the subsyndromal behavioral and emotional risk it assesses. The purpose of the present study was to determine whether the BESS screener identifies similar clusters of students to the full BASC-2. It was hypothesized that the BESS would demonstrate a similar cluster solution to the BASC-2; however, one or more clusters with milder problems were expected, with less differentiation in terms of problem type. Predictive validity was examined by using the clusters as predictors of behavioral and academic outcomes.

Method

Participants and Procedure

A total of 1,935 students across Grades 3 to 8 in six schools (four elementary, two middle) in the Northeast completed the Behavioral and Emotional Screening System Student self-report form (BESS Student). Passive parental consent was used by the six schools, and less than 1% chose not to participate. Forty-nine percent of participating students were female; 61% were Hispanic, 24% Caucasian non-Hispanic, 12% Black/African American, and 3% Other races/ethnicities. Data were collected as part of a universal screening program for behavioral and emotional risk; screening instruments were completed during the students’ homeroom periods using scannable forms.

Measures

The BASC-2 BESS Student consists of 30 items to assess for behavioral and emotional risk among students in Grades 3 through 12 (Kamphaus & Reynolds, 2007). Each item on the BESS Student uses a four-point response scale (never, sometimes, often, almost always). Previous research (Dowdy et al., 2011) indicated that the BESS Student consists of four factors: inattention/hyperactivity, internalizing problems, school problems, and personal adjustment. The psychometric properties of the BESS Student version are generally acceptable, including adequate split-half reliability (.90-.93) and test–retest reliability (.80). Student demographic characteristics, office disciplinary referrals (ODRs), suspensions, and Pennsylvania System of School Assessment (PSSA) scores were obtained from the district.

Results and Discussion

Due to the large sample size, a k-means cluster analysis procedure was used (Aldenderfer & Blashfield, 1984). The four subscales were standardized prior to clustering, and were used as the clustering variables. Cluster solutions from 2 to 7 clusters were considered based on work with the full BASC and BASC-2 (e.g., DiStefano & Kamphaus, 2006). The final solution resulted in four clusters: Well-Adapted, Internalizing/Adjustment Problems, Mild Externalizing Problems, and General Problems-Severe (Table 1). Chi-square analyses revealed significant differences in the composition of the clusters by race/ethnicity, χ²(18) = 42.52, p < .001; English language learner status, χ²(3) = 10.12, p < .05; gender, χ²(3) = 22.26, p < .001; special education status, χ²(3) = 10.23, p < .05; and risk classification on the BESS, χ²(6) = 1451.39, p < .001. Hispanic students were overrepresented in the Internalizing/Adjustment Problems and General Problems-Severe clusters, whereas African American students were overrepresented in the Mild Externalizing Problems cluster. Females and English language learners were more likely to be classified in the Internalizing/Adjustment Problems cluster, whereas students in special education were more likely to be in the General Problems-Severe cluster. Almost all (99%) of those in the General Problems-Severe cluster were classified as at-risk on the BESS, and none of those in the Well-Adapted cluster were considered at-risk. The BESS classified 39.5% of students in the Internalizing/Adjustment Problems cluster and 2.3% of those in the Mild Externalizing Problems cluster as at-risk of behavioral and emotional problems.

Table 1.

Descriptive Information by Cluster on Demographics, Subscales, and Outcomes.

	Cluster 1	Cluster 2	Cluster 3	Cluster 4	F	df
	Well-Adapted	Internalizing/Adjustment Problems	Mild Externalizing Problems	General Problems-Severe	F	df
N	801	383	529	222
Cluster %	41.40%	19.79%	27.34%	11.47%
Demographics
Middle schoo1	60.2%	55.1%	69.6%	65.3%
Hispanic/Latino	59.6%	64.0%	59.4%	67.6%
Black/African American	10.6%	8.9%	18.1%	9.9%
White	27.0%	24.0%	20.0%	21.6%
English language learner	10.1%	16.2%	11.0%	13.5%
Female	49.4%	57.7%	42.0%	47.7%
Special education	19.7%	23.2%	20.6%	29.3%
At risk on BESS	0.0%	39.5%	2.3%	99.1%
Clustering scale means
School problems	–0.70	–0.10	0.48	1.54	760.31	3,1931
Internalizing problems	–0.67	0.84	–0.14	1.33	737.14	3,1931
Inattention/hyperactivity	–0.71	–0.00	0.42	1.58	775.91	3,1931
Personal adjustment	0.63	– 1.10	0.25	–0.96	710.94	3,1931
Outcome measures
English language arts	997.80^a	969.76^b	970.00^b	949.37^b	18.35	3,1838
Mathematics	934.35^a	912.06^b	906.60^b	898.01^b	17.77	3,1838
Disciplinary incidents	0.53^a	0.75^a	1.38^b	1.71^b	18.06	3,1805
Suspensions	0.05^a	0.09^a	0.16^b	0.32^c	14.84	3,1805

Note. BESS = Behavioral and Emotional Screening System.

Tests with same superscript for each outcome measure were not statistically significant.

All other tests above were statistically significant, p < .05.

Predictive validity evidence suggested that cluster membership was associated with standardized achievement scores (Math and English Language Arts) and ODRs (Table 1). Those in the Well-Adapted cluster had higher achievement scores than the other three clusters. Students in the Mild Externalizing Problems and General Problems-Severe clusters had higher disciplinary incidents than the other two groups; however, those in the General Problems-Severe cluster had more suspensions than those in the Mild Externalizing cluster.

Results of this study identified four clusters of students using the BESS Student, each characterized by distinctive patterns of behavioral/emotional risk. Differences in risk classification and behavioral and academic outcome measures provided validity evidence for these subgroups. Consistent with previous studies (DiStefano & Kamphaus, 2006; DiStefano et al., 2010), the clusters that emerged from the brief BESS form (i.e., Well-Adapted, Internalizing/Adjustment Problems, and General Problems-Severe) showed a high degree of overlap with those identified by the full BASC-2, suggesting that the less comprehensive BESS may effectively assess similar constructs. However, a milder externalizing category was found in our study, with few students in this cluster being identified as at-risk on the BESS; however, these students were involved in more disciplinary incidents and had lower achievement test scores than the Well-Adapted group. This finding suggests that the BESS Student form may be better at identifying behavioral and emotional risk in the internalizing domain. Future research is needed to replicate these clusters, as well as examine their predictive and applied utility. In addition, cluster analyses of the new BASC-3 (Reynolds & Kamphaus, 2015) and BASC-3 BESS (Kamphaus & Reynolds, 2015) must be conducted to determine whether the same clusters emerge for the updated system. However, the correlation between the scale scores of the new and previous version of the Student BESS are .95 (Kamphaus & Reynolds, 2015), suggesting that these results are likely to extend to the new BASC-3 BESS system.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported in part by a Faculty Innovation Grant from Lehigh University to Drs. Bridget Dever, Craig Hochbein, and George White.

References

Achenbach

T. M.

McConaughy

S. H.

(1992). Taxonomy of internalizing disorders of childhood and adolescence. In Reynolds

W. M.

(Ed.), Internalizing disorders in children and adolescents (pp. 19-60). New York, NY: Wiley.

Aldenderfer

M. S.

Blashfield

R. K.

(1984). Cluster analysis. Newbury Park, CA: Sage.

Brestan

E. V.

Eyberg

S. M.

(1998). Effective psychosocial treatments of conduct-disordered children and adolescents: 29 years, 82 studies, and 5,272 kids. Journal of Clinical Child Psychology, 27, 180-189.

Bruhn

A. L.

Woods-Groves

Huddle

(2014). A preliminary investigation of emotional and behavioral screening practices in K-12 schools. Education & Treatment of Children, 37, 611-634.

Chin

J. K.

Dowdy

Quirk

M. P.

(2013). Universal screening in middle school: Examining the behavioral and emotional screening system. Journal of Psychoeducational Assessment, 31, 53-60.

Clark

L. A.

Watson

Reynolds

W. S.

(1995). Diagnosis and classification of psychopathology: Challenges to the current system and future directions. Annual Review of Psychology, 46, 121-153.

Dever

B. V.

Kamphaus

R. W.

(2013). Social and emotional assessment of children in schools. In Geisinger

K. F.

Bracken

B. A.

Carlson

J. F.

Hansen

J. C.

Kuncel

N. R.

Reise

S. P.

Rodriguez

M. C.

(Eds.), APA handbook of testing and assessment (Volume 3: Testing and Assessment in School Psychology and Education) (pp. 129-148). Washington, DC: American Psychological Association.

DiStefano

C. A.

Kamphaus

R. W.

(2006). Investigating subtypes of child development: A comparison of cluster analysis and latent class cluster analysis in typology creation. Educational and Psychological Measurement, 66, 778-794. doi:10.1177/0013164405284033

DiStefano

C. A.

Kamphaus

R. W.

Mindrila

D. L.

(2010). A typology of teacher-rated child behavior: Revisiting subgroups over 10 years later. School Psychology Quarterly, 25, 152-163. doi:10.1037/a0020913

10.

Dowdy

Twyford

Chin

DiStefano

Kamphaus

Mays

(2011). Factor structure of the BASC-2 Behavioral and Emotional Screening System Student Form. Psychological Assessment, 23, 379-387. doi:10.1037/a0021843

11.

Hudziak

J. J.

Wadsworth

M. E.

Heath

A. C.

Achenbach

T. M.

(1999). Latent class analysis of child behavior checklist attention problems. Journal of the American Academy of Child & Adolescent Psychiatry, 38, 985-991.

12.

Kamphaus

R. W.

Reynolds

C. R.

(2007). Behavior Assessment System for Children—Second Edition (BASC-2): Behavioral and Emotional Screening System (BESS). Bloomington, MN: Pearson.

13.

Kamphaus

R. W.

Reynolds

C. R.

(2015). Behavior Assessment System for Children—Third Edition (BASC-3): Behavioral and Emotional Screening System (BESS). Bloomington, MN: Pearson.

14.

Kamphaus

R. W.

Rowe

E. W.

Dowdy

E. T.

Henry

C. N.

(2006). Classification and diagnosis concepts. In Kamphaus

R. W.

Campbell

J. M.

(Eds.), Psychodiagnostic assessment of children: Dimensional and categorical approaches (pp. 1-27). Hoboken, NJ: John Wiley.

15.

Lahey

B. B.

Miller

T. L.

Gordon

R. A.

Riley

A. W.

(1999). Developmental epidemiology of the disruptive behavior disorders. In Quay

Hogan

(Eds.), Handbook of disruptive behavior disorders (pp. 23-48). New York, NY: Springer.

16.

Najman

J. M.

Heron

M. A.

Hayatbakhsh

M. R.

Dingle

Jamrozik

Bor

Williams

G. M.

(2008). Screening in early childhood for risk of later mental health problems: A longitudinal study. Journal of Psychiatric Research, 42, 694-700.

17.

Reynolds

C. R.

Kamphaus

R. W.

(2004). Behavioral Assessment System for Children (2nd ed.). Circle Pines, MN: American Guidance Service.

18.

Reynolds

C. R.

Kamphaus

R. W.

(2015). Behavioral Assessment System for Children (3rd ed.). Circle Pines, MN: American Guidance Service.

19.

Stiffler

M. C.

Dever

B. V.

(2015). Mental health screening at school: Instrumentation, implementation, and critical issues. New York, NY: Springer.

20.

Vander Stoep

Mccauley

Thompson

K. A.

Herting

J. R.

Kuo

E. S.

Stewart

D. G.

Kushner

(2005). Universal emotional health screening at the middle school transition. Journal of Emotional and Behavioral Disorders, 13, 213-223.

21.

Wittchen

H. U.

(1996). What is comorbidity: Fact or artifact? British Journal of Psychiatry, 168, 7-8.