Assessing the Factor Structure and Measurement Invariance of the BASC-3 Behavioral and Emotional Screening System Student Form Across Race/Ethnicity and Gender

Abstract

The BASC-3 Behavioral and Emotional Screening System Student Form (BESS SF) is the latest iteration of a widely used instrument for identifying students at behavioral and emotional risk. Measurement invariance across race/ethnicity and gender for the latest BESS SF has not yet been established. Using a sample of 737 U.S. urban fourth- to eighth-grade students, we tested competing models of the BESS SF to determine the best-fitting factor structure. We also tested for measurement equivalence by race/ethnicity (i.e., White, Black, Latinx) and gender (i.e., boys, girls). Consistent with prior findings, we identified that a bifactor structure of the BESS SF best fit the data and supported measurement equivalence across race/ethnicity and gender. These findings provide further support for using the BESS SF to conduct universal behavioral and emotional screening among diverse students. More research is needed in schools serving students with greater racial/ethnic and socioeconomic diversity.

Keywords

emotional/behavioral disorders social-emotional diversity

Schools have been challenged with identifying and evaluating students with behavioral and emotional problems and often serve as the primary mental health support system for racial/ethnic minority youth (Cauce et al., 2002; Farmer et al., 2003; U.S. Congress, 2004). However, traditional school-based methods of assessing behavioral and emotional risk among students (e.g., office discipline referrals) have been insufficient. For example, these traditional methods are subject to reporter biases that disproportionately over-refer male and racial/ethnic minority students, under-refer students with internalizing problems, and have little predictive validity (Girvan et al., 2017; Kauffman, 1999; Naser et al., 2018a; Skiba et al., 2011; Sullivan & Bal, 2013). This is particularly problematic, as racial/ethnic minority children and youth are at greater risk for impaired socioemotional development due to unique risk factors (e.g., experiencing racism) and are ultimately less likely to be linked to mental health interventions (Alegria et al., 2010; Berry et al., 2021). Improving how schools identify students at behavioral and emotional risk is imperative for addressing child mental health and treatment disparities.

One remedy for these shortcomings could be the implementation of school-based universal screening, which can more effectively and accurately identify students at behavioral and emotional risk compared with traditional methods and can link such students to targeted evidence-based interventions (Eklund & Dowdy, 2014; Miller et al., 2015; Naser et al., 2018a). Recent research has advocated for student-reported universal screening measures for social, emotional, and behavioral risk as a way to address inequity in mental health care access and the disproportionate use of punitive discipline practices and special education referrals with Black, Latinx, and Native American youth (Dever et al., 2016). Furthermore, integrating self-report screeners can more thoroughly identify students with emotional problems without solely relying on teachers’ report, which are more likely to identify and refer students that exhibit disruptive behaviors (Splett et al., 2019). To effectively address mental health treatment disparities, establishing self-report school-based screeners that adequately assess behavioral and emotional risk across race/ethnicity and gender is needed.

The Behavioral and Emotional Screening System

The original (BASC-2) Behavioral and Emotional Screening System Student Form (BESS SF) was developed as a universal screening tool for behavioral and emotional risk using items from the extensive Behavioral Assessment System for Children (BASC; Kamphaus & Reynolds, 2007). Prior psychometric testing of the original screener demonstrated measurement invariance across race/ethnicity (i.e., White, Black, Latinx) among a sample of high school students (Harrell-Williams et al., 2015). Nearly a decade later, the BASC-3 BESS SF was created to revise the previous form and assess behavioral and emotional risk using three factors (personal adjustment, self-regulation, and internalizing) and a series of miscellaneous school-related risk items (Reynolds & Kamphaus, 2015). The latest BESS SF was normed on a nationally representative sample; however, no known studies have tested whether this assessment assesses behavioral and emotional risk constructs equally across racial/ethnic groups.

Measurement invariance, or the determination that the psychometric properties of an instrument measure a construct equally across different subgroups, must be established to ensure the BESS SF does not have gender, racial/ethnic, and/or cultural biases. A systematic review identified that several child and adolescent self-report measures of psychopathology fail to demonstrate measurement invariance across cultures, particularly those measuring social, emotional, and behavioral problems (Stevanovic et al., 2017). Previous invariance research on the prior version of the BESS SF focused on high school students and demonstrated measurement invariance by race/ethnicity (Harrell-Williams et al., 2015) and partial metric and scalar invariance by gender (Kim & Kamphaus, 2018). However, without establishing measurement invariance for the latest BESS SF among other age groups, school psychologists may be unable to compare behavioral and emotional risk scores across groups.

Our study expands prior findings by testing for the best-fitting factor structure of the new BESS SF among a sample of fourth- through eighth-grade students. We also build upon extant literature by including the miscellaneous school-related items in the factor structure of the BESS SF, which are used to calculate the global risk score and had prior support for inclusion in the factor structure of the previous BESS SF (Dowdy et al., 2011). Although these school-related behavioral and emotional risk items are not a subscale available in the scoring software, these items provide useful information for school psychologists’ interpretation of behavioral and emotional risk within the school environment. For example, students with emotional and behavioral disorders are significantly more likely to have lower perceptions of school climate (Salle et al., 2018). Despite the predictive utility of BESS SF global risk scores on student outcomes (King et al., 2012), behavioral symptoms that contribute to elevations in global risk may differ across respondents. For example, one student may have higher risk due to elevated internalizing risk whereas another student may have higher risk due to elevated self-regulation risk. Therefore, prior studies have supported modeling previous versions of the BESS SF using a bifactor structure, as this structure accounts for both global risk and underlying sources of risk (Dowdy et al., 2011; Naser et al., 2018b). Even with empirical support for modeling the BESS SF using a bifactor structure, few studies have tested the factor structure of the BESS SF including miscellaneous school-related items as a latent construct. Therefore, identifying the best-fitting structure including these items is necessary to improve interpretation of these school-related risk items in tandem with the primary factors (i.e., internalizing, self-regulation, personal adjustment). In addition, specifying a theoretically supported, well-fitting model is needed before conducting invariance analyses (Van de Schoot et al., 2012).

The Current Study

In this study, we first identified the best-fitting factor structure of the latest BESS SF by testing competing models and tested measurement invariance across race/ethnicity, gender, and school using the best-fitting factor structure. Considering previous empirical support on the previous BASC-2 BESS SF for using a bifactor structure compared with other competing models (Dowdy et al., 2011; Naser et al., 2018b), we hypothesized that the bifactor model would provide the best fit compared with a non-hierarchical four-factor model and a second-order hierarchical model (Hypothesis 1). Next, as studies on the previous BESS SF (Harrell-Williams et al., 2015; Kim & Kamphaus, 2018) supported full and partial invariance across race/ethnicity and gender, respectively, we also expected the latest BESS SF to be invariant across White, Black, and Latinx students and across boys and girls (Hypotheses 2 and 3). Finally, we tested invariance across schools and expected invariance given the schools in our sample are all urban public schools in the same midwestern state (Hypothesis 4).

Method

Participants and Procedures

This study used archival data collected from prior BESS SF administrations to 737 U.S. elementary (i.e., fourth and fifth grades; n = 290) and middle school (i.e., sixth through eighth grades; n = 447) students at three separate urban schools in northeast Ohio. During the fall semesters between 2016 and 2018, schools administered the BESS SF for schoolwide screening purposes. Schools provided the authors a de-identified archival dataset with student ID numbers and race/ethnicity data for 601 students. When preparing the final dataset, we omitted duplicate students that could emerge when students happened to be tested twice in different years and included only the first BESS SF administration in analyses. As schools were unable to provide race/ethnicity data for all students in our sample, we conducted independent-samples t tests to determine whether behavioral and emotional risk scores differed between students with race/ethnicity data and those without. Results indicated there were no significant differences in risk scores across each subscale between students with race/ethnicity data and those without, ts(733–734) = −0.07 to 0.58, ps = .09–.78. All three schools shared similar characteristics as urban schools representing students diverse in socioeconomic status (i.e., high- and low-income families) and race/ethnicity. Most respondents identified as female (n = 381; 51.7%) and identified as White (34.6%), African American/Black (27.5%), Hispanic/Latinx (19.3%), Multiracial (15.8%), and Asian or Pacific Islander (2.8%). All demographic information is presented in Table 1. This study was exempt from review by the university’s Institutional Review Board.

Table 1.

Student Demographics by School for Study Participants.

Demographic Category	All schools (N = 737)		School 1 (n = 172)		School 2 (n = 372)		School 3 (n = 193)
Demographic Category	M (SD)	Range	M (SD)	Range	M (SD)	Range	M (SD)	Range
Age (years)	11.2 (1.5)^a	8–14	11.0 (1.7)	8–14	10.8 (1.4)	8–14	12.3 (1.0)	10–14
Grade	6.0 (1.5)	4–8	5.9 (1.7)	4–8	5.5 (1.4)	4–8	7.1 (0.8)	6–8
	n	%	n	%	n	%	n	%
Grade
Fourth	183	24.8	71	41.3	112	30.1	0	0.0
Fifth	107	14.5	0	0.0	107	28.8	0	0.0
Sixth	126	17.1	15	8.7	56	15.1	55	28.5
Seventh	176	23.9	54	31.4	53	14.2	69	35.8
Eighth	145	19.7	32	18.6	44	11.8	69	35.8
Gender
Boys	356	48.3	93	54.1	158	42.5	105	54.4
Girls	381	51.7	79	45.9	214	57.5	88	45.6
Race/ethnicity^b
White	208	28.2	35	20.3	76	20.4	97	50.3
African American/Black	165	22.4	52	30.2	71	19.1	42	21.8
Hispanic/Latinx	116	15.7	1	0.6	105	28.2	10	5.2
Multiracial	95	12.9	4	1.2	53	14.2	38	19.7
Asian or Pacific Islander	17	2.3	0	0.0	10	2.7	5	2.6
Missing race	136	18.5	78	45.3	57	15.3	1	0.5
Students eligible for free or reduced price lunch^c (% of student body)	—		15.7^d		73.0^e		55.6^f

Age did not differ by gender, t(735) = 1.91, p = .091. ^bGender distribution did not differ by race/ethnicity, with the exception of more girls (88%) than boys (12%) in the Asian/Pacific Islander ethnic group. ^cAll socioeconomic descriptive data are based on publicly available information for zip codes serving the schools and are presented to contextualize each school’s population. ^dPer estimates by schooldigger.com, academic year 2018–2019. ^eAs this is an independent school, estimates were provided by the school website, academic year 2020–2021. ^fPer estimates by National Center for Education Statistics, academic year 2018–2019.

Measure

The BESS SF is a 28-item universal screener used to assess behavioral and emotional risk among children and adolescents, ages 8 to 18 years 11 months (Reynolds & Kamphaus, 2015). The BESS SF is normed on a nationally representative sample, provides a global risk score, and is comprised of three scored subindices—internalizing risk, self-regulation risk, and personal adjustment risk—and a series of miscellaneous school-related behavioral and emotional risk items. Respondents rate items (e.g., “I have trouble sitting still,” “I get blamed for things I can’t help”) on a 4-point Likert-type scale from 1 (never) to 4 (almost always). Total raw scores are then converted to t-scores. Higher t-scores indicate greater behavioral and emotional risk and ranges of these t-scores are classified into three behavioral and emotional risk categories (i.e., 20–60 = normal risk, 61–70 = elevated risk, 71+ = extremely elevated risk; Kamphaus & Reynolds, 2007). Prior research on the BESS SF evidenced high internal consistency (α = .93–.94) and test–retest reliability (.88), as well as strong concurrent and predictive validity with measures of school climate (Naser & Dever, 2020; Reynolds & Kamphaus, 2015).

Data Analysis

First, positively worded items were reverse coded. Then, a series of confirmatory factor analysis (CFA) models were fit using Mplus, Version 8.4, specifying all items as ordered categorical (L. K. Muthén & Muthén, 2021). Research on the previous BESS SF determined a bifactor model provides the best model fit (Naser et al., 2018b). Therefore, this study compared the bifactor model (Model 1) with two competing models: a non-hierarchical four-factor model (Model 2) and a second-order hierarchical model (Model 3). Model 1 (i.e., bifactor model) has all items load on a general behavioral and emotional risk factor and also load on one of four factors (i.e., internalizing, self-regulation, personal adjustment, school problems; see Table 2). Items in Model 2 load on a four-factor structure without a general factor, whereas Model 3 has items loading on a four-factor structure of behavioral and emotional risk, with the latent factors explained by a second-order overall behavioral and emotional risk factor.

Table 2.

Standardized Factor Loadings for Bifactor Measurement Model of BASC-3 BESS SF.

Item number	Item description	General BERI (α = .90)	Internalizing (α = .85)	Self-regulation (α = .73)	Personal adjustment (α = .79)	School problems (α = .74)
5	Want to do better, but I can’t	−.253	.708	—	—	—
8	Lonely	.512	.336	—	—	—
11	Worry but I don’t know why	.106	.556	—	—	—
13	Life is getting worse and worse	.422	.560	—	—	—
16	Blamed for things I can’t help	−.171	.753	—	—	—
21	When I try hard, I fail	−.147	.748	—	—	—
23	Out of place around people	.343	.481	—	—	—
24	Trouble controlling thoughts	−.083	.717	—	—	—
26	Worry about what will happen	.141	.562	—	—	—
27	No one understands me	.300	.527	—	—	—
1	Trouble sitting still	.066	—	.577	—	—
4	Trouble paying attention	.017	—	.686	—	—
7	People tell me to slow down	−.033	—	.394	—	—
12	Talk while others talk	−.128	—	.671	—	—
18	Forget to do things	.220	—	.613	—	—
20	Trouble for not paying attention	−.078	—	.757	—	—
3	Parents trust me	−.090	—	—	.716	—
6	Others respect me	.028	—	—	.515	—
10	Liked by others	.220	—	—	.384	—
14	Parents are proud of me	−.077	—	—	.836	—
19	Happy with who I am	.534	—	—	.525	—
22	Parents listen to me	.041	—	—	.720	—
25	Good at making decisions	−.076	—	—	.530	—
28	Parents like to be with me	.166	—	—	.637	—
9	School feels good to me	.157	—	—	—	.694
17	Feel safe at school	.276	—	—	—	.649
2	Teacher is proud of me	−.083	—	—	—	.722
15	Get along with my teacher	−.112	—	—	—	.837

Note. All parameters were freely estimated. BASC = Behavioral Assessment System for Children; BESS SF = Behavioral and Emotional Screening System Student Form; BERI = Behavioral and Emotional Risk Index; α = Cronbach’s alpha.

As items were treated as ordered categorical, robust weighted least squares mean and variance-adjusted (WLSMV) estimation was used to estimate these models. The WLSMV estimator provides mean- and variance-adjusted chi-square test statistics appropriate for ordinal data. We used the chi-square test of model fit value to assess overall model fit (i.e., significant chi-square indicates poor-fitting model). Additional model fit indices were also examined including the root mean square error of approximation (RMSEA), comparative fit index (CFI), and Tucker–Lewis index (TLI). An RMSEA value ≤0.05 and CFI and TLI values above 0.95 indicate good model fit; however, RMSEA values ≤0.08 and CFI and TLI values above 0.90 evidence acceptable model fit (Brown, 2015). We compared these fit indices among the three models to determine the best-fitting model.

Once the best-fitting model was determined, measurement invariance analyses were completed following steps outlined by B. Muthén and Asparouhov (2002) for invariance using ordinal data. First, all models were fit for groups to be examined in the measurement invariance (i.e., girls and boys for gender invariance; White, Black, and Latinx for race/ethnicity invariance; each of the three schools for school invariance). Then, we compared an unconstrained (i.e., configural) with a constrained (i.e., scalar or strong invariance) model to test for measurement invariance by race/ethnicity, gender, and school. Model fit change scores were calculated by subtracting fit indices of the constrained model from the unconstrained model. In the unconstrained model, factor loadings and thresholds varied across groups (e.g., boys, girls) with factor variances fixed at one and factor means fixed at zero in three separate analyses (i.e., race/ethnicity, gender, school). Within the constrained model, factor loadings and thresholds were constrained to be equal across groups (e.g., boys, girls). When estimating these models with ordinal data, the factor loadings and thresholds are constrained in the same step rather than individually as both parameters influence the item probability curve simultaneously (L. K. Muthén & Muthén, 2021). The unconstrained and constrained models were compared based on chi-square difference tests using DIFFTEST. The DIFFTEST option is used with the WLSMV estimator to appropriately compare differences in the mean- and variance-adjusted chi-squares among models. Due to these adjustments made by the WLSMV estimator, the chi-square difference is not distributed as a chi-square. Therefore, the DIFFTEST option is a two-step process that (a) saves the derivatives of the unconstrained model needed for a chi-square difference test, and then (b) computes the chi-square difference comparing the unconstrained model and constrained model (L. K. Muthén & Muthén, 2021). This option in Mplus provides an estimate of chi-square difference test comparing the least restrictive model (i.e., unconstrained) to a model with more restrictions (i.e., constrained), rather than a simple difference test. A statistically significant p value (p < .05) indicates that the unconstrained model results in a significant improvement in fit. However, as the chi-square can be influenced by large sample sizes, such as ours, and thus can detect even minor improvements in model fit, we also examined whether freeing parameters improved model fit. If freeing parameters improved the overall model fit by at least 0.01 on the CFI and TLI and 0.015 on the RMSEA, the invariance assumption was rejected (Cheung & Rensvold, 2002).

Results

Among the final sample of respondents with race/ethnicity data, missing value analysis indicated the maximum percentage of BESS SF items missing was 1.1%. We used listwise deletion to handle missing data. Only three race/ethnicity categories had sufficient sample sizes for running invariance analyses per Meade (2005): White (n = 208), Black (n = 165), and Latinx (n = 116). For Items 3, 14, 15, and 28, there was insufficient representation of response Category 4 (almost always) across racial/ethnic categories. Specifically, no Latinx students selected Category 4 on Item 15, no Black students selected Category 4 on Item 3, and no students at School 1 selected Category 4 on Items 14 and 28. Due to the infrequent endorsement of Response Category 4, and because these zero frequencies create estimation problems using the WLSMV estimator, responses to Category 4 were collapsed into response Category 3 (often) across these four items. We also examined the item characteristic curves (ICCs) of individual items, which indicate the probability of selecting a response option. As all response options were monotonically increasing and otherwise of sufficient use, the remaining 4-point ordinal scales were retained for subsequent analyses.

To test our first hypothesis, a series of categorical CFAs using WLSMV estimation were computed for each model (see Table 3). Items that represented similar forms of behavioral and emotional risk (i.e., “I am liked by others” and “Others have respect for me”) were covaried across all models due to significantly high modification indices values (modification indices = 204.82, expected parameter change = 0.41), a significant correlation between the two items (r = .39, p < .05), and improvement in model fit after covarying these items. The bifactor model (Model 1) fit the data well and was retained for subsequent analyses (RMSEA = 0.055, TLI = 0.931, CFI = 0.942). The hierarchical model (Model 2) poorly fit the data based on absolute fit statistics (RMSEA = 0.143) and the four-factor model (Model 3) had acceptable absolute model fit (RMSEA = 0.075) but poor comparative and non-normed fit (CFI = 0.884, TLI = 0.872). Individual items, factor loadings, and factor internal consistency values are presented in Table 2.

Table 3.

Fit Indices and Tests of Measurement Invariance for Competing Models of BASC-3 BESS SF (N = 737).

Model description	χ² (df)	Δχ² (df)^a	CFI	ΔCFI^a	TLI	ΔTLI^a	RMSEA	ΔRMSEA^a
Model 1: Bifactor	1,012.88 (316)**	—	0.942	—	0.931		0.055	—
Model 2: Hierarchical	5,609.62 (350)**	—	0.565	—	0.530		0.143	—
Model 3: Four-factor	1,752.27 (344)**	—	0.884	—	0.872		0.075	—
Multiple group analysis of bifactor model by race
Unconstrained	1,598.616 (963)**	—	0.909	—	0.893	—	0.064	—
Constrained	1,661.608 (1,239)**	325.301 (354)*	0.939	−0.030	0.944	−0.051	0.046	0.018
Multiple group analysis of bifactor model by gender
Unconstrained	1,802.454 (642)**	—	0.901	—	0.884	—	0.070	—
Constrained	1,615.958 (780)**	247.161 (138)**	0.929	−0.028	0.931	−0.047	0.054	0.016
Multiple group analysis of bifactor model by school
Unconstrained	2,333.087 (975)**	—	0.873	—	0.853	—	0.075	—
Constrained	2,054.039 (1,235)**	295.198 (260)	0.924	−0.051	0.930	−0.077	0.052	0.023

Note. BASC = Behavioral Assessment System for Children; BESS SF = Behavioral and Emotional Screening System Student Form; χ² = weighted least squares mean and variance-adjusted chi-square (WLSMV); df = degrees of freedom; Δχ² = chi-square difference test for WLSMV χ²; CFI = comparative fit index; TLI = Tucker–Lewis index; RMSEA = root mean square error of approximation. Dashes indicate no model comparison (i.e., this was the model used for subsequent comparisons within invariance tests).

Columns indicate change in chi-square, CFI, TLI, and RMSEA relative to comparison model. DIFFTEST command in Mplus was used to calculate Δχ² (df).

p < .05. **p < .001.

The bifactor structure was individually tested and evidenced acceptable fit among subgroups across race/ethnicity (i.e., RMSEAs = 0.04–0.05, CFIs = 0.93–0.96, TLIs = 0.91–0.96), gender (i.e., RMSEA = 0.05, CFIs = 0.94–0.95, TLIs = 0.92–0.95), and school (i.e., RMSEAs = 0.05–0.06, CFIs = 0.93–0.95, TLIs = 0.92–0.94). Measurement invariance by race/ethnicity (Hypothesis 2), gender (Hypothesis 3), and school (Hypothesis 4) was tested using multigroup categorical CFA with the bifactor model (see Table 3). For race/ethnicity, the unconstrained model (i.e., configural) displayed acceptable model fit (CFI = 0.909, TLI = 0.893, RMSEA = 0.064). When constraints were imposed on the loadings and thresholds across races/ethnicities (i.e., scalar), model fit improved (ΔCFI = −0.030, ΔTLI = −0.051, ΔRMSEA = 0.018). Across gender, the unconstrained model also acceptably fit the data (CFI = 0.901, TLI = 0.884, RMSEA = 0.070) and constraining the factor loadings and thresholds across genders showed improved model fit (ΔCFI = −0.028, ΔTLI = −0.047, ΔRMSEA = 0.016). To rule out measurement variability between schools, we tested for invariance across schools. For school, the unconstrained model evidenced acceptable absolute fit (RMSEA = 0.075) and poor comparative and non-normed fit (CFI = 0.873, TLI = 0.853), but the constrained model improved model fit (ΔCFI = −0.051, ΔTLI = −0.077, ΔRMSEA = 0.023). Consistent with our hypotheses, these findings support measurement equivalence for boys and girls and among White, Black, and Latinx fourth- to eighth-grade students attending urban schools.

Discussion

Our findings support using the latest BESS SF to identify diverse at-risk students. By allowing students to self-report their risk levels via screeners such as the BESS SF, school psychologists can identify students at behavioral and emotional risk directly from the source, which is especially important considering the racial/ethnic and gender biases associated with relying on other informants (e.g., teachers, parents) to gauge a child’s risk level. These results support the efficacy of using the BESS SF to detect behavioral and emotional risk without the gender and racial/ethnic biases that often accompany relying solely on outside informants. By reducing the influence of gender and racial/ethnic biases, BESS SF risk scores can be more confidently interpreted and compared across gender and race/ethnicity. Moreover, as teachers are more likely to identify students with externalizing compared with internalizing symptoms (Splett et al., 2019), this self-report screener can comprehensively assess students’ social, emotional, and behavioral risk. Students’ global risk scores on the BESS SF can provide schools with an early indication of their overall risk and their scores on each risk subscales can help schools determine what school supports best fit the students’ needs. These scores on the BESS SF can then be used to link at-risk students to appropriate interventions based on their risk level while being less likely to lead to disproportionate over-referrals of male and racial/ethnic minority students.

Consistent with prior research on the factor structure of its predecessor (Naser et al., 2018b), among a diverse sample of fourth- to eighth-grade students, these findings also supported the bifactor solution for the BESS SF as the best-fitting model. In practice, the general risk score is the most salient indicator of behavioral and emotional risk and the subscales are used to potentially inform the type of risk. A supported bifactor structure supports the existence of these subscales as uniquely interpretable information in addition to the general risk factor. In contrast with the previous study, we estimated these models with ordered categorical indicators, rather than as continuous items. This suggests that student responses to the BESS SF may be best modeled as ordered categorical, which is supported by previous guidance to use categorical estimation for measures with fewer than five response categories (Rhemtulla et al., 2012).

In addition, as school psychologists often use the miscellaneous school-related risk items for further examination of behavioral and emotional risk, we extended prior findings by supporting an invariant factor structure across race/ethnicity and gender using these items as a unique factor. Often these miscellaneous school-related risk items are portrayed as less informative given their lack of inclusion in BESS SF scoring software; however, our findings support the stability of this factor across race/ethnicity and gender. Given the associations between perceptions of school climate and behavioral and emotional risk (Salle et al., 2018), these items provide novel information not assessed by the other primary factors. Therefore, school psychologists can more confidently use all items of the BESS SF, including the school-related risk items, among diverse elementary and middle school students.

Despite the relatively large sample size, we found that specific demographic groups did not select certain response categories for some items. For example, no Latinx student reported that they “almost always” get along with their teacher. In addition, zero Black students responded with “almost always” when rating whether their parents trust them. These findings indicate that Black and Latinx students may, respectively, perceive these items differently than their peers with other racial/ethnic identities. More research is needed to understand what underlies these differences. Furthermore, future iterations of the BESS SF should consider these differing response patterns when labeling response categories to ensure more inclusive and accurate assessment of behavioral and emotional risk. Our findings should be replicated to determine whether these behaviors are less frequently occurring or observed among racial/ethnic subgroups. If our findings are consistently replicated, the next BESS SF should consider three response categories rather than four. Beyond this, results indicate that the factor structure of the latest BESS SF was invariant across White, Black, and Latinx students, which replicates prior psychometric evidence with the previous BESS SF (Harrell-Williams et al., 2015).

Limitations and Future Directions

These analyses did not represent students of all racial, ethnic, and gender identities, nor were they representative of students who attend schools in suburban or rural areas. More research is needed to determine whether the BESS SF is invariant across students of other racial (e.g., Multiracial, Asian) and ethnic (e.g., Middle Eastern, North African) identities, as well as across diverse gender identities and experiences (e.g., genderqueer, non-binary, transgender). Future replication of these findings among larger samples is needed, as results could change with larger subgroup samples (Meade, 2005). Another limitation of this study was the substantial amount of missing student race/ethnicity data (i.e., 18.5%) in the larger sample. It is possible that there was variability by race/ethnicity we did not measure by omitting these students from analyses. In addition, replicating these findings would be beneficial to verify that these results are not school-specific. As there were items that needed to be covaried due to their similar nature, researchers should independently verify whether these items are indeed considered similar among children and youth using strategies such as cognitive interviewing. Finally, as this study used cross-sectional data from similar schools in one midwestern state, future research should assess for measurement invariance of the BESS SF among students attending nationally representative urban, rural, and suburban schools and using longitudinal data.

Conclusion

This study provides further evidence that the latest BESS SF assesses behavioral and emotional risk among elementary and middle school students with reduced racial/ethnic, gender, and school biases. Using this tool, school psychologists can more confidently compare risk scores across Black, White, and Latinx male and female students. These screening results can be used to detect students at behavioral and emotional risk and ultimately link them to appropriate interventions without the biases that accompany traditional methods of socioemotional risk assessment. Notably, these findings do not refute the utility of multi-informant assessment approaches, as considering teacher and parent perspectives are likely more informative than relying solely on child self-report when conducting universal screening (von der Embse et al., 2019). Although our findings support invariance, further research is needed on the latest BESS SF to identify factors that may underlie the differing response patterns across racial/ethnic groups that were identified in this study. Even still, this investigation of the psychometric properties of a widely used school-based social, emotional, and behavioral health screener provides a more accurate picture of what this tool can tell us and how screening results from this tool can inform programming.

Footnotes

Associate Editor: Evan Dart

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Evan J. Basting

Elizabeth A. Goncy

References

Alegria

Vallas

Pumariega

A. J.

(2010). Racial and ethnic disparities in pediatric mental health. Child and Adolescent Psychiatric Clinics, 19(4), 759–774. https://doi.org/10.1016/j.chc.2010.07.001

Berry

O. O.

Tobón

A. L.

Njoroge

W. F.

(2021). Social determinants of health: The impact of racism on early childhood mental health. Current Psychiatry Reports, 23(5), 1–10. https://doi.org/10.1007/s11920-021-01240-0

Brown

T. A.

(2015). Confirmatory factor analysis for applied research. Guilford Publications.

Cauce

A. M.

Domenech-Rodríguez

Paradise

Cochran

B. N.

Shea

J. M.

Srebnik

Baydar

(2002). Cultural and contextual influences in mental health help seeking: A focus on ethnic minority youth. Journal of Consulting and Clinical Psychology, 70(1), 44–55. https://doi.org/10.1037/0022-006X.70.1.44

Cheung

G. W.

Rensvold

R. B.

(2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233–255. https://doi.org/10.1207/S15328007SEM0902_5

Dever

B. V.

Raines

T. C.

Dowdy

Hostutler

(2016). Addressing disproportionality in special education using a universal screening approach. The Journal of Negro Education, 85(1), 59–71. https://doi.org/10.7709/jnegroeducation.85.1.0059

Dowdy

Twyford

J. M.

Chin

J. K.

DiStefano

C. A.

Kamphaus

R. W.

Mays

K. L.

(2011). Factor structure of the BASC–2 Behavioral and Emotional Screening System Student Form. Psychological Assessment, 23(2), 379–387. https://doi.org/10.1037/a0021843

Eklund

Dowdy

(2014). Screening for behavioral and emotional risk versus traditional school identification methods. School Mental Health, 6(1), 40–49. https://doi.org/10.1007/s12310-013-9109-1

Farmer

E. M. Z.

Burns

B. J.

Phillips

S. D.

Angold

Costello

E. J.

(2003). Pathways into and through mental health services for children and adolescents. Psychiatric Services, 54(1), 60–66. https://doi.org/10.1176/appi.ps.54.1.60

10.

Girvan

E. J.

Gion

McIntosh

Smolkowski

(2017). The relative contribution of subjective office referrals to racial disproportionality in school discipline. School Psychology Quarterly, 32(3), 392–404. https://doi.org/10.1037/spq0000178

11.

Harrell-Williams

L. M.

Raines

T. C.

Kamphaus

R. W.

Dever

B. V.

(2015). Psychometric analysis of the BASC–2 Behavioral and Emotional Screening System (BESS) Student Form: Results from high school student samples. Psychological Assessment, 27(2), 738–743. https://doi.org/10.1037/pas0000079

12.

Kamphaus

R. W.

Reynolds

C. R.

(2007). BASC-2 Behavioral and Emotional Screening System (BASC-2 BESS). Pearson Clinical Assessments.

13.

Kauffman

J. M.

(1999). How we prevent the prevention of emotional and behavioral disorders. Exceptional Children, 65(4), 448–468. https://doi.org/10.1177/001440299906500402

14.

Kim

Kamphaus

R. W.

(2018). Investigation of factor structure and measurement invariance by gender for the Behavioral and Emotional Screening System among high school students. Psychological Assessment, 30(2), 231–240. https://doi.org/10.1037/pas0000469

15.

King

Reschly

Appleton

(2012). An examination of the validity of the Behavioral and Emotional Screening System in a rural elementary school: Validity of the BESS. Journal of Psychoeducational Assessment, 30, 527–538. https://doi.org/10.1177/0734282912440673

16.

Meade

A. W.

(2005, April). Sample size and tests of measurement invariance [Conference session]. Annual Conference of the Society for Industrial and Organizational Psychology, Los Angeles, CA, United States.

17.

Miller

F. G.

Cohen

Chafouleas

S. M.

Riley-Tillman

T. C.

Welsh

M. E.

Fabiano

G. A.

(2015). A comparison of measures to screen for social, emotional, and behavioral risk. School Psychology Quarterly, 30(2), 184–196. https://doi.org/10.1037/spq0000085

18.

Muthén

Asparouhov

(2002). Latent variable analysis with categorical outcomes: Multiple-group and growth modeling in Mplus. Mplus Web Notes, 4(5), 1–22.

19.

Muthén

L. K.

Muthén

(2021). Mplus. The comprehensive modelling program for applied researchers: User’s guide, 8.

20.

Naser

S. C.

Brown

Verlenden

(2018a). The utility of universal screening to guide school-based prevention initiatives: Comparison of office discipline referrals to standardized emotional and behavioral risk screening. Contemporary School Psychology, 22(4), 424–434. https://doi.org/10.1007/s40688-018-0173-2

21.

Naser

S. C.

Dever

B. V.

(2020). A preliminary investigation of the reliability and validity of the BESS-3 teacher and student forms. Journal of Psychoeducational Assessment, 38(2), 263–269. https://doi.org/10.1177/0734282919837825

22.

Naser

S. C.

Hitti

Overstreet

(2018b). The Behavioral and Emotional Screening System Student Form: Is there evidence of a global at-risk factor in a sample of predominantly African American youth? Journal of Psychoeducational Assessment, 36(5), 446–463. https://doi.org/10.1177/0734282916686015

23.

Reynolds

C. R.

Kamphaus

R. W.

(2015). BASC-3 Behavioral and Emotional Screening System manual. Pearson.

24.

Rhemtulla

Brosseau-Liard

P. É.

Savalei

(2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17(3), 354–373. https://doi.org/10.1037/a0029315

25.

Salle

T. L.

George

H. P.

McCoach

D. B.

Polk

Evanovich

L. L.

(2018). An examination of school climate, victimization, and mental health problems among middle school students self-identifying with emotional and behavioral disorders. Behavioral Disorders, 43(3), 383–392. https://doi.org/10.1177/0198742918768045

26.

Skiba

R. J.

Horner

R. H.

Chung

C.-G.

Rausch

M. K.

May

S. L.

Tobin

(2011). Race is not neutral: A national investigation of African American and Latino disproportionality in school discipline. School Psychology Review, 40(1), 85–107. https://doi.org/10.1080/02796015.2011.12087730

27.

Splett

J. W.

Garzona

Gibson

Wojtalewicz

Raborn

Reinke

W. M.

(2019). Teacher recognition, concern, and referral of children’s internalizing and externalizing behavior problems. School Mental Health, 11(2), 228–239. https://doi.org/10.1007/s12310-018-09303-z

28.

Stevanovic

Jafari

Knez

Franic

Atilola

Davidovic

. . .Lakic

(2017). Can we really use available scales for child and adolescent psychopathology across cultures? A systematic review of cross-cultural measurement invariance data. Transcultural Psychiatry, 54(1), 125–152. https://doi.org/10.1177/1363461516689215

29.

Sullivan

A. L.

Bal

(2013). Disproportionality in special education: Effects of individual and school variables on disability risk. Exceptional Children, 79(4), 475–494. https://doi.org/10.1177/001440291307900406

30.

Individuals with Disabilities Education Improvement Act of 2004, 20 U.S.C. § 1400 et seq. (2004)

31.

Van de Schoot

Lugtig

Hox

(2012). A checklist for testing measurement invariance. European Journal of Developmental Psychology, 9(4), 486–492. https://doi.org/10.1080/17405629.2012.686740

32.

von der Embse

Kim

E. S.

Kilgus

Dedrick

Sanchez

. (2019). Multi-informant universal screening: Evaluation of rater, item, and construct variance using a trifactor model. Journal of School Psychology, 77, 52–66. https://doi.org/10.1016/j.jsp.2019.09.005