The adolescent behavioral index: Identifying students at risk for disengagement in high school

Abstract

BACKGROUND:

Proposed frameworks of college and career readiness include aspects of school climate perceptions and behavioral engagement. School climate initiatives emphasize the importance of healthy learning environments; while behavioral indicators tend to be used to identify students at-risk for disengagement, established early warning systems include few, if any, behavioral indicators. A promising and useful screening mechanism that combines behavioral indicators along with school climate perceptions may enhance identification of adolescents at risk for disengagement, academic failure, and dropout, and may ultimately improve college and career readiness.

OBJECTIVE:

The purpose of this study was to examine the relationship between self-reported adolescent perceptions of school climate and extant high school academic and behavioral data to create an index score which could be used as an early warning system for student disengagement.

METHODS:

To accomplish the stated objective, we employed structural equation modeling to estimate a multiple indicators multiple causes model using a sample of high school students (n = 5039) with and without disabilities. Extant school data included grade point average, attendance, tardies, and office discipline referrals, while school climate was measured using the Georgia Brief School Climate survey.

RESULTS:

Findings show evidence of a robust Adolescent Behavioral Index when controlled for disability status as those with disabilities are expected to have lower index scores indicating greater risk for school disengagement.

CONCLUSION:

Implications for research and practice are focused on the Adolescent Behavioral Index as a mechanism for systematic data collection that may underlie early warning systems in high schools specifically when designing college and career readiness interventions for adolescents and when identifying students who may be at risk for disengagement.

Keywords

Early warning system high school behavioral indicators multiple-indicator multiple-causes model school climate special education

1 Introduction

School climate initiatives emphasize the importance of healthy learning environments (e.g., Federal School Climate Transformation Grants, United States Department of Education, 2014). Advances in measuring school climate perceptions help to clarify the importance of school engagement among adolescents (White et al., 2014) and proposed frameworks of college and career readiness include aspects of school climate perceptions and behavioral engagement (Lombardi et al., 2018; Morningstar et al., 2017). Behavioral indicators based on attendance and discipline tend to be used to identify students at-risk for disengagement and in need of more intensive behavioral supports (e.g., McIntosh & Goodman, 2016); yet, established early warning systems include few, if any, behavioral indicators (e.g., Davis et al., 2019; Mac Iver et al., 2019). Combining behavioral indicators along with school climate perceptions may be a promising and useful screening mechanism to identify adolescents at risk for disengagement, academic failure, and dropout, and ultimately better promote and support college and career readiness.

The purpose of this study was to construct a measurement model of self-reported perceptions of school climate and extant high school academic and behavioral data for adolescents with and without disabilities. The current study is unique in that a Multiple Indicator Multiple Causes (MIMIC) model was tested to generate an index score, which we henceforth refer to as the Adolescent Behavioral Index. Formation and validation of such an index may advance school climate and college and career readiness intervention efforts concurrently, and ultimately inform early warning systems in high schools. While there is scant research on modeling academic, behavioral, and perception data together, the literature base is rich in each of these areas individually.

1.1 School climate

Measuring adolescent perceptions about the fairness of school rules, their own safety while at school, as well as how they feel about teachers and administrators in the building is the focus of school climate instruments (e.g., Bradshaw et al., 2014;Fan et al., 2011;O’Malley et al., 2015; White et al., 2014). Further, evidence suggest these perceptions decline at the secondary level (Jones & Shindler, 2016), even though positive perceptions of school climate are related to improved academic performance (Goddard et al., 2015; Jones & Shindler, 2016; McIntosh & Goodman, 2016).

Among established school climate measures, little is known about the inclusion of students with disabilities in school-wide efforts to measure school climate as well as the functioning of these measures across student subgroups. In fact, disability category is rarely reported as a study sample characteristic, and in some cases certain students were intentionally excluded from school climate studies based on their disability category (Cornell et al., 2016). La Salle and colleagues (2018) compared students with emotional disturbance who receive special education services to their peers without disabilities (La Salle et al., 2018). While this study was informative with regard to students with emotional disturbance, there was no examination of school climate responses across all students eligible for special education services that represent a range of disability categories.

More recently, adolescents with and without disabilities were the focus of a construct validity study (Rifenbark et al., 2021), where a widely-used school climate measure was examined for measurement invariance as well as the relationship to achievement for adolescents with and without disabilities. This study was particularly notable in that measurement invariance on the basis of disability was established for a widely-used school climate measure, yet, no extant school behavioral data were considered.

1.2 Attendance

Students must be present in school in order to engage in and be exposed to learning opportunities. Poor school attendance is a significant contributor to negative outcomes for students overall (Balfanz, 2016) and specifically related to student perceptions of poor school climate (Van Eck et al., 2017), poor academic performance (Baker et al., 2001; Balfanz & Byrnes, 2012; Feldman et al., 2014), behavioral difficulties (Feldman et al., 2014; McIntosh et al., 2010), and high school dropout rates (Balfanz & Byrnes, 2012;Henry et al., 2012;Neild & Balfanz, 2006;Rumberger & Lim, 2008). Poor attendance is also linked to challenges outside of school including increased juvenile crime (Baker et al., 2001), as well as adult substance abuse, criminal activity, and mental and physical health concerns (Baker et al., 2001; Henry et al., 2012; Rumberger & Lim, 2008).

High absenteeism is a sign of disengagement and potential dropout. Dropping out of school has serious negative outcomes for youth, including an increased likelihood of being unemployed, underemployed, and dependent on welfare (Belfield & Levin, 2007). Youth with disabilities who drop out are much more likely to have been arrested, stopped by police, or incarcerated (Sanford et al., 2011). The types of jobs, salary, and opportunities for career advancements among youth with disabilities has been reported as lower than among those who graduated but without disabilities (Trainor et al., 2013). Clearly, youth with disabilities who drop out of school are at a distinct disadvantage as they strive for independence and self-sufficiency in adulthood. Thus, efforts to improve attendance can be critical for schoolwide dropout prevention efforts, and adolescents with disabilities may be more in need of intensive supports in this area.

1.3 Tardies

Collecting tardies as an additional behavioral variable is a growing area of the literature and has only recently been included in large-scale statistical modeling efforts in educational research partly due to availability and access. Although limited, tardiness is associated with decreased academic performance for students in classes with high rates of tardiness (Gottfried, 2014). Most studies examining the effects of missing class time on school outcomes focus on absences and fail to account for the missed class time associated with tardies (Gottfried, 2009; Gottfried, 2014). Despite the limited research on tardies, it is a documented source of concern (Gottfried, 2014) and the subject of intervention research (Caldarella et al., 2011;Johnson-Gros et al., 2008). Moreover, students who reported feeling more connected to school had fewer instances of tardies, absences, and course failures (Churchill et al., 2021).

1.4 Office discipline referrals

Office Discipline Referrals (ODR) are defined as the documentation of an observed school rule violation (Irvin et al., 2006) and are important behavioral indicators. Improvements in behavior, as measured by ODR rates, are associated with improved academic outcomes for students with (Sanford & Horner, 2013) and without disabilities (Algozzine et al., 2011); whereas high rates of problem behavior are associated with increased dropout rates (Sweeten, 2006) as well as closely related to academic difficulty (McIntosh, Flannery et al., 2008).

For individual students, ODRs have been a) combined with other data sources to identify the function of a student’s behavior (March & Horner, 2002;McIntosh, Brown et al., 2008;McIntosh, Horner et al., 2008), b) used to assess intervention effectiveness (McIntosh et al., 2009;Todd et al., 2008), and c) used as a behavioral screener (Hawken et al., 2008). Concerns have been raised about the use of ODRs as a valid indicator of problem behaviors given the variability in referral systems across schools and the subjectivity of the discipline process (Irvin et al., 2004;Skiba et al., 2014). However, other findings indicate that ODRs are moderately valid indicators of student behavior and useful for both research and applied practice (Pas et al., 2011), and ODRs received early in the school year are moderately predictive of the school year total (McIntosh et al., 2010).

1.5 Achievement

In a recent literature synthesis, Welch and colleagues (2017) named grade point average (GPA) as one of several critical academic indicators that has substantial predictive validity of later adult outcomes such as college persistence and completion. GPA is typically used as a measure of academic progress relevant to college and career readiness rather than statewide achievement test scores or college admissions exams (American Institutes for Research, 2014) and evidence shows GPA is a stronger predictor of postsecondary achievement (Allensworth & Clark, 2020;Geiser & Santelices, 2007;Hodara & Lewis, 2017). GPA is also one of several indicators typically used in early warning systems (Allensworth & Easton, 2007), which is considered an evidence-based practice to support college and career readiness in high school (American Institutes for Research, 2017).

Combined, the aforementioned variables- attendance, tardies, ODRs- have strong support in the literature of important indicators to consider when using extant school data to identify students at-risk for disengagement and in need of more intensive behavioral supports. However, these indicators tend to be examined separately. Furthermore, only attendance has been used in early warning systems (Allensworth & Easton, 2007) and prior research demonstrating effectiveness did not include tardies or ODRs (Davis et al., 2019; Mac Iver et al., 2019). Combining these indicators along with school climate perceptions and achievement may be a promising and useful screening mechanism to identify students at risk for disengagement, academic failure, and dropout. As such, in the current study we leveraged various data sources by incorporating school climate perceptions and the observed variables of attendance, tardies, GPA, and ODRs to create the Adolescent Behavioral Index. Specifically, we addressed the research questions: (a) Can we adequately model adolescent school climate perceptions and extant school data using a MIMIC approach? and (b) do adolescents with and without disabilities differ in their index scores?

2 Method

2.1 Participants

Participants were adolescents in grades 9–12 at 13 public high schools in urban and suburban districts in a Midwestern state. Table 1 shows detailed sample characteristics. Students with (n = 784) and without (n = 4253) disabilities were included in the sample. Of the students with disabilities, the majority of them fell into either the Learning Disability (44%) or Other Health Impairment (36%) categories. Compared to the national average (National Center for Education Statistics, 2014), this sample included a high percentage of African American students overall (41% in sample, 15.7% nationally) and within special education (49% in sample, 15.3% nationally). In this study, we intentionally grouped all students with disabilities together in order to better understand how students who receive special education services compare with their peers in general education with regard to index scores.

Table 1
Sample characteristics

Group Overall SPED Non-SPED

n 5039 784 4253

Gender

Males 51% 67% 49%

Females 49% 33% 51%

Ethnicity

Caucasian 27% 25% 28%

African American 41% 49% 40%

Hispanic/Latino 24% 22% 25%

Asian 5% 2% 6%

Other 2% 1% 2%

Grade

9th 35% 40% 35%

10th 27% 26% 27%

11th 23% 23% 23%

12th 15% 10% 15%

Free/reduced lunch

Yes 61% 69% 60%

No 39% 31% 40%

Disability status

Yes 16% 100% 0%

No 84% 0% 100%

Disability

Learning disability – 44% –

Other health impairment – 36% –

Emotional disturbance – 6% –

Autism Spectrum Disorder – 5% –

Intellectual disability – 5% –

Other – 4% –

Group	Overall	SPED	Non-SPED
n	5039	784	4253
Gender
Males	51%	67%	49%
Females	49%	33%	51%
Ethnicity
Caucasian	27%	25%	28%
African American	41%	49%	40%
Hispanic/Latino	24%	22%	25%
Asian	5%	2%	6%
Other	2%	1%	2%
Grade
9th	35%	40%	35%
10th	27%	26%	27%
11th	23%	23%	23%
12th	15%	10%	15%
Free/reduced lunch
Yes	61%	69%	60%
No	39%	31%	40%
Disability status
Yes	16%	100%	0%
No	84%	0%	100%
Disability
Learning disability	–	44%	–
Other health impairment	–	36%	–
Emotional disturbance	–	6%	–
Autism Spectrum Disorder	–	5%	–
Intellectual disability	–	5%	–
Other	–	4%	–

Note. Overall n = 10,735 including survey responders and non-responders, and contains 2 more observations.

2.2 Procedures

We recruited participating schools through existing relationships with technical assistance providers connected with the National Technical Assistance Center for Positive Behavioral Interventions and Supports (PBIS). Interested administrators followed up with researchers to volunteer for the study and signed a data access agreement to release specified school data. Students took the survey using the online survey program Qualtrics on school-based computers. An administration window for data collection was determined between the researchers and school partners that was 3-4 weeks long. School administrators determined time of day and class period(s) in which to administer the survey, and they were asked to ensure that all students were given the opportunity to take the survey. The survey was designed to take 30–50 minutes, and students were given one class period to respond. Students gave assent to participate by answering yes to question 1 “I would like to take these surveys”. During administration, students were instructed to ask for help if they had any questions or wished to have one or more items read aloud by an adult. Upon this request, we instructed staff members to read aloud the specified item, then step away from the computer and allow the student to respond independently. School personnel were told to provide accommodations (e.g., extra time, translated materials) for any students and/or families for which these accommodations are routinely provided. All items were optional and students were able to discontinue the survey at any point without penalty. During the survey administration window, research team members monitored the number of responses and coordinated follow-up survey administrations with school contacts if needed (e.g., the number of respondents was considerably low after the first two weeks). Students were asked to enter their school-issued identification numbers at the beginning of the survey. After survey administration, we matched student responses with extant school data using the identification numbers. All study protocols were approved by the institutional review board for the protection of human subjects.

2.3 Measures

We collected student response data and observed variables from school data sources. Table 2 shows descriptive statistics on all study variables.

Table 2
Descriptive statistics

Georgia Brief School Climate Inventory n Strongly Somewhat Somewhat Strongly

disagree disagree agree agree

I like school. 3971 15% 16% 51% 18%

I feel successful at school. 3946 8% 16% 52% 25%

I feel my school has high standards for achievement. 3938 8% 16% 44% 32%

My school sets clear rules for behavior. 3940 7% 14% 45% 34%

The behaviors in my classroom allow the teacher to teach so I can learn. 3932 8% 17% 48% 27%

Students are frequently recognized for good behavior. 3940 15% 25% 44% 17%

I know an adult at school that I can talk with if I need help. 3931 10% 13% 42% 35%

School is a place at which I feel safe. 3946 10% 17% 49% 24%

Teachers treat me with respect. 3966 7% 14% 47% 32%

Academic and behavioral data n Mean (SD) 25^th Median 75^th

ABSENT 5039 14.74 (17.35) 4 9 18.50

TARDY 5039 8.63 (18.31) 0 2 8

ODR 5039 1.64 (5.08) 0 0 1

GPA 5037 2.29 (0.95) 1.63 2.27 3.00

Georgia Brief School Climate Inventory	n	Strongly	Somewhat	Somewhat	Strongly
I like school.	3971	15%	16%	51%	18%
I feel successful at school.	3946	8%	16%	52%	25%
I feel my school has high standards for achievement.	3938	8%	16%	44%	32%
My school sets clear rules for behavior.	3940	7%	14%	45%	34%
The behaviors in my classroom allow the teacher to teach so I can learn.	3932	8%	17%	48%	27%
Students are frequently recognized for good behavior.	3940	15%	25%	44%	17%
I know an adult at school that I can talk with if I need help.	3931	10%	13%	42%	35%
School is a place at which I feel safe.	3946	10%	17%	49%	24%
Teachers treat me with respect.	3966	7%	14%	47%	32%
Academic and behavioral data	n	Mean (SD)	25^th	Median	75^th
ABSENT	5039	14.74 (17.35)	4	9	18.50
TARDY	5039	8.63 (18.31)	0	2	8
ODR	5039	1.64 (5.08)	0	0	1
GPA	5037	2.29 (0.95)	1.63	2.27	3.00

2.3.1 Georgia brief school climate inventory

The Georgia Brief Scale (GBS; White et al., 2014) measures student perceptions of school climate and contains 9 ordinal response items each containing four response levels (e.g., (1) strongly disagree; (2) somewhat disagree; (3) somewhat agree; (4) strongly agree). The GBS is a unidimensional school climate measure intended to be brief in order for ease of administration and to measure change over time (White et al., 2014). Although brief, the one-factor measure purports to produce a global score of school climate that maps onto teaching and learning, relationships, and safety. While first developed for the state of Georgia, the GBS is nationally endorsed by the PBIS Center, has been used in more than 100 high schools in 19 states (La Salle et al., 2018), and functions similarly for students with and without disabilities (Rifenbark et al., 2021). In the current sample consisting of students in grades 9–12, the internal consistency was estimated to be 0.87 (95% C.I.: 0.86, 0.87) for the full sample, 0.88 (95% C.I.: 0.86, 0.89) for those with disabilities, and 0.87 (95% C.I.: 0.86, 0.87) for those without disabilities.

2.3.2 Disability status

“Disability” was defined as those students who have an Individualized Education Program (IEP) and receive special education services. We gathered student disability status (Yes/No) using extant school data records. Students were coded as a “Yes” if they had a current IEP. Students with disabilities were the reference category.

2.3.3 Cumulative grade point average

Cumulative grade point average (GPA) was gathered from student records and was recorded on a scale ranging from 0.0 = “F” to 4.0 = “A.” For the current sample, the mean estimate for GPA was 2.29 with a standard deviation of 0.95. The minimum GPA observed was 0.00, the median GPA observed was 2.27, and the maximum GPA observed was 4.38.

2.3.4 Attendance

Attendance (ABSENT) was gathered from extant school data as the total number of days a given student was absent in the school year and incorporated both full and half days. The modal number of absences observed was 2.

2.3.5 Office discipline referrals

Office discipline referrals (ODRs) were gathered from extant school data and represented the number of times a given student was sent to the office due to problem behavior. The modal number of ODRs observed was 0. As school records were used to gather these data, we relied on school definitions of problem behavior.

2.3.6 Tardies

Tardies (TARDY) data were gathered from school records and represent the number of times a student was late to class. The modal number of tardies observed was 0. As school records were used to gather these data, we relied on school definitions of tardies.

2.4 Data analysis

To investigate the relationship between School Climate and extant school data (e.g., ODRs) we estimated a structural equation model that incorporated both causal-indicators and effect-indicators. Broadly speaking, observed variables commonly utilized in measurement models are effect-indicators, causal-indicators, and covariates (Bollen & Bauldry, 2011). Before formally introducing the latent variable model, we briefly discuss these different types of observed variables to provide a rationale for our approach to modeling.

Covariates are observed variables that, if omitted, result in bias. In the context of our study, disability status is an important covariate to include. Specifically, disability status is neither a cause nor an effect of the intended construct, rather inferences must be adjusted based on disability status. Causal- and Effect-Indicators both result in a latent variable, and for this reason the distinction between effect-indicators and causal-indicators is not straightforward. Although, there is no absolute certainty whether observed variables should be treated as causal-indicators or effect-indicators, Bollen and Bauldry (2011) suggest theoretical and empirical methods to help decipher their nature. The definition or meaning of a latent variable that is properly modeled with effect-indicators will be similar to that of a latent variable estimated with a reduced set of effect indicators –this is the assumption of interchangeability (Bollen & Bauldry, 2011). Conversely, if a latent variable is properly modeled via causal-indicators its definition would be altered in a non-negligible way if a causal-indicator were removed.

With recursive measurement, factor loadings are interpreted as the expected change in the effect-indicators per unit change in the latent variable and correspond to the variance that is shared among the effect-indicators (Bollen, 1989). Therefore, given a unit change in the latent variable, all effect-indicators are expected to change. When measurement models utilize causal-indicators, the estimated parameters are structural and represent unique variance explained by each of the causal-indicators. Therefore, relying on this notion of directionality, we can test whether it is reasonable to treat hypothesized causal-indicators as effect-indicators.

2.4.1 School climate

All nine GBS items were treated as effect-indicators. With regard to directionality of the cause and effect, we deemed it reasonable that an increase in School Climate would then increase the probability that positive responses would be given across all nine GBS items. Therefore, student responses are manifestations of their perception of School Climate. Due to the GBS items containing four ordinal response options, we treated them as categorical.

2.4.2 Extant school data

We identified the following variables as causal-indicators of Behavior: ABSENT, TARDY, and ODRs. We based this decision on the implications inherent in causal-indicator modeling. First, if we were to drop the ODRs causal-indicator, Behavior would lose all information about behavioral problems and thus inherently change its meaning. Second, if we were to envision these observed variables as effect-indicators, we must ask whether it is reasonable to expect that given a change in Behavior, would this translate to changes across all three observed variables? We surmise that it is possible for a positive increase in Adolescent Behavior could ‘cause’ a decrease in ODRs; however, this is not necessarily the case for ABSENT and TARDY. Moreover, a change in Behavior will not cause the students to be absent or tardy on a regular basis. As such, ABSENT, TARDY, and ODRs are modeled as causal-indicators and inform Adolescent Behavior. GPA was treated as an effect-indicator of Adolescent Behavior (see model specification below and Fig. 1).

Fig. 1

Path diagram of hypothesized multiple-indicators multiple-causes model. Note. ABSENT, TARDY, and ODRs are causal-indicators of Adolescent Behavior. School Climate and Latent GPA are the modeled outcomes, each informed by effect-indicators. Arrows above each latent variable (denoted by circles) represent latent disturbance.

2.4.3 Causal-indicator modeling

Causal-indicator modeling possesses challenges with respect to model identification when trying to estimate a latent disturbance (Bollen & Bauldry, 2011). This is necessary as we acknowledge that the construct (Adolescent Behavior) may not be fully explicated. The disturbance parameter then, represents variance left unexplained by our identified set of causal-indicators and assists in determining the extent to which the construct is captured (Bollen, 1989). The Multiple Indicators Multiple Causes (MIMIC; Jöreskog & Goldberger, 1975) model not only provides an avenue for proper identification but it is also in concert with our theoretical model and is utilized in educational research (see Tighe et al., 2015).

2.4.4 Multiple indicators multiple causes

Figure 1 depicts a path diagram of our hypothesized MIMIC model. As noted earlier, ABSENT, TARDY, and ODRs are causal indicators of the Adolescent Behavioral Index. We hypothesize students’ school climate perceptions are informed (or caused) by their Adolescent Behavior; therefore, an arrow is emitted from Adolescent Behavior to School Climate. Subsequently, arrows are emitted from School Climate to each of the nine GBS items, as these are effect indicators and we set the scale for School Climate by fixing the first factor loading to 1.0. We also posited Adolescent Behavior has an effect on GPA and therefore, inserted GPA into the model as an observed latent variable (see Little, 2013). For identification purposes, the structural path from Adolescent Behavior to GPA was fixed to 1.0 in order to freely estimate the latent disturbance and successfully identify this model (Bollen & Bauldry, 2011). All models were estimated in Mplus version 8.4 (Muthen & Muthen, 2019) using its weighted least squares estimator, WLSMV with THETA parameterization, due to the GBS items being ordered variables (Bollen, 1989). The specification of our hypothesized model in Mplus can be found in the Appendix.

3 Results

To empirically investigate whether it was feasible to treat the GBS items as effect-indicators and ABSENT, TARDY, and ODRs as causal-indicators we estimated correlations among these variables. As hypothesized, all GBS items had medium to large correlation estimates ranging from 0.34 to 0.54 indicating that it was reasonable to treat them as effect-indicators of School Climate (Bollen & Bauldry, 2011). With respect to ABSENT, TARDY, and ODRs, we observed a medium effect between ABSENT and ODRs (r = 0.3) while the remaining correlations were 0.14 or smaller. Therefore, we deemed our treatment of observed variables as acceptable and proceeded to fit the MIMIC model using the entire sample. Table 3 contains the estimated correlation matrix. In terms of sufficient statistics, the mean numbers of absences and tardies were 14.74 (SD = 17.35) and 8.63 (SD = 18.31), respectively; while the mean number of ODRs was 1.64 (SD = 5.08).

Table 3
Bivariate correlation among study variables

1 2 3 4 5 6 7 8 9 10 11 12 13

1. I like school. (GBS1) 1

2. I feel successful at school. (GBS2) 0.51 1

3. I feel my school has high standards for achievement. (GBS3) 0.40 0.41 1

4. My school sets clear rules for behavior. (GBS4) 0.35 0.35 0.54 1

5. The behaviors in my classroom allow the teacher to teach so I can learn. (GBS5) 0.37 0.38 0.48 0.50 1

6. Students are frequently recognized for good behavior. (GBS6) 0.38 0.36 0.45 0.45 0.49 1

7. I know an adult at school that I can talk with if I need help. (GBS7) 0.36 0.39 0.34 0.36 0.34 0.36 1

8. School is a place at which I feel safe. (GBS8) 0.43 0.41 0.44 0.46 0.50 0.48 0.43 1

9. Teachers treat me with respect. (GBS9) 0.40 0.43 0.42 0.45 0.46 0.44 0.46 0.49 1

10. GPA 0.14 0.26 0.16 0.10 0.12 0.12 0.13 0.18 1

11. ABSENT –0.09 –0.11 –0.15 –0.13 –0.10 –0.05 –0.09 –0.10 –0.46 1

12. TARDY –0.06 –0.09 –0.07 –0.04 –0.05 –0.20 0.11 1

13. ODR –0.04 –0.05 –0.09 –0.06 –0.04 –0.05 –0.06 –0.29 0.30 0.14 1

	1	2	3	4	5	6	7	8	9	10	11	12	13
1. I like school. (GBS1)	1
2. I feel successful at school. (GBS2)	0.51	1
3. I feel my school has high standards for achievement. (GBS3)	0.40	0.41	1
4. My school sets clear rules for behavior. (GBS4)	0.35	0.35	0.54	1
5. The behaviors in my classroom allow the teacher to teach so I can learn. (GBS5)	0.37	0.38	0.48	0.50	1
6. Students are frequently recognized for good behavior. (GBS6)	0.38	0.36	0.45	0.45	0.49	1
7. I know an adult at school that I can talk with if I need help. (GBS7)	0.36	0.39	0.34	0.36	0.34	0.36	1
8. School is a place at which I feel safe. (GBS8)	0.43	0.41	0.44	0.46	0.50	0.48	0.43	1
9. Teachers treat me with respect. (GBS9)	0.40	0.43	0.42	0.45	0.46	0.44	0.46	0.49	1
10. GPA	0.14	0.26	0.16	0.10	0.12		0.12	0.13	0.18	1
11. ABSENT	–0.09	–0.11	–0.15	–0.13	–0.10		–0.05	–0.09	–0.10	–0.46	1
12. TARDY	–0.06	–0.09	–0.07	–0.04					–0.05	–0.20	0.11	1
13. ODR	–0.04	–0.05	–0.09	–0.06	–0.04			–0.05	–0.06	–0.29	0.30	0.14	1

Note. Only significant correlation coefficients are reported above at the p < 0.05 level.

3.1 MIMIC model

Upon estimating the MIMIC model, we observed a significant χ² test for model fit (χ²: 935.281_df :61; p < 0.001); however, when consulting the AFIs we find acceptable fit. Specifically, the RMSEA and SRMR were estimated to be 0.053 (90% C.I.: 0.050, 0.056) and 0.036, respectively. With respect to the incremental fit indices, the CFI and TLI were estimated to be 0.974 and 0.967, respectively. Having surpassed the recommended values for the AFIs above (e.g., greater than or equal to 0.95 for CFI and TLI), we then tested for the construction of the Adolescent Behavioral Index and examined its impact on School Climate.

Next, we estimated a series of models to evaluate the impact of each of the causal-indicators to determine the extent to which these variables decreased the disturbance parameter (ζ₁) and to better understand their impact on Adolescent Behavior. Specifically, we compared ζ₁ (i.e., the latent disturbance) when only one causal-indicator was modeled, while retaining both outcomes: School Climate and GPA. The parameter ζ₁ ranged from 0.756 (ODR only) to 0.419 (ABSENT only) indicating the importance of ABSENT on the Adolescent Behavioral Index. When modeling all three causal-indicators, each had a negative impact on the Adolescent Behavioral Index, their standardized coefficient estimates were: ABSENT (γ= –0.481, SE = 0.028), ODR (γ= –0.175, SE = 0.013), and TARDY (γ= –0.161, SE = 0.017), resulting in an unstandardized estimate of 0.414 (SE = 0.069) for ζ₁. In sum, a total of 36% of the variance in Adolescent Behavior (R² = 0.360, SE = 0.038) was explained. In terms of School Climate, the GBS survey items demonstrated near equivalence with respect to the marker variable; specifically, these loadings ranged from 0.893 to 1.128. When regressing School Climate onto the Adolescent Behavioral Index we observed a positive effect which was significantly different from zero –its standardized estimate was 0.248 (SE = 0.021).

3.2 MIMIC X disability status

Next, we fit the MIMIC model independently based on disability status to determine whether we observed the same pattern across groups. This allowed us to determine if disability status should be entered as a covariate. For students without disabilities, we found the MIMIC model to have acceptable fit to the data (χ²: 935.281_df :61; p < 0.001). The CFI and TLI were 0.972 and 0.964, respectively; while the RMSEA and SRMR were 0.055 (90% C.I.: 0.052, 0.059) and 0.037, respectively. As hypothesized, each of the three causal indicators were negative and significantly different from zero. Specifically, the standardized estimates for ABSENT, TARDY, and ODRs were –0.444 (SE: 0.030), –0.186 (SE: 0.017), and –0.176 (SE: 0014). The parameter ζ₁ for the Adolescent Behavioral Index was estimated to be 0.450 (SE: 0.080), leading to an R² of 0.328 (SE: 0.039). With respect to the impact of the Adolescent Behavioral Index on School Climate, we found this to be positive and significantly different from zero, with a standardized estimate of 0.245 (SE: 0.023).

For students with disabilities, we found acceptable data-model fit (χ²: 174.966_df :61; p < 0.001). The CFI and TLI were estimated to be 0.978 and 0.971, respectively; while the RMSEA and SRMR were estimated to be 0.049 (90% C.I.: 0.040, 0.057) and 0.039, respectively. Of the Adolescent Behavioral Index causal-indicators, we observed a non-significant standardized structural path emitting from TARDY (Est: 0.038, SE: 0.071), while ABSENT (Est.: –0.614, SE: 0.066) and ODRs (Est.: –0.133, SE: 0.004) were significantly different from zero. The parameter ζ₁ for the Adolescent Behavioral Index was estimated to be 0.184 (SE: 0.074) leading to an R² of 0.486 (SE: 0.094). Further, as anticipated, we observed a positive effect of the Adolescent Behavioral Index on School Climate, with a standardized estimate of 0.315 (SE: 0.054).

3.3 Final model

Due to the non-significant effect of TARDY on Adolescent Behavior when modeling data from students with disabilities independently, we proceeded to fit a final MIMIC model using the full sample that incorporated disability status as a covariate. This model was found to fit the data well with the CFI and TLI estimated to be 0.974 and 0.967, respectively; whereas, the RMSEA and SRMR were 0.05 (90% C.I.: 0.047, 0.053) and 0.037, respectively. In this model, the standardized effect of disability was estimated to be 0.17 (SE: 0.021), while all other structural parameters were similar to before. Table 4 shows these estimates.

Table 4
Standardized estimates of structural model

Estimate Standard error

Regressions

Adolescent behavior on:

Absent (γ₁₁) –0.44 0.03

Tardy (γ₂₁) –0.16 0.02

ODR (γ₃₁) –0.14 0.01

No disability (γ₄₁) 0.17 0.02

School climate on:

Adolescent behavior (β₁₂) 0.23 0.02

Latent GPA on:

Adolescent behavior (β₁₃) 0.92 0.05

Correlations

Absent with Tardy 0.12 0.01

Absent with ODR 0.30 0.01

Absent with No disability –0.13 0.01

Tardy with ODR 0.14 0.01

Tardy with No disability –0.01 0.02

ODR with No disability –0.16 0.01

Latent disturbances

Adolescent behavior (ζ) 0.49 0.08

School climate (ζ) 0.79 0.04

Latent GPA (ζ) 0.15 0.08

Latent R²

Adolescent behavior 0.35 0.04

School climate 0.05 0.01

Latent GPA 0.84 0.09

Model fit χ² = 959.58 df = 70

RMSEA (90% C.I.: LB, UB) 0.05 (0.047, 0.053)

SRMR 0.04

CFI 0.97

TLI 0.97

	Estimate	Standard error
Regressions
Adolescent behavior on:
Absent (γ₁₁)	–0.44	0.03
Tardy (γ₂₁)	–0.16	0.02
ODR (γ₃₁)	–0.14	0.01
No disability (γ₄₁)	0.17	0.02
School climate on:
Adolescent behavior (β₁₂)	0.23	0.02
Latent GPA on:
Adolescent behavior (β₁₃)	0.92	0.05
Correlations
Absent with Tardy	0.12	0.01
Absent with ODR	0.30	0.01
Absent with No disability	–0.13	0.01
Tardy with ODR	0.14	0.01
Tardy with No disability	–0.01	0.02
ODR with No disability	–0.16	0.01
Latent disturbances
Adolescent behavior (ζ)	0.49	0.08
School climate (ζ)	0.79	0.04
Latent GPA (ζ)	0.15	0.08
Latent R²
Adolescent behavior	0.35	0.04
School climate	0.05	0.01
Latent GPA	0.84	0.09
Model fit	χ² = 959.58	df = 70
RMSEA (90% C.I.: LB, UB)	0.05	(0.047, 0.053)
SRMR	0.04
CFI	0.97
TLI	0.97

4 Discussion

These findings support the need to examine both observed measured variables and self-reported perception data in educational research in high schools, specifically when designing college and career readiness interventions for adolescents and when identifying students who may be at risk for disengagement. This demonstrated MIMIC model posits that attendance, tardies, and office discipline referrals make unique contributions and inform the Adolescent Behavioral Index. Notably, we found adolescents with disabilities tend to have lower index scores as compared to their peers without disabilities and therefore, is an important covariate to include. Specifically, the Adolescent Behavioral Index among adolescents without disabilities are expected to be 0.17 standard deviations above adolescents with disabilities indicating that students with disabilities are expected to have lower index scores and in-turn are expected to then have lower perceptions of school climate and academic achievement. Further, we found a positive relationship between the Adolescent Behavioral Index and School Climate. In our final model that controlled for disability status, for a one-unit increase in the Adolescent Behavioral Index (i.e., indicating fewer tardies, absences, and ODRs), student perceptions of school climate are expected to increase by 0.23 standard deviations. These findings show a preliminary connection between behavioral engagement and climate, an important contribution to the school climate literature base.

Our study findings are unique in that behavioral variables are used in tandem with adolescent perception data, and modeled together to produce a potential early warning system. Established early warning systems include primarily academic data (e.g., course completion, grades), along with attendance (Allensworth & Easton, 2005, 2007; American Institute for Research, 2017; Davis et al., 2019; Mac Iver et al., 2019). In a recent report on early warning systems, 79% of high schools from a national sample reported using discipline incidents defined as suspensions and expulsions as part of their early warning system (United States Department of Education, 2016), yet published research does not include these variables (e.g., Davis et al., 2019; Mac Iver et al., 2019). Moreover, these systems do not include behavioral data that was used in the current study (tardies, ODRs), nor have other studies incorporated student perception data regarding school climate. Our findings demonstrated the promise of expanding early warning systems to include a wider range of behavioral and perception data for college and career readiness planning purposes. After all, college and career readiness is multidimensional and includes academic and non-academic skills (Lombardi et al., 2018;Morningstar et al., 2017).

Specifically, we found combining attendance with other key variables may be a more sensitive progress monitoring measure in high school engagement and college and career readiness interventions among adolescents with and without disabilities. Importantly, we chose to focus on the combination of behavioral indicators and school climate perceptions, and thus our findings are relevant to behavioral engagement rather than engagement more broadly defined (e.g., Stevenson et al., 2019). Our results indicate the necessity of expanding critical non-academic indicators and adolescent perceptions into systematic data collection and progress monitoring to promote and support behavioral engagement leading to positive college and career readiness outcomes. Overall, we were able to construct a meaningful index by controlling for disability status; therefore, the Adolescent Behavioral Index is useful for students with and without disabilities. Specifically, our findings revealed adolescents with and without disabilities differed with respect to tardies. This difference signals a potential need for schools to more closely examine definitions of tardies in order to ensure all teachers fairly apply attendance and tardy policies.

4.1 Limitations

In this study, there are several limitations to consider when interpreting the findings. First, data collection was performed by the individual schools. School personnel coordinated the dates and class periods in which the GBS survey was administered and may have inadvertently influenced which students had access to the survey. As shown in Table 1, our sample was not reflective of national trends in race and disability categories. There was a substantially higher proportional representation of African Americans represented at 41% and 49% of the study sample from general and special education, respectively (as compared to the national average of 15.7% and 15.3%, respectively). Regarding disability categories, some were well represented and reflected national trends (e.g., learning disability). Yet, other categories in the sample were far below national data (e.g., emotional disturbance, Autism Spectrum Disorder). As such, the findings may not be generalizable on a national scale. Additionally, the behavioral variables utilized in this study were collected by each school; therefore, differences between the schools that participated in the study are possible with respect to how these variables were measured. Finally, we grouped all students with disabilities into one group and did not disaggregate by disability category. This was intentional because we sought to better understand the differences in the Adolescent Behavioral Index scores between those who do and do not receive special education services across the broad range of 12 disability categories defined in the Individuals with Disabilities Education Act (2004). Also, we did not have access to students with disabilities who were not on IEPs (e.g., students on a 504 plan).

We also must disclose limitations regarding the MIMIC model. First, we were not able to freely estimate the effect of the Adolescent Behavioral Index on GPA due to modeling constraints; we elected it was more important to examine the relationship between School Climate and Adolescent Behavior as this is not well understood in the literature. Second, the data collected was cross-sectional; therefore, we were not able to examine the relationship between Adolescent Behavior and longitudinal outcomes. Third, we exhausted all feasible indicators in this model and observed a moderate latent R² for Adolescent Behavior; this index could be improved with additional causal-indicators that were outside of the scope of the current study.

4.2 Implications for future research and practice

The Adolescent Behavioral Index may be a useful tool to integrate into existing multi-tiered efforts in high schools. Particularly, the Positive Behavioral Interventions and Supports (PBIS) framework offers varying behavioral support based on systematic data collection. Researchers have suggested integrating college and career readiness efforts into existing PBIS frameworks (Morningstar et al., 2018) and specifically into defined PBIS procedures, such as the Rules within Routines matrix (Freeman et al., 2018). Adding the index to regular data collection and analysis processes by PBIS teams seems a viable approach and will ensure students with disabilities are included in such schoolwide efforts in school climate and college and career readiness.

Ultimately, the index combines key academic, behavioral, and perception data for adolescents with and without disabilities. Most of these data are routinely collected by schools (attendance, tardies, ODRs); whereas perception data collection, such as school climate, may be less routine. We recommend schools incorporate both data collection routines into school improvement efforts. Future research studies should also incorporate race, ethnicity, and gender data in order to better understand differences based on intersectional student characteristics. Given the necessity of ensuring all students are college and career ready, developing robust data mechanisms that will aid educators in making support decisions for all students is imperative. It is equally important, in future research studies, to explore the predictive validity of the Adolescent Behavioral Index on longer term outcomes in postsecondary education and employment.

Such a sophisticated early warning system is not yet a developed or operationalized mechanism; however, our study initially demonstrates appropriate statistical modeling that could underlie such a system. The impact may be that educators will more effectively use existing and relevant student-level data when considering appropriate interventions associated with school climate and college and career readiness. As such, it is important for high school practitioners to utilize such practices by integrating them into any school-wide initiatives and ensuring students with disabilities are included in these efforts.

5 Conclusion

Ultimately, adolescents with disabilities should be included in schoolwide data collection efforts in order to have access to the same college and career readiness opportunities as their peers without disabilities (Morningstar et al., 2018). Age-appropriate transition assessment, while legally mandated, is typically limited only to those youth who receive special education services. Our findings show that adolescents with disabilities were at greater risk for disengagement with regard to their index scores. As such, it is important for secondary special educators to examine other sources of data such as the academic and behavioral data examined in the current study in order to complement transition assessment and ensure alignment of transition services with broader schoolwide college and career readiness efforts.

Footnotes

Acknowledgments

The authors have no acknowledgments.

Conflict of interest

The authors report no conflict of interest.

Ethics statement

The Institutional Review Board at the University of Connecticut approved the study (protocol number H15-113).

Funding

The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through grant R324A190170 to the University of Connecticut. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education.

Informed consent

The study was determined to be no more than minimal to no risk to study participants and therefore, a parental notification was sent to families allowing students to be opted-out of the study.

Appendix: Specification of the hypothesized model in Mplus

Example Mplus Input File for MIMIC Model

! Denotes a commented section

TITLE:

Example MIMIC Model Specification;

DATA:

! Must not contain variable names

FILE = datafile.dat;

VARIABLE:

! Must be in the order that the variables appear in data file

NAMES = studid absence tardy odr gpa

gbs1 gbs2 gbs3 gbs4 gbs5 gbs6 gbs7 gbs8 gbs9;

MISSING = .; ! Establish missing data symbol or number (e.g., –999)

USEVARIABLES ARE

! Only include variables used in MIMIC model

!Causal-indicators of Adolescent Behavior

absence tardy odr

! Effect-indicators of School Climate –Outcome No. 1

gbs1 gbs2 gbs3 gbs4 gbs5 gbs6 gbs7 gbs8 gbs9

! Grade point average –Outcome No. 2

gpa;

CATEGORICAL ARE ! Treat GBS items as categorical

gbs1 gbs2 gbs3 gbs4 gbs5 gbs6 gbs7 gbs8 gbs9;

ANALYSIS:

! Employ the Weighted Least Squares estimator with theta parameterization

ESTIMATOR = WLSMV;

PARAMETERIZATION = THETA;

MODEL:

!!!! Left-hand side of “by” denotes the latent variable (X-Side)

!!!! Right-hand side of by models effect-indicators (Y-Side)

! School Climate

!! The first loading will be fixed to 1.0 by default

SchClim by

gbs1-gbs9;

!! This allows the latent disturbance to be freely estimated

SchClim^*;

! Adolescent Behavior

!! By specifying no effect-indicators, a phantom latent variable is created

AdolBeh by;

!! Phantom variable is then regressed onto causal-indicators

!!!! Right-hand side of ON denotes dependent variable (Y-Side)

!!!! Left-hand side of ON denotes causal-indicators (X-Side)

AdolBeh on absence tardy odr;

!! This allows the latent disturbance to be freely estimated

AdolBeh^*;

! Latent GPA

LGPA by

gpa@1.0; ! Fixes the factor loading to 1.0

gpa@0.0; ! Fixes the residual to 0.0

!! All variance information is pushed into latent space and is freely estimated

LGPA^*;

[gpa@0.0]; ! Fixes the manifest intercept to 0.0

!! All mean information is pushed into latent space and is freely estimated

[LGPA^*];

!! Estimate covariances among all three causal-indicators

absence WITH tardy odr;

tardy WITH odr;

!! Orthogonalize the two outcome constructs

LGPA WITH SchClim@0.0;

!! Regress School Climate on Adolescent Behavior

SchClim on AdolBeh;

LGPA ON AdolBeh@1.0; !! For identification purposes, this structural path is fixed to 1.0

OUTPUT:

!! Request solution conditioning variables standardized on the X-Side

!! (e.g., latent variable) & Y-Side (e.g., manifest variables)

STDYX;

!! Request details on freely estimated parameters in the model

TECH1;

References

Algozzine,

, Wang,

, Violette,

A. S.

(2011) Reexamining the Relationship Between Academic Achievement and Social Behavior. Journal of Positive Behavior Interventions, 13(1), 3–16. https://doi.org/10.1177/1098300709359084

Allensworth,

, Clark,

(2020). High school GPAs and ACT scores as predictors of college completion: Examining assump- tions about consistency across high schools. Educational Researcher. https://doi.org/10.3102/0013189X20902110

Allensworth,

E. M.

, Easton,

J. Q.

(2005). The on-track indicator as a predictor of high school graduation. Chicago, IL: Univer- sity of Chicago, Consortium on Chicago School Research.

Allensworth,

E. M.

, Easton,

J. Q.

(2007). What matters for stay- ing on-track and graduating in Chicago public high schools: A close look at course grades, failures, and attendance in the freshman year. Chicago, IL: University of Chicago, Consor- tium on Chicago School Research.

American Institutes for Research. (2014). Measurement Practice Guide. College and Career Readiness and Success Center. Washington, DC: American Institutes for Research.

American Institutes for Research (2017)Evidence-based prac- tices to support college and career readiness in high school: Early warning indicators. College and Career Readiness and Success Center.Washington, DC: United States Department of Education.

Baker,

M. L.

, Sigmon,

J. N.

, Nugent,

M. E.

(2001). In Sig-mon J. N., Nugent M. E. and United States Office of Juvenile Justice and Delinquency Prevention(Eds.), Truancy reduction keeping students in school.Washington, DC: U.S. Dept. of Jus- tice, Office of Justice Programs, Office of Juvenile Justice and Delinquency Prevention.

Balfanz,

(2016) Missing school matters. Phi Delta Kappan98(2), 8–13.

Balfanz,

, Byrnes,

(2012) The Importance of Being in School: A Report on Absenteeism in the Nation’s Public Schools. Education Digest: Essential Readings Condensed for Quick Review78(2), 4.

10.

Belfield,

C. R.

, Levin,H. M. (Eds.). (2007). The price we pay: Economic and social consequences of inadequate education. Brookings Institution Press.

11.

Bollen,

K. A.

(1989). Structural equation modeling with latent variables. New York, NY: John Wiley & Sons.

12.

Bollen,

K. A.

, Bauldry,

(2011) Three Cs in measurement models: Causal indicators, composite indicators, and covariates. Psychological Methods16(3), 265.

13.

Bradshaw,

C. P.

, Waasdorp,

T. E.

, Debnam,

K. J.

, Johnson,

S. L.

(2014) Measuring school climate in high schools: A focus on safety, engagement, and the environment. Journal of School Health84, 593–604.

14.

Caldarella,

, Christensen,

, Young,

K. R.

, Densley,

(2011) Decreasing Tardiness in Elementary School Students Using Teacher-Written Praise Notes. Intervention in School and Clinic, 47(2), 104–112. https://doi.org/10.1177/1053451211414186

15.

Churchill,

E. D.

, Rogers,

M. R.

, Pristawa,

K. A.

(2021) High schoolers’ and middle schoolers’ connections in their schools: Relation to tardiness, absences, disciplinary referrals, and failed courses. National Youth-At-Risk Journal4(2), 22–43.

16.

Cornell,

, Shukla,

, Konold,

T. R.

(2016). Authoritative school climate and student academic engagement, grades, and aspirations in middle and high schools. AERA Open, 2. https://doi.org/10.1177/2332858416633184

17.

Davis,

M. H.

, Mac Iver,

M. A.

, Balfanz,

R. W.

, Stein,

M. L.

, Fox,

J. H.

(2019) Implementation of an early warning indicator and intervention system. Preventing School Failure: Alternative Education for Children and Youth63(1), 77–88.

18.

Fan,

, Williams,

C. M.

, Corkin,

D. M.

(2011) A multilevel analysis of student perceptions of school climate: The effect of social and academic risk factors. Psychology in the Schools48(6), 632–647.

19.

Feldman,

M. A.

, Ojanen,

, Gesten,

E. L.

, Smith-Schrandt,

, Brannick,

, Totura,

C. M. W.

, Alexander,

, Scanga,

, Brown,

(2014) The effects of middle school bullying and victimization on adjustment through high school: Growth modeling of achievement, school attendance, and disciplinary trajectories. Psychology in the Schools,51(10), 1046–1062https://doi.org/10.1002/pits.21799

20.

Freeman,

, Kern,

, Lombardi,

, Swain-Bradway,

, Sugai,

(2018). Stronger together: Delivering college and career readiness skills to all through a Schoolwide Positive Behavioral Interventions and Supports framework. In Flannery,K. B., Hershfeldt, P., Freeman, J. (Eds). Lessons Learned on Implementation of PBIS in High School Current Trends and Future Directions. Eugene, OR: University of Oregon Press.

21.

Geiser,

, Santelices,

M. V.

(2007). Validity of High-School Grades in Predicting Student Success beyond the Freshman Year: High-School Record vs. Standardized Tests as Indicators of Four-Year College Outcomes. (CSHE 6.07). Berkeley, CA: Center for Studies in Higher Education.

22.

Goddard,

, Goddard,

, Kim,

(2015) School instructional climate and student achievement: An examination of group norms for differentiated instruction. American Journal of Education122(1), 111–131.

23.

Gottfried,

M. A.

(2009) Excused Versus Unexcused: How Student Absences in Elementary School Affect Academic Achieve-ment. Educational Evaluation and Policy Analysis,31(4), 392–415https://doi.org/10.3102/0162373709342467

24.

Gottfried,

M. A.

(2014) The achievement effects of tardy classmates: evidence in urban elementary schools. School Effectiveness and School Improvement,25(1), 3–28https://doi.org/10.1080/09243453.2012.728135

25.

Hawken,

L. S.

, Vincent,

C. G.

, Schumann,

(2008) Response to intervention for social behavior: Challenges and opportunities, Journal of Emotional and Behavioral Disorders16(4), 213–225.

26.

Henry,

, Knight,

, Thornberry,

(2012) School Disengage- ment as a Predictor of Dropout, Delinquency, and Problem Substance Use During Adolescence and Early Adulthood., Journal of Youth and Adolescence,41(2), 156–166https://doi.org/10.1007/s10964-011-9665-3

27.

Hodara,

, Lewis,

(2017). How well does high school grade point average predict college performance by student urbanic-ity and timing of college entry? (REL 2017-250).Washington, DC: U.S. Department of Education.

28.

Individuals with Disabilities Education Improvement Act, 20 U.S.C. §1400 et seq. (2004).

29.

Irvin,

L. K.

, Horner,

R. H.

, Ingram,

, Todd,

A. W.

, Sugai,

, Sampson,

N. K.

, Boland,

J. B.

(2006) Using office discipline referral data for decision making about student behavior in elementary and middle schools: An empirical evaluation of validity. Journal of Positive Behavior Interventions8(1), 10–23.

30.

Irvin,

L. K.

, Tobin,

T. J.

, Sprague,

J. R.

, Sugai,

, Vincent,

C. G.

(2004) Validity of office discipline referral measures as indices of school-wide behavioral status and effects of school-wide behavioral interventions. Journal of Positive Behavior Interventions6(3), 131–147.

31.

Johnson-Gros,

, Lyons,

E. A.

, Griffin,

J. R.

(2008) Active Supervision: An Intervention to Reduce High School Tardi- ness. Education and Treatment of Children,31(1), 39https://doi.org/10.1353/etc.0.0012

32.

Jones,

, Shindler.

(2016). Exploring the school climate—student achievement connection: Making sense of why the first precedes the second. Educational Leadership and Administra- tion: Teaching and Program Development, 2735-2751.

33.

Jöreskog,

K. G.

, Gold berger,

A. S.

(1975) Estimation of a model with multiple indicators and multiple causes of a single latent variable. Journal of the American Statistical Association70(351a), 631–639.

34.

La Salle,

T. L.

, George,

H. P.

, McCoach,

D. B.

, Polk,

, Evanovich,

L. L.

(2018) An examination of school climate, victimization, and mental health problems among middle school students self-identifying with emotional and behavioral disorders. Behavioral disorders43, 383–392.

35.

La Salle,

T. P.

, McIntosh,

, Eliason,

B. M.

(2018). School cli-mate survey suite administration manual. Eugene, OR: OSEP Technical Assistance Center on Positive Behavioral Interven- tions and Supports. University of Oregon.

36.

Little,

T. D.

(2013). Longitudinal structural equation modeling. Guilford press.

37.

Lombardi,

A. R.

, Freeman,

, Rifenbark,

G. G.

(2018) Modeling college and career readiness for adolescents with and without disabilities: A bifactor approach. Exceptional Children,84(2), 159–176https://doi.org/10.1177/0014402917731557

38.

Mac Iver,

M. A.

, Stein,

M. L.

, Davis,

M. H.

, Balfanz,

R. W.

, Fox,

J. H.

(2019) An efficacy study of a ninth-grade early warning indicator intervention, Journal of Research on Educational Effectiveness12(3), 363–390.

39.

March,

R. E.

, Horner,

R. H.

(2002) Feasibility and Contributions of Functional Behavioral Assessment in Schools.. Journal of Emotional and Behavioral Disorders,10(3), 158–170https://doi.org/10.1177/10634266020100030401

40.

McIntosh,

, Flannery,

K. B.

, Sugai,

, Braun,

D. H.

, Cochrane,

K. L.

(2008) Relationships between academics and problem behavior in the transition from middle school to high school. Journal of Positive Behavior Interventions10(4), 243–255.

41.

McIntosh,

, Brown,

J. A.

, Borgmeier,

C. J.

(2008) Validity of Functional Behavior Assessment Within a Response to Intervention Framework: Evidence, Recommended Practice, and Future Directions. Assessment for Effective Intervention34(1), 6–14https://doi.org/10.1177/1534508408314096

42.

McIntosh,

, Campbell,

A. L.

, Carter,

D. R.

, Rossetto Dickey,

(2009) Differential Effects of a Tier Two Behavior Intervention Based on Function of Problem Behavior. Journal of Positive Behavior Interventions11(2), 82–93https://doi.org/10.1177/1098300708319127

43.

McIntosh,

, Frank,

J. L.

, Spaulding,

S. A.

(2010) Establishing Research-Based Trajectories of Office Discipline Referrals for Individual Students, School Psychology Review39(3), 380.

44.

McIntosh,

, Goodman,

(2016). Integrated multi-tiered systems of support: Blending RTI and PBIS. New York, NY: Guilford Press.

45.

McIntosh,

, Horner,

R. H.

, Chard,

D. J.

, Dickey,

C. R.

, Braun,

D. H.

(2008) Reading Skills and Function of Problem Behavior in Typical School Settings. The Journal of Special Education42(3), 131–147https://doi.org/10.1177/0022466907313253

46.

Morningstar,

, Lombardi,

, Fowler,

, Test,

(2017) A college and career readiness framework for secondary students with disabilities. Career Development and Transition for Exceptional Individuals40(2), 79–91https://doi.org/10.1177/2165143415589926

47.

Morningstar,

M. E.

, Lombardi,

A. R.

, Test,

(2018) Including college and career readiness within a multi-tiered systems of support framework. AERA Open4(1), 2332858418761880.

48.

Muthén,

L. K.

., Muthén,

B. O.

(1998-2019). Mplus user’s guide (8th ed.). Los Angeles, CA: Muthén & Muthén.

49.

Neild,

R. C.

, Balfanz,

(2006) An Extreme Degree of Difficulty: The Educational Demographics of Urban Neighborhood High Schools. Journal of Education for Students Placed at Risk (JESPAR)11(2), 123–141https://doi.org/10.1207/s15327671espr1102_1

50.

O’Malley,

, Voight,

, Renshaw,

T. L.

, Eklund,

(2015) School climate, family structure, and academic achievement: A study of moderation effects, School Psychology Quarterly30(1), 142.

51.

Pas,

E. T.

, Bradshaw,

C. P.

, Mitchell,

M. M.

(2011) Examining the Validity of Office Discipline Referrals as an Indicator of Student Behavior Problems. Psychology in the Schools48(6), 541https://doi.org/10.1002/pits.20577

52.

Rifenbark,

G. G.

, Lombardi,

, Freeman,

(2021) A confirmatory item factor analysis of a school climate measure for adolescents with and without disabilities. Educational Assessment26, 52–68https://doi.org/10.1080/10627197.2020.1841625

53.

Rumberger,

R. W.

, Lim,

S. A.

(2008). Why students drop out of school: A review of 25 years of research. Santa Barbara, CA: Dropout Research Project.

54.

Sanford,

A. K.

, Horner,

R. H.

(2013) Effects of Matching Instruction Difficulty to Reading Level for Students With Escape-Maintained Problem Behavior. Journal of Positive Behavior Interventions15(2), 79–89https://doi.org/10.1177/1098300712449868

55.

Sanford,

, Newman,

, Wagner,

, Cameto,

, Knokey,

A. M.

, Shaver,

(2011). The Post-High School Outcomes of Young Adults with Disabilities up to 6 Years after High School: Key Findings from the National LongitudinalTransition Study-2 (NLTS2). NCSER 2011-3004. National Center for Special Education Research.

56.

Skiba,

R. J.

, Chung,

, Trachok,

, Baker,

T. L.

, Sheya,

, Hughes,

R. L.

(2014) Parsing Disciplinary Disproportionality: Contributions of Infraction, Student, and School Character- istics to Out-of-School Suspension and Expulsion. American Educational Research Journal51(4), 640–670https://doi.org/10.3102/0002831214541670

57.

Stevenson,

N. A.

, Swain-Bradway,

, LeBeau,

B. C.

(2019). Examining high school student engagement and critical factors in dropout prevention. Assessment for Effective Intervention.https://doi.org/10.1177/1534508419859655

58.

Sweeten,

(2006) Who Will Graduate? Disruption of High School Education by Arrest and Court Involvement. Justice Quarterly23(4), 462–480https://doi.org/10.1080/07418820600985313

59.

Tighe,

E. L.

, Wagner,

R. K.

, Schatschneider,

(2015) Applying a multiple group causal indicator modeling framework to the reading comprehension skills of third, seventh, and tenth grade students. Reading and Writing28(4), 439–466.

60.

Todd,

A. W.

, Campbell,

A. L.

, Meyer,

G. G.

, Horner,

R. H.

(2008) The Effects of a Targeted Intervention to Reduce Prob-lem Behaviors: Elementary School Implementation of Check In—Check Out. Journal of Positive Behavior Interventions10(1), 46–55https://doi.org/10.1177/1098300707311369

61.

Trainor,

A. A.

, Morningstar,

M. E.

, Murray,

, Kim,

(2013) Social capital during the postsecondary transition for young adults with high incidence disabilities. The Prevention Researcher20(2), 7–11.

62.

United States Department of Education (2016). Issue Brief: Early warning systems. Washington, DC: Office of Planning, Eval-uation, and Policy Development, Policy and Program Studies Service.

63.

United States Department of Education. (2014). U.S. Department of Education Invests More than $ 70 Million to Improve School Climate and Keep Students Safe. Retrieved from: https://www.ed.gov/news/press-releases/us-department-education-invests-more-70-million-improve-school-climate-and-keep-

64.

Van Eck,

, Johnson,

S. R.

, Bettencourt,

, Johnson,

S. L.

(2017) How school climate relates to chronic absence: A multi―level latent profile analysis. Journal of School Psychology61, 89–102https://doi.org/10.1016/j.jsp.2016.10.001

65.

Welch,

M. J.

, Abulhab,

, Therriault,

S. B.

(2017). College and career readiness in Boston: Understanding and tracking com-petencies and indicators.Washington, DC: American Institutes for Research, College and Career Readiness and Success Center.

66.

White,

, La Salle,

, Ashby,

J. S.

, Meyers,

(2014) A brief measure of adolescent perceptions of school climate, School Psychology Quarterly29, 34.