Abstract
Unreliable diagnoses (e.g., based on inconsistent criteria, subjective) may be inaccurate and even inequitable. This study uses an event history approach with yearly child- and school-level data from 378,919 children in a large urban school district in the southwestern United States between 2006–2007 and 2011–2012 to investigate contextual reliability in the designation of cognitive health conditions (e.g., autism, learning disabilities). This study’s findings suggest the likelihood of designation is higher in schools with more resources (higher teacher-to-student ratio, student population with more resources at home, charter school or magnet program), controlling on student-level differences. Cross-level interactions suggest children’s likelihood of designation also may be higher if they are distinctive relative to other students in their school, sometimes even in terms of nonclinical qualities (race, English Learner status).
Introduction
A disability designation shapes a child’s educational trajectory, social experiences, and social psyche, sometimes beneficially (Leiter and Krauss 2004) but sometimes in limiting and stigmatizing ways (Morgan et al. 2010; Shifrer 2013; Shifrer, Callahan, and Muller 2013). More than 10 percent of the school-aged population in the United States qualifies for special education services through designation with 1 of 13 federal disability categories. With school designations typically precipitated by diagnosis by an educational psychologist/specialist or medical practitioner, many federal disability categories correspond with conditions defined in the Diagnostic and Statistical Manual of Mental Disorders (DSM) (Office of Special Education Programs 2015). These include, from most to least prevalent, learning disabilities, attention deficit hyperactivity disorder (ADHD), autism, intellectual disabilities, and emotional disturbance. These conditions are referenced as “disabilities” rather than “mental disorders” within schools. To recognize the overlapping domains (medical, educational) that contribute to the contested and evolving nature of these conditions, in this manuscript we use a phrase we coined, “cognitive health conditions,” as well as “disabilities.” This study aims to understand whether and how the likelihood of designation with a disability varies across schools for similarly achieving and resourced children.
Although researchers and health practitioners may be comfortable with the inherent subjectivity of diagnoses (Jutel 2013), the public’s understandings of these conditions aligns with the biomedical bent of recent versions of the DSM (Bowker and Star 1999; Vallee 2011; Vanheule 2012). In the United States, educators, parents, youth, and the general public often regard these cognitive health conditions as stable, internal, and uncontrollable (Clark 1997; Vallee 2011), magnifying the implications these designations carry for children’s life chances and well-being. Yet unreliability in designations (e.g., based on inconsistent criteria, subjective) suggests potential inaccuracy and inequity. We use event history methods on data from 2006–2007 through 2011–2012, longitudinally tracking children in a large urban district in the southwestern United States to investigate two questions: (1) Is the likelihood of designation with a cognitive health condition reliable across schools for similar children? and (2) Do school characteristics moderate the association between children’s characteristics and likelihood of designation? We hypothesize the likelihood of designation will vary for otherwise similar youth depending on their schools’ teacher resources, resources for distinctive learners, and student population resources. We also hypothesize children’s likelihood of disability designation will vary depending on the degree to which their own characteristics make them distinctive relative to their peers, in terms of their achievement level, English Learner (EL) status, and race.
Rather than assuming school-level differences reflect variation in disability incidence, this study examines unreliability as a product of the social context. Unreliability, that is, subjectivity and inconsistency (Aboraya et al. 2006), references the extent to which different individuals “arrive at the same diagnosis given an identical set of data” (Matuszak and Piasecki 2012). Interrater reliability is explored through diagnoses from multiple psychiatrists for the same patient (Aboraya et al. 2006) or for the same vignette or video (Leung 2013). Test-retest explorations of reliability focus on whether the patient receives the same diagnosis at different points in time (Aboraya 2007). Studies have also focused on the qualities of practitioners (Klin et al. 2000) or contextual qualities (Cooper et al. 1972) that produce unreliable diagnoses. This study explores contextual reliability by using regression models with controls for student characteristics in an attempt to compare differences across schools in the likelihood of disability designation for similarly resourced and achieving students.
Although this study is limited by unmeasured factors, marked data limitations related to these diagnoses make this study an important contribution to the scholarly literature and to policy. Only a handful of studies on these conditions have used multilevel data (Bal, Betters-Bubon, and Fish 2017; Hibel, Farkas, and Morgan 2010; Samson and Lesaux 2009; Shifrer 2018), leaving contextual variability poorly understood (Canino and Alegria 2008). Just as study of these conditions can benefit from theoretical frames from the sociologies of diagnosis and health, the sociology of health literature can benefit from increased attention to these central markers of health for children. Medical sociologists are encouraged to consider the implication of extramedical institutions (school, family, work) in diagnoses (Jutel 2009; Medina and McCranie 2011; Timmermans and Haas 2008). Focusing on meso-level institutions is an important step in bridging an understanding between macro- and micro-forces that produce diagnoses (Brown, Lyson, and Jenkins 2011). Schools are a social institution second only to the family in the lives of children—their contributions to childhood health should not be disregarded.
Contested Diagnoses and Disability Designations
This study focuses on disability categories established by the Individuals with Disabilities Education Act to qualify students for special education services. The determination of the appropriate category is supported by diagnoses provided by psychologists, therapists, specialists within school systems, medical practitioners, and less often, psychologists and specialists outside of schools (USDOE 2018). Even among diagnoses provided outside of schools, a teacher referral often precipitates the evaluation (Bradley, Danielson, and Doolittle 2007; Brinkman et al. 2009; Cormier 2012; Klingner and Harry 2006; Lloyd et al. 1991; Sax and Kautz 2003), and outside evaluators rely on information provided by schools and teachers (USDOE 2018), demonstrating the inextricable involvement of both the medical and educational domains in these diagnoses. This is consistent with the increasing complexity of diagnoses and shifts in the jurisdiction of medical authority (Jutel 2009; Medina and McCranie 2011; Timmermans and Haas 2008). Moreover, data on specialists’ diagnoses are limited, making special education data an invaluable means of tracking at least most diagnoses across a large number of children.
Alignment between DSM diagnoses and disability categories under the Individuals with Disabilities Education Act varies. The federal disability categories of “intellectual disability” and “autism” are explicitly included in the DSM as diagnoses. Students with the DSM diagnosis of “specific learning disorder” can receive services through the federal disability category “learning disability.” ADHD is also a DSM diagnosis, although students receive services for it under the disability categories of “other health impairment” or “emotional disturbance” (SAMHSA 2016). “Emotional disturbance” is used for “diagnosable mental, behavioral, or emotional” disorders “of sufficient duration to meet diagnostic criteria specified within the”DSM (SAMHSA 2016). The disability category “speech impairment” aligns with DSM diagnoses like language disorder, speech sound disorder, and child-onset fluency disorder (stuttering). Consistent with general data limitations surrounding these conditions, there are no estimates of the proportion of students in special education who do not have diagnoses. There also is not a clear delineation of what qualifies as a “diagnosis” (National Center for Learning Disabilities 2014). Although most of the students operationalized as having disability in this study likely have some form of diagnosis, we take the conservative route of using “designated” rather than “diagnosed” to describe our results.
The qualities of these conditions that may facilitate inconsistencies in their designation are the qualities of contested diagnoses, that is, conditions that are generally accepted but lack a widely applied biomedical definition (Brown 1995). These conditions’ diagnostic criteria are critiqued as subjective and socially rooted (Ong-Dean 2009) like other conditions in the DSM (e.g., schizophrenia) (Pickersgill 2012). Diagnoses rely on the inference of neurological difference through differences in cognitive performance and behaviors (Harry and Klingner 2006). Other “contested diagnoses,” like chronic fatigue syndrome and depression, also do not clearly associate with a physical abnormality (Kokanovic, Bendelow, and Philip 2013). Moreover, distinctions between these diagnostic categories are unclear. It can be difficult, for instance, to determine whether learning struggles are due to a learning disability, intellectual disability, or autism (Eyal 2013). Inattention, hyperactivity, and impulsivity are indicative of autism, intellectual disabilities, and ADHD (Bradley and Isaacs 2006), just as social skill deficits are characteristic of autism, learning disabilities, emotional disturbance, intellectual disability, and ADHD (Gresham 1992).
Contextual Unreliability in the Process of Diagnosis
On top of subjective categories, the designation process may facilitate contextual unreliability. Diagnoses represent the intersection of tidy clinical goals with the complications of real humans (Armstrong 2011), the collision of macro structure with micro interactions (Brown et al. 2011), and the enactment of social power (Nettleton 2006). In addition to subjective and socially rooted DSM criteria (Pickersgill 2012), vague federal guidelines for designating these conditions facilitate school-level discretion (Ferri and Connor 2005). Just as institutional emphasis on a condition can propagate more diagnoses (Brown 1995), a teacher referral nearly always results in designation with some disability (Harry and Klingner 2007). This suggests the educational realm has some authority in the case of these diagnoses (Phillips 2006; Shifrer 2013), specifically medical authority, a legitimate right to delineate and treat human deviance (Jutel 2011).
This study identifies specific aspects of schools that differentiate otherwise similar children’s likelihood of designation with a disability. We investigate inconsistency in terms of schools’ teacher resources and structural resources for diverse learners. We investigate subjectivity in terms of schools’ student population resources and by interacting characteristics of children distinctive relative to the student population. Because so few studies have used multilevel data to consider how school characteristics relate to children’s likelihood of disability designation, we generate hypotheses by making connections across previous research focused on similar school measures and outcomes.
Teacher resources
Teachers are central in the process of referring students for special education evaluation (Cormier 2012). Schools are variably resourced depending on the average experience and educational attainment, for instance, of their teachers (Allensworth, Ponisciak, and Mazzeo 2009; Berry 2004; Clotfelter, Ladd, and Vigdor 2010; Feng 2010). Teachers may have better strategies for learning struggles when they have more experience (Wiswall 2013) and higher levels of education (Murnane and Steele 2007). The benefits of smaller classrooms (i.e., higher teacher-to-student ratios) for student outcomes are relatively consistent in the previous literature (Biddle and Berliner 2002; Konstantopoulos and Chung 2009) and are attributed to teachers’ additional time and attention for individual students. Because contemporary education policy encourages special education as a last resort for struggling children (Buffum, Mattos, and Weber 2014), schools with more teacher resources may be better equipped to prevent disability designations. We hypothesize the likelihood of designation will be lower for youth in schools with more teacher resources than the likelihood for otherwise similar youth in schools with fewer teacher resources.
Structural resources for distinctive learners
The distinctive structures of charter and magnet schools arise from an educational philosophy of individuation and autonomy, a philosophy potentially antithetical to the centralized regulated nature of special education (Rhim and McLaughlin 2001). All schools are required to offer special education services, but charter schools tend to be less well equipped to accommodate the needs of children with cognitive health conditions and may avoid designations (Lange, Rhim, and Ahearn 2008). Magnet programs, aimed at raising achievement for diverse youth in innovative ways (Smrekar and Honey 2015), may strive to meet the needs of struggling children using their unique structural resources rather than special education. Symptomatic children may also be less likely to enroll in charter or magnet schools (Harris 2018). We hypothesize the likelihood of designation will be lower for youth enrolled in charter schools or magnet programs than the likelihood for otherwise similar youth in traditional schools and programs.
Student population resources
With children’s socioeconomic status closely linked to their average achievement levels (Reardon and Portilla 2016), a school’s average achievement level is largely predicated on the resources of the school’s student population. Achievement disparities are generally in much larger part a product of differences across families than differences across schools or teachers (Coleman et al. 1966; Gamoran and Long 2006; Hill 2016; Quinn 2015). Twenty-seven percent of Hispanic and 31 percent of black children lived in poverty in 2016, versus 11 percent of white children (Wilson and Schieder 2018), so lower-achieving schools also serve a disproportionate share of racial minority youth (Kannapel et al. 2005; Logan, Minca, and Adar 2012). Diagnostic processes for conditions without clear biomarkers inherently originate in social comparison, with each person compared against some normative ideal (Conrad and Barker 2010; Rafalovich 2005). Normative ideal achievement may be higher in schools with more well-resourced student populations (e.g., higher-socioeconomic-status students, a larger share of white students) than in schools with less-resourced student populations (Ready and Wright 2011). More specifically, low achievement, a central predictor of increased risk of disability diagnosis, may become even more salient when the student is surrounded by higher-achieving peers. We hypothesize the likelihood of designation will be higher for youth in schools serving a more well-resourced student population than for otherwise similar youth in schools serving a less well-resourced student population.
Achievement distinction
In addition to differentiating every student’s likelihood of diagnosis, contextually specific normative ideals may differentiate how educators interpret specific qualities of children. These cognitive health conditions are marks of deviance or distinction, ostensibly identifying youth with nonnormative responses to educational instruction (Bradley et al. 2007). With deviance, distinction, and impairment being socially constructed (Conrad and Barker 2010; Goffman 1963), the medicalization of learning struggles demands that teachers determine which students’ struggles are distinctively engrained in the child (e.g., biological, neurological). Akin to a frog-pond effect (Crosnoe 2009; Davis 1966; Espenshade, Hale, and Chung 2005), children with similar achievement levels may be perceived as low achieving only if they attend a school with peers who are relatively higher achieving. We hypothesize the likelihood of designation will be higher for low achievers in schools where they are more distinctive: schools with higher average test scores.
EL distinction
ELs are lower achieving on average (Callahan and Shifrer 2016). Some studies find ELs are disproportionately overdesignated with disabilities (Klingner, Artiles, and Barletta 2006; Sullivan 2011), whereas others find they are disproportionately underdesignated (Zehler et al. 2003). Although federal regulations prohibit the designation of a cognitive health condition in children with culturally or linguistically sourced learning difficulties (Spellings, Knudsen, and Guard 2007), there are few objective guidelines for determining whether learning struggles are environmentally or neurologically sourced (Klingner et al. 2006). Although this is the first study to examine EL designations with multilevel data and cross-level interactions, Zehler et al. (2003) used district-level data to show lower designation rates for ELs in districts with more ELs. This may indicate educators unaccustomed to working with ELs struggle more to distinguish between limited English proficiency and neurological difference. We hypothesize the likelihood of designation will be higher for ELs in schools in which they are more distinctive, that is, schools with fewer ELs.
Race distinction
Racial minorities are disproportionately designated with most of these conditions, with explanations ranging from racial bias to the confounding of neurological difference with social disadvantage (Mehta, Lee, and Ylitalo 2013; Shifrer 2018; Shifrer, Muller, and Callahan 2010). Some describe the disproportionate placement of racial minorities into special education as a means of segregating within schools (Ferri and Connor 2005; Reid and Knight 2006), something that might be more relevant in contexts with fewer racial minorities (Eitle 2002). Fish (2017) found evidence suggesting referral decisions are based on racially biased interpretations of behaviors, bias that may be enhanced in schools where racial minority youth are more distinctive. Research on other DSM conditions also shows race influences diagnosis (Loring and Powell 1988; Neighbors et al. 1989; Winstead and Sanchez 2012). We hypothesize the likelihood of designation will be higher for black children in schools in which they are more distinctive, that is, schools with a lower proportion of black children. We hypothesize the likelihood of designation will be higher for Hispanic 1 children in schools in which they are more distinctive, that is, schools with a lower proportion of Hispanic children.
Methods
Data and dependent variable
This study uses yearly data collected by the school district and state education agency on 378,919 children in grades prekindergarten through 12 in a large urban district in the southwestern United States between 2006–2007 and 2011–2012. This project was determined to be exempt from review by the institutional review board at the lead author’s university. In any one year, the district consisted of approximately 200,000 children, 12,000 teachers, and 300 schools. Averaging across school years, around 80 percent of district children were economically disadvantaged, 60 percent Hispanic, 25 percent black, and 30 percent ELs. The teaching population was also diverse, with around 25 percent of teachers white, 40 percent black, and 25 percent Hispanic. Our dependent variable is a dichotomous indicator of whether the child receives special education services each school year, as reported by the school. Schools also reported the federal disability category qualifying each child for special education. We do not stratify most analyses by disability category because results were substantively similar. We exclude 28,185 children (7.4 percent) who entered the district with a disability designation and children receiving special education services through the district despite living outside the district (n = 3,558) because it is unlikely the schools in our data set participated in their diagnosis. We exclude 53 children designated with Noncategorical Early Childhood, as the lack of specificity of this district designation for very young children prevented analytic groupings. We exclude 166 youth who qualify for services through any of the 7 of 13 federal disability categories with more objective diagnostic criteria. 2 Thus, analyses include students receiving services for 1 of the 6 remaining categories: learning disability, speech impairment, other health impairment (often used for ADHD), autism, intellectual disability, and emotional disturbance. With analyses ultimately focusing on 346,957 children, descriptive statistics on all school- and child-level measures are in Online Tables 1 and 2.
Analytic strategy
We use an event history approach, which capitalizes on more data and handles censoring better than other methods (Singer and Willett 1991). If the administrative measure codes a child as not having a disability one school year and having a disability in the subsequent school year, we consider the first as the year they were designated, that is, an “event.” Children’s risk periods begin the 1st year they were enrolled in the district between 2006–2007 and 2011–2012 and end when the child exited the district or was designated. Some children are missing from the district 1 or more school years during their risk period, but on average, children are enrolled in the district 98 percent of their risk period. A total of 572 children (0.15 percent) return to the district with a designation after an absence of 1 or more years. As it is impossible to know whether the designation occurred within the district, we censor their risk periods at their first absence. We use Weibull rather than Cox models because fit statistics indicate the former fit the data better. We adjust the standard errors because children are clustered within schools.
To investigate contextual inconsistency, we identify school-level predictors that differentiate children’s hazards of designation even with child-level controls. To investigate potential subjectivity within contexts, we use cross-level interactions to determine whether children’s hazards of designation vary depending on whether they are distinct in terms of the qualities of their school peers, controlling on other child and school differences. Interaction results are shown graphically with predicted means. Two separate models with interactions explore achievement and EL distinction. Four additional models with interactions explore the implications of race distinction for designation with disabilities black and Hispanic children each are disproportionately over- and underidentified with in this district. We stratify these analyses by disability type because racial disproportionality has been a dominant focus in academic research and policy during the past few decades and may vary by disability category (Blanchett 2006; Morgan et al. 2014; Shifrer 2018). Consistent with this literature, we consider it disproportionate overidentification if the proportion of black children, for instance, within a disability category is larger than the proportion of black children in the district. We stratify by racial group to consider the respective salient school characteristic. We post-estimate predicted probabilities at realistic school racial composition levels for black and Hispanic children in this district, within the bounds of the 10th and 90th percentile for each; we exclude a focus on white children as there are so few schools in this district in which they are not distinctive. 3 Consistent with Rodríguez (2015), we include children who experienced competing events (i.e., designation with a disability other than those of interest) with right-censored children.
Independent variables
Independent variables focus on the year preceding the first year children have a disability designation, or the first year of the risk period in which undesignated children are in a grade level in which designation was most common. All continuous school-level predictors are standardized to increase regression coefficient comparability. School teacher resources measure teacher-to-child ratio across the school and within classrooms, proportion core teachers (math, reading, language arts, science, and/or social studies) with more than a bachelor’s degree, and core and special education teachers’ average years of experience. Structural resources for distinctive learners measure whether the school was a charter or had a magnet program (applied [careers and professions]; arts or alternative pedagogy [languages, arts, Montessori]; academically intensive [Vanguard, college preparatory, and International Baccalaureate]). Student population resources measure school mean test score and proportion of students living in poverty or who are black or ELs. Proportion black and proportion Hispanic are too highly correlated to include in the same regression model—we use the former because Hispanic children are the majority in this district and white children are too small of a minority to capture variation salient to most children’s schools.
We include child-level controls for age, gender, race, economic status, EL status, average test score, zoned school attendance, and Gifted and Talented program participation. Children’s ages are standardized to reflect their age relative to others in the same grade level. District categories for child economic status include the following: not disadvantaged, eligible for reduced-price lunch, eligible for free lunch, living in poverty. Test scores are more predictive of disability designation than even behaviors (Hibel, Faircloth, and Farkas 2008; Hibel et al. 2010), although diagnoses of these conditions sometimes occur for children without learning struggles. We used children’s scores from the Stanford Achievement Test Series as well as the Spanish version (Aprenda), which were administered annually to children in kindergarten through grade 12. There were no alternate or modified versions of the Stanford test for students with disabilities, and the test had no time limit (personal communication, Kristin Dwyer from Pearson Learning Assessment Customer Service, September 25, 2014). We standardize scores within each cohort, grade level, test subject (e.g., math, reading), and test version (English or Spanish), consistent with Reardon and Galindo (2009), and we average test scores from the focus year across subjects.
Missing values
Online Table 3 details missingness rates. Because this data set is large, longitudinal, and multilevel, with highly correlated measures repeated each year at both levels, multiple imputation models do not converge. Fortunately, test scores are the only child-level measures with missing values, and children designated with learning disabilities, the largest part of the special education population (40 percent), are actually less likely to be missing scores than undesignated children. To achieve the best possible imputations, we first use the child’s average standardized test score from the year closest to the focus year. For children without test scores from any years, we use single imputation stratifying by disability status. Single imputation relies on regression modeling to generate a predicted value for each child based on his or her other characteristics. After using values from the same school from different school years to impute, remaining missing are addressed through mean/mode imputation within the same school year, differentiating depending on charter status.
Results
Contextual Inconsistency in Disability Designations
Table 1 explores potential contextual unreliability by predicting children’s hazards of designation with a cognitive health condition with measures describing their schools and controls for their own characteristics. Consistent with the increasing call for a shift in emphasis from statistical to substantive significance (Nuzzo 2014), shaded cells show hazards that increased or decreased by more than 10 percent. All statistically significant findings are “substantively significant,” excepting two instances mentioned below.
Inconsistency and Resources—Weibull Regression Models Predicting Designation with a Disability.
Note. A total of 346,957 children were used to estimate the model, with 9,382 designated with a disability. All continuous school-level variables are standardized. Substantially significant effects (hazards increased or decreased by more than 10 percent) are shaded. Dashes indicate reference categories.
p < .10. **p < .01. ***p < .001, two-tailed test.
Table 1 shows that, controlling for differences across children, the hazards of designation with a cognitive health condition increase by 24 percent (100 [1.24 − 1] percent) with every one standard deviation increase in the teacher-child ratio across the whole school and by 11 percent with every one standard deviation increase in the average teacher-child ratio across classrooms. Relative to children not in magnet programs, the hazards of designation are 13 percent higher for children in applied magnet programs and 55 percent higher for children in academically intensive magnet programs, controlling for child-level differences. These differences are not statistically significant. Finally, the hazards of designation decrease by 12 percent with every one standard deviation increase in the proportion of children in the school living in poverty; similarly, the hazards of disability designation increase by 29 percent with every one standard deviation increase in the school’s average test score, net of all child-level differences.
Distinction and Disability Designations
Figures 1 and 2 show results from models with cross-level interactions (full models in Online Tables 4 and 5), which explore whether designations relate to student distinction. Figure 1-1 shows predicted probabilities of disability designation depending on children’s average test scores and their school mean test scores. First, consistent with criteria, predicted probabilities of designation are much higher for children with the lowest test scores than for children with higher test scores, controlling for other child- and school-level differences. Probabilities of designation are higher for children in higher-achieving schools than those for children in lower-achieving schools. The steeper slope of the line for the lowest-achieving children suggests their designation risk is accelerated by attending a higher-achieving school relative to the risk for higher-achieving children. Figure 1-2 shows predicted probabilities of designation depending on children’s EL status and the standardized proportion of their school peers who are ELs. Probabilities of designation are generally higher for children who are not ELs relative to children who are ELs, controlling on all other student and school differences. The probabilities of designation are higher for ELs attending schools with fewer ELs than the probabilities for ELs attending schools with more ELs.

Achievement and English Learner distinction—Differences in predicted probabilities of designation with a disability at the interaction of child and school characteristics.

Race distinction—Differences in predicted probabilities of disability designation at the interaction of child’s race and school’s student body racial composition.
Figures 2-1 and 2-2 explore differences in the predicted probability of designation for black and Hispanic children depending on the proportion of children at school who share their race. Figure 2-1 shows that black students’ predicted probability of designation is lower if they attend a school with a higher proportion of black students, accounting for other child- and school-level differences. This pattern is evident for disabilities that black students are disproportionately over- and underdesignated with in this district. In contrast, Figure 2-2 shows that Hispanic students’ predicted probability of designation with a learning disability or speech impairment (disabilities they are disproportionately overdesignated with in raw data) increases slightly if they attend a school with a higher proportion of Hispanic students. The racial composition of the school does not appear to differentiate Hispanic children’s probability of designation with the other disabilities.
Conclusions
Disability designations shape children’s educational trajectories, social experiences, and social psyches, sometimes beneficially, but sometimes in limiting and stigmatizing ways. This study’s findings suggest these designations do not always occur reliably across school contexts. In this district, children are more likely to be designated with disability in schools with more resources, controlling for measured student-level differences. Cross-level interactions suggest children’s likelihood of designation is higher if they are distinctive relative to other students in their school, sometimes even in terms of nonclinical qualities (race, EL status). Unreliability weakens the legitimacy of the diagnosis, suggesting it reflects not only social influences on health but also social influences on diagnosis (Fuller 2011). This study’s findings extend ideas from medical sociology to diagnoses that overlap education and health realms and demonstrates parallels with other contested diagnoses. The paragraphs that follow expand on each set of findings.
Consistent with the notion that the meaning of impairment emerges from the “fabric of everyday life” (Conrad and Barker 2010), educators may interpret similar “symptoms” differently depending on the context. First, aligning with our hypothesis that children in schools with more well-resourced student populations would be more likely to be designated with disabilities, we find children’s likelihood increases in schools with higher average test scores and decreases in schools with higher proportions of students living in poverty. Other studies have similarly found an elevated likelihood of diagnosis in higher-achieving schools (Hibel et al. 2010; Ramey 2015). It is also possible that parents of children in schools with more-well-resourced student populations are more likely to advocate for disability designations that benefit their children in terms of additional time on tests or performance-enhancing stimulants (King, Jennings, and Fletcher 2014; Ong-Dean 2009).
In most cases, we also find support for our hypotheses that distinction increases a child’s likelihood of designation with a cognitive health condition. Cross-level interactions show a low average test score is even more predictive of designation for a child in a school with higher average test scores. Previous findings are mixed on the effects of average school achievement (Bal et al. 2017; Hibel et al. 2010; Sullivan and Bal 2013), but these studies did not use cross-level interactions. Teachers’ perceptions and understandings may become bounded such that teachers come to perceive the average achievement levels at their schools as the normative base and lower achievement levels as aberrant. In addition, social pressures at high-achieving schools may motivate parents to be more interventionist and seek extra resources for their lower-achieving children (Blanchett 2010; Ong-Dean 2009). Similarly, supporting our racial distinction hypothesis, we find a heightened likelihood of disability designation for black students in schools with a lower proportion of black students. This aligns with district-level findings from Eitle (2002). Teacher referrals of black youth for evaluation appear to be informed by racially biased interpretations of behavior (Fish 2017), and racial distinction may amplify this bias.
We also find ELs are more likely to be designated with disability in schools where they are more distinctive. Gottfried (2016) shows students’ academic and social-psychological outcomes were better in classrooms with higher proportions of “same needs” peers, perhaps suggesting teachers more easily identify and respond to students’ individual needs when they are familiar. Educators in schools with fewer ELs may be less accustomed to their learning patterns and more likely to attribute ELs’ learning struggles to cognitive deficiency. Counter to our racial distinction hypothesis, Hispanic children’s likelihood of designation with learning disabilities or speech impairments is slightly lower in schools with a lower share of Hispanic peers. Educators in these schools may be more likely to interpret Hispanic children’s learning struggles through a stereotyped lens of the challenges Hispanic children face (e.g., linguistic differences), such that they receive services through bilingual programming rather than special education. Educators in schools with a larger share of Hispanic children may be able to take a more nuanced view, distinguishing some Hispanic children’s learning struggles as sourced in neurological difference. Important to note, in unadjusted analyses, Hispanic children are disproportionately underidentified with most disabilities in this district, and school racial composition did not differentiate likelihood of designation with these disabilities. With racial segregation potentially particularly salient for the special education placements of Hispanic children (Perez, Skiba, and Chung 2008), it is possible results would be different in a school district where Hispanic children are not the majority racial group.
The school-level variables that differentiate children’s likelihood of designation, after accounting for differences across children, suggest disability designations are contextually unreliable, but the direction of estimated effects is not always consistent with our expectations. Counter to our hypothesis of school teacher resources reducing likelihood, children in schools with higher teacher-to-student ratios are more likely to be designated with disability. Previous studies did not consider teacher-to-student ratios (Sullivan and Bal 2013; Tejeda-Delgado 2009) or were focused on a British sample (Ford, Goodman, and Meltzer 2004). Counter to our expectation that teachers do not refer students for special education evaluation because they are answering the call to address students’ struggles through other means, it may be these teachers are not addressing students’ struggles at all. In this way, referring a student for evaluation may represent more effort on the part of teachers, such that teachers with fewer students are more likely to invest that effort. Counter to our hypothesis of lowered likelihood with more school resources for distinctive learners, we find children in charter schools and magnet programs are more likely to be designated relative to similarly resourced and achieving children in traditional schools. The specialized focus of these schools/programs may extend into an increased willingness to individualize children’s learning experiences, consistent with the philosophy underlying special education services. Charters and magnets also may be responding to past criticisms that they are ill prepared to meet the needs of children with cognitive health conditions.
Some limitations merit mention. Students are not randomly sorted across schools, such that this study’s findings may be confounded by systematic differences in the children who attend each school as well as differences across their parents. For instance, children with cognitive health conditions just may be more likely to attend schools with higher teacher-to-student ratios, a selective difference not captured by the measures in these data. Most important, this study lacks a measure of clinical child characteristics. This is an important point for future research once data are available with more specific measures of children as well as measures of context. Nonetheless, even extraschool designations of these conditions infer neurological difference on the basis of qualities that are subjective and socially rooted (Carrier 1983; Dudley-Marling 2004; Leiter 2007; Vellutino et al. 2004), drawing on information provided by schools, teachers, and parents (USDOE 2018). Similar criticisms of subjectivity are levied at many conditions in the DSM (Crowe 2000; Pickersgill 2014; Vallee 2011). With sociologists of mental health called to reconsider using measures from other disciplines (Horwitz 2002), data with more nuanced measures of these conditions might be collected, with continuous indicators of symptoms, for instance, rather than a dichotomous indicator of diagnosis.
Some of our measures lack validity and precision. For one, the data provide a school measure of disability designation. Although these designations are often based on diagnoses, it is possible some children are designated despite no diagnosis or that diagnosed children are not designated at school. The institutional labels that correspond with designations are an important aspect of diagnosis (Dobransky 2011), in this case facilitating study at a scale otherwise impossible. The measures also do not indicate whether schools or parents initiated the disability designation or the diagnosis that preceded it. Previous research documents that most referrals for special education evaluation are initiated by teachers rather than parents (Bradley et al. 2007; Cormier 2012; Klingner and Harry 2006; Sax and Kautz 2003), but this may vary depending on disability type. Our inability to discern whether variation across schools in children’s likelihood of disability designation is a product of school or parent initiation does not change the study’s main premise that these designations are contextually unreliable, and it parallels the literature on the centrality of social stratification for both health and education outcomes. Nonetheless, the potential involvement of parents, instead of or in addition to teachers, should not be discounted.
These results are not generalizable to the United States or even to other similar school districts in the United States. The one large federal U.S. data set with similar disability data (the Early Childhood Longitudinal Study) offers more measures describing parents but less rich measures of schools. These findings should be replicated once suitable data are available. Nonetheless, this narrowed focus may increase the applicability of findings for minorities, as 73 percent of black and 78 percent of Hispanic students attend schools in which more than half of the student population consists of racial minorities, in contrast to 40 percent of white students (Aud et al. 2010). Given the data limitations on this topic in the United States, the use of these data still marks a major contribution to the study of disability designations as well as a theoretically and methodologically rich examination of social diagnosis.
These designations might occur more reliably if processed through a cross-school or cross-district team to ensure personnel involved have a broader perspective of normative achievement levels and to mask student race. Diagnosticians also might consider achievement data disaggregated by social characteristics to better incorporate an understanding of the social factors that shape educational performance. Ultimately, though, wholly reliable designations may be difficult to achieve in the absence of biological diagnostic criteria. With broad calls for increased transparency surrounding the subjective and socially constructed nature of many psychiatric diagnoses (Dammann 1997), reform might begin by reframing the discourse surrounding these conditions. Insurance payment in mental health treatment outside of schools requires a diagnosis (Kirk and Kutchins 1988), just as special education services in schools require a diagnosis. This “diagnostic determinism” (Brown 1990) perpetuates assumptions that disorders are discrete and categorical—that these conditions result from intraindividual biological dysfunction (Sonuga-Barke 1998). With acknowledgment of remaining gaps in scientific knowledge, children might incorporate useful insights from a diagnosis while not feeling the diagnosis seals their destiny or captures their complexity.
Supplemental Material
SMH2019_R_R3_OnlineTables – Supplemental material for A Multilevel Investigation into Contextual Reliability in the Designation of Cognitive Health Conditions among U.S. Children
Supplemental material, SMH2019_R_R3_OnlineTables for A Multilevel Investigation into Contextual Reliability in the Designation of Cognitive Health Conditions among U.S. Children by Dara Shifrer and Rachel Fish in Society and Mental Health
Footnotes
Notes
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project benefited from support from the Houston Education Research Consortium (HERC), which is directed by Dr. Ruth López Turley and funded by the Laura and John Arnold Foundation and Houston Endowment Inc. This project benefited from the assistance of school district staff and HERC staff, particularly Noe Perez, Holly Heard, and Jing Li. This study also benefited from suggestions from Drs. Angela Frederick, Anna Mueller, Jennifer Pearson, and Carrie Shandra.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
