Measurement of Childhood Serious Emotional Disturbance: State of the Science and Issues for Consideration

Abstract

Surveys suggest that between 4.3% and 11.5% of children living in the United States have a serious emotional disturbance (SED). SED is defined in the Federal Register, and federal block grants are allocated to states based on the prevalence of SED. Accurate measurement of SED is critical, yet surveys have used different methodologies and instrumentation to obtain prevalence estimates. Two expert panels were convened by the Substance Abuse and Mental Health Services Administration (SAMHSA) in the fall of 2014 to discuss the current state of the field in operationalizing and measuring SED. The aims of this article are to (a) provide an update on SED prevalence rates from survey research, (b) summarize key discussion points that emerged during the SAMHSA expert panel meetings, and (c) make specific recommendations for next steps in measuring the prevalence of SED. Issues addressed in this article are important not only for federal allocation of service dollars to meet the needs of children with SED, but also for broader mental health surveillance efforts within the context of large national research surveys.

Keywords

serious emotional disturbance impairment child mental health epidemiology

The prevalence of serious emotional disturbance (SED) among children and adolescents in the United States has relevance for both service planning and broader mental health surveillance efforts (Centers for Disease Control and Prevention [CDC], 2013). States estimate the prevalence of SED as one component of their request for federal resources to support mental health services. To help to enable this process, there is a federal mandate to produce national and state estimates of SED as specifically defined in the Federal Register (FR). Pursuant to section 1912(c) of the Public Health Service Act, and as amended by Public Law 102-321, “children with a serious emotional disturbance” are persons (a) from birth up to age 18, (b) who currently or at any time during the past year (c) have had a diagnosable mental, behavioral, or emotional disorder of sufficient duration to meet diagnostic criteria specified within the Diagnostic and Statistical Manual of Mental Disorders (3rd ed., rev.; DSM-III-R; American Psychiatric Association [APA], 1987) (d) that resulted in functional impairment which substantially interferes with or limits the child’s role or functioning in family, school, or community activities. These disorders include any mental disorder (including those of biological etiology) listed in the DSM-III-R or their International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9–CM; Central Office on ICD-9–CM, 1997) equivalent (and subsequent revisions), with the exception of DSM-III-R “V” codes, substance use, and developmental disorders (e.g., mental retardation, autism), which are excluded, unless they co-occur with another diagnosable SED (Substance Abuse and Mental Health Services Administration [SAMHSA], Center for Mental Health Services [CMHS], 1993).

The term SED was first used in association with the original federal efforts to address the needs of children and youths with serious mental conditions and their families—the Child and Adolescent Service System Program (CASSP). The definition specified criteria including diagnosis, functional impairment, duration, and service needs. In 1992, the Alcohol, Drug Abuse, and Mental Health Administration (ADAMHA) Reorganization Act established a Block Grant for Community Mental Health Services to be administered by the CMHS within the newly created SAMHSA. This block grant allows funds to be allocated to states to support the provision of services to children with SED and adults with serious mental illness. In an associated effort, SAMHSA convened a technical advisory group that developed a formal definition of SED, first published for public comment in the November 6, 1992, issue of the FR (57 FR 53118) with the final notice published on May 20, 1993 (SAMHSA, CMHS, 1993). The definition was intended to help identify and estimate the size of the SED population in each state as a measure of needed block grant funding. The FR definition was also intended to distinguish SED from conditions covered by other funding streams, such as substance use disorders (SUDs) and developmental disorders. This definition of SED continues to be used today.

Despite the public health relevance of SED, there is no current way to track trends in SED over time in the United States. A 2013 article in the Morbidity and Mortality Weekly Report summarized current surveillance efforts designed to capture indicators of children’s behavioral health in national surveys from 2005 to 2011 (CDC, 2013). Available national indicators included self-reported diagnoses such as attention-deficit/hyperactivity disorder (ADHD) and Tourette syndrome as well as the prevalence of substance abuse and suicide. Recent publications using this national survey data suggest potential changes in the prevalence of these behavioral health phenomena. For example, a study using Medical Expenditure Panel Survey data from 1998 to 2012 found decreases in the number of children (aged 6–17 years) with severe mental health impairment (Olfson, Druss, & Marcus, 2015). Another recent study used National Survey of Children’s Health data from 2003 to 2011 to document the increasing prevalence of parent-reported ADHD (Visser et al., 2014). These studies’ findings highlight the utility of consistently measuring the prevalence of childhood behavioral health indicators over time. Unfortunately, national surveillance efforts do not currently capture both the presence of any childhood mental disorder and associated functional impairments to allow for SED estimation over time.

Research in the past decade has produced national prevalence estimates for child mental disorders and, separately, for impairment associated with child mental disorders. Some of these same studies offer specific, separate estimates of SED. Published estimates of SED vary, due in large part to the use of different operational definitions, measures, study designs, and samples with overlapping but not identical age ranges. Without the use of consistent operational definitions and agreed-upon, gold-standard methods of measuring diagnoses and functional impairment, it is difficult to determine accurately the true prevalence of children who qualify for mental health services, such as including children with SED, or whether the rates of SED over time are stable, increasing, or decreasing. This is not only true at the national level but also for comparing rates within and across states.

As a mandated federal statistic, estimates of SED for use by the federal government have to consider the rigor and guidelines for statistical surveys required by the Office of Management and Budget (OMB, 2006). This requirement elevates the need for attention to the instruments and methods used to generate estimates of SED. Currently, no national- or state-level estimates of SED exactly align with the 1993 FR notice definition for two major reasons. First, available studies and publications were not designed explicitly for this purpose. Second, there are more broad-based challenges in the field around measurement, instrumentation, and study design. The purpose of this article is twofold: (a) to describe what might be necessary to create a platform to meet the federal mandate to produce national and state estimates of SED as defined by the FR and (b) to articulate what areas of instrumentation and methodological research might facilitate the ultimate surveillance of SED in national survey efforts. The article discusses the strengths and limitations of measures relevant to assessing childhood SED along with the methodological and study design challenges associated with estimating SED.

National Prevalence Estimates of SED

Four large-scale epidemiological datasets have been used to produce prevalence estimates of childhood mental disorders with functional impairment in the United States. These studies include a supplemental study to the National Health and Nutrition Examination Survey (NHANES; Merikangas et al., 2010), the National Comorbidity Survey Adolescent Supplement (NCS-A; Kessler et al., 2012) and the NCS-A school sample (Green et al., 2015; Li, Green, Kessler, & Zaslavsky, 2010), Methods for the Epidemiology of Child and Adolescent Mental Disorders (MECA; Lahey et al., 1996; Shaffer et al., 1996), and the Great Smoky Mountains Study of Youth (GSMS; Costello et al., 1996). Two additional studies have a screening scale within the National Health Interview Survey (NHIS) to estimate the prevalence of SED (Mark & Buck, 2006; Pastor, Reuben, & Duran, 2012).

Not all of these studies used the term SED, and the studies did not all consistently include “functional impairment which substantially interferes with or limits the child’s roles or functioning,” as defined by SAMHSA. In short, none of the study SED definitions aligned exactly with the 1993 FR notice definition. For example, no study estimated SED from birth to 17 years, likely due to the lack of one validated instrument to cover this entire age range. In addition, not all of the studies excluded SUDs or developmental disorders from their estimates. Moreover, the mental disorders measured in each study differed, as did the number of individuals (e.g., parents, youths) asked to report upon symptoms. The NHANES study, for example, measured only a small number of disorders, included only one informant per disorder, and thus yielded lower estimates of SED than did the other studies. Time reference periods also varied from the past 3 months to the past year. Moreover, none of the existing national studies was designed to generate state-level estimates, which would require a significant increase in sample size. These studies’ key characteristics are summarized in Table 1 along with their ability to align with the 1993 FR notice definition of SED.

Table 1.

Characteristics of Large Epidemiological Studies Assessing the Prevalence of Childhood SED and Consistency With the 1993 Federal Register Notice Definition.

Survey	Age rangea	Sample size	Most recent year fielded	Included past 12-month DSM diagnosis	Excluded SUDs and/or developmental disorders	Nationally generalizable	Support state estimates
NHANES (special study)	8–5	3,042	2004	Yes (DISC-IV)	Yes	Yes	No
NCS-A	13–17	6,483b (adolescent/parent pairs)	2002	Yes (CIDI-A)	No	Yes	No
MECA	9–17	1,285	1991	No (DISC-2.3 past 6 months)	No	No	No
GSMS	9–13 (age at baseline)	1,420	1999 (included youths up to age 17)	No (CAPA; past 3 months)	Yes	No	No
NHIS	4–17	10,367	2001 (25-item SDQ); 2012 (5-item SDQ)	No (screening only)	NA	Yes	No

Note. SED = serious emotional disturbance; DSM = Diagnostic and Statistical Manual of Mental Disorders; SUDs = substance use disorders; NHANES = National Health and Nutrition Examination Survey; DISC-IV = Diagnostic Interview Schedule for Children Version IV; NCS-A = National Comorbidity Survey Adolescent Supplement; CIDI-A = Composite International Diagnostic Interview Adolescent; MECA = Methods for the Epidemiology of Child and Adolescent Mental Disorders; GSMS = Great Smoky Mountains Study of Youth; CAPA = Child and Adolescent Psychiatric Assessment; NHIS = National Health Interview Survey; SDQ = Strengths and Difficulties Questionnaire.

Several of these studies included respondents older than 17 years. The purpose of this table is to show coverage for children and youths from birth to 17 years old. bThe NCS-A included a larger adolescent-only sample; however, the household study of SED included only the adolescent-parent paired data.

The greatest variation across the studies with published SED estimates lies in their operational definitions of “functional impairment which substantially interferes with or limits the child’s roles or functioning” (p. 6; henceforth referred to in this article as “functional impairment”). Most of the studies used the functional impairment indicators in the diagnostic instrument that was employed to assess mental disorders; however, the cut points and definitions within each instrument varied. For example, the GSMS defined “any significant functional impairment” as one rating of either partial or severe impairment within the incapacity section of the Child and Adolescent Psychiatric Assessment (CAPA; G. Keeler, personal communication, October 17, 2012). This method generated a prevalence estimate of 27.3% for any disability (13.8% related to family, 12.7% to educational, and 6.2% to peer functional impairments; Ezpeleta, Keeler, Erkanli, Costello, & Angold, 2001). Both NHANES and NCS-A defined functional impairment as having partial impairment ratings across multiple domains of functioning or the presence of at least one severe impairment rating. In the NHIS studies, “serious overall emotional or behavioral health difficulties” were considered, those cases where a parent responded “yes, definite difficulties” to the one Strengths and Difficulties Questionnaire (SDQ) impact item. Analyses of the MECA study used a definition of SED contained in the Alcohol, Drug Abuse, and Mental Health Administration Reorganization Act of 1992 (Pub. L. No. 102-321). This definition employed a concept of diagnosis (diagnostic criteria specified within the DSM-III-R) plus functional impairment, and also included children “who would have met functional impairment criteria during the referenced year without the benefit of treatment or other support services.” Impairment related to SED was estimated using three cutoff scores (50, 60, and 70) of the Child Global Assessment Scale (CGAS; Narrow et al., 1998; Shaffer et al., 1996). The MECA study was the only study that relied on a supplemental impairment scale, separate from a diagnostic or screening instrument, to establish and operationalize functional impairment.

Table 2 summarizes estimates from the NHANES, NCS-A, GSMS, NHIS, and MECA studies for the presence of any mental disorder and mental disorders with various levels of functional impairment (including SED). Estimates across these studies cannot be statistically compared directly with one another because of study methodological differences. These differences include not only the varying definitions of functional impairment discussed earlier but also the different disorder inclusionary criteria. For example, the NHANES study included eating disorder in addition to ADHD, conduct, anxiety, and mood disorders, but not SUDs. The NCS-A included a much more comprehensive assessment of disorders from the Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM-IV; APA, 1994), including substance use and dependence (not included in the 1993 FR notice definition of SED). Furthermore, the GSMS estimates pertained to the past 3 months, while the NHANES and NCS-A estimates were for the past 12 months. Finally, child age ranges varied across study samples. Although the range of estimates may be interpreted as quite small, it bears repeating that these estimates were drawn from different subpopulations with respect to disorder coverage, child age range, and informants used, as discussed earlier.

Table 2.

Prevalence of Any Mental Disorder and SED Based On the NHANES, NCS-A, GSMS, and NHIS Studies.

Mental disorder	Prevalence estimate (%)
Mental disorder	NHANES special study (12–15 years), past 12 months	NHANES special study (8–11 years), past 12 months	NCS-A (13–17 years), past 12 months	NCS-A school sample (13–17 years), past 12 months	GSMS (9–16 years), past 3 months	NHIS (4–17 years), past 12 months	MECA ADAMHA (9–17 years), past 6 months
Any DSM-IV disorder	13.4	12.8	42.6	Not provided	13.3	NA	NA, not provided
SED (and associated impairment definition)	11.5 Level A or B: intermediate impairment rating on 2 of 6 items or at least 1 severe rating	11.0 Level A or B: intermediate impairment rating on 2 of 6 items or at least 1 severe rating	8.0 Moderate impairment in most areas of living or severe in 1 area	7.3 (Green et al., 2015) 5.7 (Li, Green, Kessler, & Zaslavsky, 2010)	6.8 Any Axis I plus “any significant functional impairment”	4.3–5.1 (from 2001–2007) score of 6 or greater on the 5-item SDQ	12.1% to 22.6% a (lower estimate if CGAS <50; higher estimate if CGAS < 70)

Note. SED = serious emotional disturbance; NHANES = National Health and Nutrition Examination Survey; NCS-A = National Comorbidity Survey Adolescent Supplement; GSMS = Great Smoky Mountains Study of Youth; NHIS = National Health Interview Survey; MECA= Methods for the Epidemiology of Child and Adolescent Mental Disorders; ADAMHA = Alcohol, Drug Abuse, and Mental Health Administration; DSM-IV = Diagnostic and Statistical Manual of Mental Disorders, 4th edition; SDQ = Strengths and Difficulties Questionnaire; CGAS = Child Global Assessment Scale.

The MECA study used the Diagnostic and Statistical Manual of Mental Disorders, 3rd revised edition (DSM-III-R).

SED Estimates for Current State Service Planning

In the absence of studies to support direct state-level estimates, proxy prevalence estimates are used by states in their mental health block grant applications. Annual state-level estimates of SED are provided by SAMHSA’s Uniform Reporting System (URS) to help states estimate the extent of service needs associated with this target population within their particular state. Each state-specific URS estimate uses as its basis, a national estimate generated in 1996 from existing studies of mental health in children and youths (Friedman, Katz-Levey, Manderscheid, & Sondheimer, 1996). The estimate was not based on a single nationally representative epidemiological study, but instead was derived as part of a work group convened to estimate the national prevalence of SED. Based on a review of the available data, the work group recommended a “less conservative” estimate of SED to be in the range of 9% to 13% and a “more conservative” estimate of SED to be in the range of 5% to 9%. For the purpose of service planning needs, the work group recommended using the higher range of 9% to 13%. Prevalence rates in the research literature at that time did not extend below age 8 years, even though the definition of SED includes those from birth. This recommended prevalence range of SED (9%–13%) has been adjusted to derive a range of SED for each state by adjusting the national estimate by yearly state poverty levels. The URS has continued to use this estimation methodology each year. The methodology, described in detail in a July 17, 1998, FR notice (SAMHSA, CMHS, 1998), acknowledged that the range used for these prevalence calculations was imperfect because of the use of existing data sources to approximate a range of SED not based on the SAMHSA-defined SED. Almost two decades later, there is still no study that can estimate SED consistently with the 1993 FR notice definition at the national or state level.

SED Expert Panels

In the fall of 2014, SAMHSA convened two expert panels to discuss several issues relevant to generating national and state estimates of childhood SED. These meetings brought together experts with critical knowledge around the history of the 1993 FR notice definition and clinical and measurement expertise in childhood mental disorders and their associated functional impairments. The goal for Panel 1 was to operationalize the SED definition for the production of national and state prevalence estimates. The goal for Panel 2 was to discuss instrumentation and measurement issues when estimating national and state prevalence of childhood SED.

Prior to the meetings, panel members were provided detailed information about the history of the SED definition and federal block grant requirements. The experts were also provided with background information summarizing available diagnostic and impairment tools and how they aligned with the 1993 FR notice definition of SED. Members were asked to consider several questions prior to the in-person meetings, with an emphasis on noting measurement and methodological challenges and recent advances in the field that would be relevant to generating national and state estimates of SED. During the two separate, day-long panel meetings, the following topics were discussed:

Panel 1 discussed the disorders in the Diagnostic and Statistical Manual of Mental Disorders (5th ed.; DSM-5; APA, 2013) that should be included and excluded from the definition of SED, age considerations for inclusion and exclusion criteria, and the components of impairment that might need to be included to assess SED.

Panel 2 discussed diagnostic, impairment, and other instruments best suited for estimating SED; instrument development work necessary to meet the different 1993 FR notice criteria; and methodological issues to consider when designing a study to generate national or state estimates (these panel members also made recommendations about future steps necessary to produce accurate estimates of SED).

The discussions from the two panel meetings contained diverse opinions and did not always reach consensus. Expert panel members’ thoughts are summarized in the following paragraphs, along with the emerging recommendations.

Summary of Expert Panel 1 (September 8, 2014)

Presence of a Past Year Mental Disorder

The first part of the 1993 FR notice definition of SED pertained to the presence of a DSM-consistent mental disorder in the past year. This original definition of SED was based on the criteria in the DSM-III-R (APA, 1987). Since that time, updated versions of the DSM have been released, including the recent DSM-5 (APA, 2013), which experts acknowledged leads to the need to update the SED diagnostic inclusion and exclusionary criteria. This was considered particularly important due to the addition of disruptive mood dysregulation disorder (DMDD), which would be included in the definition of SED, as well as larger DSM-5 changes to criteria for specific mental disorders, such as posttraumatic stress disorder (PTSD) in young children. Experts noted that some diagnostic tools have been updated for DSM-5. Some of these diagnostic instruments may account for DSM-5 changes via their scoring algorithms, essentially realigning DSM-IV symptoms to match DSM-5 disorder criteria. Expert panel members indicated that it is unlikely that changes to scoring alone will be sufficient to fully account for DSM-5 revisions because some disorders have new symptoms not present in DSM-IV (e.g., PTSD), others have more highly specified criteria (e.g., manic episode), and DMDD is a new disorder that may require new module development.

The 1993 FR notice definition of SED is broadly inclusive of all mental disorders, with a few noted exclusions (neurodevelopmental disorders and SUDs). Meanwhile, many large-scale epidemiological studies do not assess all mental disorders likely included in the SED definition. Panel 1 members noted that many studies exclude the assessment of low-prevalence conditions, such as psychotic disorders or obsessive compulsive disorder. Panel 1 also noted, however, that low-prevalence disorders may be important for estimating the prevalence of SED, particularly if these disorders are highly impairing. As such, some low prevalence, but highly impairing disorders may be critical for any study that aims to estimate the prevalence of SED. Acknowledging the difficulty of assessing conditions known to have low prevalence (because of the requirement for a very large sample size), the expert panel members also indicated that an assessment of symptoms may be sufficient for an accurate estimate of SED. For example, they suggested that priority assessment areas might include psychotic symptoms or experiences, mania and hypomania, suicidality, and sleep and regulatory behaviors for young children. These symptoms are often precursors of disorders that develop in late adolescence and early adulthood.

Epidemiological studies have also sometimes excluded some mental disorders that are highly comorbid with other disorders. Panel 1 participants noted that large-scale studies exist to help inform the decision about which childhood mental disorders might be the most critical to measure within a study to estimate SED (e.g., determining the most prevalent and least comorbid mental disorders). The ideal study design to estimate SED would then include an assessment of common DSM-5 childhood mental disorders, supplemented by the measurement of highly disabling symptoms that are associated with low-prevalence, non-comorbid disorders.

Impact of Child Age on the Establishment of a Mental Disorder

The 1993 FR notice definition of SED includes children from their birth to 17 years of age. However, no published studies have estimated the prevalence of SED in children younger than 6 years. Although a diagnostic manual is available for young children (i.e., the Diagnostic Classification of Mental Health and Developmental Disorders of Infancy and Early Childhood [DC:0-3 R]; Zero to Three, 2005), the wide age range defined by SED poses a measurement challenge. No one diagnostic interview is designed to measure the presence of a mental disorder among individuals aged 0 to 17 years. Separate diagnostic tools will have to be used to assess the presence of a mental disorder among children from birth to older adolescence. Members of Panel 1 noted that many well-validated diagnostic tools are available across this age range, and some of these tools have been updated to meet DSM-5 criteria. Adding further complexity, some expert panel members recommended that the definition of SED include youths 0 to 21 years. This is the population of focus for current SAMHSA (2015) mental health block grants. The addition of 18- to 21-year-olds would require further attention to appropriate measures for that age range.

The general mental disorder inclusionary and exclusionary criteria to assess the presence of SED will not differ significantly based on child age; however, different disorders may be more or less of a measurement priority for various age groups. For instance, expert panel members noted that attachment disorders would likely be a priority for young children 1 to 3 years old; meanwhile, mood disorders would be a high priority for adolescents.

Functional Impairment

The second part of the 1993 FR notice definition of SED states the requirement for “functional impairment which substantially interferes with or limits the child’s role or functioning in family, school, or community activities” (SAMHSA, CMHS, 1993). Although this component of the definition has strong face validity within a service context, it lacks operational specificity for measurement purposes. No universally defined cut point is available for establishing when a child’s mental health condition is considered to “substantially interfere” with functioning. In great part, this happens because what is considered substantial impairment has not been operationally defined. The measurement of functional impairment is less well developed than the measurement of childhood mental disorders, but it continues to be critically important.

The importance of functional impairment (in addition to symptoms) for making a mental health diagnosis has changed over time and still differs across classification systems. The DSM has incorporated elements of functional impairment into its criteria for many mental disorders, and it has been adapted over time so that symptoms and elements of impairment are more closely related than when the DSM was originally conceived in 1952. In comparison, the International Classification of Diseases, 10th revision (ICD-10; World Health Organization, 2015) has kept disorders and disabilities conceptually disentangled over time so that symptoms and functional impairment are increasingly separate from each other; one could, for example, have a mental disorder functional impairment without meeting symptom criteria for a disorder. Disentangling the impact of the symptoms of disorders versus impairment in day-to-day functioning related to specific disorders is critical for assessing SED and estimating its prevalence. The 1993 FR notice definition is clear that a child must meet symptom criteria for a disorder and have impaired functioning related to the disorder. As such, children with functional impairment that does not stem directly from symptoms of a specific mental disorder would not be classified as having SED. Following from this, assessment of functional impairment must account for the underlying mental disorder that causes the functional impairment. This issue is critical for the provision of clinical services and for any research that aims to document the prevalence of SED (Rapee, Bogels, van der Sluis, Craske, & Ollendick, 2012; Warner, 2006).

To give an illustration, population-based epidemiological research has reported high rates of child psychiatric disorders (even close to half of the population) when a separate measure of impairment is not explicitly included in the identification of cases (i.e., only the impairment items that are part of the diagnosis of the mental disorders are used). When in addition to the diagnosed mental disorders a separate assessment of impairment is endorsed, however, the estimates drop to 17%–20% (Canino, Fisher, Alegria, & Bird, 2013).

Consistent with the inclusion and importance of functional impairment in the definition, some expert panel members recommended that the term SED be changed to a “serious mental health impairment” or “serious mental, emotional, or behavioral impairment,” terms that may be considered less stigmatizing by young people.

Summary of Expert Panel 2 (November 12, 2014)

Measurement of Past Year Disorders

Recognizing that SED covers individuals who are aged 0 to 17, one issue that quickly emerged during Panel 2’s meeting was the question of whether valid and reliable instruments are available for measuring mental disorders in infants and toddlers. Despite the fact that an SED estimate for children under 6 years of age has not been published, Panel 2 members emphasized that there has been tremendous growth in knowledge about how to assess the presence of mental disorders in early childhood. Diagnostic tools now exist to assess the presence of mental disorders in children beginning at 2 years of age based on parent reports (Egger & Emde, 2011). For example, the Preschool Age Psychiatric Assessment (PAPA) is a psychiatric diagnostic interview developed for parents of children aged 2 to 5 years old. Other instruments are available for children 12 to 36 months (Carter, Briggs-Gowan, Jones, & Little, 2003; Carter, Little, Briggs-Gowan, & Kogan, 1999; Copeland, Angold, Costello, & Egger, 2013; Egger, 2009; Egger et al., 2006; Lavigne, LeBailly, Hopkins, Gouze, & Binns, 2009; Wichstrom et al., 2012). In addition, child-report instruments for children aged 4 or older are now seen as developmentally appropriate for reporting psychiatric symptoms (Copeland et al., 2013; Egger, 2009; Egger et al., 2006). Although measures now exist to assess mental disorders from early childhood through adolescence, it should be noted that information is not as extensive regarding how estimates generated from early childhood instruments may compare with estimates generated from instruments designed for school-aged children or adolescents.

Diagnostic Interviews

Several instruments have been developed to assess mental disorders in children and adolescents, relying primarily on the presence of symptoms aligned with the DSM. For each instrument discussed by the expert panel members, Table 3 provides information on the age range covered, type of informant, administration method, total time to administer, interview format, validated mode, and diagnostic time frame. Collectively, these diagnostic measures cover the age range from 12 months to 18 years; however, no single instrument covers the entire age range from infancy to 17 years of age. Table 4 shows the age coverage for the validated diagnostic instruments and illustrates the overlap in age coverage across the instruments. Each instrument maps to DSM symptom criteria; however, only the CAPA, the K-SADS-PL (Kiddie Schedule for Affective Disorders and Schizophrenia for School-Age Children–Present and Lifetime Version), and the MINI-KID (Mini International Neuro-Psychiatric Interview for Children and Adolescents; Sheehan et al., 1998; Sheehan et al., 2010) have been updated for DSM-5 to date, while the Diagnostic Interview Schedule for Children (DISC) is being updated. The K-SADS-PL relies on clinical judgment to assess symptoms and assign diagnoses, but the other instruments listed in Table 3 rely on automated scoring algorithms that take into account all of the symptoms endorsed across all modules, regardless of mode of administration.

Table 3.

Examples of Diagnostic Instruments for Childhood Mental Disorders.

Instrument	Age range	Informant	Administered by	Time to Administer	Interview Format	Validated mode	Diagnostic time frame
K-SADS-PL	6–18 years	Child (6–18) Parent	Clinician	1.05 hr child 1.5 hr parent	Semistructured interview—paper and pencil version		LifetimePast yearCurrent
K-SADS-COMP	6–18 years	Child (6–18)Parent	Clinician—computer assisted	1.0 hr child1.5 hr parent	Semistructured interview—web-based administration		LifetimeCurrent
K-SADS-COMP	6–18 years	Child (11–18)Parent (6–18)	Self-administered—computer-assisted	1.0 hr child1.5 hr parent	Web-based administration with video clips for youths		Current
DISC-IV	6–17 years	Child (9–17)Parent (of child 6–17)Also a young child and young adult version	Trained lay interviewera	90–120 min	Structured interview	CAPI English and SpanishCATI EnglishACASI English	Past yearCurrent (past 4 weeks; optional lifetime)
CAPA Version 4.2	6–18 years	Child (6–18)Parent	Trained lay interviewer	1 hr parent1 hr child	Semistructured interview	PAPI in-person	LifetimeCurrent
PAPA	2–5 years	Parent	Trained lay interviewer	1 hr parent2 hr child	Semistructured interview	PAPI in-person English	Current (past 3 months)Lifetime for some symptoms
NCS-A CIDI	13–17 years	Child (13–17)Parent	Trained lay interviewer	2–3 hr child1 hr parent	Structured interview		LifetimePast year
ITSEA	12–36 months	Parent or child care provider	Self-administered	25–30 min	Paper and pencil		Past month
MINI-KID	6–17 years	Parent and child		30 min	Structured interview	Face-to face interview	CurrentLifetime

Note. K-SADS-COMP = Kiddie-Schedule of Affective Disorders and Schizophrenia-Computerized; K-SADS-PL = Kiddie Schedule for Affective Disorders and Schizophrenia–Present and Lifetime Version; DISC-IV = Diagnostic Interview Schedule for Children Version IV; CAPI = computer-assisted personal interviewing; CATI = computer-assisted telephone interviewing; ACASI = audio computer-assisted self-interviewing; CAPA = Child and Adolescent Psychiatric Assessment; PAPI = paper-and-pencil interviewing; PAPA = Preschool Age Psychiatric Assessment; NCS-A = National Comorbidity Survey Adolescent Supplement; CIDI = Composite International Diagnostic Interview; ITSEA = Infant Toddler Social Emotional Assessment; MINI-KID= Mini International Neuro-Psychiatric Interview for Children and Adolescents.

Trained lay interviewers are typically BA-level staff who receive prescribed training from the developer.

Table 4.

Selected Instruments’ Age Coverage.

	Children or youths’ age in years
Instrument	0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18
Mental disorder
K-SADS-PL
DISC-IV
CAPA
PAPA
NCS-A CIDI
ITSEA
MINI-KID
Impairment
CIS
CGAS
BIS
CAFAS
PECFAS

Note. Mental disorder: K-SADS-PL = Kiddie Schedule for Affective Disorders and Schizophrenia–Present and Lifetime Version; DISC-IV = Diagnostic Interview Schedule for Children Version IV; CAPA = Child and Adolescent Psychiatric Assessment; PAPA = Preschool Age Psychiatric Assessment; NCS-A = National Comorbidity Survey Adolescent Supplement; CIDI = Composite International Diagnostic Interview; ITSEA = Infant Toddler Social Emotional Assessment; MINI-KID= Mini International Neuro-Psychiatric Interview for Children and Adolescents. Impairment: CIS = Columbia Impairment Scale; CGAS = Children’s Global Assessment Scale; BIS = Brief Impairment Scale; CAFAS = Child and Adolescent Functional Assessment Scale; PECFAS = Preschool and Early Childhood Functional Assessment Scale.

Measurement of Functional Impairment

Measures to assess functional impairment were first developed in the early 1980s. Available scales of functional impairment range from those with highly documented psychometric support (Achenbach & Rescorla, 2001; Bird et al., 2005; Bird et al., 1993; Canino et al., 2013; Francis, Ebesutani, & Chorpita, 2012; Shaffer et al., 1983) to those that have not been critically reviewed but are in widespread use (Hodges & Gust, 1995; Winters, Collett, & Myers, 2005). These scales represent a continuum of constructs from symptoms to functional impairment to contextual factors, with the latest scales focusing on the measurement of functional impairment separate from symptomatology (Winters et al., 2005). A key challenge for impairment scales has been the need for instruments that do not vary systematically in relation to a specific diagnosis (Pickles et al., 2001; Winters et al., 2005) or by the reporting source (e.g., youth, parent, teacher; Kramer et al., 2004; Wille, Bettge, Wittchen, Ravens-Sieberer, & BELLA Study Group, 2008). For example, one study comparing multiple impairment tools found significantly higher rates of SED among youths with externalizing disorders when compared with youths with internalizing disorders (Francis et al., 2012). This study raised questions about the optimal assessment strategy to measure impairment and identify SED, as well as the need for caution when only one measure of functional impairment is used to determine whether an individual has a disorder, as some disorders are by their very nature more impairing than others.

Panel 2 concluded that work on developing and refining measures of functional impairment is greatly needed, especially the creation of measures tailored to a child’s age and developmentally appropriate expectations. Even within the same DSM mental disorder diagnosis, impairment may be defined very differently depending on the child’s age. Social and other demands placed upon children differ across age and setting. Panel members stressed that this work is acutely needed for measures that assess impairment associated with internalizing problems (anxiety, depressive disorders) for children who are too young to self-report. It was also noted that measures to assess functional impairment in early childhood are particularly lacking. Experts indicated that young child functional impairment may manifest itself as the impact of a child’s symptoms on a parent’s or family’s functioning (e.g., a parent’s ability to work outside the home). Consequently, measures of functional impairment in school-aged children may not be immediately translatable to young children. A measure of functional impairment for young children will likely need to consider family impact (Towe-Goodman, Franz, Copeland, Angold, & Egger, 2014).

Expert panel members recommended that measures of functional impairment be more strongly operationalized with concrete criteria organized by developmental periods and age-specific examples. They also suggested that further work be done to establish severity cut points that define the presence of “SED” within available impairment measures. For the purpose of generating an accurate national estimate of SED, Panel 2 experts suggested that an ideal impairment measure would have indicators across the three domains of functioning described in the 1993 FR notice (i.e., home, school, and community). This is important because some measures of functional impairment result in a single global summary score that cannot disentangle functioning within or across multiple settings.

Many panel experts also recommended that research assessing SED should include an impairment measure that is separate from the diagnostic interview used to assess the presence of a mental disorder. Impairment measures should ideally include both self-reports by youths and parent reports; however, the panel members noted that adolescent or child self-report measures of functional impairment are currently almost completely absent in the field. The panel also suggested that teacher reports of impairment can be useful for assessing children’s functioning in the school environment, especially for the externalizing disorders; however, caution is warranted in relying exclusively on reports from special education teachers because their frame of reference may differ from teachers in a mainstream classroom. Specifically, special education teachers may be more likely to inflate a child’s level of functioning if they have limited opportunities to observe the child’s behavior in a normative setting, and all teachers may underestimate impairment from internalizing disorders.

Measures of Functional Impairment

To measure functional impairment, Panel 2 discussed several candidate measures, including the CGAS (Shaffer et al., 1983), the Columbia Impairment Scale (CIS; Bird et al., 1993), and the Brief Impairment Scale (BIS; Bird et al., 2005; Shaffer et al., 1983; see Table 5). Future developmental work was recommended by panel experts for each functional impairment measure. For example, one recommendation was that the CGAS should include examples with developmental specificity for each cut point (e.g., behaviors or specific functioning examples of how a given score might look for a young child vs. an adolescent). Other opportunities for further research on measures of functional impairment are also described in Table 5.

Table 5.

Examples of Diagnostic Instruments for Measuring Child Impairment.

Instrument	Age range	Informant	Administered by	Time to administer	Interview format	Opportunities for additional research
CIS	7–17 years	ChildParent	Trained lay interviewers	5 min	Structured questionnaire	Capitalize on inclusion in the Medical Expenditure Panel Survey for secondary data analysisCompare and contrast CIS and CGAS resultsDetermine the utility of a composite versus domain-specific score for determining significant functional impairmentExamine cross-cultural applicability (available for Spanish version)
CGAS	4–16 years	Clinician-trained lay interviewer	Not applicable (1 min based on results of diagnostic assessment with trained lay interviewers)	10–30 min	Single scale with a range of 1–100; 10 anchor points with simplified descriptions for lay interviewer	Determine cut point necessary to establish serious functional impairmentOperationalize scores to provide more behaviorally grounded descriptors across the developmental periods of childhoodCompare reliability across reporters (clinician, parent, lay, interviewer, child)Examine cross-cultural applicability for other than Spanish-speaking populations
BIS	4–17 years	Parents	Clinician	5 min	Structured questionnaire	Examine use of the BIS with parents of preschool-aged children, and consider integration of a family burden of illness scaleDevelop a child-report version, and develop psychometricsDevelop a short formExamine cross-cultural applicability for other than Spanish-speaking populations
CAFAS	5–19 years	Parents	Clinician/lay interviewer (agencies’ state staff)	10–30 min	97 items over 8 scales	Describe scale development information (construct conceptualization, item pool creation, initial data collection, and item selection)Create and validate a standardized scoring system for SED
PECFAS	3–7 years	Parents	Clinician/lay interviewer (agencies’ state staff)	10–30 min	7 scales (same scales as CAFAS minus substance abuse)	Describe scale development informationCreate and validate a standardized scoring system for SED

Note. CIS = Columbia Impairment Scale; CGAS = Children’s Global Assessment Scale; BIS = Brief Impairment Scale; CAFAS = Child and Adolescent Functional Assessment Scale; SED = serious emotional disturbance; PECFAS = Preschool and Early Childhood Functional Assessment Scale.

Methodological Challenges Discussed by Both Panels

The broad age range covered by the 1993 FR notice’s SED definition and the associated diverse measurement requirements create several methodological challenges for producing national and state estimates as well as for any future survey efforts toward national surveillance. This section focuses on three challenges discussed by the expert panel participants: the type of reporter, the time reference period, and “seam effects.”

Type and Number of Informants

For a more accurate determination of mental disorders and impairment, both parent and child reports are generally recommended in the literature (Bidautrussell et al., 1995; Collishaw, Goodman, Ford, Rabe-Hesketh, & Pickles, 2009; Grills & Ollendick, 2002; Jensen et al., 1999; Rothen et al., 2009; van der Meer, Dixon, & Rose, 2008), and this was acknowledged by both panels. Most typically, children younger than 9 years of age are assessed by parent reports only. Children 9 years or older can be assessed by both parent reports and child self-reports, yielding more opportunities for symptom endorsement. Expert panel participants acknowledged that obtaining reports from both parent and child may be difficult in the context of a national survey. Expert panel members also noted research findings that indicate that parent reports and child reports will often disagree (Kramer et al., 2004; Wille et al., 2008). Consequently, they recommended that mental disorder status be estimated by both parent and child whenever possible.

Reporter decisions may also be driven by the type of mental disorder assessed. For certain diagnoses, the parent report may be more accurate than the child report (e.g., ADHD). Meanwhile, the child report may be more accurate for other diagnoses (e.g., substance use, mood disorder; Grills & Ollendick, 2002; Jensen et al., 1999). However, even with these disorders, there will be cases where a parent will report on substance use or suicidal behavior that the youth did not disclose. Validation studies of the MINI-KID with the K-SADS showed higher accuracy in general for information provided by the child due to parental underreporting of hallucinations, suicidal ideation, and substance use (Rothen et al., 2009). Unfortunately, expert panel members felt that the evidence base was not currently strong enough to confidently guide recommendations to collect diagnosis-specific information from only one informant. Consequently, the need to have more than one reporter for both mental disorder status and impairment adds a degree of complexity to any study designed to accurately estimate SED. Stated broadly, the number of reporters and the identity of reporters (in terms of their relationship to the child) will likely influence SED prevalence estimates. As a rule of thumb, the greater the number of reporters, the higher the prevalence estimates because the diagnostic algorithms of most instruments count symptoms endorsed by any reporter. It should also be noted that prevalence estimates will vary based on whether multiple-reporter information is combined at the level of symptoms or disorder.

Time Reference Period

The 1993 FR notice defined SED to be the presence of an impairing mental disorder in the past 12 months. Not all diagnostic and impairment tools use a 12-month time reference period, which could create a mismatch between the reference period within a diagnostic tool and impairment measure and with the 1993 FR notice definition. Expert panel members indicated that certain measures without a 12-month reference period might still be considered for inclusion in a study designed to estimate SED. Research suggests that respondents generally report on their recent experiences and memories (Patten, 2003; Patten et al., 2012). Consequently, panel experts suggested that estimates for past 12-month, past 3-month, and past month mental disorders will be very similar because of respondent recall problems. This is further true for disorders that require symptom onset prior to a specific age. In such cases, the recall period could be quite long. The recall bias will most typically lead to disorder underestimates for longer time reference periods (Compton & Lopez, 2014; Copeland, Shanahan, Costello, & Angold, 2011).

Seam Effects

Although the goal of SED estimation is to have comparable SED estimates across the age groups of interest, the estimation of mental disorders is affected by the number of reporters involved and by the variation in the measure by child age. Reporter variation (parent only, child only, both parent and child) will create differences in the prevalence estimates (i.e., seam effects) across child age just by virtue of the number of reporters and whether endorsement is needed from all reporters or just one. As noted previously, the field standard is to consider a child meeting the diagnostic criteria for a mental disorder by counting positive symptom reports from either parent report or child self-report. Consequently, children with two reporters will be more likely to meet the criteria for SED than children with only one reporter.

In addition, the use of different instruments across the different age groups may also contribute to seam effects. Estimates of mental disorders based on a measure established for very young children will likely not be directly comparable with estimates generated from an instrument designed for adolescents. Expert panel members noted that this is further complicated by the fact that natural progressions in the prevalence of mental disorders exist across different age groups. These types of estimate changes should be expected and not assumed to be exclusively due to methodological differences in study design for different age groups. However, sudden, non-trendable differences at the “seam” (i.e., at the age where a different instrument was being used) are likely to be related to the differences in instruments or study design.

Any study estimating the prevalence of SED will have to consider seam effects that may be due to child age, reporter type, and measurement tool(s). Expert panel members suggested that federal or state estimates of SED might ultimately need to be provided separately by various age cohorts so as not to imply that an overarching estimate applies equally from birth to 17 years.

Recommendations

No current or past study is well-positioned to generate national or state estimates of SED as defined by SAMHSA and described in the 1993 FR notice. Studies that have produced national estimates used different definitions and measures, relying on different methods and focusing on different age ranges. Many well-validated clinician-administered instruments do exist, particularly with regard to the assessment of mental disorders; however, few were developed for use in a national survey context. The expert panels convened by SAMHSA in 2014 discussed these issues, the challenges of measuring SED, and what would be needed to produce accurate national and state estimates. Taking these into consideration, short-term and longer term recommendations emerged not only to create a platform to meet the federal mandate to estimate SED but also to enhance the instruments and methods useful to estimate SED and in turn to help facilitate national surveillance efforts. These recommendations are particularly timely as the federal government considers ways to expand the collection of behavioral health data. Recommendations described here complement discussions of the steering committee for the workshop on integrating new measures of SED in children into SAMHSA’s data collection programs (see National Academies of Sciences, Engineering, and Medicine, 2016).

Recommendations for Short-Term Action

Adapt existing instruments to measure child mental disorders for use in a large-scale survey context. Several valid and reliable instruments for measuring past year mental disorders for children 6 to 18 years of age are available; however, most of these instruments were designed outside of the context of large-scale, ongoing surveillance efforts. Consequently, measures and tools may need to be adapted to suit this survey context.

Conduct small-scale studies to validate the use of existing instruments in various modes beyond in-person administration. Many large epidemiological studies require an instrument administration mode (e.g., by telephone, with lay interviewers, via the web) that differs from the way in which an instrument was originally designed to be administered. Across measures for both early and later childhood, further validation studies may be needed for specific instruments using specific modes of collection within the household context of nationally representative samples.

Perform cognitive testing to examine whether instrument items are understood similarly across survey modes and respondents. Panel members discussed the importance of multiple reporters to assess both mental disorders and functional impairment. Varying respondents may not comprehend or assess instruments items comparably either within or across survey modes. Children of various ages may also not understand instrument questions in a similar way.

Strengthen existing measures of functional impairment through additional research. Panel members noted the need for more work to determine the characteristics of substantial impairment in functioning across the key developmental stages of childhood, particularly for children from birth to 4 years.

Create clearly operationalized behavioral anchors in existing impairment scales. Panel members also noted that many measures of functional impairment (as well as the items included within them) may need to be tailored with behavioral anchoring to operationalize key functional constructs by child age and developmental stage.

Capitalize upon available secondary data to establish cut points on existing impairment scales to characterize various levels of functional impairment. Moreover, and importantly, an agreed-upon, scale-specific cut point is needed to establish an impairment level necessary to “substantially interfere with or limit” a child’s daily functioning.

Consider updates to SED definition and terminology. Panelists were asked to ground their discussions in the 1993 FR SED definition. During the meeting, however, panelists noted that the SED definition was outdated and had serious limitations. For example, the 1993 SED definition uses DSM-III-R terminology and excludes conditions such as SUD and developmental disabilities. Exclusion of SUDs is contrary to the current integration approach that combines mental and SUDs into a “behavioral health” category. Panelists also suggested that the FR definition could be improved by more concretely operationalizing the criteria defining functional impairment. In addition, panelists noted that there is significant concern, particularly expressed by youth and families, with the phrase “Serious Emotional Disturbance.” Youth and family members stated that this term is negative and gives rise to prejudice and discrimination. Panel members suggested that the term could be revised to include impairment using terminology such as “Serious Mental Health Impairment or Condition.” Others suggested incorporating language used in other countries such as “mental ill-health.” What is clear is that it would be useful to youth and families, and the research and clinical practice communities, to revisit and update the definition and terminology associated with SED.

Recommendations for Longer Term Action

Develop new instruments (or expand upon current ones) to fill existing gaps in the field. Instruments to facilitate national surveillance efforts are particularly needed for young children and to be used across age groups.

Develop or adapt clinical instruments for infants and toddlers that are easily administered to parents without the need for clinical observation. Panel members noted that there have been many recent advances in the measurement of early childhood mental health resulting in a handful of standardized instruments designed for children who are aged 2 to 5 years and even 12 to 36 months; however, these early childhood tools have not yet been used in large national studies.

Develop or adapt instruments to assess both mental disorders and functional impairment from infancy through young adulthood in a consistent, standardized, and comparable manner. Existing instruments do not seamlessly cover the full age range of children from birth to 18 years and may not cover the suggested increase in the SED definition to age 22 years. Ultimately, the development of tools for use from birth to young adulthood will facilitate the study of behavioral health across development.

Refine existing measures of impairment to ensure that they are culturally competent. Expert panel members noted that perceptions of impairment can vary by culture, given different culturally specific behavioral expectations of children. Little is known about how impairment measures might be adapted to consider this cultural variation.

Create versions of existing diagnostic interviews and measures of functional impairment that can be administered in languages other than English. Many large-scale surveys in the United States collect information from Spanish-speaking respondents. However, Spanish-language versions of some child mental health instruments are absent and would require new development to accurately reflect the prevalence of SED within the increasingly diverse U.S. population.

Conclusion

It is particularly complex to measure the behavioral health of children. A child’s age and developmental level complicate the measurement of both mental disorders and functional impairment. The best reporter or reporters of mental health symptoms and impairment vary across age ranges. This reporter and instrument variation will necessarily create differences in SED prevalence estimates across age merely as a result of the study design. Moreover, given the differences in symptom manifestation by children’s developmental stage and chronological age, it is unlikely that one diagnostic and functional impairment measure could be used to estimate SED. Consequently, caution should be taken around any attempt to generate a single prevalence estimate of childhood SED. This may not be a realistic goal. Instead, SED prevalence estimates might be more accurate and clinically useful if reported separately for key age ranges (infants and toddlers, preschool age, school age, adolescents, and young adults), using the best instruments and methods for each group. An accepted weakness of this approach would be that there could be methodologically induced differences in SED between ages.

There appears to be consensus in the field on the methodological challenges and need for further research and developmental work prior to the production of accurate prevalence estimates of SED at the national or state level for children from their birth to 18 years. Future measurement, development, and psychometric testing of instruments (for both mental disorders and functional impairment) are needed across the age span. Also, methodological studies are needed to examine the impact of factors such as instrument administration mode, type of reporter, and the various time reference periods used across measures. These preparatory steps are essential to create a study design for producing national and state estimates of SED that are consistent with the SAMHSA definition provided in the 1993 FR notice and standards required for official statistics produced by statistical agencies (OMB, 2006). These efforts will lay the necessary groundwork to understand the changing resource needs of those children who could most benefit from mental health care and services.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

References

Achenbach

T. M.

Rescorla

L. A.

(2001). Manual for the ASEBA school-age forms & profiles. Burlington: University of Vermont, Research Center for Children, Youth & Families.

American Psychiatric Association. (1987). Diagnostic and statistical manual of mental disorders (3rd ed., Rev.). Washington, DC: Author.

American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author.

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Arlington, VA: American Psychiatric Publishing.

Bidautrussell

Reich

Cottler

L. B.

Robins

L. N.

Compton

W. M.

Mattison

R. E.

(1995). The Diagnostic Interview Schedule for Children (PC-DISC V3.0): Parents and adolescents suggest reasons for expecting discrepant answers. Journal of Abnormal Child Psychology, 23, 641–659. doi:10.1007/bf01447667

Bird

H. R.

Canino

G. J.

Davies

Ramirez

Chavez

Duarte

(2005). The Brief Impairment Scale (BIS): A multidimensional scale of functional impairment for children and adolescents. Journal of the American Academy of Child & Adolescent Psychiatry, 44, 699–707. doi:10.1097/01.chi.0000163281.41383.94

Bird

H. R.

Shaffer

Fisher

Gould

M. S.

Staghezza

Chen

J. Y.

Hoven

(1993). The Columbia-Impairment-Scale (CIS): Pilot findings on a measure of global impairment for children and adolescents. International Journal of Methods in Psychiatric Research, 3, 167–176.

Canino

G. J.

Fisher

P. W.

Alegria

Bird

H. R.

(2013). Assessing child impairment in functioning in different contexts: Implications for use of services and the classification of psychiatric disorders. Open Journal of Medical Psychology, 2, 29–34. doi:10.4236/ojmp.2013.21006

Carter

A. S.

Briggs-Gowan

M. J.

Jones

S. M.

Little

T. D.

(2003). The Infant-Toddler Social and Emotional Assessment (ITSEA): Factor structure, reliability, and validity. Journal of Abnormal Child Psychology, 31, 495–514. doi:10.1023/a:1025449031360

10.

Carter

A. S.

Little

Briggs-Gowan

M. J.

Kogan

(1999). The Infant-Toddler Social and Emotional Assessment (ITSEA): Comparing parent ratings to laboratory observations of task mastery, emotion regulation, coping behaviors, and attachment status. Infant Mental Health Journal, 20, 375–392. doi:10.1002/(sici)1097-0355(199924)20:4<375::aid-imhj2>3.0.co;2-p

11.

Centers for Disease Control and Prevention. (2013). Mental health surveillance among children—United States, 2005-2011. Morbidity and Mortality Weekly Report, 62(Suppl. 2), 1–40.

12.

Central Office on ICD-9-CM. (1997). International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM). Chicago, IL: American Hospital Association.

13.

Collishaw

Goodman

Ford

Rabe-Hesketh

Pickles

(2009). How far are associations between child, family and community factors and child psychopathology informant-specific and informant-general? Journal of Child Psychology and Psychiatry, 50, 571–580. doi:10.1111/j.1469-7610.2008.02026.x

14.

Compton

W. M.

Lopez

M. F.

(2014). Accuracy in reporting past psychiatric symptoms: The role of cross-sectional studies in psychiatric research. JAMA Psychiatry, 71, 233–234. doi:10.1001/jamapsychiatry.2013.4111

15.

Copeland

W. E.

Angold

Costello

E. J.

Egger

(2013). Prevalence, comorbidity, and correlates of DSM-5 proposed disruptive mood dysregulation disorder. American Journal of Psychiatry, 170, 173–179. doi:10.1176/appi.ajp.2012.12010132

16.

Copeland

W. E.

Shanahan

Costello

E. J.

Angold

(2011). Cumulative prevalence of psychiatric disorders by young adulthood: A prospective cohort analysis from the Great Smoky Mountains Study. Journal of the American Academy of Child & Adolescent Psychiatry, 50, 252–261. doi:10.1016/j.jaac.2010.12.014

17.

Costello

E. J.

Angold

Burns

B. J.

Erkanli

Stangl

D. K.

Tweed

D. L.

(1996). The Great Smoky Mountains Study of youth: Functional impairment and serious emotional disturbance. Archives of General Psychiatry, 53, 1137–1143. doi:10.1001/archpsyc.1996.01830120077013

18.

Egger

H. L.

(2009). Psychiatric assessment of young children. Child and Adolescent Psychiatric Clinics of North America, 18, 559–580. doi:10.1016/j.chc.2009.02.004

19.

Egger

H. L.

Emde

R. N.

(2011). Developmentally sensitive diagnostic criteria for mental health disorders in early childhood: The Diagnostic and Statistical Manual of Mental Disorders-IV, the Research Diagnostic Criteria-Preschool Age, and the Diagnostic Classification of Mental Health and Developmental Disorders of Infancy and Early Childhood-Revised. American Psychologist, 66, 95–106. doi:10.1037/a0021026

20.

Egger

H. L.

Erkanli

Keeler

Potts

Walter

B. K.

Angold

(2006). Test-retest reliability of the Preschool Age Psychiatric Assessment (PAPA). Journal of the American Academy of Child & Adolescent Psychiatry, 45, 538–549. doi:10.1097/01.chi.0000205705.71194.b8

21.

Ezpeleta

Keeler

Erkanli

Costello

E. J.

Angold

(2001). Epidemiology of psychiatric disability in childhood and adolescence. Journal of Child Psychology and Psychiatry and Allied Disciplines, 42, 901–914. doi:10.1111/1469-7610.00786

22.

Francis

S. E.

Ebesutani

Chorpita

B. F.

(2012). Differences in levels of functional impairment and rates of serious emotional disturbance between youth with internalizing and externalizing disorders when using the CAFAS or GAF to assess functional impairment. Journal of Emotional and Behavioral Disorders, 20, 226–240. doi:10.1177/1063426610387607

23.

Friedman

Katz-Levey

Manderscheid

Sondheimer

(1996). Prevalence of serious emotional disturbances in children and adolescents. In Manderscheid

Sonnenschein

(Eds.), Mental Health, United States, 1996 (pp. 71–88). Rockville, MD: Center for Mental Health Services.

24.

Green

J. G.

Alegria

Kessler

R. C.

McLaughlin

K. A.

Gruber

M. J.

Sampson

N. A.

Zaslavsky

A. M.

(2015). Neighborhood sociodemographic predictors of Serious Emotional Disturbance (SED) in schools: Demonstrating a small area estimation method in the National Comorbidity Survey (NCS-A) Adolescent Supplement. Administration and Policy in Mental Health and Mental Health Services Research, 42, 111–120. doi:10.1007/s10488-014-0550-8

25.

Grills

A. E.

Ollendick

T. H.

(2002). Issues in parent-child agreement: The case of structured diagnostic interviews. Clinical Child and Family Psychology Review, 5, 57–83. doi:10.1023/a:1014573708569

26.

Hodges

Gust

(1995). Measures of impairment for children and adolescents. The Journal of Mental Health Administration, 22, 403–413. doi:10.1007/bf02518634

27.

Jensen

P. S.

Rubio-Stipec

Canino

Bird

H. R.

Dulcan

M. K.

Schwab-Stone

M. E.

Lahey

B. B.

(1999). Parent and child contributions to diagnosis of mental disorder: Are both informants always necessary? Journal of the American Academy of Child & Adolescent Psychiatry, 38, 1569–1579. doi:10.1097/00004583-199912000-00019

28.

Kessler

R. C.

Avenevoli

Costello

Green

J. G.

Gruber

M. J.

McLaughlin

K. A.

. . . Merikangas

K. R.

(2012). Severity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication Adolescent Supplement. Archives of General Psychiatry, 69, 381–389. doi:10.1001/archgenpsychiatry.2011.1603

29.

Kramer

T. L.

Phillips

S. D.

Hargis

M. B.

Miller

T. L.

Burns

B. J.

Robbins

J. M.

(2004). Disagreement between parent and adolescent reports of functional impairment. Journal of Child Psychology and Psychiatry, 45, 248–259. doi:10.1111/j.1469-7610.2004.00217.x

30.

Lahey

B. B.

Flagg

E. W.

Bird

H. R.

SchwabStone

M. E.

Canino

Dulcan

M. K.

. . . Regier

D. A.

(1996). The NIMH Methods for the Epidemiology of Child and Adolescent Mental Disorders (MECA) Study: Background and methodology. Journal of the American Academy of Child & Adolescent Psychiatry, 35, 855–864. doi:10.1097/00004583-199607000-00011

31.

Lavigne

J. V.

LeBailly

S. A.

Hopkins

Gouze

K. R.

Binns

H. J.

(2009). The prevalence of ADHD, ODD, depression, and anxiety in a community sample of 4-year-olds. Journal of Clinical Child & Adolescent Psychology, 38, 315–328. doi:10.1080/15374410902851382

32.

Green

J. G.

Kessler

R. C.

Zaslavsky

A. M.

(2010). Estimating prevalence of serious emotional disturbance in schools using a brief screening scale. International Journal of Methods in Psychiatric Research, 19, 88–98. doi:10.1002/mpr.315

33.

Mark

T. L.

Buck

J. A.

(2006). Characteristics of U.S. youths with serious emotional disturbance: Data from the National Health Interview Survey. Psychiatric Services, 57, 1573–1578. doi:10.1176/appi.ps.57.11.1573

34.

Merikangas

K. R.

J. P.

Brody

Fisher

P. W.

Bourdon

Koretz

D. S.

(2010). Prevalence and treatment of mental disorders among US children in the 2001-2004 NHANES. Pediatrics, 125, 75–81. doi:10.1542/peds.2008-2598

35.

Narrow

W. E.

Regier

D. A.

Goodman

S. H.

Rae

D. S.

Roper

M. T.

Bourdon

K. H.

. . . Moore

(1998). A comparison of federal definitions of severe mental illness among children and adolescents in four communities. Psychiatric Services, 49, 1601–1608. doi:10.1176/ps.49.12.1601

36.

National Academies of Sciences, Engineering, and Medicine. (2016). Measuring Serious Emotional Disturbance in children: Workshop summary (K. Marton, Rapporteur. Committee on National Statistics and Board on Behavioral, Cognitive, and Sensory Sciences, Division of Behavioral and Social Sciences and Education, Board on Health Sciences Policy, Institute of Medicine). Washington, DC: The National Academies Press.

37.

Office of Management and Budget. (2006). Standards and guidelines for statistical surveys. Retrieved from https://www.whitehouse.gov/sites/default/files/omb/inforeg/statpolicy/standards_stat_surveys.pdf

38.

Olfson

Druss

B. G.

Marcus

S. C.

(2015). Trends in mental health care among children and adolescents. New England Journal of Medicine, 373, 1079. doi:10.1056/NEJMc1507642

39.

Pastor

P. N.

Reuben

C. A.

Duran

C. R.

(2012). Identifying emotional and behavioral problems in children aged 4-17 years: United States, 2001-2007 (Vol. 48, National Health Statistics Reports, pp. 1–15). Washington, DC: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Health Statistics.

40.

Patten

S. B.

(2003). Recall bias and major depression lifetime prevalence. Social Psychiatry and Psychiatric Epidemiology, 38, 290–296. doi:10.1007/s00127-003-0649-9

41.

Patten

S. B.

Williams

J. V. A.

Lavorato

D. H.

Bulloch

A. G. M.

D’Arcy

Streiner

D. L.

(2012). Recall of recent and more remote depressive episodes in a prospective cohort study. Social Psychiatry and Psychiatric Epidemiology, 47, 691–696. doi:10.1007/s00127-011-0385-5

42.

Pickles

Rowe

Simonoff

Foley

Rutter

Silberg

(2001). Child psychiatric symptoms and psychosocial impairment: Relationship and prognostic significance. British Journal of Psychiatry, 179, 230–235. doi:10.1192/bjp.179.3.230

43.

Rapee

R. M.

Bogels

S. M.

van der Sluis

C. M.

Craske

M. G.

Ollendick

(2012). Annual Research Review: Conceptualising functional impairment in children and adolescents. Journal of Child Psychology and Psychiatry, 53, 454–468. doi:10.1111/j.1469-7610.2011.02479.x

44.

Rothen

Vandeleur

C. L.

Lustenberger

Jeanpretre

Ayer

Gamma

. . . Preisig

(2009). Parent-child agreement and prevalence estimates of diagnoses in childhood: Direct interview versus family history method. International Journal of Methods in Psychiatric Research, 18, 96–109. doi:10.1002/mpr.281

45.

Shaffer

Fisher

Dulcan

M. K.

Davies

Piacentini

SchwabStone

M. E.

. . . Regier

D. A.

(1996). The NIMH Diagnostic Interview Schedule for Children Version 2.3 (DISC-2.3): Description, acceptability, prevalence rates, and performance in the MECA study. Journal of the American Academy of Child & Adolescent Psychiatry, 35, 865–877. doi:10.1097/00004583-199607000-00012

46.

Shaffer

Gould

M. S.

Brasic

Ambrosini

Fisher

Bird

Aluwahlia

(1983). A Children’s Global Assessment Scale (CGAS). Archives of General Psychiatry, 40, 1228–1231. doi:10.1001/archpsyc.1983.01790100074010

47.

Sheehan

D. V.

Lecrubier

Sheehan

K. H.

Amorim

Janavs

Weiller

. . . Dunbar

G. C.

(1998). The Mini-International Neuropsychiatric Interview (M.I.N.I.): The development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. Journal of Clinical Psychiatry, 59(Suppl. 20), 22–33.

48.

Sheehan

D. V.

Sheehan

K. H.

Shytle

R. D.

Janavs

Bannon

Rogers

J. E.

. . . Wilkinson

(2010). Reliability and validity of the Mini International Neuropsychiatric Interview for Children and Adolescents (MINI-KID). Journal of Clinical Psychiatry, 71, 313–326. doi:10.4088/JCP.09m05305whi

49.

Substance Abuse and Mental Health Services Administration. (2015). Substance abuse and mental health block grants. Retrieved from http://www.samhsa.gov/grants/block-grants

50.

Substance Abuse and Mental Health Services Administration, Center for Mental Health Services. (1993). Final notice [Final definitions for: (1) Children with a serious emotional disturbance, and (2) adults with a serious mental illness]. Federal Register, 58, 29422–29425.

51.

Substance Abuse and Mental Health Services Administration, Center for Mental Health Services. (1998). Children with serious emotional disturbance: Estimation methodology. Federal Register, 63, 38661–38665.

52.

Towe-Goodman

N. R.

Franz

Copeland

Angold

Egger

(2014). Perceived family impact of preschool anxiety disorders. Journal of the American Academy of Child & Adolescent Psychiatry, 53, 437–446. doi:10.1016/j.jaac.2013.12,017

53.

van der Meer

Dixon

Rose

. (2008). Parent and child agreement on reports of problem behaviour obtained from a screening questionnaire, the SDQ. European Child & Adolescent Psychiatry, 17, 491–497. doi:10.1007/s00787-008-0691-y

54.

Visser

S. N.

Danielson

M. L.

Bitsko

R. H.

Holbrook

J. R.

Kogan

M. D.

Ghandour

R. M.

. . . Blumberg

S. J.

(2014). Trends in the parent-report of health care provider-diagnosed and medicated attention-deficit/hyperactivity disorder: United States, 2003-2011. Journal of the American Academy of Child & Adolescent Psychiatry, 53, 34–46.e2. doi:10.1016/j.jaac.2013.09.001

55.

Warner

L. A.

(2006). Medical problems among adolescents in US mental health services: Relationship to functional impairment. Journal of Behavioral Health Services & Research, 33, 366–379. doi:10.1007/s11414-006-9022-6

56.

Wichstrom

Berg-Nielsen

T. S.

Angold

Egger

H. L.

Solheim

Sveen

T. H.

(2012). Prevalence of psychiatric disorders in preschoolers. Journal of Child Psychology and Psychiatry, 53, 695–705. doi:10.1111/j.1469-7610.2011.02514.x

57.

Wille

Bettge

Wittchen

H. U.

Ravens-Sieberer

, & BELLA Study Group. (2008). How impaired are children and adolescents by mental health problems? Results of the BELLA study. European Child & Adolescent Psychiatry, 17, 42–51. doi:10.1007/s00787-008-1005-0

58.

Winters

N. C.

Collett

B. R.

Myers

K. M.

(2005). Ten-year review of rating functional scales, VII: Scales assessing impairment. Journal of the American Academy of Child & Adolescent Psychiatry, 44, 309–338. doi:10.1097/01.chi.0000153230.57344.cd

59.

World Health Organization. (2015). ICD-10 classification of mental and behavioural disorders. Retrieved from http://www.who.int/substance_abuse/terminology/icd_10/en/index.html

60.

Zero to Three. (2005). Diagnostic classification of mental health and developmental disorders of infancy and early childhood (DC:0-3). Washington, DC: Zero to Three Press.