Abstract
In March 2014, the Centers for Disease Control and Prevention (CDC) released their most recent estimate of the prevalence of autism spectrum disorder (ASD) among children aged 8 years (CDC, 2014). Their startling finding was that 1 in 68 children met criteria for ASD in 2010, compared with 1 in 88 in 2008 (CDC, 2012), 1 in 110 using combined data from 2006 and 2004 (CDC, 2009), and 1 in 150 in 2002 (CDC, 2007). This dramatic increase has raised many concerns regarding possible causes, with much attention given to toxic exposure in the perinatal period (Rodier, 2011) and broadening of the diagnostic criteria (Newschaffer et al., 2007). While these and other hypotheses certainly are worthy of further exploration, we believe that this apparent increase should raise as many concerns about the study methods themselves as it does about other reasons for the observed change in prevalence.
The CDC surveillance studies have performed an important service for policy makers, system administrators, and advocates. Prior to the first surveillance study that used data from 2000, the only consistent count of the number of children with autism in the United States came from the report each state provided to the US Department of Education of the number of children served in the autism category of special education (Shattuck, 2006; Shattuck and Grosse, 2007). This count did not start until 1990, when autism became a separate disability category under US special education law. These types of administrative counts are fraught with problems. They are subject to the idiosyncrasies of state reporting systems, there is no validation of diagnosis, and the potential for missed cases is substantial.
CDC-sponsored surveillance studies of ASD responded to the limitations of administrative data by putting in place a cost-efficient, population-based protocol for estimating ASD prevalence. The surveillance protocol is based on one the CDC developed for use in the Atlanta, Georgia, metropolitan area (Yeargin-Allsopp et al., 2003). In the original study using this protocol with data from 1996, the medical and education records of all children aged 3–10 years in a 5-county area were examined, looking for any indication in the records that children may have had ASD, either because they had that diagnosis in their record or because there were enough other “red flags” that suggested its presence. An expert clinician then reviewed the records of all children with suspected ASD and made a case determination based on those records. That study found that 1 in 294 children met criteria for ASD, and that prevalence peaked at age 8, which is why CDC surveillance studies since then have estimated prevalence in 8-year-olds. Of great importance, 98% of children with an ASD diagnosis in their medical records, 99% of children in the autism category of special education, and 100% of children with both were classified as autism cases. That is, there were almost no false positives. Yet, the study clinicians did not directly observe a single child to validate their record review process. On the other hand, 18% of children meeting research criteria for autism did not have that diagnosis in their records.
This study was rightly lauded as a landmark in estimating the prevalence of autism. Its methods became the backbone of the multi-site studies the CDC then funded to continue its important surveillance work. In 2007, a report was published from six sites that used the same methods as the CDC had in Atlanta, but this time examining the records of children who were 8 years old in 2000 (CDC, 2007). They found that 1 in 150 children met criteria for ASD, almost twice the prevalence found in the original study, but almost exactly the same as was observed when the CDC conducted a door-to-door prevalence study in Brick Township, New Jersey, in which they clinically assessed a large portion of the population of children in person (Bertrand et al., 2001). In this 6-site study, prevalence across sites varied from 1 in 222 to 1 in 101. The boy:girl ratio varied twofold across sites. The prevalence of intellectual disability among children with ASD ranged from 40% to 62%.
The study expanded to 14 sites for the surveillance of children who were 8 years old in 2008 (CDC, 2012). In that study, which found an overall prevalence of 1 in 88, prevalence across sites ranged from 1 in 208 to 1 in 47. Racial and ethnic differences in prevalence were observed, with traditionally underserved minority children less likely to meet criteria for a diagnosis. Only 38% of children had a documented intellectual disability, although this ranged from 13% to 54% across sites.
The most recent study, examining records of children who were 8 years old in 2010, relied on data from 11 sites (CDC, 2014). The range of prevalence across sites was similar to the prior study, ranging from 1 in 46 in New Jersey to 1 in 175 in Alabama. Racial and ethnic prevalence differences remained. This time, only 31% of children had a documented intellectual disability, ranging from 18% to 37% across sites.
To those who read these studies carefully, the cross-site variation should be as startling and alarming as the apparent increase in prevalence over time. Why should ASD respect geo-political boundaries? Could it be that there are characteristics of the physical environment in New Jersey that result in a greater prevalence of autism than in Alabama? Why would the proportion of children with ASD and intellectual disability vary so greatly across sites? Why would one observe racial and ethnic disparities that differed by site?
The discussion sections of these papers and subsequent commentaries suggested that local policies, resources and awareness may drive observed differences in prevalence. We agree these are likely sources of these observed differences. But if they are, then these studies are not measuring true prevalence and should not be advertised as such. They instead measure the extent to which clinicians and educators test for and document the symptoms of autism, regardless of whether the practitioner ultimately assigns that diagnosis. For example, as awareness of the importance of social ability and its precursors increases (in large part due to increased awareness of autism), clinicians may be more likely to make notations about eye contact or joint attention. They may note related impairments in the chart. Even if they do not ultimately diagnose the child with an ASD, the expert clinical reviewer at the surveillance site may still think there is enough information to warrant a diagnosis. In other words, in a number of cases, the community clinician has not assigned a diagnosis, but the CDC clinician is overriding this decision based on a chart review. It is interesting to note that 80% of children in the most recent study had a previous diagnosis or classification of autism, almost exactly the same as in the original study conducted in Atlanta. Based on that statistic, community practitioners were no better in 2010 in identifying cases than they were in 1996. That is, in both years, they identified only 4 out of 5 CDC-determined cases.
In a “true” prevalence study, the information a child has in their clinical or educational record is irrelevant. Researchers identify some population or population-based sample and clinically assess individuals in person to determine the presence of ASD. The CDC did not rely on this in-person strategy, presumably because of the high costs. The result, however, is that the data they have collected may be uninterpretable as it relates to prevalence. Simply put, without direct assessments of children, we will not know the extent to which the CDC-determined “cases” include false positives, or the extent to which children who it was determined do not have autism are really false negatives. Social impairments and repetitive behaviors are present in many other childhood psychiatric disorders and developmental disabilities (Casey et al., 2013). The flaws in this methodology certainly could explain the great variation in prevalence, clinical presentation, and racial disparities by site.
Tracking ASD is no easy task. In addition to the changes in diagnostic criteria, ASD is clinically complex and has no established biomarkers (Lai et al., 2014). The CDC surveillance studies have resulted in rich datasets from which much important research has been published regarding disparities in diagnosis (Giarelli et al., 2010; Mandell et al., 2009), age of diagnosis (Shattuck et al., 2009), and clinical presentation (Maenner et al., 2013). We question, however, whether they should be used any longer to provide meaningful estimates of prevalence. In fact, we believe it is a mistake to do so.