Abstract
In 2016, the US Preventive Services Task Force concluded that there was “insufficient” (“I” statement) evidence to support universal primary care screening for autism spectrum disorder. The statement led to controversy among research and clinical communities. Although a number of papers have since been published arguing for the potential benefit of autism spectrum disorder screening, none adequately address the potential harms of autism spectrum disorder screening. This evidence gap may relate to confusion regarding how the US Preventive Services Task Force conceptualizes and evaluates potential harm. In this commentary, we explore how the US Preventive Services Task Force operationalizes harm and discuss how the potential for harm was described in the “I” statement on autism spectrum disorder screening. This information can serve as a guide for investigators working to study the benefits and harms of autism spectrum disorder screening in order to fill the research gaps cited by the US Preventive Services Task Force report. Finally, we recommend future research directions for exploring harms of autism spectrum disorder screening, filling cited research gaps, and ultimately ensuring that the benefits of autism spectrum disorder screening truly outweigh the harms for all children and their families.
What is the US Preventive Services Task Force?
The US Preventive Services Task Force (USPSTF), supported by the US Department of Health and Human Services’ Agency for Healthcare Research and Quality, is a volunteer panel of prevention, evidence-based medicine, and primary care experts that reviews research to inform evidence-based recommendations about clinical preventive services directed toward undiagnosed individuals (USPSTF, 2015). For each preventive service reviewed, the USPSTF releases a systematic review of the literature and official recommendation including a letter grade (A, B, C, D, or I) based on the strength of the evidence and the balance of benefits and harms (see Figure 1). For example, an “A” indicates substantial evidence supporting benefits of the service; a “D” indicates substantial evidence there is no net benefit or that harms outweigh benefits. An “I” indicates insufficient evidence to assess the balance of benefits and harms, (i.e. evidence is lacking, of poor quality, or conflicting). Although the USPSTF does not enforce these recommendations, they often inform recommendations from professional practice organizations and policy through the USPSTF’s annual report to Congress.
USPSTF Letter Grades (reproduced from USPSTF, 2012).
The 2016 USPSTF report on autism screening
In the most recent report on autism spectrum disorder (ASD) screening for children 18–30 months, the USPSTF concluded in its “I” statement that “current evidence is insufficient to assess the balance of benefits and harms of screening for ASD in young children for whom no concerns of ASD have been raised” (Siu et al., 2016, p. 691). The report notes, “clinicians should use their clinical judgment to decide if screening in children without overt signs and symptoms is appropriate” (Siu et al., 2016, p. 693). In response, clinical and research communities published a number of commentaries raising significant questions about the statement (Broder-Fingert et al., 2018; Coury, 2015; Dawson, 2016; Fein et al., 2016; Marks & Glascoe, 2016; Robins et al., 2016; Veenstra-VanderWeele & McGuire, 2016), some arguing the “I” statement could itself cause harm by decreasing early screening for ASD, resulting in an overall delay in access to early intervention (Marks & Glascoe, 2016; Robins et al., 2016), and worse outcomes for children and families (Fein et al., 2016).
While these commentaries raised important concerns, they focused heavily on the benefits of screening, citing evidence that earlier treatment is effective and arguing that universal screening would increase early identification and improve outcomes. Only one commentary mentioned potential harms, asserting that the benefits of ASD screening are likely to outweigh the risks (Fein et al., 2016). However, the USPSTF noted that data on the effectiveness of early ASD interventions were primarily derived from children identified from standard, primary care settings. How effectiveness data applies to a population of children identified as a result of universal screening is less clear. While children identified via universal screening and standard primary care practice may be similar, the USPSTF posits that these populations could be different, and thus treatment effectiveness could also be different. If these populations are in fact different, then children who screen positive as a result of universal screening might also experience harms disproportionately, particularly those who screen positive but do not meet diagnostic criteria. Thus, without direct evidence that ASD treatment benefits universal screen-detected children, and in the absence of data on children identified as a result of universal screening, the USPSTF issued an “I” statement. This commentary explores the USPSTF’s operationalization of harm and discusses themes outlined in their “I” statement on ASD screening to guide investigators working to fill the research gaps cited by the USPSTF.
How does USPSTF define harms?
Four potential harms are described in the USPSTF (2015) Procedures Manual: direct harms, classification errors, opportunity costs, and overdiagnosis. We discuss how each applies to ASD screening.
Direct harms
The USPSTF defines direct harms as harm from “the screening test. . . subsequent diagnostic tests, . . .and early treatment of screen-detected asymptomatic disease” (USPSTF, 2015, p. 40). Potential direct harms of ASD screening include time, effort, and anxiety associated with diagnostic testing. For example, the diagnostic process for ASD is time consuming, requiring multiple visits with long wait-times. This can lead to family stress and loss of wages (i.e. parent takes off work).
Classification errors
Classification errors include false-positive and false-negative errors. False-positive errors (being diagnosed when you don’t actually have ASD) could expose families to stigma and unnecessary, time-intensive, costly services. False-negative errors can delay needed services through inappropriate reassurance. The USPSTF assesses both “what is known about the number of false-positives” (USPSTF, 2015, p. 40) and evidence of “psychological harm from labeling” (p. 43). Data suggest parental stress associated with the ASD diagnostic process (Crane et al., 2016) may place unnecessary burden and anxiety on families of children with false-positive errors. Research on the negative predictive value of ASD screening tools is limited (Siu et al., 2016); thus, little is known about the false-negative rate. In the event of classification errors, families who receive a false-negative screen may forgo evaluation if future “red-flags” arise, delaying a diagnosis and treatment.
Opportunity costs
The USPSTF considers opportunity costs, “the time and effort required by both patients and the health care system to implement the preventive care service” (USPSTF, 2015, p. 43). Regarding ASD screening, the USPSTF noted, “Even good-quality studies of screening had a high dropout rate between. . . screening and diagnosis, suggesting that the process may be difficult for some families” (Siu et al., 2016, p. 695). When families drop out of the screening-to-diagnosis cascade of care, appointments are missed, wait-times lengthen, and revenue is lost. These systemic harms may be particularly important for low-income, minority, or rural families who may not have equitable access to services (Liptak et al., 2008; Magaña et al., 2013). If widespread screening leads to increased demand on already overwhelmed systems, it is possible that disparities could widen. Thus, although there is an important argument that universal screening could alleviate disparities, the impact of this practice on the system—and “clogging the system” as a potential harm—is also an important consideration.
Overdiagnosis
Overdiagnosis refers not to medical errors (false-positives/false-negatives) but to correct diagnoses that will not provide benefit to the patients who receive them. The concept of overdiagnosis can be explained by an analogy to smoke detectors. If you need to evacuate your building on a cold winter day because a battery ran low, that’s a false-positive. If the fire alarm sounded because someone scorched a full bag of popcorn in the microwave, however, then the alarm is not to blame. The smoke was real, and the alarm classified it appropriately. The problem is that the harms (standing in the cold) outweighed the benefits (escaping the odor of burnt popcorn). Similarly, overdiagnosis applies to classifications that are technically correct but confer no benefit.
Overtreatment of ASD remains possible for several reasons. First is the heterogeneity in ASD. Some individuals with ASD, especially those with milder symptoms, may benefit less from services that result from an ASD diagnosis. Next, growing awareness and de-stigmatization surrounding ASD, particularly in high-income countries, may lead to diagnoses and treatment for children and adolescents who would have remained undiagnosed in the past. Changes in the diagnostic criteria of ASD over time may have led to reduced thresholds for a diagnosis, contributing to increases in prevalence rates. Finally, clinicians may not strictly adhere to diagnostic criteria; their clinical judgment may be affected by a desire to link particular patients and families to services as well as racial, ethnic, and gender biases regarding who is and is not exhibiting sufficient symptoms of ASD to justify a clinical diagnosis (Merten et al., 2017)
Thus, the potential for overdiagnosis and overtreatment cannot be ruled out. First, the USPSTF calls for ASD treatment studies to be conducted with “children identified through screening” who “are likely to be younger and have milder symptoms” (Siu et al., 2016, p. 695). Such studies may find positive outcomes for only some clinical subgroups. While it is argued that overdiagnosis and overtreatment resulting from universal ASD screening will not outweigh the benefits because ASD diagnosis, by definition, requires symptoms to impact daily functioning, the USPSTF report suggests that it is not clear whether all individuals will benefit from treatment of those symptoms.
Where do we go from here?
Exploring and operationalizing harms is an important step in understanding the USPSTF statement on ASD screening. While a number of questions related to research and practice could result in filling gaps in knowledge about the issues raised concerning harms above, we selected and will highlight two questions, in particular, that we believe will most easily and logically build on existing research and/or for which data may soon be accessible based on trends toward wide-spread screening in large health systems over the last decade. Although not fully exhaustive, two key research questions to be addressed include: (1) what are the harms of screening asymptomatic populations (to children and parents/families) and how likely are they to occur? and (2) how does universal screening impact service systems? To address the first key research question, harms resulting from classification errors may be identified with population-based diagnostic accuracy studies. Studies that follow populations of screen-negative children in addition to screen-positive children would provide valuable information regarding screening sensitivity, specificity, and negative predictive value. Investigations of treatment effects for children across the whole spectrum of ASD severity who were diagnosed through universal screening would be valuable for assessing potential harms related to overdiagnosis. A range of methods could be considered to answer these questions. Qualitative methods could explore potential direct harms of screening through investigation of experiences of families who experience classification errors and broadly explore whether psychological or material harms occur when asymptomatic populations undergo screening (e.g. results are communicated with poor clarity or a lack of empathy; Balogh et al., 2015). Studies investigating parent/caregiver reactions to the USPSTF recommendation on ASD screening and how it impacts their perception of or desire for screening would provide information on parent preferences, engagement, and overall service experiences.
To address the second key research question, community-engaged implementation science research on service system-level functioning is necessary. Data on the impact of screening on the healthcare and educational systems would address gaps in understanding about harms related to opportunity costs. Various methods, including but not limited to failure modes and effects analysis and systems dynamics modeling, could be useful for studying the impact of screening on the service-system. The relative impacts of universal ASD screening versus standard developmental surveillance on service systems could be rigorously assessed in a future randomized controlled trail (RCT) in which asymptomatic children are randomized to one of these two screening practices and assessed for differences in patient-centered outcomes, rates of referral for diagnostic evaluation, rates of diagnosis, use of ASD-related treatments, and positive responses to treatment.
Previous commentaries have pointed out that the USPSTF systematic approach may not fit developmental and behavioral screening, such as screening for ASD, given that a positive screen does not result in immediate treatment and, rather, additional assessment. The USPSTF regularly refines their methodological and analytical framework; thus, the next assessment may involve a new and/or different conceptualization of harm. It is important for the USPSTF to consider the ways in which developmental and behavioral diagnoses like ASD differ from other medical conditions. Research outlining whether universal screening leads to more streamlined, efficient, and accurate processes for identifying children who benefit from earlier intervention and/or whether universal screening identifies a unique population needing some intervention, regardless of whether they meet ASD diagnostic criteria would greatly help the USPSTF make a more informed decision on ASD screening; however, these are not questions for the USPSTF to address. The USPSTF simply reviews the evidence that exists. It is also important for ASD intervention researchers to report whether study participants were first identified with ASD directly from universal screening or due to symptoms noted by caregivers/providers, allowing for improved systematic review. The above should inform future data collection, lead to improved policy and practice, and ultimately ensure that the benefits of ASD screening truly outweigh the harms for all children and their families.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
