The Toddler Autism Symptom Inventory: Use in diagnostic evaluations of toddlers

Abstract

Although symptoms of autism are present early in life and early diagnosis can lead to better outcomes, there is a dearth of validated caregiver-report interviews designed for children under the age of 3 years. We developed the Toddler Autism Symptom Inventory, a semi-structured interview designed to assess the presence and absence of skills and symptoms in children aged 12–36 months. Reliability and validity of items and a cutoff score for likelihood of autism spectrum disorder were established. Specificity and sensitivity of this cutoff were confirmed with a cross-validation sample. The Toddler Autism Symptom Inventory effectively identified most children with autism without excessive false positives. The Toddler Autism Symptom Inventory is a developmentally appropriate caregiver interview for use in diagnostic evaluations of children under age 3 years that offers clearly operationalized diagnostic criteria and a cutoff for autism likelihood for very young children.

Lay abstract

Determining whether a young child has an autism spectrum disorder requires direct observation of the child and caregiver report of the child’s everyday behaviors. There are few interviews for parents that are specifically designed for children under 3 years of age. The Toddler Autism Symptom Inventory is a new interview that asks caregivers of children age 12–36 months about symptoms of possible autism spectrum disorder. The Toddler Autism Symptom Inventory uses a cutoff score to indicate likelihood for autism spectrum disorder; this cutoff score appears to accurately identify most children who are diagnosed with autism spectrum disorder without identifying too many who do not have autism spectrum disorder. The Toddler Autism Symptom Inventory interview can help clinicians to determine whether a young child shows symptoms suggestive of an autism spectrum disorder.

Keywords

autism spectrum disorder developmental disorders diagnosis toddlers

Symptoms of autism spectrum disorder (ASD) usually emerge within the first 3 years of life (Ozonoff & Iosif, 2019). Whereas typically developing (TD) children point, show objects to others, look toward others, and respond to their name, these behaviors are often reduced or absent as early as the first birthday in children later diagnosed with ASD (Osterling & Dawson, 1994; Ozonoff et al., 2010). In addition, young children later diagnosed with ASD show early difficulty in disengaging visual attention, reduced or decreasing eye contact, reduced expression of positive affect and anticipation during a social interaction, atypical play behaviors, repetitive movements, and motor delays (Jones & Klin, 2013; Leonard et al., 2014; Ozonoff et al., 2008; Sacrey et al., 2018; Zwaigenbaum et al., 2005). Although repetitive behaviors and a preference for routine are not uncommon in TD children (Evans et al., 1997; Harrop et al., 2014), by 12–24 months of age, stereotypical and restricted and repetitive behavior (RRB) frequency is higher in high-risk children diagnosed with ASD than high- and low-risk children not diagnosed with ASD (Wolff et al., 2014). The divergence of both social-communicative and RRB symptoms starting by age 12 months suggests that this may be an appropriate time to begin detection of elevated ASD likelihood in the general population.

In spite of some providers’ hesitation about diagnosing children with an ASD in the first 2 years of life, diagnoses assigned under the age of 2 years have been shown to be stable and reliable (Chawarska et al., 2007; Kleinman, Ventola, et al., 2008), with one recent study suggesting that diagnoses are stable for most children as young as 14 months of age (Pierce et al., 2019). Children with less clear or milder manifestations of ASD, however, may not be able to be detected until later in life. The finding of diagnostic stability holds even in children with cognitive, social, and language abilities under the age equivalent of 1 year (Hinnebusch et al., 2017).

Although ASD symptoms are present and recognizable early in life, the median age of diagnosis in the United States in 2014 is still around 51 months (Maenner et al., 2020). Use of ASD-specific screeners in young children has been shown to aid in early identification and to reduce disparities in age of diagnosis, provided that the child is referred for a developmental evaluation after a positive screening result (Chlebowski et al., 2013; Herlihy et al., 2014; Robins et al., 2014; Sánchez-García et al., 2019; Wetherby et al., 2008).

Early diagnoses are essential for optimizing outcome; children who access intervention services early have better outcomes than those who access services later (MacDonald et al., 2014; Rogers et al., 2014). Thus, reliable and valid screening and evaluation tools for assessing ASD symptoms in young children are important for achieving the best possible outcomes.

Diagnosis of ASD in toddlers

While screening tools are invaluable for identifying children at elevated likelihood for ASD, they do not provide a formal diagnosis of ASD, which is often required for children to receive access to the specialized intervention approaches that have been associated with the most favorable outcomes. Full diagnostic evaluations, which are more time- and cost-intensive than screeners, allow a trained clinician to gather more information about a child’s functioning, and ideally include both direct observation and caregiver report (Kim & Lord, 2012a; Sacrey et al., 2018). Direct observation allows a clinician to assess a child’s social interactions with familiar and unfamiliar adults, which is crucial. Most commonly used is the Autism Diagnostic Observation Schedule–Second Edition (ADOS-2; Lord et al., 2012), an empirically supported observational measure that can be used to determine presence and severity of ASD. Several factors indicate the need for caregiver report in addition to direct observation: the child’s behavior may not be typical for him or her in an unfamiliar setting and caregiver report provides information about the child’s everyday functioning. In addition, caregiver report includes behaviors that may not be observable in an unfamiliar office or during a time-limited observation, including interaction with peers, response to displays of emotion by others, and repetitive play with a favorite toy.

Caregiver report, which is generally quite accurate (Miller et al., 2017; Sacrey et al., 2018), can be gathered using semi-structured interviews. Caregivers often are able to report a child’s daily functioning with high accuracy (Miller et al., 2017), and when differences between parents and clinicians are found, parent report shows greater ability to differentiate children with ASD from those without ASD (Sacrey et al., 2018). The most widely used and evidence-supported caregiver report interview is the Autism Diagnostic Interview–Revised (ADI-R; Rutter et al., 2003), a standardized interview used in research and clinical diagnostic evaluations when ASD is suspected. Toddler algorithms have been developed, extending use of the ADI-R down to 12 months chronological or 10 months developmental age, with sensitivity ranging from 67% to 100% and specificity between 64% and 94% for a Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM-IV) diagnosis (Kim et al., 2013). The ADI-R toddler algorithms are based on verbal ability and age, and the lowest-functioning published sample includes children under 20 months of age and non-verbal children age 21–47 months (Kim & Lord, 2012b). However, this combined sample is heavily weighted toward older non-verbal children (n = 318, age 21–47 months with ASD) rather than younger children (n = 43, under 20 months of age with ASD). Thus, the reported validity of the interview may not accurately describe how it functions with children under 20 months of age. In addition, the ADI-R does not align with the Diagnostic and Statistical Manual of Mental Disorders (5th ed.; DSM-5; American Psychiatric Association, 2013) and is not designed to specifically ask about ASD symptoms as they manifest in toddlers; most questions on the ADI-R apply to individuals of all ages, and behaviors that may characterize toddlers with ASD in particular are not addressed. For example, the ADI-R does not specifically ask caregivers about some of the pre-verbal social communication behaviors that are important for early detection of ASD, including integrating eye contact with gestures, noticing another’s positive emotions and joining in them, pointing to show something the child has done or is doing (as opposed to pointing to express interest in something at a distance), or engaging in back-and-forth babbling. While other semi-structured caregiver interviews are available, including the Diagnostic Interview for Social and Communication Disorders (DISCO-11) and the Developmental, Dimensional, and Diagnostic Interview (3di), to our knowledge there are no published, validated semi-structured interviews specifically designed and validated for caregivers of toddlers suspected of having ASD (Randall et al., 2018). A relatively brief interview that incorporates how ASD symptoms may manifest in toddlers has a validated cutoff score indicating elevated likelihood of ASD, and is aligned with DSM-5 symptoms will aid in streamlining diagnostic evaluations for very young children.

Challenges associated with early diagnosis

Several important challenges are present when young children are referred for diagnostic evaluations. First, caregivers of children later diagnosed with ASD most often report language concerns, which are also common caregiver concerns in children with global delay and language disorders (Coonrod & Stone, 2004; Richards et al., 2016). While many children with other developmental disorders (DDs) or even typical development may demonstrate some behaviors seen in children who are subsequently diagnosed with ASD, determining whether the number, frequency, and severity of ASD symptoms are sufficient for an ASD diagnosis can be difficult.

Second, the DSM-5 ASD diagnosis describes criteria for individuals of all ages; there are no specific criteria based on the individual’s age or developmental level. These criteria can be challenging to apply to toddlers (Barton et al., 2013; Matson et al., 2012). Operationalizing these criteria into specific, observable toddler-level behaviors is essential for accurate identification of ASD in early childhood.

Third, toddlers with delayed cognitive skills and other circumstances such as inconsistent caregiving or a lack of consistent exposure to peers may be particularly difficult to diagnose. For example, children who do not yet have the requisite cognitive abilities, such as imitation and representation (Nielsen & Dissanayake, 2004), are unlikely to demonstrate pretend play because of developmental delays. If a young child spends much of their time at home or with only adults, it will be difficult to judge their difficulty forming and maintaining age-appropriate peer relationships.

A caregiver interview addressing the toddler-specific behaviors that are characteristic of ASD, using DSM-5 criteria, may help clinicians identify the specific skills and symptoms that are needed for diagnosis. An operationalized measure will also allow for more careful examination of behaviors common in ASD that are also observed in children with other DDs or limited experiences.

The aim of this study was to develop a caregiver interview for use as part of the ASD diagnostic evaluation in toddlers age 12–36 months. Reliability and validity demonstrating the measure’s ability to identify children with ASD are reported.

Methods

Participants

Participants were recruited by pediatricians involved in large cross-site studies of the early detection of ASD. Caregivers completed one or more age-appropriate screening tools: the Modified Checklist for Autism in Toddlers–Revised with Follow-up (M-CHAT R/F) at age 15 or 18 months (Robins et al., 2009, 2014), First Year Inventory–Lite (FYI-L) at 12 and 15 months (Baranek et al., 2014), or Infant-Toddler Checklist (ITC) at 12 months (Wetherby & Prizant, 2002). All children screened before 18 months were rescreened with the M-CHAT R/F at 24 or 36 months. Children (aged 12–36 months) identified by a positive screen or by ASD-specific concerns raised by a participating pediatrician were referred for a cost-free developmental/diagnostic evaluation. Exclusionary criteria for children included a previous diagnosis of an ASD and significant sensory or motor impairments that would preclude diagnostic testing (e.g. blindness, deafness, and severe cerebral palsy).

Caregivers of children who attended an evaluation (n = 336) were administered an extensive, semi-structured pilot interview during the evaluation. Forty-two interviews (19 children with ASD, 23 non-ASD; mean age 27.25 months, standard deviation (SD) = 4.14) were excluded for having more than four missing responses. Data were gathered over 5 years. Data from the first 204 children were used to develop the current version of the Toddler Autism Symptom Inventory (TASI) and the total score cutoff indicating likelihood of ASD (Sample 1). Data from the next 90 children were used to cross-validate this cutoff (Sample 2). Demographic and other characteristics of each sample are reported in Table 1.

Table 1.

Comparison of participant samples on demographic characteristics and diagnoses.

	Sample 1 (development; n = 204)	Sample 2 (cross-validation; n = 90)
Mean age, months (SD; range)	19.06 (3.71; 13–32)	19.53 (4.63; 12–36)	t(292) = −0.931, p = 0.353
Sex			χ²(1) = 0.06, p = 0.807
Male (%)	133 (65%)	60 (66%)
Race			χ²(7) = 29.91, p < 0.001
White	121 (59%)	33 (37%)
Black/African American	32 (16%)	37 (41%)
Asian	16 (8%)	3 (3%)
American Indian/Alaska Native	2 (1%)	1 (1%)
Bi-/multi-racial	12 (6%)	8 (9%)
Native Hawaiian/Pacific	0	1 (1%)
Islander
Not reported	18 (9%)	7 (8%)
Other	3 (1%)	0
SES: annual income			χ²(8) = 11.70, p = 0.165
<12,000	17 (8%)	13 (14%)
12,001–24,000	19 (9%)	10 (11%)
24,001–36,000	18 (9%)	10 (11%)
36,001–48,000	15 (7%)	5 (6%)
48,001–60,000	10 (5%)	7 (8%)
60,001–72,000	4 (2%)	5 (6%)
72,001–84,000	8 (4%)	1 (1%)
84,001–96,000	6 (3%)	2 (2%)
>96,001	85 (42%)	25 (28%)
CBE diagnostic groups			χ²(2) = 6.558, p = 0.038
ASD	53 (26%)	19 (21%)
DD	81 (40%)	50 (55%)
Global delay	36	27
Language disorder	39	21
Other	1	0
Sub-threshold autism	5	2
TD/ND	70 (34%)	21 (24%)

SD: standard deviation; CBE: clinical best estimate; ASD: autism spectrum disorder; DD: developmental disorder; TD/ND: typically developing or no-diagnosis; SES: socio-economic status.

TASI item and cutoff score development

The TASI is a semi-structured interview developed to be used by clinicians during diagnostic evaluations of very young children. The TASI was produced in a four-step process. First, a team of expert clinicians, including psychologists (M.L.B., D.L.R., W.L.S., and D.A.F.) and a developmental-behavioral pediatrician (T.D.-M.), collaborated to identify behaviors seen in toddlers age 12–36 months with any form of ASD. Observable behaviors relating to each Diagnostic and Statistical Manual of Mental Disorders (4th ed., text rev.; DSM-IV-TR) symptom were first generated. Behaviors commonly associated with ASD that are not apparent in very young children, such as changing behavior to suit specific social contexts, were not included. The pilot interview comprised 63 such items.

Next, data were collected on this 63-item pilot interview during diagnostic evaluations at three sites. Caregiver responses were classified as indicating symptom presence or absence.

Third, and occurring concurrently with data collection, our team evaluated TASI items for fit with the DSM-5 diagnostic criteria. Items that did not align with these diagnostic criteria, or that were found to be frequently misunderstood by caregivers, were removed. One example of such an item is how much the child seems to enjoy praise and attention, which was often misunderstood.

Fourth, mapping each TASI item onto a DSM-5 symptom was a challenge, as has previously been found with other measures (Barton et al., 2013). Extensive discussion led to consensus about assigning each item to the most relevant DSM-5 symptom. In general, our mapping of behaviors onto DSM-5 symptoms largely aligns with other work with similar measures (Evers et al., 2020). For example, while the authors of the 3di classified responses to a parent’s facial expression as a behavior aligned with the social-emotional reciprocity symptom of the DSM-5, as the TASI does, other authors have argued that this item better fits within the DSM-5 non-verbal communication symptom (Evers et al., 2020). Examination of the TASI’s mapping of symptoms compared to other measures revealed only one difference compared to the 3di (smile to familiar adult; 3di classified this in non-verbal communication while the TASI classifies it in social-emotional reciprocity), one in the ADOS-2 (response to joint attention; in social-emotional reciprocity symptom on ADOS-2 (Evers et al., 2020), while in non-verbal communication symptom of TASI), and none compared to the DISCO-11.

Finally, the percent of caregivers endorsing each TASI item was examined. One item (child uses no gestures) was removed from the total score calculation because it was almost never endorsed, and one item (how child responds to a hurt or sad adult) was removed due to its frequent endorsement in children with DD and TD.

Each TASI item was aligned with one ASD symptom in the DSM-5. Within the social-emotional reciprocity symptom, for example, the TASI includes questions about a child’s responses to others’ emotions, response to their name, initiation of joint attention for the purpose of showing items of interest or things the child has done, and engaging in back-and-forth babbling. Behaviors in the non-verbal communication symptom include making eye contact when requesting or during physical or social games, use of gestures, and whether the child follows an adult’s point. The relationship symptom includes questions regarding a child’s interest in other children, how the child usually plays when other children are present or when approached by another child, and whether the child engages in spontaneous imitation or pretend play. The RRB symptom of stereotyped and repetitive movements or speech includes questions about any atypical play behaviors such as lining up toys, immediate and delayed echolalia, and repetitive hand, finger, and body movements. The insistence on sameness section includes questions about how a child responds to a change in routine or in the environment, and whether the child attempts to impose their routines on others. The restricted, fixated interests symptom includes two questions about whether the child has any intense or atypical interests. Finally, the sensory table includes 29 examples of sensory symptoms often observed in toddlers with ASD and sorts them into three categories: sensory-seeking (e.g. spins car wheels while watching), hyper-sensitivity (e.g. startles easily, and covers ears), and hypo-sensitivity behaviors (e.g. does not react to painful stimuli) across sensory modalities of visual, tactile, auditory, and smell and taste. Each symptom (e.g. sensory-seeking) only requires one behavior to be scored as present.

The final TASI is composed of 37 interview items and a table for eliciting information about sensory symptoms. The TASI interview form, administration and scoring manual, and scoring tool are accessible online at https://mchatscreen.com/tasi/ .

The TASI also includes specific questions addressing regression of skills, which are not included in the TASI total score. On average, the TASI interview takes about 40 min to administer and score.

Diagnostic procedures

Diagnostic evaluations were conducted by a clinical team composed of a licensed psychologist or developmental-behavioral pediatrician, all of whom were autism specialists, and a graduate student or other trainee. Final diagnosis was a DSM-5 clinical best estimate (CBE) assigned by the clinical team; CBE has long been regarded as the gold standard in diagnosing ASD (Klin et al., 2000; Ventola et al., 2006). Diagnosis was based on data from the Mullen Scales of Early Learning (MSEL), the ADOS-2, the Vineland Adaptive Behavior Scales–2 (VABS-2), the Child Behavior Checklist (CBCL), demographic and history forms, and the TASI pilot interview. Each DSM-5 symptom was determined to be either present or absent based on these data, and only children who met full DSM-5 criteria were given an ASD diagnosis. The TASI scores were not used in CBE decision-making and clinicians knew that TASI was in development, but were free to use caregiver answers as part of their clinical judgment.

Children not meeting criteria for an ASD diagnosis were evaluated for other neurodevelopmental disorders, including Language Disorder, Global Developmental Delay, and Developmental Coordination Disorder. All children with these diagnoses, and all those who demonstrated sub-threshold ASD symptoms that did not meet DSM-5 criteria, were included in the other DD group. The third group was composed of typically developing or no-diagnosis (TD/ND) group; these were children who demonstrated some mild delays but did not meet full diagnostic criteria for any disorder as well as children without any apparent delays.

Legal guardians provided informed consent for themselves and their child. Following the evaluation, results and recommendations were discussed with caregivers, and a full written report followed. All procedures were approved by the Institutional Review Boards at the University of Connecticut, Drexel University, and Georgia State University.

Diagnostic measures

The MSEL is a developmental assessment designed for use with children age 0–68 months (Mullen, 1995). Visual Reception, Fine Motor, Receptive Language, and Expressive Language domains were assessed. The MSEL provides T-scores for each domain as well as a composite score (Early Learning Composite, ELC).

The ADOS-2 (Lord et al., 2012) is the most widely used observational measure of symptoms of ASD. It is a play-based measure designed to elicit social and communicative behaviors and provides opportunities for children to engage in RRBs. Calibrated severity scores (CSS; Esler et al., 2015; Gotham et al., 2009) for the Toddler Module and Module 1 were calculated.

Psychometric evaluation of the TASI

Inter-rater reliability

A third sample was used to calculate inter-rater reliability of the TASI interview. This sample was composed of 15 evaluated children aged 12–36 months, not in Samples 1 or 2. These children were evaluated after the collection of Sample 2. They were included sequentially, without reference to diagnosis or specific age; interviews in which the audio was not clear enough to understand the clinician and parent were excluded. The mean age (20 months) was similar to the two main samples (19 and 19.5 months). Approximately half of the sample were children with ASD (eight children, 53%), whose mean ELC (62) and CSS (8.4) were similar to the ASD groups in the two main samples (ELC of 64 and 65 and CSS of 7.9 and 7.9). Two clinicians (K.L.C. and D.A.F.) independently reviewed videotapes of TASI administrations and independently scored each item on the TASI. Hyper-sensitivity and hypo-sensitivity were scored together as a single category indicating atypical sensory response. Cohen’s κ and percent agreement are shown for each individual item and for ASD likelihood classification (see section “Validity” below) (Table 3).

Validity

Sensitivity (ability to detect ASD) was operationalized as the percent of children with a CBE diagnosis of ASD who were identified by the TASI cutoff. Specificity was operationalized as the proportion of children without ASD who were correctly identified by the cutoff as not having ASD (Dow et al., 2017). Positive predictive value (PPV; the confidence that a positive score indicates the disorder) was defined as the percent of children who met the TASI cutoff for an ASD diagnosis and independently received a CBE of ASD among all children who met the TASI cutoff. Negative predictive value (NPV; the confidence that a negative score indicates the disorder is not present) was defined as the percent of children who did not meet the TASI cutoff and who did not have a CBE diagnosis of ASD among all children below the TASI cutoff. Validity statistics were first calculated for the initial 204 children (Sample 1; Tables 4 and 5) based on the receiver operating characteristic (ROC) curve for total TASI score (Figure 1); the cutoff and other psychometric properties were then verified in a cross-validation sample (Sample 2; Tables 4 and 5).

Figure 1.

Receiver operating characteristic curve for Sample 1.

Validity was also calculated for subsamples of children 23 months and younger and 24–36 months of age (Table 6), for subsamples of boys and girls, and for the TASI’s ability to differentiate ASD from other DDs (Table 7). Finally, agreement between the ADOS-2 and TASI scores were examined in children with ASD.

Community involvement

Participating children and families were not involved in study design, implementation, analysis, or interpretation.

Results

Sample

In Sample 1 (n = 204), 53 children were given a CBE diagnosis of DSM-5 ASD. In Sample 2, the cross-validation sample, (n = 90), 19 children received an ASD diagnosis (Table 1). Proportion of diagnostic classification (ASD vs DD vs TD/ND) was different in the two samples, with a greater proportion of DD children in Sample 2.

Samples 1 and 2 differed in racial/ethnic makeup, with a greater proportion of Black/African American children in the cross-validation sample (Table 1). There were no significant differences in annual income between Samples 1 and 2.

Samples 1 and 2 were largely comparable in ADOS-2 and MSEL scores (Table 2). The ASD groups in Samples 1 and 2 did not differ on any ADOS-2 or MSEL score, whereas the other two diagnostic groups (DD and TD/ND) had a few small differences between Samples 1 and 2; the only difference that remained significant after Bonferroni’s correction was the finding of more RRBs in the DD Sample 2 group than Sample 1.

Table 2.

Comparison of participant samples on assessment results.

	Sample 1 (development; n = 204)	Sample 2 (cross-validation; n = 90)
ASD group			t(70)	p ^a
MSEL ELC^b	64.00 (16.10)	65.68 (16.65)	−0.388	0.699
Visual reception^c	32.28 (12.37)	33.16 (12.37)	−0.265	0.792
Fine motor^c	34.19 (13.46)	37.16 (11.90)	−0.849	0.399
Receptive language^c	25.91 (10.71)	26.37 (11.71)	−0.158	0.875
Expressive language^c	27.45 (9.23)	30.79 (9.53)	−1.340	0.184
ADOS CSS	8.02 (1.89)	8.05 (1.55)	−0.070	0.944
SA domain CSS	8.02 (2.05)	7.84 (1.39)	0.347	0.729
RRB domain CSS	7.34 (1.94)	7.16 (1.54)	0.368	0.714
TASI total	13.35 (6.00)	13.68 (6.17)	−0.284	0.777
DD group			t(129)	p
MSEL ELC^b	71.43 (11.51)	68.76 (11.21)	1.304	0.195
Visual reception^c	38.93 (11.84)	37.58 (9.57)	0.713	0.499
Fine motor^c	40.44 (11.50)	36.78 (11.63)	1.764	0.080
Receptive language^c	29.12 (8.78)	28.24 (7.78)	0.584	0.560
Expressive language^c	29.69 (6.95)	29.26 (7.76)	0.330	0.742
ADOS CSS	2.96 (1.71)	2.90 (1.71)	0.201	0.841
SA domain CSS	3.53 (2.06)	3.28 (1.80)	0.693	0.489
RRB domain CSS	1.99 (1.88)	3.22 (2.42)	−3.071	0.003
TASI total	5.23 (3.89)	6.46 (4.12)	−2.679	0.008
TD/ND group			t(89)	p
MSEL ELC^b	93.96 (13.33)	97.29 (12.20)	−1.023	0.309
Visual reception^c	48.76 (9.52)	49.71 (10.81)	−0.392	0.696
Fine motor^c	49.13 (8.59)	53.10 (8.43)	−1.864	0.066
Receptive language^c	45.58 (12.44)	45.43 (9.84)	0.051	0.959
Expressive language^c	43.73 (7.85)	46.24 (7.95)	−1.282	0.203
ADOS CSS	1.74 (0.91)	2.29 (0.96)	−2.366	0.020
SA domain CSS	2.01 (1.12)	2.38 (1.12)	−1.314	0.192
RRB domain CSS	1.91 (1.69)	3.05 (2.22)	−2.155	0.040
TASI total	3.44 (2.85)	3.00 (2.29)	0.806	0.423

ASD: autism spectrum disorder; MSEL: Mullen Scales of Early Learning; ELC: Early Learning Composite; ADOS-2: Autism Diagnostic Observation Schedule–Second Edition; SA: social affect; RRB: restricted repetitive behavior; TASI: Toddler Autism Symptom Inventory; CSS: calibrated severity scores; DD: developmental disorder; TD/ND: typically developing or no-diagnosis.

Bonferroni’s corrected p-value = 0.00556.

MSEL ELC standard score (M = 100, SD = 15).

MSEL domain T-score (M = 50, SD = 10).

Reliability

Cohen’s κ was calculated to compare the two independent evaluators’ TASI scores for each item and for the cutoff score that determined ASD likelihood classification (Table 3).

Table 3.

Inter-rater reliability for TASI diagnostic cutoff score and items (reliability sample).

DSM-5 symptom mapping	TASI item	Cohen’s κ (95% CI)	Cohen’s κ classification^a	Percent agreement
	Diagnostic cutoff score	1.000	Very good	100
	Language delay	0.632 (−0.015 to 1.279)	Good	92.86
A1	1 Smile back to caregiver	0.762 (0.325 to 1.199)	Good	93.33
A1	2 Smile back to other	1.000	Very good	100
A1	3 Response to happy	1.000	Very good	100
A1	4 Response to hurt adult^b
A1	5 Response to name	0.667 (0.261 to 1.073)	Good	86.67
A1	6 Approach to play	1.000	Very good	100
A1	7 Show done/doing	1.000	Very good	100
A1	8 Show interest	0.865 (0.612 to 1.118)	Very good	93.33
A1	9 Back-and-forth talking/babbling	0.625 (0.155 to 1.095)	Good	83.33
A1	10 Social games	0.831 (0.517 to 1.145)	Very good	92.31
A2	11 Look when requesting	0.759 (0.316 to 1.202)	Good	92.86
A2	12 Look back when showing	0.800 (0.435 to 1.165)	Good	90.00
A2	13 Look during physical play	1.000	Very good	100
A2	14 Look during social play	0.629 (−0.020 to 1.278)	Good	92.31
A2	15 Gestures^b
A2	16 Gesture frequency	1.000	Very good	100
A2	17 Follow point	1.000	Very good	100
A3	18 Interest in children	1.000	Very good	100
A3	19 Usual play	0.851 (0.573 to 1.129)	Very good	92.86
A3	20 Approaching child	0.865 (0.612 to 1.118)	Very good	93.33
A3	21 Spontaneous imitation	1.000	Very good	100%
A3	22 Pretend play	1.000	Very good	100
A3	23 Pretend play with others	0.750 (0.295 to 1.205)	Good	91.67
B1	24 Inflexible play	0.851 (0.573 to 1.129)	Very good	92.86
B1	25 Flapping	0.629 (−0.020 to 1.278)	Good	92.31
B1	26 Repetitive hand movements	0.851 (0.573 to 1.129)	Very good	92.86
B1	27 Rocking	0.837 (0.533 to 1.141)	Very good	92.86
B1	28 Unusual body movements	0.847 (0.563 to 1.131)	Very good	92.31
B1	29 Repeat vocalizations	0.621 (−0.036 to 1.278)	Good	90.91
B1	30 Immediate echolalia^c			100
B1	31 Delayed echolalia	1.000	Very good	100
B2	32 Routine change	0.750 (0.295 to 1.205)	Good	91.67
B2	33 Minor changes^c			100
B2	34 Minor, indirect changes^c			100
B2	35 Impose routines	1.000	Very good	100
B3	36 Strong interest	0.455 (0.016 to 0.894)	Moderate	73.33
B3	37 Atypical object	0.837 (0.533 to 1.141)	Very good	92.86
B4	Sensory seeking	0.851 (0.573 to 1.129)	Very good	92.86
B4	Hyper- and hypo-sensitivity	0.759 (0.316 to 1.202)	Good	92.86

TASI: Toddler Autism Symptom Inventory; CI: confidence interval.

Altman (1991).

Item not included in TASI total score.

κ not able to be calculated due to insufficient variability.

Inter-rater reliability of all TASI items was good or very good (κ >0.60; Altman, 1991), except for item 36: “Is there anything that your child is interested in that seems like all he or she wants to do?,” which had moderate reliability. At the level of ASD likelihood classification, discussed below, the raters achieved 100% agreement. Thus, Cohen’s κ was in the “very good” range (κ = 1.00).

Validity

A total score cutoff indicating likelihood of ASD was derived using ROC analyses. Using Sample 1, an ROC curve (Figure 1) indicated excellent discrimination between ASD and non-ASD groups (area under the curve (AUC) = 0.92; Table 4). An optimal cutoff that maximized sensitivity while maintaining specificity above 0.80 was determined to be a score of 7 or more out of a total possible score of 40 (Table 4). A second ROC curve (Figure 2) calculated for Sample 2 showed a similarly high AUC (0.89; Table 4). Validity for alternate cut-off points, which may be helpful for purposes in which an emphasis on higher sensitivity or specificity is required, is reported in Table 4.

Table 4.

Sensitivity and specificity of alternate cutoff points for ROC of Samples 1 and 2.

	Sample 1		Sample 2
AUC	0.916		0.890
Cutoff score	Sensitivity	Specificity	Sensitivity	Specificity
1 or more	98.11	6.62	100	2.82
2 or more	98.11	20.53	100	16.9
3 or more	96.23	33.11	100	23.94
4 or more	96.23	52.98	100	36.62
5 or more	96.23	66.23	100	47.89
6 or more	94.34	74.17	100	60.56
7 or more	88.68	81.46	89.47	67.61
8 or more	86.79	90.07	89.47	74.65
9 or more	81.13	92.05	78.95	81.69
10 or more	67.92	92.72	63.16	83.1
11 or more	60.38	92.72	57.89	88.73
12 or more	58.49	94.04	57.89	92.96
13 or more	52.83	96.03	52.63	95.77
14 or more	45.28	96.69	47.37	95.77
15 or more	35.85	98.01	36.84	97.18
16 or more	32.08	99.34	31.58	97.18
17 or more	24.53	99.34	26.32	97.18
18 or more	20.75	100	15.79	98.59
19 or more	18.87	100	15.79	0

AUC: area under the curve.

Figure 2.

Receiver operating characteristic curve for Sample 2.

Validity is summarized in Table 5. In Sample 1, with the cutoff of 7 or more, sensitivity was 88.68 and specificity was 81.46. PPV for ASD was 62.67 and NPV was 95.35. The sensitivity of Sample 2 was slightly higher (89.47), with a concomitant decrease in specificity (67.61). PPV for ASD in this sample was 42.50 and NPV was 96.00.

Table 5.

TASI sensitivity, specificity, PPV, and NPV for Sample 1 (development) and Sample 2 (cross-validation) using a cutoff of 7.

	Sensitivity (95% CI)	Specificity (95% CI)	PPV_ASD (95% CI)	PPV_anyDD (95% CI)	NPV (95% CI)
Sample 1 (n = 204)	88.68 (76.97 to 95.73)	81.46 (74.33 to 87.31)	62.67 (54.24 to 70.39)	86.67 (78.11 to 92.21)	95.35 (90.58 to 97.76)
Sample 2 (n = 90)	89.47 (66.86 to 98.70)	67.61 (55.45 to 78.24)	42.50 (33.80 to 51.69)	95.00 (83.33 to 98.63)	96.00 (86.50 to 98.90)

TASI: Toddler Autism Symptom Inventory; CI: confidence interval; PPV: positive predictive value; NPV: negative predictive value.

As discussed in Pandey et al. (2008), false positives are primarily undesirable because they can create unnecessary alarm. However, if the false positive cases are primarily children with other developmental disorders rather than TD children, this concern is somewhat obviated. Therefore, we also calculated PPV_anyDD where false positives in this analysis were defined as children in the TD/ND group. These values were calculated for both Sample 1 (86.67) and Sample 2 (95.00; Table 5), indicating that the non-ASD children exceeding the cutoff score usually had another developmental disorder.

In order to determine whether the TASI’s psychometrics are good in children under 24 months of age, for whom other measures often fall short, we repeated several analyses in this subsample. Our sample was young; 89% of our sample was under 24 months of age (32% were under 18 months of age). Validity statistics were calculated for children age 12–23 months old and compared to those for children age 24–36 months old (Table 6). Sensitivity and specificity of the younger group were found to be quite similar to those of the older group; however, PPV was lower in the younger group. The AUC value for children under 24 months of age was found to be 0.89 compared to 0.97 in children 24–36 months of age.

Table 6.

Sensitivity, specificity, PPV, and NPV for subsamples of children younger and older than 24 months of age using a cutoff of 7.

	Sensitivity (95% CI)	Specificity (95% CI)	PPV_ASD (95% CI)	PPV_anyDD (95% CI)	NPV (95% CI)
Children <24 months (n = 261)	88.89 (77.37 to 95.81)	76.81 (70.46 to 82.38)	50.00 (43.41 to 56.59)	88.54 (81.36 to 93.19)	96.36 (92.55 to 98.26)
Children ⩾24 months (n = 33)	88.89 (65.29 to 98.62)	80.00 (51.91 to 95.67)	84.21 (65.67 to 93.70)	94.74 (73.90 to 99.13)	85.71 (61.32 to 95.78)

CI: confidence interval; PPV: positive predictive value; NPV: negative predictive value.

TASI psychometrics were also calculated separately for girls and boys. Within the whole sample, the AUC values for girls (AUC = 0.95, standard error (SE) = 0.021) and boys (AUC = 0.89, SE = 0.027) were comparable.

We also sought to demonstrate that the TASI effectively differentiates children with ASD from those with other DDs and that the psychometrics above were not simply driven by the ability to differentiate ASD from TD. Not surprisingly, AUC and specificity were found to be slightly lower for discriminating ASD from other DD’s than for discriminating ASD from non-ASD (above), but remained satisfactory (AUC >0.85, Table 7).

Table 7.

Sensitivity and specificity of alternate cutoff points for ASD versus DD ROC of Samples 1 and 2.

	Sample 1		Sample 2
AUC	0.900		0.853
Cutoff score	Sensitivity	Specificity	Sensitivity	Specificity
1 or more	98.1	7.4	100	2
2 or more	98.1	13.6	100	6
3 or more	96.2	24.7	100	14
4 or more	96.2	46.9	100	26
5 or more	96.2	63	100	36
6 or more	94.3	70.4	100	50
7 or more	88.7	77.8	89.47	58
8 or more	86.8	86.4	89.47	66
9 or more	81.1	88.9	78.95	76
10 or more	67.9	90.1	63.16	76
11 or more	60.4	90.1	57.89	84
12 or more	58.5	92.6	57.89	90
13 or more	52.8	95.1	52.63	94
14 or more	45.3	95.1	47.37	94
15 or more	35.8	97.5	36.84	96
16 or more	32.1	98.8	31.58	96
17 or more	24.5	98.8	26.32	96
18 or more	20.8	100	15.79	98
19 or more	18.9	100	15.79	100

AUC: area under the curve; DD: developmental disorder; ROC: receiver operating characteristic.

Finally, agreement between the ADOS-2 and TASI was evaluated. Of the 72 children diagnosed with ASD as per CBE, only one scored in the non-ASD range on the ADOS-2; this child had a TASI score that fell in the elevated likelihood range. Six of the 72 children with ASD did not meet the TASI cutoff score; all of these children were identified by the ADOS-2.

Discussion

The TASI is a relatively brief, easily scored parent interview addressing the symptoms and behaviors often observed in toddlers with ASD. A cutoff score was developed and tested across two samples. The TASI form itself offers clinicians an opportunity to identify not only whether the child meets the cutoff for elevated ASD likelihood but also whether each operationalized DSM-5 criterion is present or absent based on caregiver report. We evaluated the validity of the cutoff against a DSM-5 CBE diagnosis in children aged 12–36 months (32% of whom were under 18 months of age), replicated these psychometrics in a cross-validation sample, and demonstrated inter-rater reliability of interview items and TASI ASD likelihood classification.

The TASI fills a gap in available interviews designed for use with very young children with suspected ASD, and has the advantage of operationalizing DSM-5 symptoms by providing specific behaviors and examples relevant to toddlers. This operationalization provides support in identifying specific language and/or play delays, which are clear challenges in early ASD diagnosis. The sample was referred based on positive results on standardized ASD screening or pediatric provider concern about ASD, making it comparable to clinically referred community samples. Children were included regardless of developmental level and represented a sample younger than most other studies of caregiver interviews. Furthermore, the TASI is efficient relative to other ASD semi-structured interviews, taking an average of 40 min to complete. The TASI’s ease of administration and immediate, simple scoring should make it a useful addition to available tools for assessment of ASD in clinical and research contexts.

Reliability

Inter-rater reliability of TASI items was strong; all Cohen’s κ values were “good” or “very good” (κ >0.60; Altman, 1991), except for one item (#36) on intense interest in a certain activity (Is there anything that your child is interested in that seems like all he or she wants to do?). Low reliability of this question may indicate that assessing whether the intensity of a young child’s interests is age-appropriate based exclusively on caregiver report is challenging. Brief intense interests in activities (“read it again!” and “read it again!”) can be developmentally appropriate in children this age (Evans et al., 1997), and it is difficult to determine whether the intensity of a specific interest exceeds typical behavior using only caregiver-report information. At the level of TASI ASD likelihood classification, inter-rater reliability was found to be 100%. Thus, item-level and ASD likelihood classification reliability of the TASI is comparable to other widely used diagnostic measures, including the ADI-R (Chakrabarti & Fombonne, 2001; Lord et al., 1994) and ADOS-2 (Lord et al., 2012; Zander et al., 2016), suggesting that it has sufficient reliability for eliciting caregiver report of symptoms in this population. A further strength of the TASI is that strong reliability was possible without any intensive training; raters used a scoring manual that includes specific examples and scoring guidance.

Validity

The TASI appears to achieve our goal of adequate sensitivity in a young sample, but due to the overlap in symptoms commonly observed in ASD and other DDs in young children, it also produces some false positives. The TASI’s validity (Table 4) is comparable to other diagnostic measures with mixed age samples (Charman & Gotham, 2013; Kim & Lord, 2012b); however, direct comparisons with other measures such as the ADI-R, whose algorithms also differ based on age and verbal abilities (Kim et al., 2013), are challenging due to these sample differences. Using a cutoff of 7, sensitivity ranged from 88% to 89%, and specificity ranged from 67% to 81%. Sample differences may have contributed to the observed differences in specificity between groups. The TASI’s normative sample includes a larger sample of very young children (children <24 months) and more children with developmental delays than normative samples for other measures (Kim & Lord, 2012b). In spite of the difficulty associated with identifying symptoms of ASD in this young age group, including the confounding effects of adaptive skills and developmental level (Zwaigenbaum et al., 2016), as well as the high frequency of language concerns in children with other developmental disorders (Coonrod & Stone, 2004; Richards et al., 2016), validity remained good even in the subsample of children under 24 months (Table 6), with relatively similar sensitivity and specificity, but lower PPV in the younger sample. PPV of the TASI cutoff for ASD indicates that 43%–63% (in the two samples) of children who meet the cutoff get a CBE diagnosis of ASD. Low PPV values are common in measures of symptoms in very young children (Kleinman, Robins, et al., 2008), since many ASD symptoms in young children overlap with the clinical presentation of other developmental disorders or delays. However, PPV for having any diagnosable developmental disorder, including ASD, was found to be 87%–95%. Therefore, there were very few children (12 total across both samples, n = 294) showing typical development or non-diagnosable mild delays who were inaccurately classified by the TASI cutoff score. Detecting other neurodevelopmental disorders is still a misclassification for an instrument designed to detect autism and should be taken into account as such by users. However, most false positives did warrant another diagnosis and appropriate intervention referral.

NPV, or the probability that a child whose score did not meet the ASD threshold on the TASI will not have ASD, is important when considering that a non-ASD diagnosis may limit access to early intervention. Failing to identify a child with ASD and thus disqualifying that child from accessing early intensive services is a key concern. The TASI was found to have high NPVs, such that fewer than 5% of children who score as non-ASD will in fact receive a CBE diagnosis of ASD, suggesting that the TASI can appropriately rule out ASD. This NPV is higher than that of other commonly used instruments (Charman & Gotham, 2013), perhaps due to the TASI’s focus on symptoms most common to a narrow age range of very young children.

Results indicate that even when differentiating children with ASD from those with other delays (DD), the TASI performs well (Table 7). The TASI was also found to work well in both boys and girls.

Finally, research or clinical evaluations may more strongly favor either detecting all children presenting with symptoms even at a cost to specificity, or classifying only the most certain cases by maximizing specificity (e.g., for genetic studies). Validity for alternate cutoff scores is reported in Table 4. For clinical or research purposes that seek to maximize sensitivity, a cutoff score of 5 or more yields sensitivity of 96%. To prioritize specificity, a cutoff score of 9 or more yields specificity of 92%. In settings in which specificity is more important, a cutoff score of 8 or 9 would likely be appropriate. The identified cutoff of 7 was determined based on the psychometrics observed in Sample 1; differences were noted in Sample 2, and further work with other samples should continue to elucidate the most appropriate cutoff score.

It has been noted that diagnostic accuracy with young children improves when observation is supplemented by parent report (Kim & Lord, 2012a; Sacrey et al., 2018). Our data indicate that neither the TASI nor the ADOS-2 catches all children determined to have ASD, although the ADOS-2 in the hands of expert diagnosticians only missed one child, who was accurately identified by the TASI. In other situations, children attending an evaluation may be shy, tired, ill, and so on; these factors may result in an inaccurate sample of the child’s typical behavior. Therefore, parent report is essential. In situations where the opportunity for direct observation is limited, a caregiver interview that can be conducted via telehealth may enable clinicians to still conduct ASD evaluations and may ensure continued access to intervention services.

In addition to a clear cutoff score, the TASI also provides explicit operationalization of the DSM-5 symptoms as they manifest in toddlers. The TASI’s accompanying scoring sheet enables a clinician to clearly visualize which DSM-5 symptoms are present based on caregiver report.

Limitations

Several limitations should be noted. DSM-5 diagnostic criteria are written to capture individuals of all ages; there may be different perspectives within the field on how to best operationalize individual criteria for children under 3 years of age. Very few differences were noted between the TASI and other measures (Evers et al., 2020), suggesting that our group’s consensus in mapping TASI items to diagnostic criteria was very similar to others’ assignments.

Another limitation is sample ascertainment. Our sample was gathered by screening positive on an ASD screener, or by provider concern during a primary care check-up. Different types of samples (younger siblings of affected children, those with other risk factors such as prematurity) may yield different results. The TASI includes questions about regression that are not included in the calculation of TASI total score; however, clinicians may interpret a positive response to regression questions as a cue to complete a more comprehensive interview about regression (Ozonoff & Iosif, 2019).

In addition, Sample 1 (development), Sample 2 (cross-validation), and Sample 3 (inter-rater reliability) were not collected during the same time period, or randomly assigned to development and cross-validation groups, or matched for sample characteristics. Samples 1 and 2 were generally very similar in characteristics (Table 2); there was only one significant difference between DD groups in ADOS RRB CSS, which we assume is a random finding. Instead, data collected first were used to develop the cutoff score, while data entered later were assigned to the cross-validation sample. This resulted in small demographic differences between the samples, as well as time of data collection as a confound. Due to the young age of the children in our sample, children and parents were not separated; although this may have resulted in some bias in scoring the TASI, the responses elicited from parents were not changed by the interviewer based on direct observation. Future work should validate the TASI in situations where the parent is interviewed before the clinician observes the child.

Finally, our current sample is not reflective of the current US census or of the specific sites. It is essential that new tools such as the TASI are developed with data from a diverse sample in terms of race, ethnicity, and socio-economic factors, and its performance within specific groups assessed. The TASI should be evaluated in other samples in order to further validate or revise the recommended cutoff score.

Directions for future work will include examining the TASI’s performance in various racial/ethnic groups, and with larger samples. It will also be important to examine the TASI’s performance of children ascertained in different ways, such as using different screeners, or in specific high-risk groups (e.g. baby sibs of affected children and premature children). Validity should be further studied by comparing TASI results with those of other diagnostic interviews such as the ADI-R, when both are given experimentally to a common sample. Additional psychometrics of the TASI including item loadings in factor analysis and internal consistency will also be examined.

Conclusion

The TASI is a novel caregiver-report interview designed to identify the presence and absence of ASD symptoms in children 12–36 months old. This interview aids in differentiating ASD from other developmental disorders and from typical development. The TASI is brief, semi-structured, and offers a clear operationalization of DSM-5 diagnostic criteria for young children. The TASI allows for both the calculation of a total sum to evaluate elevated likelihood of ASD, and a clear determination of whether each individual DSM-5 symptom is present or absent based on caregiver report.

The TASI cutoff score offers good psychometrics, suggesting it can be a useful way to assess elevated likelihood of ASD. The TASI shows good reliability and validity, and is an improvement over existing caregiver-report diagnostic measures in its ease of administration, clear scoring aligned with DSM-5, and brevity. Furthermore, false positives detected by the ASD likelihood cutoff score are very likely to have a diagnosable neurodevelopmental disorder, and thus require referral to appropriate intervention services.

Combining information from caregiver report instruments and direct observation improves accuracy of these tests to identify those with a clinical best estimate diagnosis of ASD (Risi et al., 2006). The data presented here indicate that while neither the TASI nor the ADOS-2 alone accurately identified every child with ASD, their use in conjunction reduced the number of misses to zero. The TASI is only one source of information regarding a child’s functioning, and thus should always be used in conjunction with other diagnostic tools including a complete developmental history, developmental testing, and structured behavioral observations. The TASI, when used in conjunction with other measures, will allow for early and accurate diagnoses, thus permitting children to access early intervention services.

Footnotes

Acknowledgements

The authors are deeply grateful to all of the children, families, pediatric providers, and colleagues who have been a part of our work. The authors also thank Lauren B. Adamson, PhD, Sarah Dufek, PhD, Sherira Fernandes, PhD, Elizabeth Karp, PhD, and Aubyn Stahmer, PhD, for their thoughtful contributions to the TASI. Data used in the preparation of this manuscript were submitted to the National Institute of Mental Health (NIMH) Data Archive (NDA). NDA is a collaborative informatics system created by the National Institutes of Health to provide a national resource to support and accelerate research in mental health (data set identifier: DOI: 10.15154/1519083). This manuscript reflects the views of the authors and may not reflect the opinions or views of the NIH.

Declaration of conflicting interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Drs D.L.R., M.L.B., and D.A.F. are co-owners of M-CHAT LLC, which licenses use of the M-CHAT in electronic products. Dr D.L.R. sits on the advisory board of Quadrant Biosciences Inc.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by funds from the Eunice Kennedy Shriver National Institute of Child Health and Human Development (R01HD039961) and the National Institute of Mental Health (R01MH115715).

ORCID iD

Kirsty L Coulter

References

Altman

D. G.

(1991). Practical statistics for medical research. Chapman and Hall.

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.).

Baranek

G. T.

Watson

Crais

E. R.

Turner-Brown

L. M.

Reznick

(2014). The First Years Inventory – Lite Version 3.1b (FYI-Lite v3.1b). University of North Carolina at Chapel Hill.

Barton

M. L.

Robins

D. L.

Jashar

Brennan

Fein

(2013). Sensitivity and specificity of proposed DSM-5 criteria for autism spectrum disorder in toddlers. Journal of Autism and Developmental Disorders, 43(5), 1184–1195. https://doi.org/10.1007/s10803-013-1817-8

Chakrabarti

Fombonne

(2001). Pervasive developmental disorders in preschool children. Journal of the American Medical Association, 285(24), 3093–3099.

Charman

Gotham

(2013). Measurement issues: Screening and diagnostic instruments for autism spectrum disorders—Lessons from research and practise. Child and Adolescent Mental Health, 18(1), 52–63. https://doi.org/10.1111/j.1475-3588.2012.00664.x

Chawarska

Klin

Paul

Volkmar

(2007). Autism spectrum disorder in the second year: Stability and change in syndrome expression. Journal of Child Psychology and Psychiatry and Allied Disciplines, 48(2), 128–138. https://doi.org/10.1111/j.1469-7610.2006.01685.x

Chlebowski

Robins

D. L.

Barton

M. L.

Fein

(2013). Large-scale use of the modified checklist for autism in low-risk toddlers. Pediatrics, 131(4), e1121–e1127. https://doi.org/10.1542/peds.2012-1525

Coonrod

E. E.

Stone

W. L.

(2004). Early concerns of parents of children with autistic and nonautistic disorders. Infants and Young Children, 17(3), 258–268. https://doi.org/10.1097/00001163-200407000-00007

10.

Dow

Guthrie

Stronach

S. T.

Wetherby

A. M.

(2017). Psychometric analysis of the systematic observation of red flags for autism spectrum disorder in toddlers. Autism, 21(3), 301–309. https://doi.org/10.1177/1362361316636760

11.

Esler

A. N.

Bal

V. H.

Guthrie

Wetherby

Weismer

S. E.

Lord

(2015). The autism diagnostic observation schedule, toddler module: Standardized severity scores. Journal of Autism and Developmental Disorders, 45(9), 2704–2720. https://doi.org/10.1007/s10803-015-2432-7

12.

Evans

D. W.

Leckman

J. F.

Carter

Reznick

J. S.

Henshaw

King

R. A.

Pauls

(1997). Ritual, habit, and perfectionism: The prevalence and development of compulsive-like behavior in normal young children. Child Development, 68(1), 58–68.

13.

Evers

Maljaars

Carrington

S. J.

Carter

A. S.

Happé

Steyaert

Noens

(2020). How well are DSM-5 diagnostic criteria for ASD represented in standardized diagnostic instruments? European Child and Adolescent Psychiatry, 30, 75–87. https://doi.org/10.1007/s00787-020-01481-z

14.

Gotham

Pickles

Lord

(2009). Standardizing ADOS scores for a measure of severity in autism spectrum disorders. Journal of Autism and Developmental Disorders, 39(5), 693–705. https://doi.org/10.1007/s10803-008-0674-3

15.

Harrop

McConachie

Emsley

Leadbitter

Green

(2014). Restricted and repetitive behaviors in autism spectrum disorders and typical development: Cross-sectional and longitudinal comparisons. Journal of Autism and Developmental Disorders, 44(5), 1207–1219. https://doi.org/10.1007/s10803-013-1986-5

16.

Herlihy

L. E.

Brooks

Dumont-Mathieu

Barton

M. L.

Fein

Chen

C. M.

Robins

D. L.

(2014). Standardized screening facilitates timely diagnosis of autism spectrum disorders in a diverse sample of low-risk toddlers. Journal of Developmental and Behavioral Pediatrics, 35(2), 85–92. https://doi.org/10.1097/DBP.0000000000000014

17.

Hinnebusch

A. J.

Miller

L. E.

Fein

D. A.

(2017). Autism spectrum disorders and low mental age: Diagnostic stability and developmental outcomes in early childhood. Journal of Autism and Developmental Disorders, 47, 3967–3982. https://doi.org/10.1007/s10803-017-3278-y

18.

Jones

Klin

(2013). Attention to eyes is present but in decline in 2–6-month-old infants later diagnosed with autism. Nature, 504(7480), 427–431. https://doi.org/10.1038/nature12715

19.

Kim

S. H.

Lord

(2012a). Combining information from multiple sources for the diagnosis of autism spectrum disorders for toddlers and young preschoolers from 12 to 47 months of age. Journal of Child Psychology and Psychiatry and Allied Disciplines, 53(2), 143–151. https://doi.org/10.1111/j.1469-7610.2011.02458.x

20.

Kim

S. H.

Lord

(2012b). New autism diagnostic interview-revised algorithms for toddlers and young preschoolers from 12 to 47 months of age. Journal of Autism and Developmental Disorders, 42, 82–93. https://doi.org/10.1007/s10803-011-1213-1

21.

Kim

S. H.

Thurm

Shumway

Lord

(2013). Multisite study of new Autism Diagnostic Interview-Revised (ADI-R) algorithms for toddlers and young preschoolers. Journal of Autism and Developmental Disorders, 43(7), 1527–1538. https://doi.org/10.1007/s10803-012-1696-4

22.

Kleinman

J. M.

Robins

D. L.

Ventola

P. E.

Pandey

Boorstein

H. C.

Esser

E. L.

Fein

(2008). The modified checklist for autism in toddlers: A follow-up study investigating the early detection of autism spectrum disorders. Journal of Autism and Developmental Disorders, 38(5), 827–839. https://doi.org/10.1007/s10803-007-0450-9

23.

Kleinman

J. M.

Ventola

P. E.

Pandey

Verbalis

A. D.

Barton

Hodgson

Fein

(2008). Diagnostic stability in very young children with autism spectrum disorders. Journal of Autism and Developmental Disorders, 38(4), 606–615. https://doi.org/10.1007/s10803-007-0427-8

24.

Klin

Lang

Cicchetti

D. V.

Volkmar

F. R.

(2000). Brief report: Interrater reliability of clinical diagnosis and DSM-IV criteria for autistic disorder: Results of the DSM-IV autism field trial. Journal of Autism and Developmental Disorders, 30(2), 163–167. https://doi.org/10.1023/a:1005415823867

25.

Leonard

H. C.

Elsabbagh

Hill

E. L.

Baron-Cohen

Bedford

Bolton

Volein

(2014). Early and persistent motor difficulties in infants at-risk of developing autism spectrum disorder: A prospective study. European Journal of Developmental Psychology, 11, 18–35. https://doi.org/10.1080/17405629.2013.801626

26.

Lord

Rutter

Dilavore

P. C.

Risi

Gotham

Bishop

S. L.

(2012). Autism Diagnostic Observation Schedule, second edition (ADOS-2). Western Psychological Services.

27.

Lord

Rutter

Le Couteur

Storoschuk

Magill

Mulloy

Anderson

(1994). Autism Diagnostic Interview-Revised: A revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. Journal of Autism and Developmental Disorders, 24(5), 659–685.

28.

MacDonald

Parry-Cruwys

Dupere

Ahearn

(2014). Assessing progress and outcome of early intensive behavioral intervention for toddlers with autism. Research in Developmental Disabilities, 35(12), 3632–3644.

29.

Maenner

M. J.

Shaw

K. A.

Baio

Washington

Patrick

DiRienzo

Dietz

(2020). Prevalence of autism spectrum disorder among children aged 8 years: Autism and developmental disabilities monitoring network, 11 Sites, United States, 2016. Morbidity and Mortality Weekly Report Surveillance Summaries, 69, 1–23. https://doi.org/10.15585/mmwr.ss6706a1

30.

Matson

J. L.

Hattier

M. A.

Williams

L. W.

(2012). How does relaxing the algorithm for autism affect DSM-V prevalence rates? Journal of Autism and Developmental Disorders, 42(8), 1549–1556. https://doi.org/10.1007/s10803-012-1582-0

31.

Miller

L. E.

Perkins

K. A.

Dai

Y. G.

Fein

D. A.

(2017). Comparison of parent report and direct assessment of child skills in toddlers. Research in Autism Spectrum Disorders, 41, 57–65. https://doi.org/10.1016/j.rasd.2017.08.002

32.

Mullen

E. M.

(1995). Mullen scales of early learning manual. American Guidance Service.

33.

Nielsen

Dissanayake

(2004). Pretend play, mirror self-recognition and imitation: A longitudinal investigation through the second year. Infant Behavior and Development, 27(3), 342–365. https://doi.org/10.1016/j.infbeh.2003.12.006

34.

Osterling

Dawson

(1994). Early recognition of children with autism: A study of first birthday home videotapes. Journal of Autism and Developmental Disorders, 24(3), 247–257.

35.

Ozonoff

Iosif

A. M.

(2019). Changing conceptualizations of regression: What prospective studies reveal about the onset of autism spectrum disorder. Neuroscience and Biobehavioral Reviews, 100, 296–304. https://doi.org/10.1016/j.neubiorev.2019.03.012

36.

Ozonoff

Iosif

A.-M.

Baguio

Cook

I. C.

Hill

M. M.

Hutman

Young

G. S.

(2010). A prospective study of the emergence of early behavioral signs of autism. Journal of American Academy of Child and Adolescent Psychiatry, 49(3), 256–266.

37.

Ozonoff

Macari

Young

G. S.

Goldring

Thompson

Rogers

S. J.

(2008). Atypical object exploration at 12 months of age is associated with autism in a prospective sample. Autism, 12(5), 457–472. https://doi.org/10.1177/1362361308096402

38.

Pandey

Verbalis

Robins

D. L.

Boorstein

Klin

Babitz

. . . Fein

(2008). Screening for autism in older and younger toddlers with the Modified Checklist for Autism in Toddlers. Autism, 12(5), 513–535.

39.

Pierce

Gazestani

V. H.

Bacon

Barnes

C. C.

Cha

Nalabolu

Courchesne

(2019). Evaluation of the diagnostic stability of the early autism spectrum disorder phenotype in the general population starting at 12 months. Journal of the American Medical Association Pediatrics, 173(6), Article 578. http://archpedi.jamanetwork.com/article.aspx?doi=10.1001/jamapediatrics.2019.0624

40.

Randall

Egberts

K. J.

Samtani

Scholten

R. J. P. M.

Hooft

Livingstone

Williams

(2018). Diagnostic tests for autism spectrum disorder (ASD) in preschool children. Cochrane Database of Systematic Reviews, 2018(7), Article CD009044. https://doi.org/10.1002/14651858.CD009044.pub2

41.

Richards

Mossey

Robins

D. L.

(2016). Parents’ concerns as they relate to their child’s development and later diagnosis of autism spectrum disorder. Journal of Developmental and Behavioral Pediatrics, 37(7), 532–540. https://doi.org/10.1097/DBP.0000000000000339

42.

Risi

Lord

Gotham

Corsello

Chrysler

Szatmari

Pickles

(2006). Combining information from multiple sources in the diagnosis of autism spectrum disorders. Journal of American Academy of Child and Adolescent Psychiatry, 45(9), 1094–1103. https://doi.org/10.1097/01.chi.0000227880.42780.0e

43.

Robins

D. L.

Fein

D. A.

Barton

M. L.

(2009). The Modified Checklist for Autism in Toddlers, Revised, with Follow-Up (M-CHAT-R/F). Self-published www.mchatscreen.com.

44.

Robins

D. L.

Casagrande

Barton

Chen

C.-M. A.

Dumont-Mathieu

Fein

(2014). Validation of the Modified Checklist for autism in toddlers, revised with follow-up (M-CHAT-R/F). Pediatrics, 133(1), 37–45.

45.

Rogers

S. J.

Vismara

Wagner

A. L.

McCormick

Young

Ozonoff

(2014). Autism treatment in the first year of life: A pilot study of infant start, a parent-implemented intervention for symptomatic infants. Journal of Autism and Developmental Disorders, 44(12), 2981–2995. https://doi.org/10.1007/s10803-014-2202-y

46.

Rutter

LeCouteur

Lord

(2003). Autism Diagnostic Interview, Revised Los Angeles: Western Psychological Services.

47.

Sacrey

L.-A. R.

Zwaigenbaum

Bryson

Brian

Smith

I. M.

Roberts

Garon

(2018). Parent and clinician agreement regarding early behavioral signs in 12- and 18-month-old infants at-risk of autism spectrum disorder. Autism Research, 11(3), 539–547. https://doi.org/10.1002/aur.1920

48.

Sánchez-García

A. B.

Galindo-Villardón

Nieto-Librero

A. B.

Martín-Rodero

Robins

D. L.

(2019). Toddler screening for autism spectrum disorder: A meta-analysis of diagnostic accuracy. Journal of Autism and Developmental Disorders, 49(5), 1837–1852. https://doi.org/10.1007/s10803-018-03865-2

49.

Ventola

P. E.

Kleinman

Pandey

Barton

Allen

Green

Fein

(2006). Agreement among four diagnostic instruments for autism spectrum disorders in toddlers. Journal of Autism and Developmental Disorders, 36(7), 839–847. https://doi.org/10.1007/s10803-006-0128-8

50.

Wetherby

A. M.

Prizant

B. M.

(2002). Communication and symbolic behavior scales: Developmental profile. Paul H Brookes Publishing Co.

51.

Wetherby

A. M.

Brosnan-Maddox

Peace

Newton

(2008). Validation of the Infant-Toddler Checklist as a broadband screener for autism spectrum disorders from 9 to 24 months of age. Autism, 12(5), 487–511. https://doi.org/10.1177/1362361308094501

52.

Wolff

J. J.

Botteron

K. N.

Dager

S. R.

Elison

J. T.

Estes

A. M.

Piven

(2014). Longitudinal patterns of repetitive behavior in toddlers with autism. Journal of Child Psychology and Psychiatry and Allied Disciplines, 55(8), 945–953. https://doi.org/10.1111/jcpp.12207

53.

Zander

Willfors

Berggren

Choque-Olsson

Coco

Elmund

Bölte

(2016). The objectivity of the Autism Diagnostic Observation Schedule (ADOS) in naturalistic clinical settings. European Child and Adolescent Psychiatry, 25(7), 769–780. https://doi.org/10.1007/s00787-015-0793-2

54.

Zwaigenbaum

Bryson

Rogers

Roberts

Brian

Szatmari

(2005). Behavioral manifestations of autism in the first year of life. International Journal of Developmental Neuroscience, 23(2–3), 143–152. https://doi.org/10.1016/j.ijdevneu.2004.05.001

55.

Zwaigenbaum

Bryson

S. E.

Brian

Smith

I. M.

Roberts

Szatmari

Vaillancourt

(2016). Stability of diagnostic assessment for autism spectrum disorder between 18 and 36 months in a high-risk cohort. Autism Research, 9(7), 790–800. https://doi.org/10.1002/aur.1585