Development of a brief version of the Autism Detection in Early Childhood

Abstract

While autism spectrum disorder screening tools provide a useful resource for practitioners, the reality is they are underused. The justifications often provided include the time required for administration and the training involved. A brief tool with good psychometric properties that require minimal training is required. This study examined the development and the psychometric properties of a brief version of the Autism Detection in Early Childhood. The data showed the potential of the brief version of Autism Detection in Early Childhood for screening children age 12–36 months. Our dataset comprised 106 Diagnostic and Statistical Manual of Mental Disorders, 5th edition autism spectrum disorder, 86 non-typical development and 78 typical development participants age 12–36 months. Analyses comparing autism spectrum disorder and non-typical development groups supported the use of five critical items (i.e. response to name, social smiling, gaze switch, response to verbal command and use of gestures) to form the brief version Autism Detection in Early Childhood. The brief version of Autism Detection in Early Childhood’s optimal cutoff score of 4 had sensitivity of 0.81, specificity of 0.78, positive predictive value of 0.81 and negative predictive value of 0.78. However, the results would need to be viewed as preliminary given the nature of the study sample and the findings might not be generalisable to samples with higher levels of cognitive functioning.

Keywords

Autism Detection in Early Childhood autism spectrum disorder brief screening Diagnostic and Statistical Manual of Mental Disorders (5th edition)early identification

Research by the Centers for Disease Control and Prevention indicates that only about 18% of children who have autism spectrum disorder (ASD) are diagnosed by age 3 years (Centers for Disease Control and Prevention (CDC), 2012), although an updated large-scale study indicated that about 50% of their sample was diagnosed by age 3 years (Soke et al., 2017). An understanding of the presentation of ASD in infancy is a crucial first step to improve early detection of children who might be considered at risk of the disorder and in need of further assessment, intervention and services. Given early identification and intervention can dramatically improve outcomes for people with ASD (Dawson and Burner, 2011), there is a pressing need to identify children with ASD as early as practical (Reichow, 2012).

The American Academy of Pediatrics (AAP) recommended that all children should be screened for ASD at 18 and 24 months (Johnson et al., 2007). However, not all primary care providers are adhering to this recommendation. Reported rates of routine screening for ASD among paediatricians and family practice physicians range from 22% (Pierce et al., 2011) to 28% (Gillis, 2009) for visits between 12 and 24 months. This rate increases to 59% at 24-month visits (Arunyanart et al., 2012). Some of the reported perceived barriers include familiarity with or knowledge of ASD-specific screening tools, lack of specific ASD training and lack of time and resources (e.g. Gillis, 2009; Gura et al., 2011).

A study looking at general medical practitioners’ consultations in European countries found that the mean length of consultation for all consultations was 10.7 (standard deviation (SD) = 6.7) min (Deveugele et al., 2002). In Australia, it was found that the mean length of general medical practitioners’ consultations for the past decade was about 15–16 min, with the median being 13–14 min (Britt et al., 2013; Gravelle et al., 2016). Such findings suggest that medical practitioners would need a screening tool that can be administered quickly, thereby leaving them enough time to attend to parents’ concerns and to perform any physical/medical tests on their patients.

More time-efficient versions of some of the ASD screening tools have been developed, such as the Autism Spectrum Quotient–Children Version (AQ-Child for 4–11 years old; Auyeung et al., 2008) and the Quantitative Checklist for Autism in Toddlers (Q-CHAT for 18–24 months old; Allison et al., 2008). However, these time-efficient or ‘brief’ versions have been criticised because they are based on parental reports which are known to be less reliable than direct observation in toddlers (Barton et al., 2012). A recent study by Havdahl et al. (2017) found that parental report may overlook children whose parents are not concerned that their child may have ASD. Furthermore, these tools are not suitable for children below 18 months of age. Oner et al. (2014) developed a Three-Item Direct Observation Screen (TIDOS) that can be administered by trained paediatric professionals to identify ASD based on Diagnostic and Statistical Manual of Mental Disorders, 4th edition, text rev. (DSM-IV-TR), but it is also not suitable for children below 18 months. The Social Attention and Communication Study SACS; Barbaro and Dissanayake, 2010) is another observational ASD screening and surveillance tool for 12- to 24-month-old children that can be used by community nurses. Crais et al. (2014) reported that there are few ASD-specific tools available to screen infants below 18 months. Given parents report signs emerging from as young as 12 months (Stone et al., 2004), the need for a brief and direct observation screening tool should be viewed as a useful component of a comprehensive screening process which includes multiple sources of information such as from parent reports and clinicians’ observations.

Various screening instruments are available to help clinicians determine the presence of an ASD, ranging from parent checklists to structured interviews and observational tools. Observational screening tools may allow the clinician to experience the child’s social and communicative behaviours firsthand, which can inform clinical judgment and identify the child’s strengths and weaknesses (Volkmar et al., 2014). Some of the observational screening tools include the Screening Tool for Autism in Two-Year-Olds (STAT; Stone et al., 2004), the Autism Detection in Early Childhood (ADEC; Young, 2007) and Systematic Observation of Red Flags (SORF; Dow et al., 2017), all of which have been validated for early screening use. A broad overview of some of these observation screening tools is provided by Young and Nah (2016).

This study reports the development and psychometric properties of a brief version of an ASD-specific screening tool, the ADEC (Young, 2007). The ADEC was designed as a direct observational screening tool to identify young children between 12 and 36 months of age referred for developmental concerns and at risk of developing ASD (i.e. as a Level 2 screening tool, not a Level 1 population screening tool). Published psychometric evaluations of the ADEC have demonstrated that it is a reliable and valid screening tool (Hedley et al., 2015; Nah et al., 2014a, 2014b; Young and Nah, 2016).

Method

Participants

The dataset included 270 participants 12–36 months of age. In all, 197 participants were involved in Nah et al.’s (2014b) study which examined the validation of the ADEC full version. We included the 197 participants in our present dataset together with an additional 73 participants collected for this study to increase the sample size and to permit a thorough examination of the ADEC brief version. All 270 participants were recruited either (a) via general advertising in mass media, child-care centres, government and private developmental clinics where children were suspected of having communication and/or developmental delay and invited to participate in an autism screening university research study (N = 130) or (b) from participants in a university-based autism research centre in South Australia (N = 140). The ethnic background of the sample was predominantly Caucasian (95.8%). No other demographic data were available. Because these participants were not formally diagnosed at the time of screening, they had not commenced any formal intervention programmes.

Of the 270 participants, 95 children had autistic disorder (AD), 28 children had pervasive developmental disorder – not otherwise specified (PDD-NOS), 69 children had other developmental disorders based on DSM-IV-TR diagnosis and 78 were considered typically developing. This study used archival and also prospective data collection to evaluate Diagnostic and Statistical Manual of Mental Disorders, 5th edition (DSM-5; American Psychiatric Association (APA), 2013) criteria with children with DSM-IV-TR clinical diagnoses. All DSM-IV-TR clinical diagnoses had been independently confirmed by two independent practitioners who had been recognised by the state’s autism association. Informed consent was obtained from the participants’ legal guardians, and appropriate ethics approvals were obtained prior to conducting this study.

A best estimate clinical (BEC) DSM-5 ASD diagnosis was made on each participant by the first author using all available information and assessment results (excluding ADEC data). For 72% of cases that were diagnosed based on the DSM-IV-TR criteria but not DSM-5, DSM-5 BEC was done retrospectively. It should be noted that a small number of these participants (12%) were followed up 2 years later and diagnostic stability was reported in another study where all the ASD participants (100%) continued to retain the ASD diagnosis (Nah et al., 2014a). The first author had extensive training and experience in the assessment and diagnosis of ASD and is formally recognised by the region’s autism association to conduct ASD assessments. Based on BEC DSM-5 ASD diagnosis, participants were assigned to one of three groups: (a) ASD, (b) non-typical development (non-TD) which included language and developmental delay, hearing loss and learning difficulty and (c) TD. Sample characteristics (based on BEC DSM-5 ASD diagnosis) of the participants are presented in Table 1. There were significantly more males in the ASD than the non-TD group, χ²(1) = 6.16, p < 0.05.

Table 1.

Means, SDs and ranges for sample characteristics.

BEC DSM-5 diagnosis
Measure	ASD (n = 106; 84 males, 22 females)			Non-typ dev (n = 86; 54 males, 32 females)			Typ dev (n = 78; 40 males, 38 females)
Measure	Mean	SD	Range	Mean	SD	Range	Mean	SD	Range
Chronological age (months)	28.7	5.5	14–36	23.1	7.1	12–36	23.5	6.9	12–36
ADEC	17.4	6.2	4–29	6.4	4.4	0–23	2.0	2.1	0–8
ADOS Revised	14.8	5.5	6–23	6.5	4.5	1–18	–	–	–
ADOS-Toddler	15.2	5.3	7–24	5.3	3.8	1–9	–	–	–
ADI-R-T	16.9	5.1	6–27	8.2	4.5	1–15	–	–	–
Nonverbal DQ	64.1	17.9	36–104	81.6	18.3	18–111	–	–	–
VABC	64.3	9.0	47–94	75.5	8.9	53–89	–	–	–

Dashes indicate that data are not available.

BEC: best estimate clinical; DSM-5: Diagnostic and Statistical Manual of Mental Disorders, 5th edition; ASD: autism spectrum disorder; typ dev: typical development; SD: standard deviation; ADEC: Autism Detection in Early Childhood; ADOS Revised: Autism Diagnostic Observation Schedule–Revised total algorithm score; ADI-R-T: Autism Diagnostic Interview–Revised Toddler total algorithm score; nonverbal DQ: Mullen Scales of Early Learning or Bayley-III developmental quotient; VABC: Vineland Adaptive Behavior Composite standard score.

The inter-rater reliability data for DSM-5 ASD diagnosis for a subset of individuals (21%) were obtained via review of de-identified findings by another experienced psychologist trained in ASD assessments. This psychologist also had extensive training and experience in the assessment and diagnosis of ASD and is formally recognised by the region’s autism association to conduct ASD assessments. The second rater was blinded to the researcher’s DSM-5 BEC diagnosis. The inter-rater reliability for DSM-5 diagnosis of ASD between the researcher’s BEC diagnosis and the independent diagnosis was high (k = 0.96, p < 0.001).

Diagnostic evaluation procedures and measures

The procedures and materials used in this study have been previously described in detail (Nah et al., 2014b). To summarise, parents and healthcare professionals who were concerned that their child or client presented with a risk of developing ASD were assessed, where possible, with a battery of tests such as the Autism Diagnostic Observation Schedule (ADOS), Autism Diagnostic Interview–Revised (ADI-R), developmental (Mullen Scales of Early Learning (MSEL) or Bayley-III) and adaptive functioning (Vineland Adaptive Behavior Scale) assessments. The participants’ nonverbal Developmental Quotient (NVDQ) was calculated using the age equivalent scores from the visual reception and fine motor scales of the MSEL and divided by the child’s chronological age. There were two participants who were administered the Bayley-III instead of the MSEL. These two participants already had Bayley-III scores available and thus were not administered the MSEL. We included the data from these two participants to add to our sample. The ADEC was administered independently of the diagnostic assessment. The ADEC administration was independent of the diagnostic assessment, and the administrators were blind to the results of the diagnostic evaluation. Likewise, the diagnostic assessor who administered the ADOS and ADI-R was blind to the ADEC assessment result. Children who were, upon presentation and in the absence of parental concern, deemed by the clinician to be typically developing were only administered the ADEC and their intellectual/adaptive functioning was assumed to be within the average range. BEC DSM-5 diagnoses were made using all available information and assessment results to generate diagnoses independent of the ADEC. To guide the decision-making of the BEC DSM-5 diagnosis, we also relied on the supplementary tables provided by (Huerta et al., 2012) where items from the ADOS or/and the ADI-R are mapped onto the DSM-5 criteria. DSM-5 guidelines were then followed to determine whether each participant met or did not meet the DSM-5 criteria for ASD. Because the ADOS-Toddler (ADOS-T; Lord et al., 2012) module was released during the data collection phase of this study, some of the more recent participants (specifically those below 30 months old, N = 20) were administered the ADOS-T. Module 1 was used for the participants using the ADOS-Generic (Lord et al., 2000). The ADI-R Toddler research algorithm (Kim and Lord, 2012) was also used in our analyses.

ADEC

The ADEC is a 16-item observation checklist developed to identify ASD in young children between the ages of 12 and 36 months. Instructions for administering the ADEC and a training DVD are provided in the manual (Young, 2007). The ADEC can be administered in 10–15 min by an administrator with limited clinical training. The assessor interacts with the child with the aim of eliciting 16 developmentally appropriate behaviours. The specific behaviours that are observed during the administration of the ADEC are (a) response to name, (b) imitation, (c) ritualistic play, (d) joint attention and social referencing, (e) eye contact, (f) functional play, (g) pretend play, (h) reciprocity of smile, (i) reaction to common sounds, (j) gaze monitoring, (k) following verbal commands, (l) delayed language, (m) anticipation of social advances, (n) nestling, (o) use of gestures and (p) task switching. Each of this behaviour is elicited from discrete activities of the ADEC. Response scores for each item range from 0 (appropriate) to 2 (inappropriate), with a possible maximum score of 32. The ADEC has been found to be a reliable and valid screening tool (Hedley et al., 2015; Nah et al., 2014a, 2014b). The ADEC also had high sensitivity (1.0), specificity (0.89) and predictive values (positive predictive value (PPV) = 0.84, negative predictive value (NPV) = 1.0) when using a cutoff score of 11 in identifying young children referred for possible risk for ASD within the validation sample (Nah et al., 2014b). However, it was noted that the study sample consisted of children with ASD who had very low nonverbal DQ. Nevertheless, the authors included the nonverbal DQ as a covariate in their analyses. A factor analytic examination of the construct validity of the ADEC indicated a one-factor solution (social communication; Nah et al., 2014b) which was consistent with the DSM-5 ASD criterion of ‘persistent deficits in social communication and social interaction’ (APA, 2013). However, the factor analysis did not indicate another factor which might be consistent with the other ASD criterion of ‘restricted, repetitive patterns of behaviour, interests, or activities’. This is in contrast to the recent research (e.g. Dow et al., 2017) and factor analyses with the ADOS-T (Lord et al., 2012) that restricted, repetitive patterns of behaviour, interests or activities may be present in young children. The entire ADEC administration was used in the development of the brief version of ADEC (BADEC).

ADOS-Generic

The ADOS (Lord et al., 2000) is a semi-structured assessment of communication, social interaction and play or imaginative use of materials for individuals suspected of having ASD. Overall ratings are made at the end of the observation. These ratings can then be used to formulate a diagnosis through the use of a diagnostic algorithm. The ADOS has demonstrated strong psychometric properties, with inter-rater reliability measured by mean exact agreement shown to be more than 88% for ADOS Modules 1–4 (Lord et al., 2000). The revised algorithm was used in this study.

ADOS-T

The ADOS-T module (Lord et al., 2012) was developed to improve sensitivity and specificity of the ADOS as a diagnostic instrument for very young children with nonverbal mental ages below 16 months and it demonstrated excellent sensitivity and specificity in the validation study. The Toddler module follows the same structure as other modules of the ADOS, in which the examiner presents semi-structured and motivating activities for the child and observes the child’s responses as well as their attempts to maintain the interaction. Symptoms relevant to a diagnosis of ASD are scored from 0 to 3 on the ADOS-T, with higher numbers indicating more abnormality. Two diagnostic algorithms have been derived for the ADOS-T: (a) for children between 12 and 20 months and nonverbal children who are 21–30 months and (b) for verbal children between 21 and 30 months. Based on cutoffs applied to total scores, the ADOS-T produces two classifications: ASD and nonspectrum.

ADI-R Toddler

The ADI-R (Le Couteur et al., 2003) is a standardised, semi-structured clinical interview for caregivers of children and adults with suspected ASD. In this study, we used the new ADI-R algorithms (Kim and Lord, 2012) developed to extend the use of the ADI-R to toddlers and young preschoolers ranging from 12 to 47 months of age and down to a nonverbal mental age of 10 months. Using the new algorithms for toddlers and preschool children has been shown to improve sensitivity and specificity compared to the previously developed algorithm (Kim et al., 2013; Kim and Lord, 2012). Similar to the ADOS-T, the ADI-R-T also provides two classifications: ASD and nonspectrum.

Results

Data analysis

A five-item measure was targeted to ensure it would take less than 5 min to administer and to score. The best five items from the ADEC were determined by examining the area under the curve (AUC) for each item. The AUC is a measure of the overall predictive validity, where an AUC = 0.5 indicates random prediction of the independent variable and the minimum acceptable AUC should be at least 0.75 (Douglas et al., 2008).

Main analyses were conducted using the ASD versus non-TD group. Where possible, supplementary analyses were also conducted using the ASD versus non-ASD (i.e. non-TD combined with typical groups). Research with younger children often contrasts ASD with mixed TD and non-TD individuals (as in studies with baby siblings; Kim and Lord, 2012). We computed the sensitivity, specificity, PPV and NPV associated with the different BADEC cutoff scores. Glascoe (2005) proposed that the minimum sensitivity for a screening tool should be between 70% and 80%. Furthermore, to avoid over-referral, specificity should be close to 80%. We also compared BADEC performance for ASD and other groups and examined correlations between total scores on the BADEC and the ADEC, the ADOS revised algorithm score/ADOS-T total score and the ADI-R-T total score. Finally, binary logistic regression analysis was used to examine the predictive validity of the BADEC score for DSM-5 classification of ASD after controlling for NVDQ and adaptive functioning.

Analysis for the 12–36 months age group

Assumptions of parametric tests

Skewness and kurtosis for all the variables (age, ADEC, BADEC, ADOS/ADOS-T, ADI-T, NVDQ and adaptive functioning scores) were within acceptable ranges (skewness was within the range of −0.45 to 0.72 and kurtosis values were within range of −1.26 to −0.36).

Receiver operating characteristic curve analysis of ADEC items

The mean (and SD), and the AUC (95% confidence interval (CI)) for each ADEC item, are presented in Table 2. The five ADEC items with the highest AUC for this age group were as follows: (a) response to name, 0.81, 95% CI (0.75, 0.87); (b) reciprocity of smile, 0.79, 95% CI (0.72, 0.86); (c) gaze switch, 0.78, 95% CI (0.71, 0.85); (d) following verbal commands, 0.76, 95% CI (0.69, 0.83) and (e) use of gestures, 0.76, 95% CI (0.69, 0.83). Thus, these five items were used to form our BADEC. Using these five items, the AUC value was 0.91, 95% CI (0.87, 0.95), Cronbach’s α = 0.82 and the corrected item-total correlations ranged from 0.49 to 0.60.

Table 2.

Mean, standard deviation and AUC values for each ADEC item.

ADEC item	Mean (SD)		AUC (95% CI)	Mean (SD)	AUC (95% CI)
ADEC item	ASD	Non-TD	AUC (95% CI)	Non-ASD	AUC (95% CI)
1. Response to name	1.36 (0.76)	0.41 (0.58)	0.81 (0.75, 0.87)	0.25 (0.50)	0.85 (0.80, 0.90)
2. Imitation	1.08 (0.81)	0.48 (0.72)	0.69 (0.61, 0.77)	0.36 (0.64)	0.73 (0.67, 0.80)
3. Ritualistic play	0.60 (0.74)	0.24 (0.45)	0.62 (0.53, 0.70)	0.15 (0.37)	0.65 (0.58, 0.73)
4. Joint attention and social referencing	1.22 (0.85)	0.29 (0.51)	0.78 (0.71, 0.85)	0.18 (0.41)	0.81 (0.75, 0.87)
5. Eye contact	0.96 (0.79)	0.33 (0.54)	0.71 (0.63, 0.78)	0.21 (0.45)	0.76 (0.69, 0.82)
6. Functional play	1.14 (0.84)	0.35 (0.61)	0.75 (0.67, 0.82)	0.21 (0.51)	0.79 (0.73, 0.85)
7. Pretend play	1.60 (0.73)	0.86 (0.90)	0.70 (0.62, 0.79)	0.68 (0.84)	0.76 (0.70, 0.82)
8. Reciprocity of smile	1.41 (0.78)	0.53 (0.65)	0.79 (0.72, 0.86)	0.36 (0.56)	0.83 (0.78, 0.89)
9. Reaction to common sounds	0.71 (0.79)	0.14 (0.41)	0.71 (0.63, 0.78)	0.09 (0.32)	0.73 (0.66, 0.79)
10. Gaze monitoring	1.25 (0.87)	0.36 (0.65)	0.75 (0.68, 0.82)	0.23 (0.55)	0.79 (0.73, 0.85)
11. Following verbal commands	1.19 (0.83)	0.37 (0.58)	0.76 (0.69, 0.83)	0.24 (0.50)	0.80 (0.74, 0.86)
12. Delayed language	1.33 (0.87)	0.70 (0.84)	0.67 (0.59, 0.75)	0.45 (0.75)	0.75 (0.68, 0.81)
13. Anticipation of social advances	0.66 (0.85)	0.33 (0.61)	0.59 (0.51, 0.67)	0.23 (0.53)	0.63 (0.55, 0.70)
14. Nestling	0.63 (0.80)	0.18 (0.39)	0.63 (0.55, 0.71)	0.14 (0.36)	0.65 (0.58, 0.72)
15. Use of gestures	1.31 (0.90)	0.40 (0.67)	0.76 (0.69, 0.83)	0.26 (0.57)	0.79 (0.73, 0.85)
16. Task switching	1.12 (0.77)	0.49 (0.55)	0.71 (0.63, 0.78)	0.28 (0.48)	0.78 (0.72, 0.84)

The five ADEC items with the highest AUC are in bold.

ADEC: Autism Detection in Early Childhood; SD: standard deviation; AUC: area under the curve; CI: confidence interval; ASD: autism spectrum disorder; TD: typical development.

Supplementary analysis using the ASD versus non-ASD (i.e. non-TD combined with typical groups) revealed the same four ADEC items with the highest AUC values (i.e. response to name, reciprocity of smile, gaze switch and following verbal commands), followed by three items with the same AUC, namely, functional play, gaze monitoring and use of gestures (Table 2). Using the result we obtained from our main analysis (i.e. the five ADEC items: response to name, reciprocity of smile, gaze switch, following verbal commands and use of gestures) and applying it to the ASD versus non-ASD (i.e. non-TD combined with typical groups) comparison, the AUC value was 0.95, 95% CI (0.92, 0.97), Cronbach’s α = 0.86 and the corrected item-total correlations ranged from 0.61 to 0.76.

Derivation of cutoff score

The optimal cutoff score for the BADEC (Table 3) was 4, sensitivity was 0.81, specificity was 0.78 and the AUC was 0.80. PPV and NPV were also calculated. PPV measures the proportion of children who screen positive (i.e. at risk) who actually have ASD, and NPV measures the proportion of children who screen negative (i.e. not at risk) who do not have ASD. Using the cutoff score of 4, PPV was 0.81 (i.e. 82/101 true positives) and NPV was 0.78 (i.e. 67/86 true negatives). Thus, 19 children were over identified as having ASD (false positive), and 19 children with ASD were missed (false negative). For the full ADEC version, the optimal cutoff score was 11, sensitivity was 0.87 and specificity was 0.84, PPV was 0.87, NPV was 0.84 and the AUC was 0.85.

Table 3.

Sensitivity and specificity associated with the different BADEC cutoff scores (ASD vs non-TD).

BADEC score	Sensitivity	Specificity	Positive predictive value	Negative predictive value	AUC (95% CI)
1	1.0	0.24	0.61	1.0	0.62 (0.54, 0.70)
2	0.99	0.56	0.73	0.98	0.77 (0.70, 0.85)
3	0.91	0.66	0.76	0.86	0.79 (0.72, 0.86)
4	0.81	0.78	0.81	0.78	0.80 (0.73, 0.86)
5	0.74	0.87	0.87	0.74	0.81 (0.74, 0.87)
6	0.62	0.94	0.93	0.68	0.78 (0.72, 0.85)
7	0.55	0.95	0.93	0.65	0.75 (0.68, 0.83)
8	0.48	0.99	0.98	0.62	0.73 (0.66, 0.80)
9	0.27	1.0	1.0	0.54	0.63 (0.56, 0.71)
10	0.17	1.0	1.0	0.51	0.58 (0.50, 0.67)

BADEC: brief version of Autism Detection in Early Childhood; AUC: area under the curve; CI: confidence interval. The selected cutoff score with associated sensitivity, specificity, PPV and NPV are shown in bold.

Supplementary analysis using the ASD versus non-ASD (i.e. non-TD combined with typical groups) revealed that the optimal cutoff score for the BADEC could be 3 (sensitivity = 0.91, specificity = 0.81, PPV = 0.74 and NPV = 0.94) or 4 (sensitivity = 0.81, specificity = 0.88, PPV = 0.81 and NPV = 0.88), depending on whether a higher sensitivity or specificity is preferred.

Diagnostic validity

Because there was a significant group difference in NVDQ, F(1, 93) = 23.81, p < 0.001, a one-way analysis of covariance (ANCOVA) with NVDQ as the covariate compared the different diagnostic groups, based on the DSM-5 diagnoses (i.e. ASD and non-TD), on the BADEC total score to determine diagnostic validity. The ANCOVA indicated significant group differences for mean BADEC total scores, F(1, 90) = 33.01, p < 0.001, η² = 0.27.

Concurrent validity

The BADEC correlated significantly with the ADEC full versions (r = 0.93, 95% CI (0.91, 0.95)), the ADOS revised algorithm scores (r = 0.82, 95% CI (0.69, 0.89)), the ADOS-T total score (r = 0.84, 95% CI (0.61, 0.93)) and the ADI-R-T score (r = 0.42, 95% CI (0.19, 0.61)). When we removed those (five) key ADEC items from the full ADEC version (i.e. the ‘abridged’ version), the BADEC still correlated significantly with the ‘abridged’ ADEC version (r = 0.85, 95% CI (0.81, 0.88)). The full ADEC version was also correlated significantly with the ADOS revised algorithm scores (r = 0.87, 95% CI (0.78, 0.93)), the ADOS-T total score (r = 0.88, 95% CI (0.70, 0.95)) and the ADI-R-T score (r = 0.44, 95% CI (0.21, 0.62)). Thus, the BADEC performed similarly to the ADEC in terms of its concurrent validity with other ASD diagnostic tools such as the ADOS and the ADI-R.

Predictive validity

Using a binary logistic regression analysis, the BADEC score was used to predict DSM-5 ASD classification of participants. Given the significant difference between groups in NVDQ and adaptive functioning (p < 0.001), we first controlled for participants’ NVDQ and adaptive functioning. The full model containing the predictor (the BADEC total score) was statistically significant, χ² (1, N = 66) = 48.70, p < 0.001, odds ratio = 2.47, 95% CI (1.47, 4.14). The model as a whole explained between 52.2% (Cox and Snell R²) and 69.7% (Nagelkerke R²) of the variance in DSM-5 classification. The odds of receiving an ASD classification increased between 1.47 and 4.14 times for every one-unit increase in the BADEC total score.

Discussion

In addition to strong psychometric features, screening tools must be practical for use by busy medical professionals and allied health professionals. Medical professionals have reported a variety of concerns such as the time and training required to use current ASD screening tools in their practice (e.g. Gillis, 2009; Gura et al., 2011). Indeed, these beliefs were supported by pilot data we collected using a survey of 30 general medical practitioners practising in Australia. About 93% of general medical practitioners reported that they did not routinely screen for ASD in children below 3 years of age. Approximately 41% and 33% of respondents reported ‘time required to learn how to administer ASD screening tool’ and ‘time required to administer in my practice’, respectively, as important barriers to ASD screening. About 70% of respondents reported that they would be likely to ‘use an ASD screening tool in their practice if the tool were easy to administer and took about 5 min’.

In this study, we developed and evaluated a brief version of an ASD-specific screening tool, the ADEC, for use by busy frontline medical professionals as a rapid screener to guide referral for further assessment. The analyses supported the use of five key behaviours (namely ADEC items: response to name, reciprocity of smile, gaze switch, following verbal commands and use of gestures) to form BADEC.

The results indicated the following: (a) the BADEC correlated well with the ADEC version and other ASD diagnostic tools such as the ADOS and ADI-R; (b) the BADEC’s optimal cutoff score of 4 produced sensitivity exceeding 80%, although specificity was slightly lower (78%), PPV of 0.81 and NPV of 0.78; (c) the BADEC discriminated participants with ASD and participants with non-TD, after controlling for participants’ nonverbal DQ, with estimates of effect size indicating that 27% of the variance in the BADEC score was associated with the group factor (i.e. whether ASD or non-TD) and (d) the BADEC predicted DSM-5 ASD classification, after controlling for participants’ nonverbal DQ and adaptive functioning. These findings suggest that the BADEC should be useful and effective in the screening process of ASD, especially if time does not permit medical and allied health professionals to use the (relatively longer) ADEC version. It should be noted that due to the clearly operationalised items, the use of the ADEC/BADEC does not require specialised or expert training, unlike the STAT. Inter-rater reliability supports consistency and thus fidelity in its use. Hence, healthcare professionals should be able to use it without any fidelity issue.

In our study, we found that the sensitivity of the BADEC (0.81) was slightly lower than the ADEC version (when using a cutoff score of 11). A study by Gabrielsen (2015) found that brief clinical observations may not provide enough information about atypical behaviours to reliably detect autism risk. The sensitivity of the ADEC version in this study was also slightly lower than the validation sample used in Nah et al.’s (2014b) study which was 1.0. This could be due to the fact that Nah et al. (2014b) validated the ADEC with the previous DSM-IV-TR AD classification, whereas in this study the ADEC was compared against the current DSM-5 ASD classification. There is only one study to date that has examined the ADEC in a DSM-5 ASD sample (Hedley et al., 2015), and the authors found that the full ADEC version (with a cutoff score of 11) returned good sensitivity (0.93–0.94) but poorer specificity (0.62–0.64) for BEC diagnosis of ASD. In our study, when we compared the ASD versus a clinical non-TD sample, the sensitivity of the ADEC was slightly lower (0.87), but the specificity was higher (0.84) when compared to Hedley et al.’s (2015) study.

Some researchers have described similar problems with ASD screening before age 2, and especially before 18 months (Barton et al., 2012), where there was a greater possibility of a higher false-positive rate (and therefore low PPV) for the younger children (Chawarska et al., 2007; Pandey et al., 2008). This may be because some young children show early developmental variations which may resolve later, perhaps explaining why false-positive rates may be higher for this age group (Swinkels et al., 2006). It is also possible that milder variants of ASD, and children with a higher level of cognitive development, could be missed at a young age (Dietz et al., 2006), although Barbaro and Dissanayake (2010) found very high PPV for children less than 24 months using the SACS surveillance methodology. Nevertheless, it may be necessary to repeat the screening when the children are at 24 months of age as recommended by the AAP (Johnson et al., 2007).

We acknowledge that a high false-positive rate is likely to cause unnecessary concerns to parents. However, given that research suggests that most of the children who falsely screen positive for ASD at 18 months are often at risk of other development disorders (Pandey et al., 2008; Pierce et al., 2011), the BADEC may still be useful in screening for younger individuals who may benefit from some form of early intervention. In addition, given parents are reporting signs emerging from as young as 12 months (Stone et al., 2004), the BADEC could fill the gap in this area by providing a brief and direct observation tool, pending on results of future studies to examine the psychometric properties of the BADEC for the 12–18 months age group. It should be noted that in the ADEC validation article (Nah et al., 2014b), they did examine diagnostic validity of the ADEC for children below age 24 months (albeit with a small sample) and results indicated a significant group difference between the ASD group and the developmental delay group. That said, the use of any screening tool is to identify children who should be referred for further examination. Hence, a Level 2 screening tool should err on the side of false positives as these children will be referred for a more thorough examination, and if the screen was falsely positive, these fears will be allayed. Conversely, children missed in the screening process will not typically be referred and intervention may be delayed.

Possible limitations

In our study, there were significantly more males in the ASD than the non-TD group, given that ASD diagnoses are more common in males than females (APA, 2013). Hence, the lack of gender matching should be considered as a possible limitation in this study. In addition, the sample was predominantly Caucasian (about 96%) which might limit the generalisability of our findings to other ethnic/cultural groups. In this study, we were not able to use the ADOS revised algorithm total score, the ADOS-T total score and the ADI-R-T total score as predictors in the regression analysis given that the BEC DSM-5 ASD diagnosis was made based, in part, on information from these scores. As a result, we were not able to compare how well the BADEC score predicts DSM-5 classification versus the ADOS/ADOS-T and ADI-R/ADI-R-T scores.

Given the low cognitive functioning of this sample, it is not known how the BADEC will perform with a higher cognitive functioning sample. Therefore, the results would need to be viewed as preliminary given the nature of the study sample and the findings might not be generalisable to samples with higher levels of cognitive functioning. However, controlling for DQ statistically suggests that the ADEC is sensitive to ASD not intellectual disability. The risk of a false negative on a Level 2 screening is a great concern and this limitation needs to be addressed. Another possible limitation was that the typically developing sample in our study was not administered any developmental/adaptive tests and their functioning was assumed to be within the average range. Nevertheless, the main focus of our analyses in this study was to compare the ASD and the non-TD sample rather than with the TD sample.

Conclusion

This study represents the first step in the development of a brief ASD screening instrument (BADEC) designed to help busy medical and allied health professionals in the referral pathway for ASD. In this study, we provided a psychometric examination of the BADEC for its intended age group of 12–36 months. Medical and allied health professionals may eventually find the BADEC to be a time-efficient and suitable screening tool to help them identify young children presenting with possible ASD in their practice settings, although future studies would need to be conducted to examine its effectiveness in clinical practice and time taken to administer it. The results would also need to be viewed as preliminary and the findings might not be generalisable to the more intellectually able ASD populations.

Supplemental Material

AUT757563_Lay_Abstract – Supplemental material for Development of a brief version of the Autism Detection in Early Childhood

Supplemental material, AUT757563_Lay_Abstract for Development of a brief version of the Autism Detection in Early Childhood by Yong-Hwee Nah, Robyn L Young and Neil Brewer in Autism

Footnotes

Funding

The author(s) received no financial support for the research, authorship and/or publication of this article.

ORCID iD

Yong-Hwee Nah

References

Allison

Baron-Cohen

Wheelwright

et al . (2008) The Q-CHAT (Quantitative Checklist for Autism in Toddlers): a normally distributed quantitative measure of autistic traits at 18-24 months of age: preliminary report. Journal of Autism and Developmental Disorders 38: 1414–1425.

American Psychiatric Association (APA) (2013) Diagnostic and Statistical Manual of Mental Disorders: Fifth Edition (DSM-5). Arlington, TX: APA.

Arunyanart

Fenick

Ukritchon

et al . (2012) Developmental and autism screening: a survey across six states. Infants & Young Children 25: 175–187.

Auyeung

Baron-Cohen

Wheelwright

et al . (2008) The Autism Spectrum Quotient–Children’s Version (AQ-Child). Journal of Autism and Developmental Disorders 38: 1230–1240.

Barbaro

Dissanayake

(2010) Prospective identification of autism spectrum disorders in infancy and toddlerhood using developmental surveillance: the social attention and communication study. Journal of Developmental and Behavioral Pediatrics 31: 376–385.

Barton

Dumont-Mathieu

Fein

(2012) Screening young children for autism spectrum disorders in primary practice. Journal of Autism and Developmental Disorders 42: 1165–1174.

Britt

Miller

Henderson

et al . (2013) A Decade of Australian General Practice Activity 2003–04 to 2012–13: General Practice Series No. 34. Sydney, NSW, Australia: Sydney University Press.

Centers for Disease Control and Prevention (CDC) (2012) Prevalence of autism spectrum disorders: autism and developmental disabilities monitoring network, 14 sites, United States, 2008. Morbidity and Mortality Weekly Report 61: 1–19.

Chawarska

Klin

Paul

et al . (2007) Autism spectrum disorder in the second year: stability and change in syndrome expression. Journal of Child Psychology and Psychiatry 48: 128–138.

10.

Crais

McComish

Humphreys

et al . (2014) Pediatric healthcare professionals’ views on autism spectrum disorder screening at 12–18 months. Journal of Autism and Developmental Disorders 44: 2311–2328.

11.

Dawson

Burner

(2011) Behavioral interventions in children and adolescents with autism spectrum disorder: a review of recent findings. Current Opinion in Pediatrics 23: 616–620.

12.

Deveugele

Derese

van den Brink-Muinen

et al . (2002) Consultation length in general practice: cross sectional study in six European countries. BMJ 325: 472–477.

13.

Dietz

Swinkels

SHN

van Daalen

et al . (2006) Screening for autistic spectrum disorder in children aged 14–15 months. II: population screening with the Early Screening of Autistic Traits Questionnaire (ESAT): design and general findings. Journal of Autism and Developmental Disorders 36: 713–722.

14.

Douglas

Guy

Reeves

KA, et al.

(2008) HCR-20 violence risk assessment scheme: overview and annotated bibliography. Available at: http://escholarship.umassmed.edu/psych_cmhsr/335/

15.

Dow

Guthrie

Stronach

et al . (2017) Psychometric analysis of the Systematic Observation of Red Flags for autism spectrum disorder in toddlers. Autism 21: 301–309.

16.

Gabrielsen

Farley

Speer

et al . (2015) Identifying autism in a brief observation. Pediatrics 135: 330–338.

17.

Gillis

(2009) Screening practices of family physicians and pediatricians in 2 southern states. Infants & Young Children 22(4): 321–331.

18.

Glascoe

(2005) Screening for developmental and behavioral problems. Mental Retardation and Developmental Disabilities Research Reviews 11: 173–179.

19.

Gravelle

Scott

Sivey

et al . (2016) Competition, prices and quality in the market for physician consultations. Journal of Industrial Economics 64: 135–169.

20.

Gura

Champagne

Blood-Siegfried

(2011) Autism spectrum disorder screening in primary care. Journal of Developmental and Behavioral Pediatrics 32: 48–51.

21.

Havdahl

Bishop

Surén

et al . (2017) The influence of parental concern on the utility of autism diagnostic instruments. Autism Research 10: 1672–1686.

22.

Hedley

Nevill

Monroy-Moreno

et al . (2015) Efficacy of the ADEC in identifying autism spectrum disorder in clinically referred toddlers in the US. Journal of Autism and Developmental Disorders 45: 2337–2348.

23.

Huerta

Bishop

Duncan

et al . (2012) Application of DSM-5 criteria for autism spectrum disorder to three samples of children with DSM-IV diagnoses of pervasive developmental disorders. American Journal of Psychiatry 169: 1056–1064.

24.

Johnson

Myers

and Council on Children with Disabilities (2007) Identification and evaluation of children with autism spectrum disorders. Pediatrics 120: 1183–1215.

25.

Kim

Lord

(2012) New autism diagnostic interview–revised algorithms for toddlers and young preschoolers from 12 to 47 months of age. Journal of Autism and Developmental Disorders 42: 82–93.

26.

Kim

Thurm

Shumway

et al . (2013) Multisite study of new autism diagnostic interview–revised (ADI-R) algorithms for toddlers and young preschoolers. Journal of Autism and Developmental Disorders 43: 1527–1538.

27.

Le Couteur

Lord

Rutter

(2003) The Autism Diagnostic Interview – Revised (ADI-R). Los Angeles, CA: Western Psychological Services.

28.

Lord

Luyster

Gotham

et al . (2012) Autism Diagnostic Observation Schedule – Toddler Module Manual. Los Angeles, CA: Western Psychological Services.

29.

Lord

Risi

Lambrecht

et al . (2000) The autism diagnostic observation schedule-generic: a standard measure of social and communication deficits associated with the spectrum of autism. Journal of Autism and Developmental Disorders 30: 205–223.

30.

Nah

Young

Brewer

(2014a) Using the Autism Detection in Early Childhood (ADEC) and Childhood Autism Rating Scales (CARS) to predict long term outcomes in children with autism spectrum disorders. Journal of Autism and Developmental Disorders 44: 2301–2310.

31.

Nah

Young

Brewer

et al . (2014b) Autism Detection in Early Childhood (ADEC): reliability and validity data for a level 2 screening tool for autistic disorder. Psychological Assessment 26: 215–226.

32.

Oner

Munir

(2014) Three-item Direct Observation Screen (TIDOS) for autism spectrum disorder. Autism 18: 733–742.

33.

Pandey

Verbalis

Robins

et al . (2008) Screening for autism in older and younger toddlers with the modified checklist for autism in toddlers. Autism 12: 513–535.

34.

Pierce

Carter

Weinfeld

et al . (2011) Detecting, studying, and treating autism early: the one-year well-baby check-up approach. Journal of Pediatrics 159: 458–465.

35.

Reichow

(2012) Overview of meta-analyses on early intensive behavioral intervention for young children with autism spectrum disorders. Journal of Autism and Developmental Disorders 42: 512–520.

36.

Soke

Maenner

Christensen

et al . (2017) Brief report: estimated prevalence of a community diagnosis of autism spectrum disorder by age 4 Years in children from selected areas in the United States in 2010: evaluation of birth Cohort effects. Journal of Autism and Developmental Disorders 47: 1917–1922.

37.

Stone

Coonrod

Turner

et al . (2004) Psychometric properties of the STAT for early autism screening. Journal of Autism and Developmental Disorders 34: 691–701.

38.

Swinkels

Dietz

van Daalen

et al . (2006) Screening for autistic spectrum in children aged 14 to 15 months. I: the development of the Early Screening of Autistic Traits Questionnaire (ESAT). Journal of Autism and Developmental Disorders 36: 723–732.

39.

Volkmar

Siegel

Woodbury-Smith

et al . (2014) Practice parameter for the assessment and treatment of children and adolescents with autism spectrum disorder. Journal of the American Academy of Child and Adolescent Psychiatry 53: 237–257.

40.

Young

(2007) Autism Detection in Early Childhood: ADEC. Camberwell, VIC, Australia: Australian Council of Educational Research.

41.

Young

Nah

(2016) Examining Autism Detection in Early Childhood (ADEC) in the early identification of young children with autism spectrum disorders (ASD). Australian Psychologist 51: 261–271.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.01 MB