Tele-assessment of young children referred for autism spectrum disorder evaluation during COVID-19: Associations among clinical characteristics and diagnostic outcome

Abstract

Since the onset of the COVID-19 pandemic, there has been a rapid acceleration of innovative research on health services delivery, including real-world clinical implementation and evaluation of tele-assessment for the diagnosis of autism spectrum disorder. Extending this promising work, the present study examined clinical characteristics and diagnostic outcome for young children receiving autism spectrum disorder tele-assessment during the COVID-19 pandemic. The standard tele-assessment procedure included caregiver clinical interview, administration of the TELE-ASD-PEDS (TAP; a novel caregiver-mediated remote autism spectrum disorder observational assessment tool), Vineland-3, and diagnostic feedback and family counseling. Overall, our findings suggest that a definitive autism spectrum disorder diagnosis can be determined for most young children evaluated using standard tele-assessment procedures. While TAP scores and measures of adaptive functioning and Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition autism spectrum disorder symptoms are highly associated with autism spectrum disorder diagnostic outcome, a specific pattern of Restricted and Repetitive Behaviors independently predicted differential autism spectrum disorder outcome. Furthermore, our findings suggest that the TAP is useful for evaluation of at-risk children above 36 months of age with delayed language. These results add to an increasing body of research supporting use of tele-assessment, and specifically the TAP, for diagnosis of young children referred for autism spectrum disorder evaluation.

Lay abstract

The diagnosis of autism spectrum disorder (ASD) has traditionally been made through in-person evaluation. While the COVID-19 pandemic disrupted access to ASD services, there has been remarkable growth in research focused on novel ASD diagnostic practices, including the use of telemedicine. We implemented a standard ASD tele-assessment evaluation procedure, including use of a novel remote clinician-coached, caregiver-delivered ASD assessment tool (TELE-ASD-PEDS; TAP), with the goal of continuing to provide diagnostic services to young children and their families during the pandemic. We examined the relationship between child characteristics and diagnostic outcome for 335 children, ages 14–78 months, who received ASD tele-assessment conducted by psychologists and pediatricians in an outpatient clinic of a Midwestern academic medical center. We found that clinicians could make a determination about ASD diagnosis for most children (85%) evaluated using tele-assessment. Child clinical characteristics, including TAP scores and clinician ratings of ASD symptoms, were related to diagnostic outcome (i.e. diagnosis of ASD, no ASD, and Unsure about ASD). When all clinical characteristics were examined together, the presence of specific repetitive behaviors predicted ASD diagnosis. We also found that the TAP is effective for making an ASD diagnosis when used as part of comprehensive tele-assessment evaluation in children ⩾ 36 months of age with delayed language. Our study adds to an increasing body of research supporting use of tele-assessment for diagnosis of ASD. Although further research is needed, telemedicine may help families from different backgrounds and geographic locations to access high-quality diagnostic services.

Keywords

autism spectrum disorders diagnosis health services tele-assessment

Best practice in the diagnosis of autism spectrum disorder (ASD) includes obtaining a history of development and current characteristics, observation of behavior, and interpretation and synthesis of this information (Huerta & Lord, 2012; Hyman et al., 2020; Lord et al., 2021). Evaluation of developmental functioning (i.e. including language, cognitive, and adaptive functioning), co-morbid conditions, and family context is also critical for providing rich information about the individual, including expected prognosis, as well as to inform intervention planning. Traditional in-person ASD observational assessment tools are standardized using specific toys and materials and rely on an experienced clinician to create social situations to elicit and observe ASD-related behaviors (C. Lord et al., 2012). However, the need for adaptation in healthcare service delivery during the COVID-19 pandemic has led to unprecedented innovation in ASD diagnostic methods, including dissemination of novel remote assessment tools and telemedicine procedures. While this innovation has allowed for ongoing service delivery during a prolonged global pandemic, it also provided an opportunity for expansion and evaluation of remote procedures that have the potential to reduce long-standing inequities in ASD service access, including for those from socioeconomically disadvantaged backgrounds and underserved geographic locations (Drahota et al., 2020; Murphy & Ruble, 2012; Thomas et al., 2012).

Prior to COVID-19, there was limited attention devoted to the development and dissemination of remote assessment tools for ASD diagnosis. Several studies, however, documented promising results using (1) the “store and forward method” in which pre-recorded home videos are reviewed by clinicians to aid in the diagnostic decision-making process (Kanne et al., 2018; Nazneen et al., 2015; Sutantio et al., 2020) and (2) “real time” video technology that allows a remote expert clinician to participate in a diagnostic evaluation conducted by a trained provider in the same clinical setting as the child and their caregivers (Juárez et al., 2018). Together, these initial studies provided foundational evidence that use of telemedicine for diagnostic evaluation of children with ASD may be a feasible and accurate approach.

Since the onset of the pandemic, (Alfuraydan et al., 2020; Ellison et al., 2021; Stavropoulos et al., 2022; Valentine et al., 2021) studies examining telemedicine for ASD screening and diagnostic evaluation have demonstrated evidence of acceptability to families and clinicians (Alfuraydan et al., 2020; Ellison et al., 2021; Stavropoulos et al., 2022; Valentine et al., 2021), feasibility of clinical implementation (Alfuraydan et al., 2020; Matthews et al., 2021), diagnostic accuracy when compared with traditional in-person evaluation (Stavropoulos et al., 2022), and improved service access (i.e. increased service availability with reduced waiting time; Valentine et al., 2021). Furthermore, ASD diagnostic tools designed for remote administration, often with clinicians coaching caregivers to deliver a standard assessment protocol, have been developed and implemented during COVID-19 (Berger et al., 2022; Corona et al., 2020; Dow et al., 2021; Jang et al., 2021; Wagner et al., 2020).

One promising remote ASD assessment tool that has been widely implemented during COVID-19 is the TELE-ASD-PEDS (TAP; Wagner, Stone, et al., 2021). The TAP is designed for use by clinicians with expertise in ASD evaluation in toddlers and young children. Remote clinicians coach caregivers to complete a brief set of activities using toys and materials available within many home environments. When implemented as part of a comprehensive ASD evaluation, including a developmental history and symptom-focused interview, clinicians and caregivers report satisfaction with this method (Corona et al., 2020; Reisinger et al., 2022; Wagner et al., 2020; Wagner, Weitlauf, et al., 2021). Additional studies have demonstrated evidence of diagnostic utility of the TAP. Wagner and colleagues (2020) reported that, in 89% of ASD tele-assessment evaluations employing the TAP, a definitive decision regarding ASD diagnosis was made. In a larger study of seven ASD diagnostic clinics across the United States using the TAP as part of a remote evaluation protocol, a confirmed diagnostic decision regarding the presence or absence of ASD was made in 86% of evaluations (Wagner, Weitlauf, et al., 2021).

The rapid growth in innovation, uptake, and evidence for the effectiveness of ASD tele-assessment practices represents a significant step forward in both the clinical and scientific arenas. ASD tele-assessment offers families a unique and valuable opportunity for direct engagement in the evaluation process (Corona et al., 2020; Nazneen et al., 2015), has the potential to improve timely access to diagnostic services (Committee on Pediatric Workforce, 2015) and reduce care costs, travel time, and caregiver missed work (Boxer & Ellimoottil, 2019; Committee on Pediatric Workforce, 2015). Novel tele-assessment tools and approaches have been disseminated and tested during the COVID-19 pandemic with highly promising results regarding acceptability, feasibility, and accuracy (Alfuraydan et al., 2020; Jang et al., 2022; Matthews et al., 2021; Stavropoulos et al., 2022; Wagner et al., 2020). However, there currently exists a gap in our understanding of the efficacy of tele-assessment across the heterogenous ASD phenotype. In order to most effectively deploy tele-assessment models and tools using a data-driven approach, research is needed to identify for whom specific ASD tele-assessment tools and evaluation models are most efficacious. As such, the present study examines the association between clinical characteristics and diagnostic outcome for young children receiving ASD tele-assessment using the TAP during the COVID-19 pandemic. Specifically, we examine (1) associations between child clinical characteristics and ASD diagnostic outcome, (2) clinical predictors of ASD diagnostic outcome, and (3) associations between child clinical characteristics and diagnostic outcome in younger versus older children to determine efficacy of the TAP for children > 36 months of age.

Methods

Clinic setting, processes & evaluation protocol

The Division of Developmental Medicine within the Department of Pediatrics at Indiana University School of Medicine is set within a large Midwestern children’s hospital and academic medical center. Several clinics within the Division provide evaluation and care management services for children and adolescents with ASD and related neurodevelopmental and behavioral disabilities. In March 2020, in-person evaluation services were suspended due to COVID-19 precautions, and faculty within the Division developed clinical telemedicine processes and obtained training on remote ASD assessment tools. In May 2020, two clinics within the Division staffed by licensed psychologists or general pediatricians with specialized training and experience in evaluation and diagnosis of ASD in young children began offering telemedicine diagnostic evaluations for young children referred for ASD evaluation. All clinicians who conducted ASD evaluations in the present study received training in remote ASD evaluation and the TAP by the developers of the instrument at the Treatment & Research Institute for Autism Spectrum Disorder (TRIAD) at the Vanderbilt Kennedy Center. Training included completion of two webinars on telemedicine assessment of ASD. The first author RMK also participated in a series of additional remote small group sessions on use of the TAP and led all study clinicians in a clinical workgroup focused on providing peer consultation (i.e. focused on remote ASD evaluation procedures, administration and scoring of the TAP, and clinical documentation and billing/coding) during the first few months of the COVID-19 pandemic.

Beginning in April 2020, clinic administration reviewed all children referred for ASD evaluation to identify those who may be eligible for ASD tele-assessment using the TAP. Although the TAP is designed for children under 36 months of age (Wagner, Stone, et al., 2021), we offered this service to all children 48 months and under, and older children with significant language impairment (i.e. ages 49–83 months with current language level ranging from no functional word use to occasional simple 2-word phrases). This decision was made based on (1) application of Autism Diagnostic Observation Schedule-2 (ADOS-2) principles in which module choice is made based upon the individual’s language level (C. Lord et al., 2012), and (2) favorable trial administration of the TAP with children with this clinical profile. Children ⩽ 48 months referred for ASD evaluation were not screened for language level prior to evaluation scheduling. However, for referred children > 48 months, a caregiver telephone screening was conducted by a speech-language pathologist to determine if the child met language inclusion criteria prior to scheduling the evaluation.

All eligible children previously scheduled or on the waitlist for ASD evaluation were contacted by the clinic scheduling team to offer telemedicine ASD evaluation. The scheduling team used a verbal script (adapted from https://vkc.vumc.org/vkc/triad/TAP) to ensure standardized delivery of information. Caregivers who consented to telemedicine were provided verbal and written instructions on how to prepare for their evaluation, including accessing the virtual telemedicine platform and a telephone hotline for assistance with technology problems. Caregivers who declined telemedicine were informed their child would remain on the waitlist for in-person evaluation and would be contacted when scheduling resumed.

The 90- to 120-minute tele-assessment was conducted via Zoom Health remote platform and was led by a licensed psychologist or pediatrician. All evaluations were conducted remotely from the clinician’s office (clinic or off-site) to the child’s home or similar setting (i.e. some caregivers reported traveling to a nearby location such as a family member’s home to access needed technology). The standard evaluation protocol included comprehensive caregiver clinical interview (including medical, family, and developmental history and ascertainment of Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) ASD symptoms), administration and scoring of the TAP (Wagner, Stone, et al., 2021), remote administration of caregiver-report Vineland-3 (Sparrow et al., 2016), and diagnostic feedback and family counseling on recommendations. An evaluation report was provided to caregivers and referring primary care provider (PCP), and caregivers were offered the opportunity for ongoing care navigation support through the clinic’s social work team.

Participants

A total of 335 children, ages 14–78 months of age (M = 37.9 months; SD = 10.8 months), completed a telemedicine ASD evaluation and were included in the study. Children and families that required a language interpreter were excluded from the study (n = 20; Primary language: Spanish: 85%; Burmese: 10%; Arabic: 5%) given that our clinic processes and evaluation procedures required modification to provide quality clinical care to this population. See Table 1 for child and family demographic characteristics. Notably, our proportion of non-White (29%) and Latinx (10%) participants are somewhat higher than expected given state census data (i.e. non-White: 16%; Latinx: 8%; United States Census Bureau, n.d.).

Table 1.

Demographic and clinical evaluation characteristics.

		ASD Diagnostic Outcome			p
	TotalN = 335	ASD Presentn = 185	ASD Absentn = 102	ASD Unsuren = 48	p
Age in months, mean (SD)	37.8 (10.8)	37.2 (11.0)	38.3 (10.8)	39.1 (11.3)	0.463
Sex, n (%)					0.974
Male	243 (73)	133 (72)	75 (74)	35 (73)
Female	91 (27)	51 (30)	27 (27)	13 (27)
Primary insurance status, n (%)					0.143
Public (Medicaid)	192 (58)	106 (58)	66 (65)	20 (44)
Private	119 (36)	67 (36)	29 (29)	23 (50)
None	20 (6)	11 (6)	6 (6)	3 (7)
aRace, n (%)					0.332
White	197 (71)	105 (70)	63 (72)	29 (69)
Black or African American	45 (16)	20 (13)	15 (17)	10 (24)
More than one race	20 (7)	12 (8)	7 (8)	1 (2)
Asian	14 (5)	11 (7)	1 (1)	2 (5)
American Indian/Alaskan Native	3 (1)	2 (1)	1 (1)	0 (0)
aEthnicity, n (%)					0.510
Non-Hispanic or Latinx	248 (90)	133 (38)	79 (93)	36 (90)
Hispanic or Latinx	28 (10)	18 (12)	6 (7)	4 (10)
aFamily income, n (%)					0.681
Less than $40,000	67 (48)	34 (45)	25 (56)	8 (42)
$40,000–$49,999	11 (8)	8 (11)	3 (7)	0 (0)
$50,000–$59,999	11 (8)	6 (8)	3 (7)	2 (11)
$60,000–$74,999	17 (12)	10 (13)	3 (7)	4 (21)
$75,000 or more	33 (24)	17 (23)	11 (24)	5 (26)
Medically underserved area, n (%)	245 (73)	134 (72)	72 (71)	39 (81)	0.369
Rural designation, n (%)	59 (18)	33 (18)	18 (18)	8 (17)	0.982
Diagnostic certainty, n (%)					< .001
Uncertain	43 (14)	10 (6)	11 (12)	22 (47)
Certain	268 (86)	159 (94)	84 (88)	25 (53)
Referred for in-person evaluation, n (%)	99 (30)	24 (13)	33 (32)	42 (88)	< .001
bTechnology barriers, n (%)	48 (15)	24 (14)	18 (19)	6 (13)	0.509
cFamily/home set up barriers, n (%)	63 (20)	29 (17)	21 (22)	13 (28)	0.247

ASD: autism spectrum disorder; ASD diagnostic status denotes clinician’s final diagnosis based on ASD telehealth evaluation. Medically underserved area and rural designation refers to dichotomous (i.e. yes/no) designation for county in which child resides.

Descriptive statistics calculated based on n = 154 (i.e. those who received evaluation and had a returned caregiver survey); data not available for those who did not return a caregiver survey.

Technology barriers include audio/video distortions, poor device capability/failure, and caregiver challenges operating the device or platform.

Family/home set up barriers include problems with placement of video technology, child non-compliance, environmental distractions, and inadequate materials for TAP assessment.

Licensed psychologists (N = 7) and pediatricians (N = 4) with significant experience in the evaluation of children with ASD and related neurodevelopmental disabilities (mean years of experience: 13; range: 2–35 years), submitted data (mean evaluations = 30). Psychologists conducted 84% of the evaluations (n = 268); pediatricians conducted 16% of evaluations (n = 53). Prior to COVID-19, none of the clinicians had experience with use of telemedicine for ASD evaluation.

Measures

Clinician telemedicine survey

A survey to capture clinician experience was developed for a broader study on telemedicine neurodevelopmental evaluation and is described in McNally Keehn et al. (2021). For the present study, clinician ratings of diagnostic certainty and technology and family/home set-up barriers encountered during the evaluation were extracted from this survey. Diagnostic certainty was rated following the evaluation on a 4-point Likert-type scale (Completely Uncertain (1)—Completely Certain (4)); ratings were dichotomized for analysis (i.e. Certain = Completely or Somewhat Certain; Uncertain = Somewhat or Completely Uncertain). Technology (i.e. audio/video distortions, poor device capability/failure, caregiver challenges operating the device or platform) and family/home set up barriers (i.e. problems with placement of video technology, child non-compliance, environmental distractions, and inadequate materials for TAP assessment) were rated for presence, as well as perceived disruption to the evaluation process (i.e. both items rated on a dichotomous “yes/no” scale).

Caregiver telemedicine survey

A survey to capture caregiver experience with tele-assessment was similarly developed for a broader study and is described in McNally Keehn et al. (2021). Items related to child/family demographic information were extracted for the present study.

TAP rating form

The TAP (Wagner, Stone, et al., 2021) is a remote assessment tool designed to aid in the diagnosis of ASD in young children ages 14–36 months. The TAP takes approximately 15–30 minutes to complete and is administered and scored by a trained clinician experienced in the assessment of ASD in young children. It can be administered remotely in the home or clinic setting. The clinician leads the child’s caregiver through a series of play-based activities (i.e. using specific standard verbal prompts) designed to elicit opportunities to observe social communication skills such as interactive play, imitation, joint attention, and requesting, as well as the presence of restricted and repetitive behaviors. Following administration, item-level ratings are made by the clinician across seven key symptoms (i.e. socially directed speech and sounds; frequent and flexible eye contact; unusual vocalizations; unusual or repetitive play; unusual or repetitive body movements; combines gestures, eye contact and speech/vocalization; unusual sensory exploration) using both dichotomous (yes/no) and Likert (3 = behaviors characteristic of ASD clearly present; 2 = behaviors characteristic of ASD present at subclinical levels; 1 = behaviors characteristic of ASD not present) scoring procedures. A total score is calculated by summing the seven Likert scale item scores and then used to determine categorical ASD risk classification. TAP total scores ⩾ 11 are considered in the At-Risk range with higher scores being indicative of greater ASD symptoms. Both the TAP categorical risk (i.e. At-Risk; not At-Risk) and total score were utilized in the present study to quantify ASD symptoms. The rating form also gathers information on ASD diagnostic outcome (ASD Absent, Present, Unsure) and whether the clinician recommended in person evaluation for diagnostic clarification.

DSM-5 ASD symptom rating scale

A clinician-report DSM-5 ASD symptom rating scale was adapted with permission from Vanderbilt University Medical Center. The scale includes ratings across DSM-5 (American Psychiatric Association, 2013) ASD symptom categories. Each symptom category also includes ratings of associated developmental markers often observed in toddlers and young children with ASD (e.g. Atypical Nonverbal Social Behavior: atypical or inconsistent eye gaze, limited index finger pointing, limited conventional gesture use, physical direction of others, challenges communicating wants and needs, limited range of facial expressions). Each rating is scored on a 3-point scale (i.e. 0 = typical behavior given developmental profile; 1 = unclear if behavior is atypical; 2 = symptom clearly present/clear atypicality). For the present study, all items were dichotomized into a categorical rating of present/absent (i.e. ratings of 0 were scored as “symptom absent” and ratings of 1 or 2 were scored as “symptom present”). A continuous variable for each DSM-5 symptom subdomain was calculated by summing developmental marker items (i.e. Atypical Nonverbal Social Behavior sum = sum of “symptom present” ratings for developmental markers within this domain). Finally, a summary variable for Social Communication (SC) and Restricted and Repetitive Behaviors (RRB) was created by summing the symptom subdomain scores.

Vineland-3

The Vineland Adaptive Behavior Scales, Third Edition—Comprehensive Parent/ Caregiver Form (Vineland-3; Sparrow et al., 2016) was administered via remote link emailed to caregivers prior to their child’s evaluation. In the present study, the Vineland-3 Adaptive Behavior Composite (ABC; i.e. measure of overall adaptive functioning) and Communication, Daily Living Skills, Socialization, and Motor Skills subdomain scores were utilized. Standard scores (ABC and subdomain scores) have a mean of 100 and standard deviation of 15.

Procedures

All study procedures were approved by the Indiana University Institutional Review Board. Data were collected from May 2020 through July 2021. Members of the research team gathered data from the child’s electronic health record (e.g. age, sex, zip code, and insurance type); child zip code was used to determine whether the child’s county of residence was designated as a medically underserved area (MUA; Health Resources and Services Administration, n.d.) or rural region (Indiana Office of Community and Rural Affairs, n.d.). Caregivers were emailed a unique secure link for remote completion of the Vineland-3 ahead of their child’s tele-assessment. On the day of evaluations, clinicians were sent unique secure links via email to complete the clinician survey for each eligible child on their clinic schedule. Clinicians also completed the TAP Rating Form and DSM-5 ASD Symptom Rating Scale. On the day following their child’s evaluation, caregivers received a secure link via email to complete the Caregiver Telemedicine Survey. Clinicians completing surveys for > 85% of scheduled evaluations were provided a $25 gift card for each month of participation. Caregivers were provided a $25 gift card for completed surveys.

Data analysis

Data analyses were performed using SAS version 9.4 (SAS Institute, Inc., Cary, NC, USA). Continuous variables are reported as means and standard deviations, and categorical variables are reported as absolute frequencies and percentages. Chi-square test or Fisher’s exact test were used to analyze comparisons among ASD diagnostic outcome for categorical variables (i.e. child sex, race, insurance type, MUA and rural status). Analysis of variance (ANOVA) was used to analyze comparisons among ASD diagnostic outcome (i.e. absent, present, unsure) for continuous variables (i.e. child age, TAP total score, and Vineland-3 standard scores. Pair-wise comparisons were analyzed using Fisher’s Protected Least Significant Differences to control the overall significance level at 5%. Multivariable logistic regression models were fitted to estimate the probability of ASD outcome (present vs unsure) with covariates of TAP total score, Vineland-3 standard scores, and DSM-5 ASD Symptom subdomain scores. Results from models were used to calculate the odds ratios (ORs) and 95% confidence intervals (CIs). Finally, sensitivity and specificity analyses were conducted to assess the performance of TAP categorical risk in discriminating ASD diagnostic outcome for all groups and again by age (i.e. younger/older groups). p values < .05 were considered statistically significant.

Community involvement

This study was led by a diverse interdisciplinary group of clinicians, researchers, and staff members including caregivers of individuals with neurodevelopmental disabilities (including ASD). All members of the study team provided input into study design, data collection and analysis, and interpretation of the findings.

Results

Of 335 children who received a telemedicine ASD evaluation, 55% (n = 185) were diagnosed with ASD (i.e. ASD Present) and 30% (n = 102) were not diagnosed with ASD (i.e. ASD Absent). In 14% (n = 48) of evaluations, the clinician could not determine whether the child met diagnostic criteria for ASD (i.e. ASD Unsure) based on information obtained during the evaluation. A significantly larger proportion (i.e. 88%; n = 42) of children with an ASD Unsure diagnostic outcome were referred for follow-up in person evaluation, χ²(2, N = 335) = 102.2, p < .00; additional next step recommendations for the ASD Unsure group included remote care navigation support with a social worker (n = 31; 65%) and remote clinical consultation (i.e. with a physician, psychologist, or speech-language pathologist; n = 2; 4%). Clinician diagnostic certainty varied across ASD diagnostic status χ² (2, N = 335) = 102.2, p < 0.001, with the highest ratings in the ASD Present group, with 94% of clinicians reporting that they were certain (i.e. Completely or Somewhat Certain), as compared to 88% in the ASD Absent and 53% in the ASD Unsure groups. There were no significant demographic differences (i.e. across age, sex, insurance status, race, ethnicity, family income, and MUA and rural designation variables) across ASD diagnostic outcome groups (all ps > 0.14; see Table 1). Overall, clinicians reported the presence of technology barriers in 15% (n = 48) of evaluations and that these challenges disrupted the clinical information obtained in 38% (n = 18) of those evaluations where barriers were present (see Table 1). Similarly, family/home set up barriers were reported in 20% (n = 63) of evaluations, with clinicians reporting that these barriers were disruptive to the clinical information obtained in 44% (n = 28) of those evaluations. There were no significant differences across diagnostic groups in the proportion of evaluations with technology or family/home set up barriers (ps > 0.24).

Associations between child clinical characteristics and diagnostic outcome

First, we examined associations between child clinical characteristics across diagnostic outcome groups (ASD Present, Absent, Unsure; see Table 2). There was a significant difference in the distribution of children scoring in the At-Risk range on the TAP across diagnostic outcome groups, χ² (2, N = 335) = 183.2, p < .001. All children (i.e. 100%; n = 185) who received a clinical diagnosis of ASD scored in the At-Risk range on the TAP (i.e. total scores ⩾ 11). Eighty-one percent (n = 39) of children with an “Unsure” ASD outcome scored in the At-Risk range while 29% (n = 29) of children with a confirmed non-ASD diagnosis scored in the At-Risk range. Similarly, a significant difference was found in TAP total scores across ASD diagnostic status groups χ² (2, N = 335) = 183.2, p < .001. Follow-up pairwise comparisons revealed that TAP total scores were significantly higher for ASD Present as compared to ASD Absent (t =-7.7; p < .001) and ASD Unsure (t = 4.7; p < .001); TAP total scores for ASD Unsure were significantly higher than ASD Absent (t =-3.0; p < .001).

Table 2.

Associations between child clinical characteristics and diagnostic outcome.

	ASD Diagnostic Outcome			P
	ASD PresentN = 185	ASD AbsentN = 102	ASD UnsureN = 48	P
TAP ASD risk, n (%)	185 (100)	29 (29)	39 (81)	<0.001
TAP Total Score	17.4 (2.4)	9.7 (1.7)	12.7 (2.7)	<0.001
^aVineland-3 SS
Communication	60.8 (18.7)	70.5 (15.1)	70.1 (14.4)	0.006
Daily living	70.7 (16.1)	78.7 (13.8)	79.0 (9.7)	0.005
Socialization	68.3 (12.5)	79.0 (15.0)	78.4 (14.2)	<0.001
Motor	80.4 (14.0)	81.6 (14.9)	80.4 (12.7)	0.904
ABC	67.5 (11.7)	75.3 (11.6)	78.4 (9.0)	<0.001
DSM-5 ASD Symptom Domain/Subdomain
Social Communication Total	14.4 (1.9)	6.4 (3.7)	10.4 (3.3)	<0.001
Social Reciprocity	4.9 (0.8)	2.1 (1.5)	3.6 (1.5)	<0.001
Nonverbal Social Behavior	5.1 (1.1)	2.1 (1.6)	3.7 (1.4)	<0.001
Social Relationships	4.4 (0.7)	2.3 (1.5)	3.2 (1.4)	<0.001
Restricted and Repetitive Behavior Total	9.7 (2.8)	5.1 (2.9)	6.4 (2.3)	<0.001
Stereotyped Speech, Motor, Object Use	3.3 (0.7)	1.5 (1.1)	2.5 (0.9)	<0.001
Nonfunctional Routines	1.6 (1.5)	0.9 (1.1)	1.0 (1.2)	<0.001
Atypical Sensory Behavior	3.6 (1.1)	2.2 (1.4)	2.5 (1.2)	<0.001
Restricted/Fixated Interests	1.3 (0.9)	0.5 (0.8)	0.4 (0.6)	<0.001

Data presented as means and standard deviation unless otherwise noted.

ASD: autism spectrum disorder; TELE-ASD-PEDS: TAP; SS: Standard Score; ABC: Adaptive Behavior Composite.

TAP ASD Risk refers to categorical TAP scores that are in the At-Risk range (i.e. TAP Total score ⩾ 11). ASD diagnostic status denotes clinician’s rating of ASD based on ASD telehealth evaluation.

Not all children had Vineland-3 data available (Total n = 135; ASD Present n = 66, ASD Absent n = 45; ASD Unsure n = 24).

There was a significant association between Vineland-3 domain and subdomain standard scores (except Vineland-3 Motor scores (p = 0.903)) and clinician-rated DSM-5 ASD symptom domains and subdomains (all ps < 0.005) with ASD Diagnostic Status group (see Table 2). Follow-up pairwise comparisons revealed significantly lower Vineland-3 domain and subdomain scores (i.e. poorer caregiver-reported adaptive skills) in the ASD Present as compared to ASD Absent group (all ps < 0.004) and ASD Unsure group (all ps < 0.02); there were no significant differences in adaptive skills between the ASD Absent and Unsure group (all ps > 0.85). However, significant differences among all three ASD Diagnostic Status groups on DSM-5 Social Communication Total and Subdomain (i.e. Social Reciprocity, Nonverbal Social Communication, and Social Relationships) scores were found. Specifically, these scores were higher (i.e. increased clinician-reported ASD symptoms) in the ASD Present as compared to the ASD Absent (all ps < 0.0001) and ASD Unsure (all ps < 0.0001) groups. The same pattern was found with higher scores in the ASD Unsure as compared to the ASD Absent (all ps < 0.0001) group. Similarly, follow-up comparisons of clinician-reported DSM-5 RRB Total scores revealed significant findings, with higher scores in ASD Present as compared to the Absent and Unsure groups, as well as ASD Unsure as compared to ASD Absent (all ps < 0.01). However, across DSM-5 RRB subdomains (Stereotyped Speech Motor and Object Use, Nonfunctional Routines, Atypical Sensory Behavior, and Restricted/Fixated Interests) scores were significantly greater for ASD Present as compared to ASD Absent (all ps < 0.01) and ASD Unsure (all ps < 0.01) groups. There were no significant differences between ASD Absent and ASD Unsure (all ps > 0.16), with the exception of the Stereotyped Speech, Motor, and Object Use subdomain (p < 0.0001).

Predictors of diagnostic outcome between ASD present and ASD unsure

Next, to determine specific predictors of diagnostic outcome between the ASD Present and Unsure groups, associations between clinical characteristics and diagnostic outcome were examined using univariable (i.e. unadjusted) and multivariable (i.e. adjusted) logistic regression models (see Table 3). All unadjusted variables were significant independent predictors of ASD diagnostic outcome (all p < 0.04) with the exception of DSM-5 Nonfunctional Routines Subdomain scores (p = 0.87). In the adjusted multivariable regression model, TAP Total score (p = 0.02) and DSM-5 RRB subdomains of Stereotyped Speech, Motor, Object use (p = 0.01) and Restricted/Fixated Interests (p = 0.03) remained significant predictors of outcome. More specifically, for every point increase in TAP total scores, a child was 1.7 (95% CI = 1.1–4.6) times more likely to be diagnosed with ASD than have an ASD Unsure outcome. Similarly, for every point increase in clinician-reported DSM-5 Stereotyped Speech, Motor, and Object Use and Restricted/Fixated Interests scores, children were 14.8 (95% CI = 1.9–116.7) and 6.0 (95% CI = 1.2–29.3) times more likely to be diagnosed with ASD, respectively. This analysis was replicated with the entire sample (i.e. without the Vineland-3 as this measure was not collected for all participants; see Table 4; Online Supplement). Findings were similar with TAP Total score (p < 0.001) and DSM-5 RRB subdomain of Restricted/Fixated Interests (p = 0.01) remaining significant predictors of ASD diagnostic outcome.

Table 3.

Association between child clinical characteristics and diagnostic outcome for ASD Present and ASD Unsure groups (N = 90).

Predictor	Unadjusted model		Adjusted model
Predictor	OR (95% CI)	p ^a	OR (95% CI)	p ^b
TAP Total Score	1.9 (1.5, 2.5)	<0.001	1.7 (1.1, 2.6)	0.016
Vineland-3 SS
Communication	1.0 (0.9, 1.0)	0.037	0.9 (0.8, 1.0)	0.224
Daily living	1.0 (0.9, 1.0)	0.025	1.1 (0.9, 1.2)	0.314
Socialization	0.9 (0.9, 1.0)	0.003	0.9 (0.8, 1.1)	0.215
DSM-5 ASD Symptom Domain
Social Reciprocity	2.4 (1.5, 4.0)	<0.001	0.7 (0.2, 2.3)	0.503
Nonverbal Social Behavior	3.3 (1.9, 5.5)	<0.001	1.2 (0.4, 3.2)	0.744
Social Relationships	2.9 (1.6, 5.0)	<0.001	1.1 (0.3, 3.3)	0.924
Stereotyped Speech, Motor, Object Use	8.2 (3.0, 22.5)	<0.001	14.8 (1.9, 116.7)	0.010
Nonfunctional Routines	1.0 (0.6, 1.5)	0.869	1.2 (0.4, 3.4)	0.791
Atypical Sensory Behavior	2.2 (1.3, 3.6)	0.003	1.0 (0.3, 3.1)	0.990
Restricted/Fixated Interests	3.5 (1.5, 7.9)	0.003	6.0 (1.2, 29.3)	0.026

p values represent significance of univariable^a and multivariable^b logistic regression analyses.

Analyses presented for s subsample (N = 90) with completed Vineland-3 scores.

ASD: autism spectrum disorder; TELE-ASD-PEDS: TAP; SS: Standard Score. TAP ASD Risk, Vineland-3 ABC, and DSM-5 Domain scores not included due to significant overlap with other variables in the model (i.e. they are categorical or summary scores of included variables).

Associations between child clinical characteristics and diagnostic outcome in younger versus older children

In order to examine how child age impacts ASD diagnostic outcome, age was dichotomized into two categories: younger (⩽36 months; n = 158) and older (⩾37 months; n = 177) children. This age distinction was made given that the TAP is recommended for use in children under 36 months of age (Wagner, Stone, et al., 2021). There were no differences between younger and older children across any child demographic factors including sex, insurance status, race, ethnicity, and MUA or rural status (all ps > 0.094; see Table 5). No significant difference was found in the distribution of ASD diagnostic outcomes (i.e. ASD Present, Absent, Unsure) between younger and older children (p = .290; see Table 5). We did, however, find a significant difference in the distribution of clinician-rated diagnostic certainty, with clinicians reporting that they were “Certain” more frequently in older children (i.e. 90% of evaluations) as compared to younger children (i.e. 82% of evaluations; χ²(1, N = 311) = 4.8, p = 0.028).

Table 5.

Demographic and clinical characteristics of child participants by age group.

	Dichotomized Child Age
	Younger(⩽36 months)N = 158	Older(⩾37 months)N = 177	p
Sex, n (%)			0.353
Male	118 (75)	125 (71)
Female	39 (25)	52 (29)
Primary Insurance Status, n (%)			0.209
Public (Medicaid)	89 (58)	103 (58)
Private	52 (34)	67 (38)
None	13 (8)	7 (4)
^aRace, n (%)			0.750
White	87 (67)	110 (73)
Black or African American	23 (18)	22 (15)
More than one race	9 (7)	11 (7)
Asian	8 (6)	6 (4)
American Indian /Alaskan Native	2 (2)	1 (1)
aEthnicity, n (%)			0.777
Non-Hispanic or Latinx	117 (89)	131 (90)
Hispanic or Latinx	14 (11)	14 (10)
aFamily Income, n (%)			0.805
Less than $40,000	37 (51)	30 (45)
$40,000–$49,999	6 (8)	5 (8)
$50,000–$59,999	7 (10)	4 (6)
$60,000–$74,999	8 (11)	9 (14)
$75,000 or more	15 (21)	18 (27)
Medically Underserved Area	110 (70)	135 (76)	0.170
Rural Designation	22 (14)	37 (21)	0.094
ASD Diagnostic Outcome			0.290
ASD Present	94 (59)	91 (51)
ASD Absent	45 (28)	57 (32)
ASD Unsure	19 (12)	29 (16)
Clinician diagnostic certainty, n (%)			0.028
Uncertain	27 (18)	16 (10)
Certain	120 (82)	148 (90)
TAP ASD risk, n(%)	126 (80)	127 (72)	0.089
TAP Total Score	15.0 (4.1)	13.8 (4.1)	0.011
bVineland-3 SS
Communication	65.4 (16.5)	65.9 (18.0)	0.891
Daily Living Skills	73.5 (15.9)	75.6 (14.2)	0.427
Socialization	73.3 (13.0)	73.9 (15.5)	0.815
Motor	84.5 (11.9)	78.7 (14.8)	0.019
ABC	70.7 (10.4)	71.8 (12.6)	0.600
DSM-5 ASD Symptom Domain/Subdomain
Social Communication Total	12.1 (4.5)	10.9 (4.5)	0.013
Social Reciprocity	4.1 (1.7)	3.7 (1.7)	0.022
Nonverbal Social Behavior	4.2 (1.9)	3.7 (1.9)	0.022
Social Relationships	3.8 (1.5)	3.4 (1.5)	0.046
Restricted and Repetitive Behavior Total	8.2 (3.7)	7.5 (3.2)	0.070
Stereotyped Speech, Motor, Object Use	2.6 (1.2)	2.6 (1.2)	0.890
Nonfunctional Routines	1.5 (1.5)	1.1 (1.2)	0.008
Atypical Sensory Behavior	3.1 (1.3)	2.9 (1.4)	0.088
Restricted/Fixated Interests	0.9 (1.0)	0.9 (0.9)	0.569

Data presented as means and standard deviation unless otherwise noted.

ASD: autism spectrum disorder; TELE-ASD-PEDS: TAP; SS: standard Score; ABC: Adaptive Behavior Composite.

Descriptive statistics calculated based on n = 154 (i.e. those who received evaluation and had a returned caregiver survey); data not available for those who did not return a caregiver survey.

Vineland-3 data available for n = 135.

When child clinical characteristics were examined between younger and older children, a significant difference was found in TAP total scores (F(1, 333) = 6.5, p = 0.011), with higher scores in younger as compared to older children. Caregiver ratings of adaptive skills were not significantly different across age groups (with the exception of Motor Skills, F(1, 132) = 5.6, p = 0.019). A significant difference between younger and older children was also found in clinician-rated DSM-5 ASD Social Communication Total scores, F(1, 323) = 6.2, p = 0.013), with younger (⩽36 months: M = 12.1, SD = 4.5) children showing a higher number of symptoms present as compared to older (⩾37 months: M = 10.9, SD = 4.5) children (see Table 5). No other significant differences were found across DSM-5 ASD RRB total symptoms or subscales (all ps > 0.070) with the exception of Nonfunctional Routines, where scores were higher in younger (⩽36 months: M = 1.5, SD = 1.5) as compared to older (⩾37 months: M = 1.1, SD = 4.5) children, F(1, 323) = 7.1, p = 0.008.

We then examined the association between age and child clinical characteristics in those with an ASD Unsure outcome. There were no significant differences between younger and older children with regard to TAP categorical risk or total scores or Vineland-3 scores (all ps ⩾ .101; see Table 6), with a the exception of a marginally significant difference in Vineland-3 Motor scores (F(1, 22) = 4.5, p = 0.045. Similar to findings above (i.e. across all diagnostic groups), a significant difference between younger and older children was found in clinician-rated DSM-5 ASD Social Communication Total scores (F(1, 46) = 7.4, p = 0.01), with younger (⩽36 months: M = 11.9, SD = 3.0) children showing a higher number of symptoms present as compared to older (⩾37 months: M = 9.4, SD = 3.2) children. No significant differences were found across DSM-5 Restricted and Repetitive Behaviors total symptoms or subdomains (all ps > 0.364).

Table 6.

Association between child clinical characteristics and age for ASD Unsure outcome.

	Dichotomized Child Age
	Younger(⩽36 months)N = 19	Older(⩾37 months)N = 29	p
TELE-ASD-PEDS ASD risk, n (%)	16 (84)	23 (79)	>0.999
TELE-ASD-PEDS total score	13.5 (3.0)	12.2 (2.4)	0.118
^aVineland-3 SS
Communication	73.1 (12.2)	67.9 (15.9)	0.399
Daily Living Skills	82.9 (8.3)	76.3 (10.0)	0.101
Socialization	79.7 (12.2)	77.4 (15.8)	0.708
Motor	86.5 (13.2)	76.1 (10.9)	0.045
ABC	76.8 (7.7)	73.4 (9.9)	0.377
DSM-5 ASD Symptom Domain/Subdomain
Social Communication Total	11.9 (3.0)	9.4 (3.2)	0.009
Social Reciprocity	4.2 (1.5)	3.2 (1.4)	0.030
Nonverbal Social Behavior	4.2 (1.3)	3.3 (1.4)	0.054
Social Relationships	3.6 (1.1)	2.9 (1.5)	0.074
Restricted and Repetitive Behavior Total	6.4 (2.7)	6.3 (2.0)	0.873
Stereotyped Speech, Motor, Object Use	2.4 (1.1)	2.6 (0.9)	0.517
Nonfunctional Routines	1.2 (1.2)	0.9 (1.1)	0.364
Atypical Sensory Behavior	2.5 (1.2)	2.5 (1.2)	0.979
Restricted/Fixated Interests	0.4 (0.6)	0.4 (0.6)	0.952

Data presented as means and standard deviation unless otherwise noted.

ASD: autism spectrum disorder; TELE-ASD-PEDS ASD Risk: TELE-ASD-PEDS total score in At-Risk range; SS: standard Score; ABC: Adaptive Behavior Composite.

Vineland-3 data available for n = 24 (Younger, n = 10; Older, n = 14).

Sensitivity and specificity of TAP categorical risk outcomes

We examined the performance of TAP Categorical Risk in differentiating ASD diagnostic outcome based on clinician best-estimate diagnosis. TAP outcome agreed with clinician best estimate diagnosis in 90% of all cases, with sensitivity of 1.0 and specificity of .72 (see Table 7). For children ⩽ 36 months, TAP outcome agreed with clinician best estimate diagnosis in 89% of all cases (i.e. based on the clinician’s best-estimate diagnosis), with sensitivity of 1.0 and specificity of .64. Finally, TAP outcome agreed with clinician best estimate diagnosis in 91% of all cases in children ⩾ 37 months, with sensitivity of 1.0 and specificity of .77 in this age group.

Table 7.

TAP categorical outcome by ASD diagnostic outcome.

ASD Diagnostic Outcome
All Children Evaluated
	ASD Present (n = 185)	ASD Absent (n = 102)
TAP ASD Risk (n = 214)	185	29
TAP non-ASD Risk (n = 73)	0	73
Younger Children (⩽ 36 months)
	ASD Present (n = 94)	ASD Absent (n = 45)
TAP ASD Risk (n = 110)	94	16
TAP non-ASD Risk (n = 29)	0	29
Older Children (⩾ 37 months)
	ASD Present (n = 91)	ASD Absent (n = 57)
TAP ASD Risk (n = 104)	91	13
TAP non-ASD Risk (n = 44)	0	44

Discussion

Although the COVID-19 pandemic created significant challenges for healthcare service delivery, it has allowed for a rapid acceleration of innovative research, including the study of real-world clinical implementation of ASD tele-assessment (Alfuraydan et al., 2020; Ellison et al., 2021; Stavropoulos et al., 2022; Valentine et al., 2021). Novel ASD diagnostic tools designed for remote administration have been developed and implemented during COVID-19 with promising success (Berger et al., 2022; Corona et al., 2020; Dow et al., 2021; Jang et al., 2021; Wagner et al., 2020). For example, recent studies have demonstrated positive results with regard to practice change (Jang et al., 2022; Wagner, Weitlauf, et al., 2021) and clinician (Wagner et al., 2020) and caregiver (Corona et al., 2020) acceptance of ASD tele-assessment using the TAP, a novel caregiver-mediated assessment tool. The objective of the present study was to examine clinical characteristics and diagnostic outcome for young children receiving ASD tele-assessment during the COVID-19 pandemic. Overall, findings suggest that a definitive ASD diagnosis (i.e. ASD present vs absent) can be determined for most young children evaluated using standard tele-assessment procedures. Categorical risk scores on the TAP, measures of adaptive functioning, and DSM-5 symptoms of ASD were strongly associated with ASD diagnostic outcome, and a specific pattern of restricted and repetitive behavior was predictive of diagnostic outcome. Finally, our results suggest that the TAP is feasible and effective for evaluation of at-risk children above 36 months of age with delayed language, perhaps lending initial support that children may most effectively be triaged into tele-assessment evaluations using the TAP based on language-level of the child and not a specific age criterion.

In our clinical implementation of ASD tele-assessment employing the TAP, 55% of children were diagnosed with ASD, 30% of children received a non-ASD diagnosis, and in 14% of evaluations, the clinician could not make a definitive diagnostic decision regarding the presence or absence of ASD. Our diagnostic outcomes vary from recent studies of TAP implementation. For example, in one clinic-referred sample, Wagner and colleagues (2020) reported that 71% of children were diagnosed with ASD while in a multi-site study these authors found that 40% of children received a diagnosis of ASD (Wagner, Weitlauf, et al., 2021). However, across both studies, rates of ASD Uncertain outcomes (i.e. Wagner et al., (2020): 18% ASD Suspected or Uncertain; Wagner, Weitlauf, et al. (2021): 15% ASD Suspected or Uncertain) were comparable to our findings (i.e. 14%), suggesting that despite probable variation in referral characteristics of children across studies, tele-assessment using the TAP is effective for a large majority of young children. As expected and in accord with previous research (Wagner et al., 2020), clinician ratings of diagnostic certainty were associated with diagnostic outcome (i.e. highest percentage of Certain ratings in those with ASD Present outcome and lowest percentage of Certain ratings in those with ASD Unsure outcome). Similar to previous research (McNally Keehn et al., 2021), we also found that demographic characteristics (i.e. child age, sex, insurance status, race and ethnicity, family income, and residing in a medically underserved or rural region) and the presence of challenges with remote technology or the home environment did not vary by diagnostic outcome, providing evidence that this modality may be an equitable and feasible method to serve many children and families.

Child clinical characteristics and diagnostic outcome

Similar to previous findings (Wagner et al., 2020), TAP categorical risk outcome and total scores were significantly related to ASD diagnostic outcome. Specifically, for all children with an ASD Present outcome, TAP scores were in the at-risk range and significantly higher than those with ASD Absent and Unsure outcomes. Not surprisingly, TAP scores (i.e. both categorical risk and total scores) were also higher for ASD Unsure as compared to ASD Absent outcomes. To understand the child phenotypic characteristics that may be associated with a definitive diagnostic determination, we examined the relationship between adaptive functioning and quantitative ratings of DSM-5 ASD symptoms with diagnostic outcome. Similar to TAP scores, caregiver-report ratings of child communication, daily living, socialization, and total (but not motor) adaptive skills, as measured by the Vineland-3, and clinician-rated measures of social communication and restricted and repetitive behaviors were significantly associated with ASD diagnostic outcome. Adaptive skills were rated lower for children in the ASD Present as compared to Absent groups, yet there was no difference among the ASD Absent and Unsure group suggesting that overall functioning, measured by self-help skills, did not differentiate these groups. Ratings of DSM-5 social communication were higher in the ASD Present as compared to Absent, as well as ASD Unsure as compared to Absent groups. This pattern was not replicated with RRB symptoms, where no significant differences were found between ASD Absent and Unsure groups with the exception of stereotyped speech, motor, and object use behaviors. As expected, all child clinical characteristics (with the exception of nonfunctional routines, likely due to low rates of this behavior across all groups) were independent predictors of outcome between ASD Present and Unsure outcomes. However, when all variables were considered together (i.e. multivariable model), TAP total scores and RRB symptom subdomains of stereotyped speech, motor, object use and restricted/fixated interests predicted diagnostic outcome between ASD Present and Unsure.

To our knowledge this is the first study to report on the relationships among child clinical characteristics and tele-assessment diagnostic outcome. Given that measures of adaptive functioning did not predict diagnostic outcome above and beyond ASD symptoms, our findings provide preliminary evidence that tele-assessment and, in particular use of the TAP, is effective across developmental level in young children at risk for ASD. Further, the similar pattern between TAP scores and ratings of DSM-5 ASD symptoms across diagnostic groups provides emerging evidence for convergent validity of the TAP. Finally, when all clinical data is combined, clinicians appear to rely on TAP scores and the presence of specific RRBs (i.e. stereotyped speech, motor, and object use and restricted/fixated interests) to differentiate those with a definitive diagnosis of ASD during tele-assessment. Past research has suggested that two distinct and stable subtypes of RRBs, specifically repetitive sensory and motor behaviors and insistence on sameness behaviors (i.e. including rigidity, routines, and restricted interests) emerge early and develop independently in ASD (Uljarević et al., 2017). Furthermore, distinct patterns of frequent and intense RRBs differentiate those with ASD from typically developing children and those with other neurodevelopmental disabilities from an early age (Morgan et al., 2008; Richler et al., 2007). Thus, our findings regarding the predictive nature of these behaviors in tele-assessment of ASD align well with research regarding the salience of RRBs in the differential diagnosis of ASD.

Child age and diagnostic outcome

In the early weeks and months of the COVID-19 pandemic, our clinical team recognized a specific need to continue providing diagnostic services for older children referred for ASD evaluation. Given our clinical experience with using level of expressive language in ADOS-2 (C. Lord et al., 2012) module selection, we applied these same conceptual guidelines to use of the TAP. We extended the age range for TAP tele-assessment above 36 months if the child’s current expressive language level ranged from no functional word use to occasional simple 2-word phrases. We found no differences in the distribution of ASD diagnostic outcome (i.e. ASD Present, Absent, and Unsure) between younger (⩽ 36 months) and older (i.e. ⩾ 37 months) children, suggesting relative equivalence in utility of tele-assessment using the TAP across toddler and preschool aged children who met this language criteria. In fact, while overall diagnostic certainty was high across all children evaluated, certainty ratings were higher in the older age group, providing further support for use of the TAP beyond the 36 month age cutoff. Importantly, we did not find differences in adaptive functioning (except motor skills) across age groups, but we did find that younger children evaluated demonstrated a higher level of DSM-5 social communication symptoms. This general pattern of findings was then replicated in children with an ASD Unsure diagnostic outcome.

Taken together, clinicians showed a similar pattern of diagnostic decision making across younger and older children. Given a higher frequency of clinicians rating their clinical confidence as “Certain” in older children, it may be that clinicians report greater comfort making a diagnosis in older children, at least within the context of tele-assessment procedures. This hypothesis is supported by the findings of Lord et al. (2012) in which clinicians were more likely to make a definitive diagnostic decision about ASD with repeated evaluations (i.e. over time and as children got older), despite high scores on the ADOS-2 at each assessment time point. Our findings cannot be explained merely by the idea that older children (i.e. across all diagnostic groups, as well as in the Unsure group specifically) presented with greater developmental and ASD symptom impairment making ASD differential diagnosis more straightforward. While some differences existed in the clinical profile of younger versus older children (i.e. greater social communication symptoms in younger children), we find promising support for the clinical utility of the TAP as part of an ASD tele-assessment for children older than 36 months who meet language impairment criteria.

Sensitivity and specificity of TAP categorical risk outcomes

TAP Categorical Risk accurately differentiated ASD outcome based on clinician best-estimate diagnosis (i.e. using data from all evaluation sources) in 90% of all children evaluated. Sensitivity of TAP Categorical Risk to identify those with ASD was 100%. Specificity, however, ranged from 64%–77% based on age group, with poorest specificity in children < 36 months of age. This finding is surprising given that the TAP was designed for evaluation of children < 36 months, again lending support for language- versus age-based use of the tool. While children with TAP scores in the ASD Risk range were all diagnosed with ASD, clinicians overrode TAP risk outcomes in 28% of children for which ASD was ruled out. Our findings are similar to those reported in Corona et al. (2021), in which an overall sensitivity of .90 and specificity of .78 was found.

Strengths, limitations & future directions

The present study has several strengths that add to the growing body of promising literature on implementation of ASD tele-assessment, and specifically use of the TAP as part of a standard diagnostic protocol. To our knowledge, this is the first study to examine the relationship between various child clinical characteristics and ability to determine ASD diagnostic outcome via tele-assessment, as well as utility of the TAP in children over 36 months of age. Given that our study took place in a real-world clinical setting with clinicians of different backgrounds and levels of training (i.e. psychologists and pediatricians) and outside the setting in which the TAP was developed, our findings are likely to generalize to other clinical settings conducting ASD tele-assessment. Furthermore, our large sample of children were diverse (with somewhat higher proportion of non-White and Latinx participants than the state census data), especially regarding family socioeconomic status, use of public health insurance, and residing in MUAs. Previous research has highlighted that telemedicine may not address health service access inequities (Weiner et al., 2021); while we cannot rule out this possibility, the present study and related work has demonstrated nonsignificant associations between demographic factors and tele-assessment satisfaction and diagnostic outcome (McNally Keehn et al., 2021; Reisinger et al., 2022).

It is also important to note several caveats to the present study that should be considered when interpreting the findings. First, while research is emerging on the utility and acceptability of the TAP, it is not yet a validated assessment tool. Similarly, our DSM-5 checklist, used to obtain a quantitative measure of DSM-5 ASD symptoms across all evaluation sources, is not validated. We did not follow up to examine diagnostic accuracy or agreement with traditional, comprehensive in-person evaluation. While clinicians in the current study were experienced in ASD diagnostic evaluation, employed the TAP as part of a more comprehensive assessment procedure, and reported high ratings of diagnostic certainty, we cannot determine if ASD tele-assessment is comparable to in-person evaluation methods. Head-to-head comparisons between streamlined remote evaluations models (including those that employ the TAP) and comprehensive gold-standard ASD evaluations (including cognitive/adaptive evaluation) will be needed to more rigorously assess accuracy of ASD diagnosis.

Our evaluation procedure represented a streamlined model designed to promote rapid access while balancing clinical quality (see Zwaigenbaum & Warren, 2020 for discussion of the benefits of streamlined models). In order to reduce evaluation duration while still obtaining a proxy of developmental status, we utilized the Parent/Caregiver Form of the Vineland-3. While there is moderate correlation between the Vineland-3 Interview and Parent/Caregiver Form (Sparrow et al., 2016), it cannot be considered a substitute for the Interview Form or an objective cognitive assessment, both of which are important for differential diagnosis (i.e. between developmental delay/intellectual disability and ASD) as well as prognosis and intervention planning (Lord et al., 2021). Furthermore, both TAP scores and the DSM-5 checklist were used to inform clinical diagnosis (as is common clinical practice), and thus were not independent observations (i.e. in the multi-variate regression models). As expected, when conducting clinical research at a time of rapid transition and global uncertainty, not all children included in the study had complete data. Demographic and Vineland-3 data were not available for every participant. Independent replication of our findings regarding clinical profiles predictive of tele-assessment diagnostic outcome is needed.

Along with recent studies documenting the feasibility, utility, and acceptability of ASD tele-assessment and TAP, the present study adds to an emerging literature documenting the promise of this evaluation method. Additional research must replicate and expand examination of telemedicine outcomes, specifically focusing on questions related to identifying for whom this method is appropriate and effective. Recent research has documented high rates of clinician and caregiver satisfaction with use of the TAP as a supportive clinical tool in tele-assessment (Reisinger et al., 2022). However, as Corona and colleagues (2020) point out, it will be critical to involve stakeholders in further design, adaptation, and evaluation of ASD tele-assessment procedures. Furthermore, and perhaps most importantly, systematic research must focus on how telemedicine may be deployed to address diagnostic inequities in populations and geographic locations most vulnerable to health care access problems.

In conclusion, despite its many challenges, the COVID-19 pandemic has allowed for a rapid acceleration of innovative research on real-world clinical implementation of ASD tele-assessment. This study examined clinical characteristics and diagnostic outcomes of young children receiving ASD tele-assessment in a large interdisciplinary academic medical center during the COVID-19 pandemic. Overall, our findings add to an increasing body of evidence supporting use of tele-assessment procedures, including caregiver-mediated remote observational assessment tools, for ASD diagnostic evaluation in young children. While TAP scores and measures of adaptive functioning and DSM-5 ASD symptoms are highly associated with ASD diagnostic outcome, a specific pattern of restricted and repetitive behaviors appears to best predict the ability to determine differential ASD diagnosis. Finally, our results suggest that the TAP is useful for evaluation of at-risk children above 36 months of age with delayed language. Our findings add to an increasing body of research supporting use of tele-assessment, and specifically the TAP, for diagnosis of young children referred for ASD evaluation. Deploying telemedicine assessment procedures is one avenue that holds potential for addressing ASD diagnostic inequities, though further research in both laboratory and real-world settings is needed.

Supplemental Material

sj-docx-1-aut-10.1177_13623613221138642 – Supplemental material for Tele-assessment of young children referred for autism spectrum disorder evaluation during COVID-19: Associations among clinical characteristics and diagnostic outcome

Supplemental material, sj-docx-1-aut-10.1177_13623613221138642 for Tele-assessment of young children referred for autism spectrum disorder evaluation during COVID-19: Associations among clinical characteristics and diagnostic outcome by Rebecca McNally Keehn, Brett Enneking, Tybytha Ryan, Cristina James, Qing Tang, Audra Blewitt, Angela Tomlin, Laura Corona and Liliana Wagner in Autism

Footnotes

Acknowledgements

We are most grateful to the children and families who participated in both clinical care and research described in this manuscript, as well as to the many clinicians in the Division of Developmental Medicine who contributed data to this effort. We also wish to thank Angela Paxton, Leah Bode, Victoria Bozinovski, Margo Ramaker, Caroline Fitterling, and Alyssa Jones for their contributions to this research. Audra Blewitt was a fellow in the Leadership Education in Neurodevelopmental Disabilities (LEND) program in the Department of Pediatrics at IUSM during the time when this research was conducted.

Declaration of conflicting interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: All authors have made substantial intellectual contributions to this manuscript. There are no biomedical financial interests or conflicts of interest to disclose.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the Robert & Helen Haddad Family Foundation and Riley Children’s Foundation (award to RMK); and the Health Resources Services Administration [T73MC00015-27-01].

ORCID iDs

Rebecca McNally Keehn

Cristina James

Laura Corona

Supplemental material

Supplemental material for this article is available online.

References

Alfuraydan

Croxall

Hurt

Kerr

Brophy

(2020). Use of telehealth for facilitating the diagnostic assessment of Autism Spectrum Disorder (ASD): A scoping review. PLOS ONE, 15(7), e0236415. https://doi.org/10.1371/journal.pone.0236415

American Psychiatric Association. (Ed.) (2013). Diagnostic and statistical manual of mental disorders (5th ed.).

Berger

N. I.

Wainer

A. L.

Kuhn

Bearss

Attar

Carter

A. S.

Ibanez

L. V.

Ingersoll

B. R.

Neiderman

Scott

(2022). Characterizing available tools for synchronous virtual assessment of toddlers with suspected autism spectrum disorder: A brief report. Journal of Autism and Developmental Disorders, 52(1), 423–434.

Boxer

R. J.

Ellimoottil

(2019). Advantages and utilization of telemedicine. mHealth, 5, 12.

Committee on Pediatric Workforce. (2015). The use of telemedicine to address access and physician workforce shortages. Pediatrics, 136(1), 202–209.

Corona

L. L.

Wagner

Wade

Weitlauf

A. S.

Hine

Nicholson

Stone

Vehorn

Warren

(2021). Toward novel tools for autism identification: Fusing computational and clinical expertise. Journal of Autism and Developmental Disorders, 51(11), 4003–4012.

Corona

L. L.

Weitlauf

A. S.

Hine

Berman

Miceli

Nicholson

Stone

Broderick

Francis

Juárez

A. P.

Vehorn

Wagner

Warren

(2020). Parent perceptions of caregiver-mediated telemedicine tools for assessing autism risk in toddlers. Journal of Autism and Developmental Disorders, 51, 476–486. https://doi.org/10.1007/s10803-020-04554-9

Dow

Holbrook

Toolan

McDonald

Sterrett

Rosen

Kim

S. H.

Lord

(2021). The Brief Observation of Symptoms of Autism (BOSA): Development of a new adapted assessment measure for remote telehealth administration through COVID-19 and beyond. Journal of Autism and Developmental Disorders, 52, 5383–5394.

Drahota

Sadler

Hippensteel

Ingersoll

Bishop

(2020). Service deserts and service oases: Utilizing geographic information systems to evaluate service availability for individuals with autism spectrum disorder. Autism, 24(8), 2008–2020.

10.

Ellison

K. S.

Guidry

Picou

Adenuga

Davis

T. E.

(2021). Telehealth and autism prior to and in the age of COVID-19: A systematic and critical review of the last decade. Clinical Child and Family Psychology Review, 24(3), 599–630.

11.

Health Resources and Services Administration. (n.d.). MUA Find. https://data.hrsa.gov/tools/shortage-area/mua-find

12.

Huerta

Lord

(2012). Diagnostic evaluation of autism spectrum disorders. Pediatric Clinics of North America, 59(1), 103.

13.

Hyman

S. L.

Levy

S. E.

Myers

S. M.

(2020). Identification, evaluation, and management of children with autism spectrum disorder. Pediatrics, 145(1), e20193447. https://doi.org/10.1542/peds.2019-3447

14.

Indiana Office of Community and Rural Affairs. (n.d.). Rural Indiana Stats: Geographic Locations. https://pcrd.purdue.edu/ruralindianastats/geographic-classifications.php

15.

Jang

White

S. P.

Esler

A. N.

Kim

S. H.

Klaiman

Megerian

J. T.

Morse

Nadler

Kanne

S. M.

(2021). Diagnostic evaluations of autism spectrum disorder during the COVID-19 pandemic. Journal of Autism and Developmental Disorders, 52, 962–973.

16.

Jang

White

S. P.

Esler

A. N.

Kim

S. H.

Klaiman

Megerian

J. T.

Morse

Nadler

Kanne

S. M.

(2022). Diagnostic evaluations of autism spectrum disorder during the COVID-19 pandemic. Journal of Autism and Developmental Disorders, 52(2), 962–973.

17.

Juárez

A. P.

Weitlauf

A. S.

Nicholson

Pasternak

Broderick

Hine

Stainbrook

J. A.

Warren

(2018). Early identification of ASD through telemedicine: Potential value for underserved populations. Journal of Autism and Developmental Disorders, 48(8), 2601–2610. https://doi.org/10.1007/s10803-018-3524-y

18.

Kanne

S. M.

Carpenter

L. A.

Warren

(2018). Screening in toddlers and preschoolers at risk for autism spectrum disorder: Evaluating a novel mobile-health screening tool. Autism Research, 11(7), 1038–1049.

19.

Lord

Charman

Havdahl

Carbone

Anagnostou

Boyd

Carr

de Vries

P. J.

Dissanayake

Divan

(2021). The Lancet Commission on the future of care and clinical research in autism. The Lancet, 399(10321), 271–334.

20.

Lord

Rutter

Dilavore

Risi

Gotham

Bishop

(2012). Autism diagnostic observation schedule, second edition (ADOS-2). Western Psychological Services.

21.

Matthews

N. L.

Skepnek

Mammen

M. A.

James

J. S.

Malligo

Lyon

Mitchell

Kiefer

S. L.

Smith

C. J.

(2021). Feasibility and acceptability of a telehealth model for autism diagnostic evaluations in children, adolescents, and adults. Autism Research, 14(12), 2564–2579.

22.

McNally Keehn

Enneking

James

Tang

Rouse

Hines

Raches

Etling

. (2021). Telehealth evaluation of pediatric neurodevelopmental disabilities during the COVID-19 pandemic: Clinician and caregiver perspectives. Journal of Developmental and Behavioral Pediatrics, 43, 262–272. https://doi.org/10.1097/dbp.0000000000001043

23.

Morgan

Wetherby

A. M.

Barber

(2008). Repetitive and stereotyped movements in children with autism spectrum disorders late in the second year of life. Journal of Child Psychology and Psychiatry, 49(8), 826–837.

24.

Murphy

M. A.

Ruble

L. A.

(2012). A comparative study of rurality and urbanicity on access to and satisfaction with services for children with autism spectrum disorders. Rural Special Education Quarterly, 31(3), 3–11.

25.

Nazneen

Rozga

Smith

C. J.

Oberleitner

Abowd

G. D.

Arriaga

R. I.

(2015). A novel system for supporting autism diagnosis using home videos: Iterative development and evaluation of system design. JMIR mHealth uHealth, 3(2), e68. https://doi.org/10.2196/mhealth.4393

26.

Reisinger

D. L.

Hines

Raches

Tang

James

McNally Keehn

(2022). Provider and caregiver satisfaction with telehealth evaluation of autism spectrum disorder in young children during the COVID-19 pandemic. Journal of Autism and Developmental Disorders, 52, 5099–5113.

27.

Richler

Bishop

S. L.

Kleinke

J. R.

Lord

(2007). Restricted and repetitive behaviors in young children with autism spectrum disorders. Journal of Autism and Developmental Disorders, 37(1), 73–85.

28.

Sparrow

S. S.

Cicchetti

D. V.

Saulnier

C. A.

(2016). Vineland adaptive behavior scales (3rd ed.). Pearson.

29.

Stavropoulos

K.-M.

Bolourian

Blacher

(2022). A scoping review of telehealth diagnosis of autism spectrum disorder. PLOS ONE, 17(2), e0263062.

30.

Sutantio

J. D.

Pusponegoro

H. D.

Sekartini

(2020). Validity of telemedicine for diagnosing autism spectrum disorder: Protocol-guided video recording evaluation. Telemedicine and e-Health, 27, 427–431. https://doi.org/10.1089/tmj.2020.0035

31.

Thomas

Zahorodny

Peng

Kim

Jani

Halperin

Brimacombe

(2012). The association of autism diagnosis with socioeconomic status. Autism, 16(2), 201–213.

32.

Uljarević

Arnott

Carrington

S. J.

Meins

Fernyhough

McConachie

Le Couteur

Leekam

S. R.

(2017). Development of restricted and repetitive behaviors from 15 to 77 months: Stability of two distinct subtypes? Developmental Psychology, 53(10), 1859.

33.

United States Census Bureau. (n.d.). QuickFacts Indiana. https://www.census.gov/quickfacts/IN

34.

Valentine

A. Z.

Hall

S. S.

Young

Brown

B. J.

Groom

M. J.

Hollis

Hall

C. L.

(2021). Implementation of telehealth services to assess, monitor, and treat neurodevelopmental disorders: Systematic review. Journal of Medical Internet Research, 23(1), e22619.

35.

Wagner

Corona

L. L.

Weitlauf

A. S.

Marsh

K. L.

Berman

A. F.

Broderick

N. A.

Francis

Hine

Nicholson

Stone

(2020). Use of the TELE-ASD-PEDS for autism evaluations in response to COVID-19: Preliminary outcomes and clinician acceptability. Journal of Autism and Developmental Disorders, 51, 3063–3072.

36.

Wagner

Weitlauf

A. S.

Hine

Corona

L. L.

Berman

A. F.

Nicholson

Allen

Black

Warren

(2021). Transitioning to telemedicine during COVID-19: Impact on perceptions and use of telemedicine procedures for the diagnosis of autism in toddlers. Journal of Autism and Developmental Disorders, 52, 2247–2257.

37.

Weiner

J. P.

Bandeian

Hatef

Lans

Liu

Lemke

K. W.

(2021). In-person and telehealth ambulatory contacts and costs in a large US insured cohort before and during the COVID-19 pandemic. JAMA Network Open, 4(3), e212618.

38.

Zwaigenbaum

Warren

(2020). Commentary: embracing innovation is necessary to improve assessment and care for individuals with ASD: A reflection on Kanne and Bishop (2020). Journal of Child Psychology and Psychiatry, 62, 143–145.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB