The implementation of the screening tool for autism in toddlers in Part C early intervention programs: An 18-month follow-up

Abstract

This study examined the feasibility of implementing the Screening Tool for Autism in Toddlers, an interactive Level-2 screen for autism spectrum disorder, within Part C Early Intervention settings. Participants included 69 Early Intervention providers (M age = 43.3 years, 93.7% females, 92.4% Whites) from nine programs who attended a one-day Screening Tool for Autism in Toddlers training workshop. Half of the providers reported using the Screening Tool for Autism in Toddlers, and reported it to be feasible and effective. Regardless of Screening Tool for Autism in Toddlers use, providers reported increased knowledge about recognizing the early signs of autism spectrum disorder following the workshop. Provider-reported self-efficacy regarding skills related to autism spectrum disorder screening increased significantly from baseline to the 18-month follow-up. Providers also described the facilitators (e.g. promotes communication with families) and barriers (e.g. certification process), that influenced their adoption. Results highlight the potential use of the Screening Tool for Autism in Toddlers within Early Intervention settings to identify autism spectrum disorder, and suggest an implementation model in which specific providers serve as a screening “point-person,” rather than expecting it to be used by all providers. Future research should aim to identify characteristics of agencies or providers that might facilitate Screening Tool for Autism in Toddlers use, as well as specific implementation plans and strategies that might promote long-term sustainability of Level-2 screening practices.

This study was registered on ClinicalTrials.gov before the time of the first study enrollee. Registration number: NCT02409303; URL: https://clinicaltrials.gov/ct2/show/NCT02409303

Lay abstract

The early detection of autism spectrum disorder can lead to access to autism spectrum disorder-specific services that have been shown to have a large impact on a child’s overall development. Although a stable diagnosis of autism spectrum disorder can be made by age 2 years, most children are not diagnosed until much later. To address this issue, this study examined the effectiveness of training Part C Early Intervention providers to use an interactive autism spectrum disorder screening tool, the Screening Tool for Autism in Toddlers. Sixty-nine providers attended a 1-day training workshop on the use of the Screening Tool for Autism in Toddlers. After the workshop, providers reported increased knowledge about recognizing the early signs of autism spectrum disorder, and about 45% of the providers reported using the Screening Tool for Autism in Toddlers with families in their caseloads 18 months after the training. These results suggest that the Screening Tool for Autism in Toddlers is feasible for use within Early Intervention settings. In addition, they suggest that specific providers might serve as a screening “point-person,” rather than expecting the Screening Tool for Autism in Toddlers to be used by all providers. Future research should aim to identify specific characteristics of agencies or providers that might be best suited for using the Screening Tool for Autism in Toddlers.

Keywords

autism implementation screening

Early identification of autism spectrum disorder (ASD), through periodic developmental monitoring and screening in infancy and toddlerhood, allows children to have access to ASD-specific behavioral interventions that have been shown to improve long-term developmental outcomes (Zwaigenbaum et al., 2015). Although there is evidence to support the reliable diagnosis of ASD by age 2 years and findings that parents often report concerns by 18 months of age, studies have shown that most children are not diagnosed with ASD or enrolled in ASD-specific services until much later (Baio et al., 2018; Rosenberg et al., 2011). National parent surveys have demonstrated that fewer than 50% of children with ASD are identified before the age of 3 years, and that one-third to one-half of the cases are identified after the age of 6 years (Sheldrick et al., 2019). Furthermore, data from the Centers for Disease Control indicate that the age of diagnosis has not changed over the past two decades, remaining, on average, at 4 years (Maenner et al., 2020). This delay to diagnosis prevents children from accessing early behavioral interventions that specifically target symptoms of ASD, which have been shown to have superior effects than less focused, or more generic, community-based treatments (Dawson et al., 2010).

Screening in primary care

There has been a rise in research on ASD screening in the last two decades since the American Academy of Pediatrics (AAP) published its recommendation for universal ASD screening practices (AAP, 2000; Johnson & Myers, 2007; Murray & Barton, 2021). Much of the focus on screening for ASD has been on its use in primary care or pediatric settings, where Level-1 ASD screening tools (i.e. measures designed for use with the general population) are used most commonly. Although physicians can have a wide reach and make a large impact in the effort for universal screening when incorporated into regular practice, doing so is difficult due to constraints on time during well-child visits and the diverse early presentation of ASD. Therefore, the subsequent process for making a referral for additional assessment or specialized treatment following positive screens is not always straightforward.

The AAP recommends that screenings be conducted consistently and regularly during infancy and toddlerhood; however, pediatric providers are not always able to carefully monitor the emergence of ASD symptoms that can occur during this phase of rapid development. Barriers may include overburdened systems, time constraints, and minimal insurance reimbursement (King et al., 2010; Mazurek et al., 2020; Ozonoff et al., 2015). Furthermore, physicians may not make referrals for ASD-specific evaluations or treatment if a child has a positive screen due to insufficient ASD specialists and long waitlists for their services. Primary care physicians (PCPs) will often refer to Part C Early Intervention (EI) instead, as a next step for families, also as recommended by the AAP (Hyman et al., 2020; Keil et al., 2014; Wallis et al., 2020). Commonly, PCPs also consider the availability, or lack thereof, of services when deciding whether or not to refer, hoping not to encourage parents to “hurry up and wait” for further ASD evaluation and EI services (Sheldrick & Garfinkel, 2017). While referral to Part C EI can often lead to services such as speech therapy or occupational therapy, it may not lead to ASD-focused intervention, which has demonstrated more robust outcomes in children with early social-communication concerns (Zwaigenbaum & Macguire, 2019).

While work can undoubtedly be done to improve the Level-1 (i.e. universal) screening practices in primary care settings, it may be helpful to re-conceptualize our screening model to also incorporate Level-2 screeners. Level-2 screeners are used to identify children at increased likelihood for ASD because they have failed Level-1 screens, are being monitored for social-communication concerns, or have an older sibling with ASD (Petrocchi et al., 2020). The “Swiss cheese model” of risk mitigation (Reason, 1990) likens human systems (e.g. ASD screening) to a stack of slices of Swiss cheese, where the holes represent system flaws. If the holes in each slice are perfectly aligned, there are no systems in place to prevent error (e.g. failure to identify a child at increased likelihood for ASD). Additional layers of defense are needed to ensure that children do not “slip through the cracks” due to imperfections in one defense. One way of mitigating the failure to detect ASD is to have additional systems in place (e.g. multiple levels of screening) that are “layered” behind each other. Therefore, at least in theory, flaws in Level-1 screening will not necessarily lead to a child slipping through the cracks, since other defenses also exist (e.g. Level-2 screening).

Even with consistent, universal screening practices in primary care, this route alone may not detect all cases of ASD. Screening in primary care alone under-identifies ASD, suggesting that many children with ASD are being missed along this route (Carbone et al., 2020; Guthrie et al., 2019; Sánchez-García et al., 2019). Establishing screening practices in multiple contexts frequented by young children, and particularly those with developmental risk factors or delays, would allow for additional layers of defense against this problem.

Screening in EI programs

In the 1980s, Congress mandated the provision of EI services through Part C of the Individuals with Disabilities Education Improvement Act (IDEA, 2004). Part C EI serves children from birth to 3 years of age who have, or are at increased likelihood of, a developmental delay or disability. EI services are designed to promote the acquisition of developmental milestones, teach families to interact with and respond to their child in developmentally appropriate ways, and promote independence (IDEA, 2004). Given that children in Part C EI already have identified developmental concerns, including social-communication delays, it is a reasonable setting for conducting further ASD screening (Eisenhower et al., 2021; Monteiro et al., 2016). However, diagnoses of ASD are often missed or delayed once referred to EI if additional screening and subsequent referral for a comprehensive ASD evaluation do not occur after a positive screen (Wiggins et al., 2015). A study by Tomlin et al. (2013) found that a majority of EI providers report not engaging in ASD screening practices whatsoever and among those who do, few report using formal tools or assessment measures. In addition, greater than half of providers reported that they did not feel like they had enough training to perform ASD screenings or talk to families about screening results, though an overwhelming majority (> 90%) reported they would be interested in receiving training in this area. The lack of early identification and access to specialized EI programs have been identified as barriers to meeting the needs of infants and toddlers with early signs of ASD in Part C settings (Twardzik et al., 2017). Improving identification of ASD within Part C EI programs can enhance connections between screening in primary care and downstream diagnoses and ASD-specific interventions.

Implementing screening practices in agencies focused on providing EI is not necessarily a straightforward process. While service coordinators working in EI report that EI providers have a role in conducting ASD-specific screening, more than 50% of the coordinators indicated that it would be difficult to complete ASD-specific screening in EI settings because of provider time constraints or lack of expertise and training (Tomlin et al., 2013). To address this concern, Pizur-Barnekow et al. (2013) proposed that one member of an EI team could be trained in conducting and interpreting the results of an ASD-specific screening. Although it may be helpful for agencies to encourage all providers to be trained in screening procedures so they can recognize early signs and make appropriate referrals, it may be more feasible for EI programs to adopt an implementation model in which certain providers serve as a screening “point-person” and relay the screening results to the family’s primary EI provider, who can discuss any concerns with the family.

The screening tool for autism in toddlers

The Screening Tool for Autism in Toddlers (STAT: Stone et al., 2000, 2004, 2008) is an interactive, play-based Level-2 screening measure comprising 12 activities that assess key social-communicative behaviors including play, communication, and motor imitation for children from 24 to 36 months. It takes 20 min to administer, and was designed at the outset for use by a wide range of community professionals working with young children in assessment or intervention settings, including psychologists, pediatricians, speech-language pathologists, social workers, preschool teachers, and EI specialists. Certification in use of the STAT requires watching an online tutorial and passing a post-test, or attending an in-person training and achieving fidelity and scoring reliability on two self-recorded STAT administrations; however, web-based training has been shown to be sufficient to train community providers from a range of backgrounds to administer the STAT effectively (Kobak et al., 2011). Previous research has found the STAT to have high sensitivity (0.92), specificity (0.85), inter-rater reliability (κ = 1.00), test–retest reliability (κ = 0.90), and concurrent validity with the Autism Diagnostic Observation Scale—Generic (ADOS-G; κ = 0.92–0.95; Lord et al., 2000; Stone et al., 2004).

This study

This study examines the implementation of the STAT by community EI providers serving toddlers from birth to 3 years. Specifically, we examined the (a) effectiveness of a one-day STAT training workshop for increasing EI providers’ self-efficacy in ASD screening skills from a baseline period to 18 months post-STAT training, (b) key implementation factors related to providers’ use of the STAT (i.e. adoption, feasibility, and perceived effectiveness), and (c) specific barriers and facilitators to STAT implementation.

Method

Participants and recruitment

EI providers were recruited from nine agencies across four counties throughout Washington State. The research team initially identified counties for participation based on the demographic diversity of the population, as well as the presence of local professional connections that could facilitate communication with EI agencies. These counties were diverse in regard to resident ethnicity, socioeconomic status, and population density. After counties were identified, the research team met with interested EI agencies and providers to offer information about the study and assess interest in participation. Agencies varied in size, and the number of enrolled providers ranged from 3 to 20 across agencies.

The current sample included 69 EI providers who attended the STAT training workshop and completed questionnaires at a minimum of one baseline and one post-training time point. See Table 1 for provider characteristics. Specific data on socioeconomic status and educational attainment levels were not recorded.

Table 1.

Provider demographics.

Baseline characteristics
Gender (%)
Female	93.7
Male	6.3
Race (%)
White	92.4
Black/African-American	0.0
American Indian/Alaska Native	0.0
Asian	1.5
Hawaiian Native/Pacific Islander	0.0
Other	6.1
Ethnicity (%)
Hispanic/Latino	3.5
Speaks Spanish fluently (%)	6.3
Age M (range)	43.3 (24.0–63.0)
Years of experience M (range)	15.5 (1.5–38.0)
Children on caseload M (range)	31.0 (6.0–84.0)
Professional background (%)
Family resource coordinator	7.2
Occupational therapist/assistant	21.7
Physical therapist/assistant	13.0
Speech language pathologist/assistant	43.5
Special educator	14.5

Overview and approach

This study was part of a larger pragmatic trial examining the implementation of a service delivery model aimed at increasing evidence-based ASD screening and ASD-specific EI practices within primary care and EI settings (see the study by Ibañez et al., 2019 for complete study protocol). This trial was pragmatic in that the research team provided an initial training to the agencies, and provided technical assistance upon request, but allowed programs to develop their own implementation plans for how or when to implement the STAT.

An interrupted time-series design was used to examine STAT use by EI providers, with the one-day STAT training workshops representing the “interruption.” Data were collected at four time points: Baseline 1 (T1, 12–18 months before STAT training), Baseline 2 (T2, 3 months before STAT training); 6-month follow-up (T3, 6 months after STAT training); and 18-month follow-up (T4, 18 months after STAT training). It has been argued that interrupted time series is one of the strongest quasi-experimental research designs—particularly when the randomization of clinicians or clinics or the sequential rollout of a program is not possible (Cook et al., 2002). In addition, the inclusion of two baseline time points controls for any externally driven trends in self-efficacy (Campbell & Stanley, 2015).

The study was reviewed and approved by the Institutional Review Board (IRB) at the University of Washington and all participants provided informed consent.

Procedures

Providers completed surveys about ASD screening self-efficacy at T1–T4. They reported on their adoption of the STAT and their perceived feasibility and effectiveness of the screener at T3 and T4. At both post-training time points, providers also completed open-ended responses to various barriers and facilitators to STAT use in their practice. Providers were compensated US$20 at each time point for completing surveys and US$5 for each eligible family they referred. Agencies were also compensated US$250 to offset the lost time for service provision for each of their providers who attended the STAT training. See Figure 1 for recruitment, eligibility, and data collection completion information.

Figure 1.

Study recruitment, enrollment, and data collection flow chart.

STAT training workshops

Providers were randomized to the timing of their training workshops at the county level. Randomization was performed using a stepped wedge approach (Ibañez et al., 2019) with four consecutive 3-month blocks. Within each 3-month time period, EI programs within the same county received a workshop followed by an optional certification process that involved sending video recordings of two STAT administrations to the research team and achieving adequate fidelity of implementation and scoring reliability. STAT training was conducted via a 1-day, in-person, interactive workshop. Each workshop was led by the study PI (the original developer of the STAT) and Co-I (a certified STAT trainer). The 8 h workshop provided an overview of the early behavioral features of ASD as well as detailed information about administration guidelines and scoring conventions for each of the 12 interactive STAT items. Numerous video examples were used to illustrate correct administration procedures for each item (e.g. number of trials) and strategies for engaging children (e.g. seating arrangements), and opportunities for independent scoring practice and feedback were provided. In addition, there were opportunities for providers to practice administering the STAT with volunteer children. Each provider also received a packet containing STAT materials as well as information related to discussing ASD concerns with caregivers/parents and eliciting information about ASD-related behaviors from caregivers.

Measures

Provider self-efficacy regarding screening practices

At all four time points, providers completed the Practices and Efficacy Survey, which asked questions about providers’ self-efficacy surrounding screening. Self-efficacy was measured with three items: how knowledgeable they feel about recognizing early signs of ASD, their comfort using ASD screening, and their comfort discussing ASD concerns with families (see Table 2). Each question was rated on a 4-point Likert-type scale with “1” indicating “Not at All” to “4” indicating “Extremely.” The mean of these three items was calculated to obtain an average screening self-efficacy score for providers. The self-efficacy items demonstrated acceptable internal consistency reliability within each time point, with an average Cronbach’s α = 0.71.

Table 2.

Item text for outcome measures.

Domain	Questions	Response options
Self-efficacy	1. How knowledgeable do you feel about recognizing the early behavioral signs of ASD?	Not at all–Extremely
	2. How comfortable do you feel using an ASD-specific screening tool?
	3. How comfortable do you feel discussing ASD concerns with caregivers of young children?
Adoption	1. Since the STAT training workshop, have you used the STAT with children for whom there are ASD concerns?	Yes/no
	2. Have you obtained formal certification to use the STAT?
	3. Have you been using the STAT formally or informally?
	4. With how many children have you used the STAT?
Feasibility	1. I have the resources I need to implement the STAT with families in my caseload	Strongly disagree–Strongly agree
	2. I understand how to use the STAT
	3. I feel comfortable discussing the need to conduct the STAT with parents
	4. The STAT is flexible enough for me to use with my current caseload
	5. The STAT fits well with my current practices
	6. I feel comfortable discussing the STAT results with parents
	7. The materials needed to implement the STAT are reasonable
	8. Use of the STAT is consistent with the mission of my work setting
	9. I have a positive attitude about using the STAT
Effectiveness	1. The STAT has been effective at identifying children at risk for ASD	Strongly disagree–Strongly agree
	2. The STAT has been useful for identifying treatment goals
	3. The STAT has been helpful for talking to parents about their child’s social-communication development
	4. The STAT has been helpful for talking to parents about ASD concerns
	5. Using the STAT has increased my confidence in identifying children with ASD

ASD: autism spectrum disorder; STAT: Screening Tool for Autism in Toddlers.

STAT adoption, feasibility, and effectiveness

At the two post-training time points, providers answered questions on the STAT Implementation Survey, which examined STAT Adoption, STAT Feasibility, and STAT Perceived Effectiveness. See Table 2 for the specific items in each category. For STAT adoption, providers answered four items assessing (a) whether they used the STAT, (b) whether they obtained formal STAT certification, (c) how they used the STAT (i.e. with or without reference to the cutoff score), and (d) with whom they used the STAT. Providers were classified as “adopters” if they answered “yes” to question (a).

For STAT feasibility, nine items examined EI providers’ attitudes about how the STAT fits with their current practices, profession, and setting, as well as the resources and materials needed to implement the STAT. These items were adapted from a validated measure for assessing pertinent aspects of implementation (Usage Rating Profile-Intervention Revised; Briesch et al., 2013). The mean of the nine items was calculated to obtain an average feasibility score. The feasibility items demonstrated high internal consistency reliability within each time point, with an average Cronbach’s α = 0.91.

For STAT effectiveness, EI providers completed five items about how effective they perceived the STAT to be in identifying children at increased likelihood of ASD and helping them provide feedback to families. The mean of these five items was calculated to obtain an average perceived effectiveness score. The effectiveness items demonstrated high internal consistency reliability for each time point, with an average Cronbach’s α = 0.90.

Barriers and facilitators to STAT usage

Providers also responded to three open-ended questions about barriers and facilitators to using the STAT and discussing screening results with families. A team of three research staff members completed a three-stage content analysis (Fowler, 2013) on the qualitative comments from the open-ended questions. The first stage of the interactive process included identification of categories by each member of the research team. Team members then met and determined coding categories for use in the analysis. During the second stage, each research team member independently coded all the comments based on the defined category structure. Also during this stage, each member shared their category codes for the responses and discussed and reconciled codes if there was a lack of agreement about category assignment. Any outlying comments were labeled as miscellaneous, and the research team reviewed and determined whether another category was needed. During the third stage, each research team member verified the categories they coded. This interactive and iterative analysis process facilitated an exhaustive list of mutually exclusive category codes (Fowler, 2013). Two additional coders with expertise on the STAT and ASD screening practices independently coded all responses and demonstrated high levels of agreement for each category (86%). These were the final codes utilized to determine themes of facilitators and barriers to STAT adoption.

Community involvement

Members of the autistic community were not involved in developing the research question, study design, measures, implementation, or interpretation and dissemination of the findings in this study.

Results

Analyses overview

Sixty-nine EI providers had at least one baseline and one follow-up time point on at least one measure and were included in the final analyses. Hierarchical linear modeling (HLM) was used to examine the trajectory of change in provider self-efficacy across time points. Descriptive statistics and measures of central tendency (i.e. mean, median) were used to examine STAT adoption, STAT feasibility, and perceived STAT effectiveness. In addition, a multiple regression analysis was conducted to determine whether certain provider characteristics predicted STAT adoption.

Provider screening self-efficacy

A two-level HLM using maximum-likelihood estimation (Singer & Willett, 2003) to avoid list-wise deletion was conducted via RStudio (nlme package) to determine whether there were differences in self-efficacy in ASD screening skills from pre-training to post-training. At T1, there were no significant differences for self-efficacy between the EI providers with and without post-treatment data (b = −0.32, SE = 0.18, p = 0.07). Providers who were missing efficacy ratings for both post-treatment time points (i.e. T3 and T4; n = 3; 66 total providers included) were dropped from this analysis, as growth models could not be completed with both post-training time points missing.

At Level-1, time effects were modeled as random effects. For provider self-efficacy, the final model had (a) a significant intercept (b = 2.51, SE = 0.10, p < 0.001), (b) no significant difference between baselines T1 and T2 (b = 0.03, SE = 0.09, p = 0.75), and (c) significant increases from baseline T2 to T3 (b = 0.24, SE = 0.08, p < 0.01) and from baseline T2 to T4 (b = 0.34, SE = 0.09, p < 0.001). STAT users reported significantly higher overall levels of self-efficacy in ASD screening than non-users (b = 0.38, SE = 0.15, p < 0.05); there were no significant interactions between STAT use group and the time vectors, p-values ⩾ 0.67. See Figure 2 for changes in self-efficacy across time points.

Figure 2.

Screening self-efficacy across time points.

Because the STAT workshop places a major emphasis on understanding the early behavioral characteristics of ASD, it was of interest to determine whether self-efficacy in this area improved for providers who were not using the STAT. Toward this end, pre- and post-training ratings for this single item were examined for the group of non-users. Descriptive statistics for one specific self-efficacy question, “How knowledgeable do you feel about recognizing the early behavioral signs of ASD?” revealed that knowledge about recognizing ASD increased after participation in the training, regardless of whether or not they adopted the STAT (Non-adopter Baseline M = 2.86 or “Somewhat knowledgeable”; Non-adopter Post M = 3.19 or “Moderately knowledgeable”).

STAT adoption

Following implementation, 30 out of 60 (50.0%) EI providers reported using the STAT for at least one post-training time point. At T3, 22 of 60 providers (36.7%) reported using the STAT to screen an average of 2.45 children each. At T4, 25 of 56 providers (44.6%) reported using the STAT to screen an average of 4.2 children each (see Figure 3). Providers reported using the STAT both formally (using standard procedures and obtaining a cutoff score; 50.0% at T3; 88.0% at T4) and informally (without using a cutoff score; 63.6% at T3% and 36.0% at T4). At the final time point, 15 of the 56 providers (26.8%) reported being formally certified in the use of the STAT.

Figure 3.

STAT adoption post-training.

Providers who did not adopt the STAT (n = 38 at T3 and n = 31 at T4) indicated that they had not done so because they (a) referred children in their caseload to receive screening from another provider who was STAT certified (n = 21 at T3 and n = 19 at T4), (b) had not worked with children with ASD concerns since the training workshop (n = 11 at T3 and n = 4 at T4), and/or (c) did not have time to complete the STAT certification process (n = 11 at T3 and n = 11 at T4). Of providers who did not adopt the STAT, 18 out of 38 (47.4%) indicated that they intended to begin using the STAT on the T3 survey and 15 out of 29 (51.7%) reported such on the T4 survey.

It was also useful to examine STAT adoption across the agency level. At least one provider from each agency attended the STAT training; however, the number of providers who attended varied greatly between agencies (M = 7.6, SD = 5.5, range = 2–19). Furthermore, the number of providers who reported using the STAT at either time point was variable across agencies (M = 3.2, SD = 2.5, range = 0–8), as was the number of providers who reported becoming certified in the STAT (M = 1.5, SD = 1.9, range = 0–5). See Table 3 for provider adoption at each agency.

Table 3.

STAT adoption by agency.

Agency	Total providers trained	Total providers trained who reported using the STAT (% of total providers)	Total providers certified (% of trained providers)
Agency 1	6	4 (66.7)	2 (50.0)
Agency 2	8	3 (37.5)	0 (0.0)
Agency 3	9	2 (22.2)	1 (50.0)
Agency 4	3	3 (100)	0 (0.0)
Agency 5	19	8 (42.1)	5 (62.5)
Agency 6	3	1 (33.3)	0 (0.0)
Agency 7	3	0 (0.0)	0 (0.0)
Agency 8	5	3 (60.0)	3 (100)
Agency 9	13	6 (46.2)	4 (66.6)
Total	69	30	15

STAT: Screening Tool for Autism in Toddlers.

With respect to provider characteristics, only years of experience in the field predicted STAT use, R² = 0.205, F (1,18) = 4.650, p = 0.045, in that providers with more experience were more likely to use the STAT. Gender, race, ethnicity, age, number of children on caseload, and professional background were not predictors of STAT use.

The majority of providers (90.9% at T3; 96.0% at T4) reported using the STAT if or when concerns arise with a child. Fewer providers endorsed using the STAT immediately after the child has entered the EI system (9.1% at T3; 24.0% at T4) or as part of the exit/transition evaluation (0% at T3; 4% at T4).

STAT feasibility

Providers who utilized the STAT reported high levels of feasibility at T3 (n = 22; M = 3.30, SD = 0.45, median = 3.33) and T4 (n = 22; M = 3.42, SD = 0.42, median = 3.20), as these scores reflected an average of “3” (i.e. “agree”) out of 4 for these items.

Providers who reported not using the STAT reported lower levels of perceived STAT feasibility at T3 (n = 36; M = 3.05, SD = 0.49, median = 3.11) and T4 (n = 31; M = 2.98, SD = 0.48, median = 3.22). However, the overall mean and median scores also reflected an average of “3” (i.e. “agree”) out of four for these items (see Figure 4).

Figure 4.

STAT feasibility.

STAT effectiveness

EI providers who used the STAT reported high levels of perceived effectiveness at T3 (n = 21; M = 3.90, SD = 0.81, median = 4.00) and T4 (n = 22; M = 4.23, SD = 0.49, median = 4.20), as the overall mean and median scores reflected an average score of “4” (or “agree”) across these items.

At T3 and T4, the majority of the 22 EI providers endorsed reasons for using the STAT that align with its original purpose: to identify a child’s risk for ASD (86.4% at T3; 92.0% at T4) and to decide whether to refer the child externally for an ASD diagnostic evaluation (68.2% at T3; 64.0% at T4). Additional reasons for using the screening tool included: to understand a child’s social-communication skills (40.9% at T3; 32.0% at T4); to help communicate with parents about a child’s strengths and needs (31.8% at T3; 40.0% at T4); to decide whether a child might benefit from ASD-specific EI (27.3% at T3; 20.0% at T4); to further assess a child referred from a doctor because of a failed first-line screener (22.7% at T3; 44.0% at T4); and to identify intervention goals for a child (18.2% at T3; 16.0% at T4).

Barriers and facilitators to STAT Use

Qualitative analysis of barriers and facilitators to STAT implementation revealed six main categories of facilitators and eight main categories of barriers. Tables 4 and 5 provide a comprehensive list of categories of barriers and facilitators with related quotes, as well as the frequency with which each code was endorsed by providers. The most commonly endorsed facilitators were that the STAT is user-friendly, that it promotes communication with families, that it informs treatment goals and intervention, and that it is validated/standardized for ASD. The most commonly endorsed barriers to STAT use were that there was a certification process, that the STAT is not applicable to all children on their caseload, and that ASD is a sensitive topic to discuss with parents.

Table 4.

Facilitators of STAT implementation.

Facilitators
Broad code	Description	Text example	No. of times endorsed
Promotes communication with families	The code relates to the STAT facilitating open dialogue with families about their clients’ functioning and symptomology, possible social-communication concerns, and treatment planning	“The STAT provides more comprehensive information [than other screeners] to give parents a plan for what comes next.”	14
User-friendly	This code relates to the STAT being user-friendly including: easy to use, simple, fast, concise, well-organized, applicable to a wide range of children in EI	“It is easy to administer in a reasonable amount of time.”	12
Informs treatment goals/intervention	This code relates to the results of the STAT helping providers assess strengths and areas for improvement in their client and forming treatment goal and intervention plans accordingly	“I like how it is broken down into the different communication behaviors (i.e. imitation, requesting etc.) because that is very helpful for setting the treatment plan and with goal writing.”	11
Validated/standardized for autism	This code relates to the STAT being a standardized and validated tool that clinicians feel confident using and accepting the results	“It is very helpful to have a norm referenced screener that is based on [child] performance rather than caregiver report.”	10
Observation-based and objective	This code relates to clinicians using the STAT because it is observation-based and objective, which allows clinicians and parents to see and document a child’s strengths and challenges in real-life situations	“I like having a more functional screener. Parents can watch the administration or video of it and see exactly what we are looking at.”	9
Promotes interdisciplinary provider communication	This code relates to the STAT promoting the discussion of at-risk children and also assessment and treatment more broadly across providers of different disciplines. It also relates to the STAT helping to facilitate communication with other settings and providers who provide formal autism evaluations	“I feel like my agency is collaborating to get kids on our caseload evaluated more quickly.”	7
Motivational, play-based, naturalistic	This code relates to the STAT being motivational, play-based, and naturalistic which allows providers to easily integrate into everyday practice. Additionally, these components make the STAT fun to use and tolerable to children and parents	“It is easy to use with items that are motivating for children that often struggle with play skills.”	3

STAT: Screening Tool for Autism in Toddlers.

Table 5.

Barriers to STAT implementation.

Barriers
Broad code	Description	Text example	No. of times endorsed
Autism is a sensitive topic to discuss with parents	The code relates to clinician discomfort bringing up the topic of autism with parents or managing parent denial or refusal to pursue further evaluation or autism-focused treatment following a positive screen	“Reviewing it with some parents who get emotional. Timing is important.”	13
Certification process	This code relates to the process to become certified in the STAT including: the time it takes to become certified, finding assistance recording videos during STAT administration, identifying children to use for practice administrations	“I am not yet STAT [certified]. The time it takes to become [certified] keeps taking a back seat so I need a trained colleague to administer the test for me.”	10
Not relevant for all children	This code relates to the STAT not being relevant for all children including: children outside the target age range, children with more advanced language abilities, children with concerns other than social communication. This code captures providers’ hesitancy to use the STAT because it is not broadly applicable to all children to their caseload and it is not applicable to all children with social-communication concerns	“[There is] limited application related to severity of child, and age of child.”	8
Integrating into existing workflow	This code relates to difficulties integrating the STAT into existing workflows within agencies, including: billing/getting reimbursed for administration, or administering the STAT in group settings, changing clinic policies, access to materials	“Not having complete access to tools at all times, changed clinics and we have a different method of intervention now.”	8
Behavior management of child/parent during administration	This code is related to the management of the child and/or parent during administration including: managing challenging behaviors, preventing parents from helping children, limiting distractions in the home	“Having parents observe and not help with verbal or physical cues despite being asked not to.”	6
Desire for more/ongoing training	This code relates to the desire for more/ongoing training in the STAT including: ongoing training to feel more confident about administration, additional training on what behaviors in typical development look like, additional training on how to interpret in-the-moment behaviors during administration	“With hindsight, I now feel that during training we should have had to practice to the STAT on typically developing kids first, to see what a variety of passing scores looks like.”	6
Results are not always in line with clinician/parent judgment	This code relates to the results of the STAT not consistently aligning with behaviors seen by the clinician or reported by the parent outside of administration	“When the STAT doesn’t indicate that the child is at risk, but I still think the child is at risk. The STAT seems to miss highly verbal children who are still at risk.”	4
Not relevant for all providers	This code relates to the STAT not being relevant for all providers including:trainees, providers who don’t work with the target age range, providers who don’t target social-communication challenges	“As a [physical therapist assistant] the STAT isn’t as applicable to use when working towards desired physical therapy goals.”	3

STAT: Screening Tool for Autism in Toddlers.

Discussion

This study is the first to pragmatically examine the implementation of a Level-2 ASD screener in EI settings and provide results of implementation outcomes 18 months post-training. Half of the providers who attended the STAT training workshop reported adopting the STAT, and those who did not reported that they often referred children with concerns to other trained providers in their agencies for screening. Promisingly, most providers who utilized the STAT continued to use it up to 18 months after being trained, even without ongoing participation or support from the research team. Participation in the STAT training workshop was associated with significant increases in provider self-efficacy in ASD screening skills, regardless of STAT use. Providers who began using the STAT in their practice reported higher levels of self-efficacy in skills related to ASD screening than those who did not. Providers who used the STAT perceived it as feasible and effective in identifying children with social-communication concerns and ASD, and these views were sustained 18 months post-STAT training. In addition, providers reported qualitatively that the STAT is user-friendly, promotes communication with families, and helps inform treatment goals for many of their clients. This finding is consistent with studies of implementation models such as the Consolidated Framework for Implementation Research (CFIR; Damschroder et al., 2009), which have demonstrated that relative advantage (i.e. the user perceives a clear advantage in the effectiveness of the tools) and patient needs and resources (i.e. the user perceives the tool to map onto the needs of their patients) are key conditions for successful implementation (Barwick et al., 2020).

Patterns emerged regarding the type of providers likely to utilize the STAT, as well as how it was used at the agency level. Providers’ training background and age were not predictors of STAT usage; however, individuals with more experience working in an EI setting were more likely to adopt the STAT. This finding is consistent with previous research indicating that some provider-level demographic factors are associated with attitudes toward adoption of evidence-based practices (EBPs; Aarons, 2004), and highlights the need to consider both agency- and provider-level factors in planning future STAT trainings and implementation efforts. Future directions of this research should also examine county-level factors (e.g. family educational levels, employment rates, household income, poverty status) to determine the degree to which they impact STAT adoption and whether future efforts to disseminate the STAT on a larger scale should include more personalized training and implementation plans based on these aspects.

Although an average of 6.9 providers were trained in the STAT at each agency, only three providers at each agency on average reported utilizing the STAT at the 18-month follow-up. Moreover, only 1.5 providers on average at each agency reported completing the formal STAT certification process. These results raise important questions regarding the best use of the STAT in community settings, where time and resource constraints limit providers’ ability to submit videotapes for review, as required to attain formal certification. Furthermore, results from our qualitative analysis of barriers and facilitators to STAT usage and informal communication with agencies indicated that the process of becoming certified in the STAT was challenging in an EI context. As a result, some providers used the STAT without becoming certified, and others referred children with social-communication concerns to the providers in their agency who were STAT certified. These findings indicate that the use of an interactive ASD screening tool such as the STAT within EI settings may be a specialized responsibility, with a few identified providers per agency taking on this role.

However, broad agency-wide training may still be beneficial, as providers reported increased self-efficacy using an ASD-specific screening tool following the STAT training workshop, which provides numerous examples of social communicative behaviors in toddlers with and without ASD. Previous research has demonstrated that relative advantage (i.e. stakeholders’ perception of the importance of the tool within the organization) is a key factor influencing implementation at the provider level (Barwick et al., 2020), further supporting a broad training model for all providers in an agency. In addition, providers with higher self-efficacy in skills pertinent to screening for ASD were more likely to report adopting the STAT, which is consistent with previous findings in the implementation science literature (Damschroder et al., 2009). Assessing provider self-efficacy at baseline and personalizing trainings to highlight components of the screening process that providers feel less confident about may be one method of increasing STAT adoption at the provider level. In addition, providing ongoing training and consultation may be necessary for providers who do not feel confident in their ability to use screening tools, even after attending the training. Providers also reported that one major barrier to STAT usage was sensitivity to discussing ASD concerns with families; subsequently, the STAT training team has begun to include information on the Transtheoretical Model of Behavior Change (Prochaska et al., 1998) in the STAT trainings, as well as strategies for providers to use with families at each stage of the model when discussing ASD concerns. Future studies are needed to determine whether this addition helps mitigate this barrier, as well as to explore other opportunities to increase providers’ comfort levels in discussing ASD with families.

These findings may also be relevant at the policy level. The state of South Carolina, for example, pursued a novel statewide initiative to increase the early identification of children at increased likelihood of ASD by implementing a two-tiered screening process (which included using the STAT in EI settings; Rotholz et al., 2017). This initiative led to large increases in children eligible to receive early applied behavior analysis services; however, researchers noted challenges in recruiting, training, and retaining providers who can screen and treat young children with ASD. Implementing systematic two-tiered screening protocols seems to be effective at the state level; however, going forward, researchers and policy makers may need to re-conceptualize how to most efficiently implement screening or assessment practices in community settings. Previous research in the field of implementation science has posited that implementation strategies that specifically target the unique aspects of an organization’s social context have demonstrated positive implementation effects (Williams & Glisson, 2020).

The importance of ASD screening and identification in EI settings cannot be overstated. It has been demonstrated that children enrolled in EI or referred for an evaluation from EI are diagnosed earlier compared to those referred from primary care or school settings (Daniels & Mandell, 2014; Yingling, 2019). It seems that some EI settings are, in fact, already playing a role in earlier diagnoses of ASD, but further research is needed to determine the mechanism of this effect and the pathways toward an eventual ASD assessment. Moreover, additional research on screening and referral in EI and the implementation of standardized screening practices may inform clearer guidelines and policy to make this trend more consistent and universal nationwide. Consideration of the unique structure of EI agencies with respect to screening practices—and the impact that provider self-efficacy has on adoption—may inform future development of training and implementation models that are both feasible for agencies and promote long-term sustainability of broad, systematic screening practices.

Limitations and future directions

Several limitations of this study must be considered in interpreting the results. Owing to a lack of brief, validated surveys in the domains of interest for this project, the measures of provider self-efficacy, adoption, and perceived effectiveness were developed by the research team, and the measure of feasibility was adapted by the research team for the purpose of this study. Although these measures demonstrated high internal consistency, future research could serve to validate similar measures for the purposes of examining screening or assessment implementation in community settings.

Furthermore, there was no direct assessment of STAT usage or fidelity, or of the impact that the STAT might have on child outcomes (i.e. rates of diagnosis, age of diagnosis, rates of referral for a full assessment). As such, it cannot be claimed that provider use of the STAT truly had an impact on child outcomes. Similarly, there are no available data to confirm providers’ reported improvements in self-efficacy or adoption of the STAT. Future studies should aim to examine whether successful STAT implementation improves the rate and age of formal assessment and eventual diagnosis. Also, as stated above, members of the autistic community were not involved in developing the research question, study design, measures, implementation, or interpretation and dissemination of the findings in this study. The inclusion of autistic self- and family advocates would help determine whether the STAT is an acceptable measure to the autistic community as well as whether EI programs are appropriate places in which to conduct Level-2 screenings.

Finally, our sample of EI providers was relatively homogeneous (92.4% Whites and 93.7% females). Although these demographics are consistent with national demographics for Part C EI providers, this lack of ethnic and gender diversity may restrict generalizability to other settings with more diverse providers (Hebbeler et al., 2007). Future research aims include examining broader implementation of the STAT with a more diverse sample of providers and assessing the generalizability of our results.

Conclusion

Our findings suggest that Part C EI agencies are appropriate settings in which to implement the STAT as a Level-2 ASD screener. This “extra layer” of screening, in addition to Level-1 screening in primary care, should be encouraged to promote broader and earlier ASD identification, which in turn could expedite receipt of ASD-specialized services and supports. Future research should aim to identify different models of implementation and examine the sustainability of screening use within the EI context. Future research might also be directed toward evaluating organizational or policy factors that influenced STAT implementation. Our study found large variability across agencies in the number of providers who were trained, who used, and who became certified in the STAT. In addition, it seemed that there was a great deal of provider autonomy and flexibility in the decision-making process regarding use of the STAT. Future directions could serve to more formally evaluate these factors via focus groups or interviews with providers and agency leadership to determine the type of program, system, or provider for whom the STAT is best suited.

Footnotes

Declaration of conflicting interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: W.L.S. is an author of the STAT and receives an author’s share of royalties from Vanderbilt University for sales.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Institute for Mental Health (grant no. R01 MH104302).

ORCID iD

Daina M Tagavi

References

Aarons

G. A.

(2004). Mental health provider attitudes toward adoption of evidence-based practice: The Evidence-Based Practice Attitude Scale (EBPAS). Mental Health Services Research, 6(2), 61–74. https://doi.org/10.1023/b:mhsr.0000024351.12294.65

American Academy of Pediatrics. (2000). Committee on Practice and Ambulatory Medicine. Recommendations for preventive pediatric health care. Pediatrics, 105(3), 645–646.

Baio

Wiggins

Christensen

D. L.

Maenner

M. J.

Daniels

Warren

Kurzius-Spencer

Zahorodny

Rosenberg

C. R.

White

Durkin

M. S.

Imm

Nikolaou

Yeargin-Allsopp

Lee

L. C.

Harrington

Dowling

N. F.

(2018). Prevalence of autism spectrum disorder among children aged 8 years—Autism and developmental disabilities monitoring network, 11 sites, United States, 2014. MMWR Surveillance Summaries, 67(6), 1–23. https://doi.org/10.15585/mmwr.ss6706a1

Barwick

Dubrowski

Damschroder

(2020). Factors associated with effective implementation: Research and practical implications. Implementation Science 3.0, 81–100.

Briesch

A. M.

Chafouleas

S. M.

Neugebauer

S. R.

Riley-Tillman

T. C.

(2013). Assessing influences on intervention implementation: Revision of the usage rating profile-intervention. Journal of School Psychology, 51(1), 81–96. https://doi.org/10.1016/j.jsp.2012.08.006

Campbell

D. T.

Stanley

J. C.

(2015). Experimental and quasi-experimental designs for research. Ravenio Books.

Carbone

P. S.

Campbell

Wilkes

Stoddard

G. J.

Huynh

Young

P. C.

Gabrielsen

T. P.

(2020). Primary care autism screening and later autism diagnosis. Pediatrics, 146(2), e20192314. https://doi.org/10.1542/peds.2019-2314

Cook

T. D.

Campbell

D. T.

Shadish

(2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin.

Damschroder

L. J.

Aron

D. C.

Keith

R. E.

Kirsh

S. R.

Alexander

J. A.

Lowery

J. C.

(2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4(1), 1–15.

10.

Daniels

A. M.

Mandell

D. S.

(2014). Explaining differences in age at autism spectrum disorder diagnosis: A critical review. Autism, 18(5), 583–597. https://doi.org/10.1177/1362361313480277

11.

Dawson

Rogers

Munson

Smith

Winter

Greenson

Donaldson

Varley

(2010). Randomized, controlled trial of an intervention for toddlers with autism: The Early Start Denver Model. Pediatrics, 125(1), e17–e23. https://doi.org/10.1542/peds.2009-0958

12.

Eisenhower

Pedraza

F. M.

Sheldrick

R. C.

Frenette

Hoch

Brunt

Carter

A. S.

(2021). Multi-stage screening in early intervention: A critical strategy for improving ASD identification and addressing disparities. Journal of Autism and Developmental Disorders, 51(3), 868–883. https://doi.org/10.1007/s10803-020-04429-z

13.

Fowler

F. J.

Jr. (2013). Survey research methods. SAGE.

14.

Guthrie

Wallis

Bennett

Brooks

Dudley

Gerdes

Miller

J. S.

(2019). Accuracy of autism screening in a large pediatric network. Pediatrics, 144(4), e20183963.

15.

Hebbeler

Spiker

Bailey

Scarborough

Mallik

Simeonsson

Nelson

(2007). Early intervention for infants and toddlers with disabilities and their families: Participants, services, and outcomes. SRI International.

16.

Hyman

S. L.

Levy

S. E.

Myers

S. M.

, & Council on Children with Disabilities, Section on Developmental and Behavioral Pediatrics. (2020). Identification, evaluation, and management of children with autism spectrum disorder. Pediatrics, 145(1), e20193447. https://doi.org/10.1542/peds.2019-3447

17.

Ibañez

L. V.

Vander Stoep

Myers

Zhou

Dorsey

Steinman

K. J.

Stone

W. L.

(2019). Promoting early autism detection and intervention in underserved communities: Study protocol for a pragmatic trial using a stepped-wedge design. BMC Psychiatry, 19(1), 169. https://doi.org/10.1186/s12888-019-2150-3

18.

Individuals with Disabilities Education Improvement Act of 2004, P.L. 108- 446, 20 U.S.C. $ 1400 et seq.

19.

Johnson

C. P.

Myers

S. M.

(2007). Identification and evaluation of children with autism spectrum disorders. Pediatrics, 120(5), 1183–1215. https://doi.org/10.1542/peds.2007-2361

20.

Keil

Breunig

Fleischfresser

Oftedahl

(2014). Promoting routine use of developmental and autism-specific screening tools by pediatric primary care clinicians. WMJ, 113(6), 227–231.

21.

King

T. M.

Tandon

S. D.

Macias

M. M.

Healy

J. A.

Duncan

P. M.

Swigonski

N. L.

Skipper

S. M.

Lipkin

P. H.

(2010). Implementing developmental screening and referrals: Lessons learned from a national project. Pediatrics, 125(2), 350–360. https://doi.org/10.1542/peds.2009-0388

22.

Kobak

K. A.

Stone

W. L.

Ousley

O. Y.

Swanson

(2011). Web-based training in early autism screening: Results from a pilot study. Telemedicine and e-Health, 17(8), 640–644. https://doi.org/10.1089/tmj.2011.0029

23.

Lord

Risi

Lambrecht

Cook

E. H.

Leventhal

B. L.

DiLavore

P. C.

Pickles

Rutter

(2000). The Autism Diagnostic Observation Schedule—Generic: A standard measure of social and communication deficits associated with the spectrum of autism. Journal of Autism and Developmental Disorders, 30(3), 205–223. https://doi.org/10.1023/A:1005592401947

24.

Maenner

M. J.

Shaw

K. A.

Baio

(2020). Prevalence of autism spectrum disorder among children aged 8 years—Autism and developmental disabilities monitoring network, 11 sites, United States, 2016. MMWR Surveillance Summaries, 69(4), 1. https://doi.org/10.15585/mmwr.ss6706a1

25.

Mazurek

M. O.

Harkins

Menezes

Chan

Parker

R. A.

Kuhlthau

Sohl

(2020). Primary care providers’ perceived barriers and needs for support in caring for children with autism. The Journal of Pediatrics, 221, 240–245. https://doi.org/10.1016/j.jpeds.2020.01.014

26.

Monteiro

S. A.

Dempsey

Broton

Berry

Goin-Kochel

R. P.

Voigt

R. G.

(2016). Early intervention before autism diagnosis in children referred to a regional autism clinic. Journal of Developmental & Behavioral Pediatrics, 37(1), 15–19. https://doi.org/10.1097/DBP.0000000000000241

27.

Murray

R. E.

Barton

E. E.

(2021). Training pediatricians to implement autism screening tools: A review of the literature. Review Journal of Autism and Developmental Disorders, 8(1), 108–117. https://doi.org/10.1007/s40489-020-00206-x

28.

Ozonoff

Young

G. S.

Landa

R. J.

Brian

Bryson

Charman

Chawarska

Macari

S. L.

Messinger

Stone

W. L.

Zwaigenbaum

Iosif

A. M.

(2015). Diagnostic stability in young children at risk for autism spectrum disorder: A baby siblings research consortium study. Journal of Child Psychology and Psychiatry, 56(9), 988–998. https://doi.org/10.1111/jcpp.12421

29.

Petrocchi

Levante

Lecciso

(2020). Systematic review of level 1 and level 2 screening tools for autism spectrum disorders in toddlers. Brain Sciences, 10(3), 180. https://doi.org/10.3390/brainsci10030180

30.

Pizur-Barnekow

Muusz

McKenna

O’Connor

Cutler

(2013). Service Coordinators’ Perceptions of Autism-Specific Screening and Referral Practices in Early Intervention. Topics in Early Childhood Special Education, 33(3), 153–161. https://doi.org/10.1177/0271121412463086

31.

Prochaska

J. O.

Johnson

Lee

(1998). The transtheoretical model of behavior change. In Shumaker

S. A.

Schron

E. B.

Ockene

J. K.

McBee

W. L.

(Eds.), The handbook of health behavior change (pp. 59–84). Springer Publishing Company.

32.

Reason

(1990). Human error. Cambridge University Press.

33.

Rosenberg

R. E.

Landa

Law

J. K.

Stuart

E. A.

Law

P. A.

(2011). Factors affecting age at initial autism spectrum disorder diagnosis in a national survey. Autism Research and Treatment, 2011, 874619. https://doi.org/10.1155/2011/874619

34.

Rotholz

D. A.

Kinsman

A. M.

Lacy

K. K.

Charles

(2017). Improving early identification and intervention for children at risk for autism spectrum disorder. Pediatrics, 139, e20161061. https://doi.org/10.1542/peds.2016-1061

35.

Sánchez-García

A. B.

Galindo-Villardón

Nieto-Librero

A. B.

Martín-Rodero

Robins

D. L.

(2019). Toddler screening for autism spectrum disorder: A meta-analysis of diagnostic accuracy. Journal of Autism and Developmental Disorders, 49(5), 1837–1852. https://doi.org/10.1007/s10803-018-03865-2

36.

Sheldrick

R. C.

Frenette

Vera

J. D.

Mackie

T. I.

Martinez-Pedraza

Hoch

Eisenhower

Fettig

Carter

A. S.

(2019). What drives detection and diagnosis of autism spectrum disorder? Looking under the hood of a multi-stage screening process in early intervention. Journal of Autism and Developmental Disorders, 49(6), 2304–2319. https://doi.org/10.1007/s10803-019-03913-5

37.

Sheldrick

R. C.

Garfinkel

(2017). Is a positive developmental-behavioral screening score sufficient to justify referral? A review of evidence and theory. Academic Pediatrics, 17(5), 464–470. https://doi.org/10.1016/j.acap.2017.01.016

38.

Singer

J. D.

Willett

J. B.

(2003). Applied longitudinal data analysis. Oxford University Press.

39.

Stone

W. L.

Coonrod

E. E.

Ousley

O. Y.

(2000). Brief report: Screening tool for autism in two-year-olds (STAT): Development and preliminary data. Journal of Autism and Developmental Disorders, 30(6), 607–612. https://doi.org/10.1023/a:1005647629002

40.

Stone

W. L.

Coonrod

E. E.

Turner

L. M.

Pozdol

S. L.

(2004). Psychometric properties of the STAT for early autism screening. Journal of Autism and Developmental Disorders, 34(6), 691–701. https://doi.org/10.1007/s10803-004-5289-8

41.

Stone

W. L.

McMahon

C. R.

Henderson

L. M.

(2008). Use of the Screening Tool for Autism in Two-Year-Olds (STAT) for children under 24 months: An exploratory study. Autism, 12(5), 557–573. https://doi.org/10.1177/1362361308096403

42.

Tomlin

Koch

S. M.

Raches

Minshawi

N. F.

Swiezy

N. B.

(2013). Autism screening practices among early intervention providers in Indiana. Infants & Young Children, 26(1), 74–88. https://doi.org/10.1097/IYC.0b013e31827842b1

43.

Twardzik

Cotto-Negrón

MacDonald

(2017). Factors related to early intervention Part C enrollment: A systematic review. Disability and Health Journal, 10(4), 467–474. https://doi.org/10.1016/j.dhjo.2017.01.009

44.

Wallis

K. E.

Guthrie

Bennett

A. E.

Gerdes

Levy

S. E.

Mandell

D. S.

Miller

(2020). Adherence to screening and referral guidelines for autism spectrum disorder in toddlers in pediatric primary care. PLOS ONE, 5(5), Article e0232335. https://doi.org/10.1371/journal.pone.0232335

45.

Wiggins

L. D.

Reynolds

Rice

C. E.

Moody

E. J.

Bernal

Blaskey

Rosenberg

S. A.

Lee

L.-C.

Levy

S. E.

(2015). Using standardized diagnostic instruments to classify children with autism in the study to explore early development. Journal of Autism and Developmental Disorders, 45(5), 1271–1280. https://doi.org/10.1007/s10803-014-2287-3

46.

Williams

N. J.

Glisson

(2020). Changing organizational social context to support evidence-based practice implementation: A conceptual and empirical review. Implementation Science, 30, 145–172.

47.

Yingling

M. E.

(2019). Participation in Part C early intervention: One key to an earlier diagnosis of autism spectrum disorder? The Journal of Pediatrics, 215, 238–243. https://doi.org/10.1016/j.jpeds.2019.06.034

48.

Zwaigenbaum

Maguire

(2019). Autism screening: Where do we go from here? Pediatrics, 144(4). https://doi.org/10.1093/pch/pxz119

49.

Zwaigenbaum

Bauman

M. L.

Choueiri

Kasari

Carter

Granpeesheh

Mailloux

Roley

S. S.

Wagner

Fein

Pierce

Buie

Davis

P. A.

Newschaffer

Robins

Wetherby

Stone

W. L.

Yirmiya

Estes

Hansen

R. L.

Natowicz

M. R.

(2015). Early intervention for children with autism spectrum disorder under 3 years of age: Recommendations for practice and research. Pediatrics, 136(Suppl. 1), S60–S81. https://doi.org/10.1542/peds.2014-3667E