Knowledge of Evidence-Based Practices and Frequency of Selection Among School-Based Professionals of Students With Autism

Abstract

Autism Focused Intervention Resources and Modules (AFIRM) are a set of self-paced, online learning modules designed to disseminate information about the 27 evidence-based practices identified through a large-scale review of focused intervention practices for individuals with autism. In this study, we used descriptive statistics and an analysis of variance to analyze more than 67,000 pre-tests completed by over 22,000 school-based AFIRM users to determine whether there are differences by occupation in the (a) selection of evidence-based practices, (b) knowledge of evidence-based practices, and (c) average number of evidence-based practices selected. Results reveal statistically significant differences between groups and have implications for providers and administrators responsible for designing and delivering professional development for school-based professionals working with students with autism.

Keywords

online learning modules school-based professionals evidence-based practices

Identification and use of evidence-based practices is a major emphasis in special education (Ganz & Ayres, 2018). Identifying evidence-based practices is especially important for students with autism, as professionals and stakeholders associated with this population have a strong history of using interventions that are unsubstantiated (Travers et al., 2016; Worley et al., 2014). Using non-evidence-based interventions for students with autism is problematic because it can lead to wasted instructional time and fiscal resources, as well as more dire consequences, such as false allegations of abuse (Ganz et al., 2018; Odom et al., 2019a). In addition, teachers can experience frustration and burnout when repeated attempts to improve students’ skills fail due to the use of ineffective interventions (Travers, 2017). Clearly, the research to practice gap in special education, which Carnine (1997) discussed and criticized more than 20 years ago, is still with us.

Current efforts to address this gap, and bring research practice into everyday practice in schools, are active under different names: implementation science, improvement science, diffusion science, and dissemination science. Although different in their details, the processes consist of (a) systematically identifying practices that have scientific evidence of efficacy, (b) translating the scientific evidence into practice and accessible information that practitioners can use in their programs, and (c) supporting the school- or system-wide implementation through professional development, coaching and performance feedback, and institutional support. This is a linked set of processes, and if any of the links are missing, the prospective of closing the research to practice gap in special education is diminished (Odom, 2009).

Fortunately, evidence-based practices for students with autism, the first link in this chain, have been identified. Over the last few decades, multiple federal laws have directed educators to use practices supported by research (e.g., Every Student Succeeds Act [ESSA], 2015; Individuals With Disabilities Education Improvement Act [IDEA], 2004; No Child Left Behind Act [NCLB], 2002). In response to this national focus on evidence-based practices, the National Professional Development Center (NPDC) on Autism Spectrum Disorder conducted a large-scale, systematic review to identify focused intervention practices that have substantial empirical evidence to warrant their use (Wong et al., 2015). Focused intervention practices are procedures that practitioners can use to affect specific behavioral or developmental outcomes for individuals with autism (Odom et al., 2010). As a result of this review, the NPDC identified 27 evidence-based practices for use with students with autism.

The identification of evidence-based practices and publication in journal articles is a critical first step to addressing the research-to-practice gap. However, such identification alone is not sufficient to support their use in applied settings (McNeill, 2019; Odom, 2009; Parsons et al., 2013). Translation of such practices into user friendly and assessable information is a second essential step (Burns, 2012). A recent survey by Brock and colleagues (2014) illustrates this point. Despite decades of research supporting the use of reinforcement (Steinbrenner et al., 2020), approximately half of nearly 500 teachers and administrators surveyed reported that they were only somewhat, a little, or not at all confident implementing this evidence-based practice (Brock et al., 2014). Without an effective method of dissemination, it is unlikely that school-based professionals will incorporate evidence-based practices into their instruction.

The NPDC recognized the need to disseminate information about evidence-based practices in a manner that would be acceptable and feasible to busy school-based professionals and, in an effort to close the research to practice gap, they created the Autism Focused Intervention Resources and Modules (AFIRM; Sam et al., 2019). In developing the modules, the AFIRM team used best practice in instructional design for adult learners, incorporating videos, pictures, case examples, and knowledge checks to increase engagement among users. Each of the 27 modules focuses on a different evidence-based practice, and users have the opportunity to earn a certificate of completion by scoring proficient on a post-test when completing the certificate track. As of May 29, 2019, AFIRM had 78,031 registered users from 179 countries, and this number has continued to increase since the modules were launched in 2015 (see Sam et al., 2019).

The research on AFIRM is very much associated with the second link in the research to practice linkage. Although modules have been developed and are being accessed, the information about current users and the state of their knowledge about practices has yet to be examined. AFIRM reports 11 unique occupation categories for users (and one “other” category), with the majority of users being special education teachers (Sam et al., 2019). Other school-based professionals who use AFIRM include administrators, general education teachers, paraprofessionals, related service providers, and technical assistance providers or coaches. Analyzing data on the selection of, and knowledge about, evidence-based practices for students with autism has implications for professional development providers and administrators who are responsible for designing trainings and other professional development opportunities for school-based professionals. As such, the purpose of this study was twofold: to (a) analyze school-based AFIRM user data to determine whether there are differences in the selection of (i.e., a need to know about) evidence-based practices by occupation, and (b) examine the scores for each evidence-based practice to determine whether there are differences in the knowledge of school-based professionals by occupation. Specific research questions that are addressed in this study include the following:

Research Question 1: What evidence-based practices are school-based professionals (by occupation) selecting to learn more about?

Research Question 2: What is the knowledge of evidence-based practices for school-based professionals by occupation?

Research Question 3: Are there differences in the knowledge of evidence-based practices for school-based professionals by occupation?

Research Question 4: How many evidence-based practices are selected by respective occupation members?

Method

Data Collection

When accessing the AFIRM modules for the first time, users create a free account on the AFIRM website (https://afirm.fpg.unc.edu). To create an account, users must enter their email address, create a username and password, select their occupation from a drop-down menu, select the age range of individuals or students with autism with which they currently work, and report the state or country in which they are located. For occupations, users can choose (a) administrator, (b) early interventionist, (c) general education teacher, (d) special education teacher, (e) technical assistance provider/coach, (f) paraprofessional/instructional assistant, (g) related service provider, (h) health care provider, (i) family member, (j) university faculty, (k) university student, or (l) other. Possible choices for age ranges include (a) 0 to 2 years, (b) 3 to 5 years, (c) 6 to 11 years, (d) 12 to 14 years, (e) over 15 years, or (f) not applicable.

Once users have created an account, they choose whether to take the certificate track or the non-certificate track for each module. For the certificate track, users take a pre- and post-test consisting of 10 multiple-choice questions for the module(s) selected. The multiple-choice items represent knowledge and understanding of the key features of the evidence-based practice. Both user and pre-test data are collected through the AFIRM website and maintained on a database. Data on user’s knowledge in the current analysis were taken from the pre-test.

Data Analysis

Overall usage data on evidence-based practices by profession, selection of evidence-based practices, and knowledge of evidence-based practices (i.e., pre-test scores) were analyzed with descriptive statistics. A one-way analysis of variance (ANOVA) was conducted to examine the differences in knowledge of evidence-based practices for school-based professionals by occupation. The school-based professional’s average knowledge was calculated for each evidence-based practice. Effect sizes were calculated as partial eta squares (η²), which are interpreted as small (.01), medium (.06), and large (.14) according to Cohen (1988). Correlations were examined using Spearman’s rho correlations among average knowledge of evidence-based practice and the average use of the evidence-based practices per month.

Results

Data for this study came from a total of 22,775 unique users who identified as a school-based professional (i.e., administrator, general education teacher, paraprofessional, related service provider, special education teacher, or technical assistance provider). Special education teachers and paraprofessionals were the largest groups by occupation, and most professionals worked with elementary-aged students and students over the age of 15. See Table 1 for more specific demographic data on users, including occupation and age range served. We excluded users who worked with students with autism between the ages of 0 and 2 because this age range is not typically served in a public school setting. In addition, all users in this data set worked within the United States and completed a pre-test for at least one evidence-based practice. User data were downloaded from the AFIRM website on May 29, 2019, and are accurate as of this date.

Table 1.

School-Based AFIRM User Demographics.

Variable	n	%
Occupation
Administrator	1,176	5.2
General education teacher	1,788	7.9
Paraprofessional	6,465	28.4
Related service personnel	2,861	12.6
Special education teacher	9,887	43.4
Technical assistance provider/coach	598	2.6
Age range served
3−5 years old	3,705	16.3
6−11 years old	8,544	37.5
12−14 years old	2,734	12.0
15 years old and over	5,860	25.7
Not applicable	1,932	8.5

Note. AFIRM = Autism Focused Intervention Resources and Modules.

The 22,775 users in this data set completed a total of 67,546 pre-tests across the 27 modules (see Table 2). We used the total number of pre-tests completed to look descriptively at the total usage across the 27 modules. To compare differences across users, the average pre-test scores were collapsed across users to meet the underlying assumptions of independence for data analysis. Because the modules were released on different dates, we calculated a rate per month metric by dividing the number of times pre-tests were taken by the number of months the module went online (see Table 2). Overall, antecedent-based interventions, visual supports, and prompting were accessed the most frequently and scripting, parent-implemented interventions, and technology-aided instruction and intervention were accessed the least.

Table 2.

Rate of Selection per Month, Average Pre-Test Scores, and Total Number of Pre-Tests Completed by EBP.

EBP	Active months	Rate per month	Rank order	Pre-test	n
Antecedent-Based Intervention (ABI)	37.6	197.05	1	55.60	7,404
Cognitive-Behavioral Intervention (CBI)	27.9	88.83	9	70.75	2,482
Differential Reinforcement (DR)	21.6	99.59	6	42.17	2,151
Discrete Trial Training (DTT)	36.3	103.33	4	70.07	3,750
Exercise (ECE)	43.5	61.69	16	58.26	2,685
Extinction (EXT)	16.6	74.21	13	40.16	1,232
Functional Behavior Assessment (FBA)	42.3	90.36	8	74.60	3,820
Functional Communication Training (FCT)	22.6	94.09	7	63.36	2,131
Modeling (MD)	40.5	72.26	15	63.32	2,924
Naturalistic Intervention (NI)	20.9	57.39	19	78.00	1,198
Picture Exchange Communication System (PECS)®	37.6	74.55	11	53.66	2,801
Parent-Implemented Intervention (PII)	19.5	38.31	26	62.49	748
Peer-Mediated Instruction and Intervention (PMII)	47.3	57.99	18	65.22	2,745
Prompting (PP)	45.3	104.08	3	55.24	4,718
Pivotal Response Training (PRT)	9.0	46.85	22	53.91	422
Reinforcement (R+)	45.3	99.97	5	71.63	4,532
Response Interruption and Redirection (RIR)	16.8	57.04	20	50.39	960
Scripting (SC)	25.7	37.02	27	75.68	953
Self-Management (SM)	32.1	74.53	12	50.81	2,396
Social Narratives (SN)	43.5	58.04	17	60.20	2,526
Structured Play Groups (SPG)	13.8	52.42	21	60.04	722
Social Skills Training (SST)	40.5	76.46	10	83.27	3,094
Task Analysis (TA)	42.3	72.43	14	66.44	3,062
Technology-Aided Instruction and Intervention (TAII)	15.3	39.34	25	87.20	600
Time Delay (TD)	45.3	39.60	24	57.57	1,795
Video Modeling (VM)	16.2	44.08	23	70.79	713
Visual Supports (VS)	45.3	109.90	2	54.68	4,982
Total					67,546

Note. EBP = evidence-based practice; Active months = number of months the module has been publicly available; Rate per month = average number of pre-tests completed per month (obtained by dividing the total number of pre-tests completed by the number of months since the module was released); rank order = the order in which the practices are selected per month, from greatest (1) to least (27); Pre-test = the average pre-test score; n = total number of pre-tests completed.

Evidence-Based Practice Selection by Occupation

When analyzing the data on the number of times pre-tests were completed for each evidence-based practice by occupation (i.e., number of times each evidence-based practice was selected), antecedent-based intervention was the top selected practice across all occupations (see Table 3). More specifically, the second top practice selected for special education teachers, general education teachers, and administrators was visual supports. For paraprofessionals and technical assistance providers, the second top selection was prompting. The third most frequently selected evidence-based practice varied across participants and included discrete trial training (special education teachers and technical assistance providers), differential reinforcement (paraprofessionals and related service providers), functional behavior assessment (administrators), and modeling (general education teachers). The least selected evidence-based practices by occupation included parent-implemented intervention (special education teachers), scripting (general education teachers, administrators, and technical assistance providers), and technology-aided instruction and intervention (paraprofessionals and related service providers). Also infrequently selected was video modeling (paraprofessionals and related service providers), time delay (general education teachers), and technology-aided instruction and intervention (administrators).

Table 3.

EBP Selection by Occupation.

EBP^a	Administrator		General education teacher		Paraprofessional		Related service		SPED teacher		TA provider
EBP^a	Use per month	Rank order	Use per month	Rank order	Use per month	Rank order	Use per month	Rank order	Use per month	Rank order	Use per month	Rank order
ABI	9.63	1	12.32	1	71.11	1	22.17	1	76.20	1	5.62	1
CBI	3.19	7	6.23	7	35.57	5	10.20	8	31.10	13	2.54	7
DR	2.87	9	4.72	16	41.86	3	12.50	3	35.28	10	2.36	8
DTT	3.80	5	4.08	18	34.25	6	11.32	4	46.87	3	3.00	3
ECE	2.37	13	4.66	17	20.63	16	7.63	16	24.19	18	2.21	10
EXT	1.75	20	3.79	20	29.22	8	8.01	14	29.34	14	2.11	11
FBA	4.49	3	7.24	4	25.57	10	10.64	6	39.46	7	2.96	4
FCT	3.09	8	5.43	14	28.34	9	14.04	2	40.97	4	2.21	9
MD	2.30	15	7.49	3	24.44	11	7.51	17	28.47	15	2.05	13
NI	1.77	19	3.11	23	22.42	13	6.13	22	22.42	21	1.53	19
PECS	2.32	14	5.64	12	19.96	17	9.47	10	35.42	9	1.73	17
PII	1.38	22	2.97	25	11.11	25	4.92	24	16.70	27	1.23	22
PMII	2.07	18	5.98	9	13.58	22	6.74	19	28.12	16	1.50	20
PP	3.40	6	5.23	15	42.75	2	9.64	9	39.80	5	3.26	2
PRT	1.44	21	3.00	24	15.99	19	6.22	21	19.10	23	1.11	23
R+	3.90	4	6.68	5	38.14	4	8.96	11	39.49	6	2.80	5
RIR	2.08	17	3.33	22	21.80	15	6.30	20	21.92	22	1.60	18
SC	0.82	27	2.06	27	11.38	24	5.13	23	16.86	26	0.78	27
SM	2.40	12	5.72	10	23.39	12	7.81	15	33.16	12	2.05	12
SN	2.21	16	5.54	13	14.73	20	8.02	13	26.28	17	1.26	21
SPG	1.31	24	3.70	21	14.45	21	8.42	12	22.80	20	1.74	16
SST	2.72	11	5.66	11	21.87	14	10.55	7	33.78	11	1.88	14
TA	2.79	10	6.10	8	18.92	18	6.77	18	36.03	8	1.82	15
TAII	0.98	26	6.29	6	9.11	27	3.93	27	17.96	25	1.05	24
TD	1.28	25	2.21	26	12.20	23	4.28	25	18.64	24	0.99	25
VM	1.36	23	3.83	19	10.02	26	4.20	26	23.68	19	0.99	26
VS	4.57	2	8.74	2	32.67	7	11.01	5	50.19	2	2.74	6

Note. EBP = evidence-based practice; Use per month = average number of pre-tests completed per month (obtained by dividing the total number of pre-tests completed by the number of months since the module was released); rank order = the order in which the practices are selected per month, from greatest (1) to least (27); SPED = special education; TA = technical assistance; ABI = Antecedent-Based Intervention; CBI = Cognitive-Behavioral Intervention; DR = Differential Reinforcement; DTT = Discrete Trial Training; ECE = Exercise; EXT = Extinction; FBA = Functional Behavior Assessment; FCT = Functional Communication Training; MD = Modeling; NI = Naturalistic Intervention; PECS = Picture Exchange Communication System®; PII = Parent-Implemented Intervention; PMII = Peer-Mediated Instruction and Intervention; PP = Prompting; PRT = Pivotal Response Training; R+ = Reinforcement; RIR = Response Interruption and Redirection; SC = Scripting; SM = Self-Management; SN = Social Narratives; SPG = Structured Play Groups; SST = Social Skills Training; TA = Task Analysis; TAII = Technology-Aided Instruction and Intervention; TD = Time Delay; VM = Video Modeling; VS = Visual Supports.

See Table 2 for more information about each EBP.

Knowledge of Evidence-Based Practices by Occupation

When considering the top three evidence-based practices that school-based professionals had the highest knowledge, all occupations consistently scored highest on technology-aided instruction and intervention, social skills training, and naturalistic interventions, with the exception of technical assistance providers, who scored highest on technology-aided instruction and intervention, social skills training, and scripting (see Table 4). All occupations consistently had the lowest knowledge of extinction and differential reinforcement. Knowledge of response interruption and redirection was also low for the majority of occupations (i.e., general education teachers, related service personnel, special education teachers, and technical assistance providers/coaches). Administrators had a low knowledge of pivotal response training and paraprofessionals scored low on self-management.

Table 4.

Pre-Test Scores by Role for Each EBP.

EBP^a	Administrator		General education teacher		Paraprofessional		Related service		SPED teacher		TA provider/coach
EBP^a	Pre-test	Rank	Pre-test	Rank	Pre-test	Rank	Pre-test	Rank	Pre-test	Rank	Pre-test	Rank
ABI	59.31	17	50.99	21	51.35	23	60.71	19	58.35	18	55.55	20
CBI	75.28	6	70.86	7	67.92	6	76.81	8	71.58	8	70.00	9
DR	44.84	26	41.08	26	40.27	26	45.93	26	43.03	26	41.96	26
DTT	73.26	7	64.93	9	66.19	8	73.94	9	71.99	7	72.84	8
ECE	59.22	19	58.42	14	56.27	16	62.92	15	58.36	17	58.44	18
EXT	38.97	27	38.10	27	39.40	27	40.30	27	41.25	27	39.71	27
FBA	76.68	4	71.63	6	70.95	5	78.16	4	76.35	5	74.08	6
FCT	71.14	9	57.80	15	55.97	18	69.94	11	66.38	12	63.20	14
MD	62.47	15	61.55	13	59.43	14	69.34	13	65.64	13	62.77	15
NI	81.08	3	76.15	3	75.19	3	81.72	3	79.70	3	79.69	4
PECS	52.53	23	48.63	24	51.47	22	58.90	21	54.40	22	52.77	24
PII	69.63	12	62.07	12	61.24	11	62.08	17	62.76	14	64.58	13
PMII	64.90	13	62.30	11	61.09	12	70.22	10	66.53	11	67.89	10
PP	55.13	21	56.88	16	53.68	19	57.92	22	56.05	20	55.20	21
PRT	49.23	25	50.00	22	52.85	20	59.82	20	53.90	23	53.00	23
R+	72.60	8	72.48	5	67.87	7	77.93	6	73.47	6	73.23	7
RIR	52.00	24	46.25	25	50.22	24	53.96	25	49.95	25	51.11	25
SC	75.71	5	72.83	4	72.66	4	78.11	5	77.10	4	80.50	3
SM	59.09	20	49.35	23	45.81	25	54.10	24	53.03	24	53.64	22
SN	61.35	16	56.27	17	57.44	15	65.19	14	60.93	15	60.91	16
SPG	62.78	14	54.51	18	60.30	13	62.67	16	59.59	16	60.83	17
SST	87.27	2	80.26	2	80.76	2	86.18	2	84.16	2	83.29	1
TA	69.83	11	62.40	10	64.65	10	69.90	12	67.22	10	65.19	12
TAII	92.00	1	85.10	1	85.83	1	89.17	1	88.18	1	83.13	2
TD	59.31	17	53.70	19	56.24	17	61.96	18	57.30	19	66.22	11
VM	70.91	10	70.00	8	65.80	9	76.91	7	71.57	9	79.38	5
VS	54.98	22	52.98	20	52.38	21	56.25	23	56.04	21	55.73	19
Average	64.65		60.50		58.78		66.54		63.87		63.07

Note. EBP = evidence-based practice; Admin = administrator; SPED = special education teacher; TA = technical assistance; Pre-test = average pre-test score; Rank = the order of the pre-test scores, from greatest (1) to least (27); ABI = Antecedent-Based Intervention; CBI = Cognitive-Behavioral Intervention; DR = Differential Reinforcement; DTT = Discrete Trial Training; ECE = Exercise; EXT = Extinction; FBA = Functional Behavior Assessment; FCT = Functional Communication Training; MD = Modeling; NI = Naturalistic Intervention; PECS = Picture Exchange Communication System®; PII = Parent-Implemented Intervention; PMII = Peer-Mediated Instruction and Intervention; PP = Prompting; PRT = Pivotal Response Training; R+ = Reinforcement; RIR = Response Interruption and Redirection; SC = Scripting; SM = Self-Management; SN = Social Narratives; SPG = Structured Play Groups; SST = Social Skills Training; TA = Task Analysis; TAII = Technology-Aided Instruction and Intervention; TD = Time Delay; VM = Video Modeling; VS = Visual Supports.

See Table 2 for more details about each EBP.

Differences in Knowledge of Evidence-Based Practices by Occupation

A one-way between-subjects ANOVA was conducted to examine whether there were significant differences in average knowledge of evidence-based practices by occupation (see Table 5). Overall, there were significant differences in average knowledge across evidence-based practices by occupation, F(5, 22769) = 167.55, p < .001, η² = .04. Post hoc comparisons were examined using the Games–Howell test due to unequal sample sizes across occupations, using a Bonferonni correction for multiple comparisons (p < .003). Post hoc comparisons indicated that the average knowledge of evidence-based practices for related service providers was significantly higher than all other occupations. In addition, both administrators and special education teachers had a statistically significant higher knowledge of evidence-based practices than general education teachers or technical assistance providers/coaches. Finally, both general education teachers and technical assistance providers/coaches had a higher knowledge than paraprofessional/instructional assistants.

Table 5.

Mean Knowledge Score of Evidence-Based Practices by Occupation and One-Way ANOVA Post Hoc Multiple Comparison Significance Findings.

Occupation	n	M	SD	Post hoc test
Related service professional (A)	2,861	67.30	15.42	A > B, C, D, E, F
Administrator (B)	1,176	64.68	16.72	B > E, F
Special education teacher (C)	9,887	64.11	15.88	C > E, F
Technical assistant provider/coach (D)	598	62.17	17.08	D > F
General education teacher (E)	1,788	60.54	16.04	E > F
Paraprofessional/instructional assistant (F)	6,465	58.39	16.02

Note. ANOVA = analysis of variance.

Number of Evidence-Based Practices Selected by Occupation

When considering the mean number of modules for which school-based professionals completed pre-tests, paraprofessionals selected to learn about the most evidence-based practices, followed by technical assistance providers/coaches, special education teachers, general education teachers, related service personnel, and finally administrators (see Table 6).

Table 6.

Number of AFIRM Modules Selected by Occupation.

Occupation	M	SD	95% CI (lower)	95% CI (upper)	Range
Administrator	2.13	2.58	1.98	2.28	1−26
General education teacher	2.68	3.46	2.52	2.84	1−27
Paraprofessional	3.41	4.14	3.31	3.51	1−27
Related service personnel	2.66	3.13	2.55	2.78	1−27
Special education teacher	2.90	3.40	2.84	2.97	1−27
Technical assistance provider/coach	3.10	3.84	2.79	3.41	1−27

Note. AFIRM = Autism Focused Intervention Resources and Modules; CI = confidence interval.

Discussion

In the research–dissemination–implementation linkage designed to close the research to practice gaps, this research fits firmly in the middle link. Research into school-based professionals’ current knowledge of the features of evidence-based practices and the utilization of dissemination information about specific practices by specific professionals will inform planning for professional development, which is a key feature of the implementation process. This study is the first to investigate these topics with different groups of school-based professionals and a substantial number of participants across the United States. By analyzing evidence-based practice selection and knowledge across occupations, we provide detailed information that can be used by professional development providers and administrators when designing and delivering trainings to school-based professionals.

Evidence-based practices that school-based professionals most frequently selected included foundational practices that are commonly used to address educational and support needs for students with autism (i.e., visual supports, prompting, reinforcement, discrete trial teaching). They also frequently selected practices that addressed behaviors that interfere with learning, such as antecedent-based intervention and functional behavior assessment. Differential reinforcement was one of the top 10 selected evidence-based practices across all occupations, with the exception of general education teachers. One possible reason for this finding is that general education teachers may be unfamiliar with differential reinforcement given that it is typically used by individuals with training in applied behavior analysis. Instead, general education teachers tend to select and learn about reinforcement (R+) more generally.

Surprisingly, technology-aided instruction and intervention, parent-implemented interventions, scripting, time delay, and video modeling were among the least frequently selected practices. Scripting and time delay are noted as foundational practices for students with autism yet are rarely selected (Sam & Hume, 2019). Parent-implemented intervention may have been selected less because of the parameters set for the sample. That is, parent-implemented interventions are often used in early intervention programs, especially if they follow a naturalistic developmental behavioral intervention orientation (Schreibman et al., 2015); professionals self-identified as working in early intervention were not included in the sample for this study. Given interest in technology by many individuals with autism, it was surprising that technology-aided instruction and intervention and visual supports were not selected as frequently as other evidence-based practices. Also, time delay, which the authors believe is a very effective practice in reducing prompt dependence, may be seen by users as just an extension of prompting (i.e., a frequently selected practice) and not selected for that reason. An alternative reason that time delay was not selected frequently is that school-based professionals may believe they already know how to implement this practice and thus do not need additional training in this area.

Knowledge of evidence-based practices across all occupations was lowest for extinction and differential reinforcement, with many professionals also having a low knowledge of response interruption and redirection. These practices are part of the broader Positive Behavioral Interventions and Supports (PBIS) initiative that has become a part of many public school programs (Sugai & Horner, 2020), suggesting that there is a national intent to address challenging behavior in an evidence-based way and the need for further training in this area. Importantly, antecedent-based intervention (another key PBIS intervention strategy) is the intervention that was selected most often across groups. These findings suggest a major professional development need is this area, which hopefully the PBIS movement is addressing, and for which the AFIRM modules may be a useful tool. In addition, equipping professionals with skills to address challenging behavior may reduce this major predictor of stress and burnout for teachers (Brunsting et al., 2014).

When analyzing statistically significant differences between groups, related service personnel had a higher knowledge than all other occupations. This finding could be attributed to the higher level of education typically required for these positions (e.g., at least a master’s degree for speech pathologist, school psychologist). Administrators and special education teachers also had a higher knowledge of evidence-based practices than general education teachers and paraprofessional/instructional assistants. This may be related to the higher level of education required for administrative positions and the focus on accommodations and behavioral strategies in many special education teacher preparation programs that are not generally included in coursework for other occupations, such as general education teachers. Given the low level of knowledge across occupations, training and professional development on evidence-based practices for students with autism remain important for school-based professionals, regardless of their role.

The finding that paraprofessionals had the lowest knowledge of evidence-based practices across all occupations is not surprising given the lack of preparation for this population and indicates a continuing need for additional training (Chopra et al., 2004; Cockroft & Atkinson, 2015; Fisher & Pleasants, 2011; Russel et al., 2015). Professional development in evidence-based practices for students with autism is especially important given that paraprofessionals often spend a large proportion of their time providing instruction to students with disabilities (Jones et al., 2012) and that paraprofessionals’ delivery of special education instruction may be less effective than teacher-delivered instruction (Giangreco et al., 2014). Despite having a low knowledge of evidence-based practices, paraprofessionals selected the highest number of evidence-based practices to learn more about which indicates a desire to improve their skills. This is encouraging and suggests that paraprofessionals are generally receptive to receiving additional training and professional development. Online learning modules seem to be especially appropriate for this population given that many paraprofessionals are not compensated for attending trainings outside of the school day and may have limited opportunities to engage in professional development.

Technical assistance providers/coaches had significantly lower knowledge scores compared with related service professionals, administrators, and special education teachers, and they also tended to select the least number of modules (exceeded only by administrators). This finding could be problematic given that technical assistance providers/coaches are often tasked with providing professional development to the professionals who have a higher baseline knowledge than them according to the pre-test scores. To be effective, technical assistance providers/coaches must be knowledgeable on the content. These findings do suggest that supervisors charged with planning professional development in their programs should first ensure that their technical assistance providers/coaches have the requisite knowledge and skills about the practices for which they will be providing training and support.

Limitations

There are several limitations to this study that should be considered when interpreting the results. First, in this study, knowledge was defined as performance on the module pre-test. Although this provides information about which evidence-based practices that school-based professionals are choosing to learn about, a fuller picture would include the knowledge gained through use of the modules, which was reported in Sam et al. (2019). Second, the pre-tests used in this study were developed by the authors of the modules, so there is a possibility that the level of difficulty may have been different across modules, which could have affected the results. Also, to date, there are no reliability data on the pre-tests given, which could certainly be a direction for future research. Third, the modules are listed in alphabetical order on the AFIRM website, and it is possible that the ordering of the modules influenced the frequency that they were selected. For example, antecedent-based intervention was the most frequently selected practice and also the first module listed on the AFIRM website. Finally, the data from this study represent a unique population in that all users chose to engage in a self-paced, technology-based professional development. As such, the results from this study may not be representative of all school-based professionals.

Future Research and Directions

Future research should investigate reasons why school-based professionals chose to learn about some practices more frequently than others. Insight into these reasons would be valuable information for administrators and other professional development providers responsible for designing and planning training opportunities for school-based professionals. In addition to selection, knowing which evidence-based practices school-based professionals are more or less knowledgeable about provides some guidance on which practices may be helpful to target in professional development. However, having a high level of knowledge about a practice may not equate to having a high level of implementation fidelity on the corresponding practice. Future research should investigate the correlation between knowledge of evidence-based practices and implementation fidelity to provide further insight into the relationship between school-based professionals’ knowledge and use of evidence-based practices. Relatedly, although prior research has demonstrated that online learning modules do increase knowledge (Sam et al., 2019), it is still unknown whether this increase in knowledge also results in an increase in fidelity of implementation. Implementation science suggests that knowledge acquisition is an essential and important first step, but for many practices, further coaching and performance feedback is necessary (Odom et al., 2019b). Given that the ultimate goal of professional development is to impact practice, future research should investigate empirically whether online learning modules are sufficient for changing practice alone or whether additional supports are needed.

Finally, research is needed on who would best be able to provide coaching or performance feedback on implementation of practices. For example, one option for training paraprofessionals in evidence-based practices is equipping special education teachers who are in supervisory roles to provide “on-the-job” coaching in the implementation of evidence-based practices. Having special education teachers deliver training to paraprofessionals could be more cost-effective than hiring outside trainers. However, despite the benefits of having special education teachers provide on-the-job training to paraprofessionals, they are often not prepared to do so (Walker & Smith, 2015). As such, developing and evaluating a model of professional development to equip special education teachers to provide training to paraprofessionals in evidence-based practices for students with autism is an important next step.

Conclusion

Dissemination, when it involves the translation of scientific knowledge into practical knowledge, is the essential linkage between research and implementation. The current study provided information about the knowledge that school-based professionals have about the specific features of evidence-based practices and the needs for training as reflected in the usage patterns of the AFIRM modules. As school leaders move forward in supporting school professionals in implementing evidence-based practices in their school and classrooms, they may use this information to target potential practices that might be most useful for specific professionals. However, we emphasize that dissemination and the availability of professional-friendly information will only get us so far in narrowing the research to practice gap. Learning the lessons from implementation science—support at the inner and outer system level, leadership, and coaching with performance feedback—may also be required for professionals’ initial and sustained use of the practices for children and youth with autism.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The research reported in this article was supported by a subcontract from Grant No. H325E170001, a project funded by the Office of Special Education Programs. Additional support for the development of the Autism Focused Intervention Resources and Modules (AFIRM) work came from Grants R324A150047, R324A170028, and R324B160038 funded by the Institute of Education Sciences. Both funding entities are part of the U.S. Department of Education, but the contents of this article do not necessarily reflect or represent the policy of that department.

ORCID iD

Kristi L. Morin

References

Brock

M. E.

Huber

H. B.

Carter

E. W.

Juarez

A. P.

Warren

Z. E.

(2014). Statewide assessment of professional development needs related to educating students with autism spectrum disorder. Focus on Autism and Other Developmental Disabilities, 29, 67–79. https://doi.org/10.1177/1088357614522290

Brunsting

N. C.

Sreckovic

M. A.

Lane

K. L.

(2014). Special education teacher burnout: A synthesis of research from 1979 to 2013. Education and Treatment of Children, 37, 681–711.

Burns

M. K.

(2012). Meta-analysis of single-case design research: Introduction to the special issue. Journal of Behavioral Education, 21, 175–184. https://doi.org/10.1007/s10864-012-9158-9

Carnine

(1997). Bridging the research-to-practice gap. Exceptional Children, 63, 513–521. https://doi.1177/001440299706300406

Chopra

R. V.

Sandoval-Lucero

Aragon

Bernal

De Balderas

H. B.

Carroll

(2004). The paraprofessional role of connector. Remedial and Special Education, 25(4), 219–231. https://doi.org/10.1177/07419325040250040501

Cockroft

Atkinson

(2015). Using the wider pedagogical role model to establish learning support assistants’ views about facilitators and barriers to effective practice. Support for Learning, 30(2), 88–104. https://doi.org/10.1111/1467-9604.12081

Cohen

(1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum.

Every Student Succeeds Act of 2015, Pub. L. No. 114-95 § 114 Stat. 1177 (2015–2016).

Fisher

Pleasants

S. L.

(2011). Roles, responsibilities, and concerns of paraeducators: Findings from a statewide survey. Remedial and Special Education, 33(5), 287–297. https://doi.org/10.1177/0741932510397762

10.

Ganz

J. B.

Ayres

K. M.

(2018). Methodological standards in single-case experimental design: Raising the bar. Research in Developmental Disabilities, 79, 3–9. https://doi.org/10.1016/j.ridd.2018.03.003

11.

Ganz

J. B.

Katsiyannis

Morin

(2018). Facilitated communication: The resurgence of a disproven treatment for individuals with autism. Intervention in School and Clinic, 54, 52–56. https://doi.org/10.1177/1053451217692564

12.

Giangreco

M. F.

Doyle

M. B.

Suter

J. C.

(2014). Teacher assistants in inclusive classrooms. In Florian

(Ed.), The SAGE handbook of special education: Two volume set (2nd ed., pp. 691–702). SAGE. https://doi.org/10.4135/9781446282236.n42

13.

Individuals With Disabilities Education improvement Act, 20 U.S.C. § 1400 (2004).

14.

Jones

C. R.

Ratcliff

N. J.

Sheehan

Hunt

G. H.

(2012). An analysis of teachers’ and paraeducators’ roles and responsibilities with implications for professional development. Early Childhood Education, 40, 19–24. https://doi.org/10.1007/s10643-011-0487-4

15.

McNeill

(2019). Social validity and teachers’ use of evidence-based practices for autism. Journal of Autism and Developmental Disorders, 49, 4585–4594. https://doi.org/10.1007/s10803-019-04190-y

16.

No Child Left Behind Act of 2001, P.L. 107-110, 20 U.S.C. § 6319 (2002).

17.

Odom

S. L.

(2009). The tie that binds: Evidence-based practice, implementation science, and early intervention. Topics in Early Childhood Special Education, 29, 53–61.

18.

Odom

S. L.

Collet-Klinenberg

Rogers

Hatton

(2010). Evidence-based practices in interventions for children and youth with autism spectrum disorders. Preventing School Failure: Alternative Education for Children and Youth, 54, 275–282.

19.

Odom

S. L.

Hall

L. J.

Suhrheinrich

(2019b). Implementation science, behavior analysis, and supporting evidence-based practices for individuals with autism. European Journal of Behavior Analysis, Online Prepublication. https://doi.org/10.1080/15021149.2019.1641952

20.

Odom

S. L.

Morin

Savage

M. N.

Tomaszewski

(2019a). Behavioral and educational interventions. In. Volkmar

(Ed.), Autism and pervasive developmental disorders (3rd ed., pp. 176–190). Cambridge University Press. https://doi.org/10.1017/9781108297769.009

21.

Parsons

Charman

Faulkner

Wallace

Witttemeyer

(2013). Commentary—Bridging the research and practice gap in autism: The importance of creating research partnerships with schools. Autism, 17, 268–280. https://doi.org/10.1177/1362361312472068

22.

Russel

Allday

Duhon

(2015). Effects of increasing distance of a one-on-one paraprofessional on student engagement. Education and Treatment of Children, 38(2), 193–210.

23.

Sam

Cox

A. W.

Savage

M. N.

Waters

Odom

S. L.

(2019). Disseminating information on evidence-based practices for children and youth with autism spectrum disorder: AFIRM. Journal of Autism and Developmental Disorders, 50, 1931–1940. https://doi.org/10.1007/s10803-019-03945-x

24.

Sam

Hume

(2019). Learning supports for students on the autism spectrum. In Jordan

Roberts

Hume

(Eds.), The SAGE handbook of autism and education (pp. 236–248). SAGE.

25.

Schreibman

Dawson

Stahmer

A. C.

Landa

Rogers

S. J.

McGee

G. G.

Kasari

Ingersoll

Kaiser

A. P.

Bruinsma

McNerney

Wetherby

Halladay

(2015). Naturalistic developmental behavioral interventions: Empirically validated treatments for autism spectrum disorder. Journal of Autism and Developmental Disorders, 45, 2411–2428.

26.

Steinbrenner

J. R.

Hume

Odom

S. L.

Morin

K. L.

Nowell

S. W.

Tomaszewski

Szendrey

McIntyre

N. S.

Yücesoy-Özkan

Savage

M. N.

(2020). Evidence-based practices for children, youth, and young adults with autism. Frank Porter Graham Child Development Institute, National Clearinghouse on Autism Evidence and Practice Review Team, The University of North Carolina at Chapel Hill. https://ncaep.fpg.unc.edu/sites/ncaep.fpg.unc.edu/files/imce/documents/EBP%20Report%202020.pdf

27.

Sugai

Horner

R. H.

(2020). Sustaining and scaling positive behavioral interventions and supports: Implementation drivers, outcomes, and considerations. Exceptional Children, 86, 120–136. https://doi.org/10.1177/0014402919855331

28.

Travers

J. C.

(2017). Evaluating claims to avoid pseudoscientific and unproven practices in special education. Intervention in School and Clinic, 52, 195–203. https://doi.org/10.1177/1053451216659466

29.

Travers

J. C.

Ayres

Simpson

R. L.

Crutchfield

(2016). Fad, pseudoscientific, and controversial interventions. In Lang

Hancock

T. B.

Singh

N. N.

(Eds.), Early intervention for young children with autism spectrum disorder (pp. 257–293). Springer. https://doi.org/10.1007/978-3-319-30925-5_9

30.

Walker

V. L.

Smith

C. G.

(2015). Training paraprofessionals to support students with disabilities: A literature review. Exceptionality, 23, 170–191. https://doi.org/10.1080/09362835.2014.986606

31.

Wong

Odom

S. L.

Hume

Cox

A. W.

Fettig

Kucharczyk

Matthew

Brock Joshua

Plavnick Veronica

Fleury Schultz

T. R.

(2015). Evidence-based practices for children, youth, and young adults with autism spectrum disorder: A comprehensive review. Journal of Autism and Developmental Disorders, 45, 1951–1966. https://doi.org/10.1007/s10803-014-2351-z

32.

Worley

J. A.

Fodstad

J. C.

Neal

(2014). Controversial treatments for autism spectrum disorders. In Tarbox

Dixon

D. R.

Sturmey

Matson

J. L.

(Eds.), Handbook of early intervention for autism spectrum disorders: Research, policy, and practice (pp. 487–510). Springer. https://doi.org/10.1007/078-1-4939-0401-3