Enhancing Professional Development for Educators: Adapting Project ECHO From Health Care to Education

Abstract

Adequately prepared educators are critical to the success of children in rural settings, but many educators receive little training on how to support students with disabilities. Professional development for educators is challenging in rural areas due to the lack of opportunities and travel requirements. Project ECHO™ was originally developed to overcome similar challenges facing physicians. The ECHO Model overcomes these challenges by creating a hub and spoke network that connects local providers with experts. These networks meet regularly over teleconferencing technology to conduct a short professional development workshop, followed by a case-study from one of the local sites. We adapted the ECHO Model for use in education and since 2014 implemented four networks: autism, assistive technology, secondary transitions, and behavioral supports. Educators found ECHO highly acceptable and reported improved skills and knowledge. ECHO for Education is effective and has the potential to improve capacity of education systems in rural settings.

Keywords

extension for community health care outcomes (ECHO) model professional learning and development communities of practice model of teacher change rural special education

According to the National Assessment of Educational Progress (NAEP), many students in the United States are not meeting grade-level expectations, causing a delay in educational goal attainment and reductions in student success. In fact, recent data suggest less than 40% of graduating high school students are prepared to meet collegiate-level requirements (National Center for Education Statistics, 2018). Moreover, these negative outcomes are further amplified among students with disabilities and require additional consideration (National Center for Education Statistics, 2018). Collectively, individuals with disabilities make up more than 13% of all public school students, or approximately 6.7 million students (Kraus, 2017). These students include a range of identifications including autism, deaf–blindness, deafness, emotional disturbance, hearing impairment, intellectual disability, multiple disabilities, orthopedic impairment, other health impairments, specific learning disability, speech or language impairment, traumatic brain injury, and visual impairment including blindness (U.S. Department of Education, 2018). Despite engagement in special educational services and targeted interventions, students with disabilities exhibit substantially worse educational outcomes as compared to their peers without disabilities and as a result, may be falling further behind in their educational goal attainment and success (McFarland et al., 2017).

While most educators receive thorough initial training and state-specific licensing, this may not prepare them adequately for educating children with disabilities (Yarbro et al., 2007). Indeed, there is no set of official or national standards for competencies for educating students with disabilities, thus leading to diverse requirements for training and ongoing professional learning and development (PLD; Yarbro et al., 2007). As a result, educators may not be able to support the specific needs of students with disabilities (Johnson et al., 2012; Lindsay et al., 2013). Furthermore, even special education teachers may not always be prepared with the specific and evolving skills necessary to identify and address student learning and behavioral needs in all disability categories or content areas (Vaughn & Swanson, 2015). Barriers to effective PLD are further amplified in rural settings where resources are limited and schools often struggle to retain well-prepared educators (Cook et al., 2015; Spatig-Amerikaner, 2012).

Fortunately, research suggests that improving educator training to work with students with disabilities ultimately leads to improved student outcomes. According to Barber and Mourshed (2007), “the quality of an education system cannot exceed the quality of its teachers” (p. 11). While teacher quality is a complex issue, it encompasses the ability of teachers to effectively educate diverse students, despite other learning challenges. However, given that the education of children with disabilities may require additional planning and coordination among numerous disciplines, educators may need additional PLD to successfully teach such students (Council for Exceptional Children [CEC], 2018). Therefore, additional teacher training is often needed for students with disabilities to succeed. Indeed, a variety of factors are likely to impact educator efficacy including subject-matter knowledge, ability to engage students in learning, teaching skills/pedagogical practices, creativity in planning and delivering instruction, and managing the classroom learning environment (Yarbro et al., 2007). Improving teacher quality is, therefore, critical to improving outcomes of students with disabilities.

Unfortunately, such PLD opportunities are not typically available in rural locations (Cook et al., 2015; Yarbro et al., 2007); therefore, limiting the success of students with disabilities in rural districts. To receive ongoing PLD, educators may need to travel out of state to attend disability-specific conferences or contact specialists to conduct intensive in-person consultation to learn about best and promising practices. These traditional approaches may carry significant financial burden to the educators and districts and take educators out of the classroom for significant periods of time (Yarbro et al., 2007). As such, educators in rural settings are often unable to obtain additional training on the full range of interventions available to support learning for all students (Kurth & Keegan, 2014). As a result, efficient and effective PLD strategies that can be deployed to rural districts are crucially needed to help all educators serve their students with disabilities.

The Need for Effective and Efficient PLD

PLD involves formal and informal strategies used across disciplines to acquire new knowledge, improve effectiveness, maintain competence, comply with professional regulation, retain employment, and/or enhance career progression (Education Council of Learning Forward, 2017; Jensen et al., 2016). PLD programs provide opportunities for professionals to learn about strategies that may change their attitudes and beliefs while expanding their skills to improve the outcomes of their students (Guskey, 2002) and may be required to maintain licensure. Current federal legislation highlights the need for PLD that is sustained, research-based, classroom-focused, and job-embedded (U.S. Department of Education, 2017; Yarbro et al., 2007).

While there is a range of PLD programs for educators, researchers suggests that to be effective PLD programs need to be based on the needs of the educators and relevant to their current situation (Fishman et al., 2003; Guskey, 2002). The Model for Teacher Change (MTC) suggests that to be effective in changing student outcomes, PLD programs should include four core features: (a) delivery of content knowledge, (b) active learning (i.e., hands-on work), (c) implementation of those skills in daily routines, and (d) follow-up and support after initial training to allow for continued adaptation (Desimone, 2009; Desimone et al., 2002; Garet et al., 2001; Guskey, 2002; Lomos et al., 2011). This model suggests that to create enduring change PLD needs to be understood as an ongoing process, rather than isolated events that may be offered infrequently.

However, many PLD programs do not provide these core features and have not been effectively adapted to improve the outcome of individuals with disabilities. For example, programs that are short-term workshops that occur outside of their professional setting often have no mechanism for the educators to apply the skills in their own work environment or receive feedback from their peers (Fishman et al., 2003). Moreover, logistical barriers often exist that prevent educators from accessing high-quality PLD (Fishman et al., 2003; Guskey, 2002; McConnell et al., 2013). Travel to conferences or workshops can take educators away from their classrooms and can be expensive. Cuts to district budgets threaten the creation and maintenance of effective, high-quality professional development workshops (Masters et al., 2010) and result in fewer opportunities for educators to receive professional development outside of their own district (Colbert et al., 2008). Furthermore, these challenges are exacerbated in rural communities where travel times are often much longer and resources for quality support during the PLD opportunities are limited (Kurth & Keegan, 2014). Identifying programs that include the components of effective PLD outlined by the MTC and that overcome the challenges endemic to rural communities is critical to ensuring the success of students with disabilities nationwide.

The ECHO Model: A Framework for Effective PLD

One way to potentially improve current PLD models is through an adaptation of PLD programs using Project ECHO (Arora et al., 2007). Project ECHO is a distance-based PLD model that was originally designed for health care providers and is based on multipoint-videoconferencing. The ECHO model (Arora et al., 2007) consists of four primary components:

(a) Use of technology, such as remote video conferencing, to leverage scarce resources.

(b) Didactic training on best-practice PLD topics.

(c) Cases presented by spoke participants to allow for case-based learning and ongoing disease co-management facilitated by the interdisciplinary hub experts.

(d) Continual program evaluation to determine the effectiveness of this PLD.

The ECHO model is designed to create an ideal professional development model by balancing the sharing of expertise with ongoing virtual interactions from all network members. Case presentations are based on current problems of practice ranging from individual student to classroom issues. Over time, ECHO clinics create a community of practice in which “all teach, all learn,” (a common ECHO saying) and new insights emerge to address system barriers for implementation of best practices. This model allows patients in rural areas to receive high-quality specialty care by their primary-care providers rather than traveling to large specialty care centers (Arora et al., 2011). Essentially, weekly ECHO sessions provide a platform for professionals to collaboratively engage in case-based, applied learning in which clinicians apply best practices to the care of individual patients and discuss ways to remove barriers to implementation and ideas for system improvement.

From its inception, Project ECHO has garnered a tremendous amount of interest (e.g., Arora et al., 2016). This led to major efforts to increase the use of the model, generalize the model beyond the health care setting, and to study the effectiveness of the ECHO model. Of particular note, the ECHO Act (“Echo act,” 2016a, 2016b) was signed into federal law to encourage the development and evaluation of the ECHO model, as well as other methods to increase health care capacity. As part of this act, a comprehensive review of research and evaluation of the ECHO model was conducted. This analysis revealed that there is strong evidence to support its use, with 211 peer reviewed publications demonstrating improved quality of care, better access to care, and lower cost with the use of ECHO in health care (Fischer et al., 2019). Thus, the ECHO model is an effective tool to improve physician efficacy by increasing opportunities for education and collaboration across professionals. However, the ECHO model had not been adapted for educational settings. Given that ECHO is a general PLD framework, we adapted this model for use with teachers working with students with disabilities.

The Adaptation of the ECHO Model for Education

The Wyoming Institute for Disabilities (WIND) at the University of Wyoming (UW) is the first organization in the world to successfully adapt the ECHO model to education with an initial focus on disabilities. In 2014, WIND created its first ECHO network: The UW ECHO in Assistive Technology (AT; Hardesty et al., 2017; Root-Elledge et al., 2016, 2018, Root-Elledge & Hardesty, 2015). This adaptation was made based on the realization that the core ECHO components are consistent with the key features of the MTC and the lack of interdisciplinary teams of AT professionals and low rates of AT assessment and implementation at local education agencies. As a result, local education agencies and the state AT Act Program had to expend significant resources to train a small number of dedicated professionals. Our initial evaluation of ECHO for Education demonstrated participation in ECHO significantly improved knowledge and skills of participants, thereby increasing capacity of educators to use AT with their students (Root-Elledge et al., 2018).

Given the success of this initial adaptation, additional ECHO networks were developed based on state needs identified through an engagement process with state stakeholders, local education agencies, and educators. Areas identified as highest need included behavioral interventions for autism spectrum disorder (ASD), positive behavior supports, administrative leadership, school nursing, secondary transitions, and career development. In this study, we report on the first four ECHO networks for Educators, which to date, provide the most comprehensive data available on ECHO for Education.

Following the original ECHO model (Arora et al., 2007), all UW ECHO networks consist of core expert teams and subject-matter experts (the Hub Team), as well as community members located at schools and collaborating agencies (Spoke participants). Hub teams include interdisciplinary professionals relevant to the topic of the network. For example, the autism hub team includes autism specialists such as board-certified behavior analysts (BCBAs), psychologists, speech-language pathologists, occupational therapists, school nurses, school and district administrators, and professionals from support agencies including community-allied health, secondary transition experts, vocational rehabilitation, and college student support services. Spoke participants also vary by network topic and may include all the above listed professionals as well as general and special educators, paraprofessional educators, family members, and staff from community agencies including departments of family services as well as children and youth with special health care needs.

This study evaluates the first 2 years of implementation of the ECHO model for Education in four networks: AT ASD, behavioral supports which focused on positive behavior intervention and supports (BS), and secondary transitions (ST). Our key outcomes were program utilization and reach, satisfaction with the model, self-reported knowledge and skill acquisition, and desire for additional PLD delivered through ECHO.

Method

Participants

All research activities were approved by the Institutional Review Board of UW and all participants were treated in accordance with the ethical standards of the American Psychological Association (2017). Participants were entered into a drawing for a $100 prize.

Data were collected from two sources. First, we collected evaluative data from four UW ECHO networks. During the 2016–2017 academic year, there were 1,731 duplicated participants (the number of individuals in attendance across all sessions regardless of participant identity) who attended at least one ECHO session. During the 2017–2018 year, there were 1,159 duplicated participants who attended at least one ECHO session (see Table 1). Note that there were fewer ECHO sessions offered in the second year of evaluation which accounts for the reduced number of participants. Participants were recruited through their local education agency (LEA), PLD programs, direct word of mouth, advertisements within their schools and districts, and through our program website. Participants from Wyoming were offered continuing education units (CEUs) at no cost for their participation. One credit was awarded for every 12 sessions attended. Spoke participants from other states did not receive free CEUs but were provided sufficient records so they could apply and pay for credits in their own states. Second, we randomly selected a subset of these participants (n = 115 in 2016–2017 and 151 in 2017-2018, both unduplicated participant counts), to survey before and after participation in the networks as part of a research study. Years of experience in education was captured to further understand the relevance of the content across levels of experience; however, additional demographics, such as race, ethnicity, gender, and age were not collected to ease participants’ concerns related to confidentiality in sparsely populated communities.

Table 1.

Overview of ECHO for Education Networks, 2014–2018.

Year	UW ECHO network	Instructional hours offered per network	# sessions/course duration	# of participants	Average # of sessions attended (SD)
2014–2018	Assistive technology	186.5 hours	129 weekly sessions, 75 minutes	1,588 duplicated 531 unduplicated	2.87 (5.82)
2016–2018	Autism	87 hours	68 weekly sessions, 75 minutes	1,247 duplicated 307 unduplicated	3.67 (5.31)
2016–2018	Behavior supports	91.5 hours	71 weekly sessions, 75 minutes	1,457 duplicated 363 unduplicated	3.76 (5.92)
2016–2018	Secondary transitions	53 hours	41 bi-weekly sessions, 75 minutes	410 duplicated 186 unduplicated	2.22 (3.26)

Note. UW = University of Wyoming; ECHO = extension for community health care outcomes.

ECHO Sessions and Networks

Attendance during the sessions was voluntary and encouraged but not required by schools/districts or agencies. Participants accessed the live ECHO sessions via Zoom videoconference technology. Participants were able to join any network they wished and participation across networks was aggregated to determine the number of CEUs earned. Participants were only able to earn credit for participating in the virtual sessions. Anyone interested in watching the recorded ECHO sessions and accessing the materials was able to set up an account via our online Learning Management System, Canvas.

Networks were offered concurrently, beginning in September and running through the end of April. Each network occurred on a different day and time, with most networks starting at 3:15 p.m. Mountain Time (MT) and ending at 4:30 p.m. MT. Topics varied by networks with some topics spanning several weeks.

Measures

Usage data

Program staff who facilitated the networks tracked attendance of each participant during the weekly sessions. Participants reported their first and last name and phone number as well as current role, district or agency of current employment, and city/county. These data are housed in a proprietary electronic tracking system used by all ECHO sites known as iECHO and were kept separately from all other evaluation and research data. Name and contact information were used only to assign CE. More detailed demographics were not collected based on feedback from our stakeholders who expressed concern about anonymity if they were from small towns.

Questionnaires

The instruments used for these analyses were internally generated by the WIND, Research and Evaluation team which includes experts in research design, implementation, and advanced statistical techniques with backgrounds in experimental psychology, social psychology, test construction, and data science. Questionnaires were self-report measures that included a pretest evaluation administered in early fall (Online Appendix A), weekly session evaluations collected immediately after each session (Online Appendix B) and a posttest administered in late spring (Online Appendix C). Pretest and posttest questionnaires included 93 to 270 questions, while weekly session evaluation included 24 questions. The number of questions completed by each participant varied based on reported participation across all four ECHO networks. These pretests and posttests measured change over the course of the 2016–2017 school year.

However, this method is biased as respondents often overestimate their baseline levels (McLeod et al., 2008; Moore & Tananis, 2009). Therefore, we included retrospective pretest (RPT) questions in both the weekly evaluations and posttest. An RPT is a test in which both the posttest and the pretest are administered at the same time after the intervention. In the RPT, respondents indicate their level of skills/knowledge after an intervention, and then they are asked to think about their level of skills and knowledge prior to the intervention and provide the RPT ratings. This allows for a similar analysis to a traditional pre/post design (e.g., paired samples t-test); however, it does not directly measure change in skills or knowledge. Nonetheless, it provides a less biased estimate of a program’s effect and to be more convenient for respondents (Bhanji et al., 2012; Coulter, 2012; Davis, 2002; Marshall et al., 2007; McLeod et al., 2008; Pratt et al., 2000). Ultimately, the use of this method allowed us to compare the differences in pretest ratings from the traditional pretest and the RPT. Note that the pretest and posttest used in the 2017–2018 school year were designed to capture different information than in previous years, so this report includes pre/post data from 2016 to 2017 only. Weekly evaluations and RPT were consistent across years, and both years are reported in this study.

Each survey was designed to capture the extent of participants’ self-reported change in knowledge and self-efficacy related to network topics. (See Online Appendices A through C for examples). Surveys were composed of Likert-type scales (range 1– or 1–6 with the highest number being most satisfied or skilled) and open-ended questions designed to assess network participation, perceived knowledge, and skills, as well as perceived self-efficacy related to knowledge and skills specific to the topic of the network. The duplicated response rate across all networks in 2016 to 2017 was 50% and unduplicated rate was 54%. The duplicated response rate in 2017 to 2018 was 44%; the unduplicated response rate was 42%.

Procedures

Prior to participating in the network, participants were asked to complete a 30-minute electronic pretest. Consent was required before completing the pretests and posttests, noting that participation was voluntary, and participants could contact the Principal Investigator at any time to drop out of the study. This was collected either at the start of the academic 2016–2017 year, or at the time of first participation if the respondent joined the network later in the school year. The pretest was individualized to each network, asking participants to indicate their general level of knowledge and skill related to the area of focus (i.e., ASD, AT). After each of the regularly scheduled network (weekly/biweekly) sessions participants were asked to complete a 3-minute survey that assessed their level of satisfaction with the program, level of knowledge and skills acquired in attending the present session, as well as perceived utility of the session material. After completion of the academic year, all participants were invited to complete the 30-minute electronic posttest.

Analytic Plan

We examined the impact of the ECHO in Education networks across four main outcomes. First, we tracked weekly attendance, the numbers of participants with specific roles and number of CEUs offered were assessed using descriptive statistics to understand how much usage the program received and the roles of those who used it. Second, we measured the average ratings of the session’s usefulness (1–5 scale) for each topic presented using descriptive statistics to understand the level of satisfaction participants had with the program.

Next, we used a paired samples t-test of self-reported skills and knowledge from weekly feedback as well as our pre–post questionnaires, including the RPT to understand the impact of the program on skills and knowledge about AT, ASD, BS, and ST. This included a question about the respondents’ ideal level of skill and knowledge to determine if the program met their learning goals (see Online Appendices). Finally, participants were asked to reflect upon changes they would like to see or recommend more broadly, as well as suggestions for both PLD and case presentations to explore the desire for future trainings. This was assessed using open-ended questions. Barriers in attending ECHO sessions were also assessed using an open-ended question about challenges they faced accessing the network. Alpha was set at .05 in all cases, and Cohen’s d was calculated as a measure of effect size for all paired samples t-test analyses. All analyses were performed in IBM SPSS Statistics 22 software.

Results

Program Utilization/Reach

During the 2016–2017 academic year, 643 unduplicated professionals (number of individuals, regardless of how many sessions were attended) from Wyoming and more than 1,088 participants from 13 other states participated in these four UW ECHO networks. During the 2017–2018 year, 947 professionals from Wyoming and more than 200 unduplicated participants from 27 other states participated in these four UW ECHO networks. Across both years, participants were from several educational professions, including general and special educators (23.50%), program and education directors (17.50%), paraprofessionals (13.1%), project coordinators (10.40%), and a number of other self-identified professions (15.40%). Remaining participants represented specialists (10.80%), related service providers (9.80%), college students (6.25%) and case managers (4.40%). Professionals represented all 23 counties and 48 LEAs in Wyoming. In the 2016–2017 year, a total of 49 cases were presented; during the 2017–2018 year a total of 38 cases were presented. See Table 1 for the number of CEU instructional hours offered through each network.

Participants’ years of experience in education ranged from 0 to 47 years. In the 2016–2017 academic year, ECHO participants had an average of 15.4 years in education. Their experience ranged from 0 to 39 years (SD = 10.8 years). In the 2017–2018 academic year, ECHO participants had an average of 13.5 years in education. Experience ranged from 0 to 47 years (SD = 11.8 years).

Satisfaction

Descriptive analyses from weekly evaluations in 2016–2017 (n = 933; duplicated) indicated that participants found the didactic training and/or PLD skills taught in the UW ECHO sessions useful (M = 4.33, SD = .77) and relevant to their training needs (M = 4.36, SD = .76). Descriptive analyses from weekly evaluations in 2017–2018 (n = 625; duplicated) again, indicated that participants largely found didactic trainings and/or PLD skills taught in the UW ECHO sessions useful (M = 4.29, SD = .70) and relevant (M = 4.34, SD = .71). In addition, the majority of participants in 2016–2017 indicated that case presentations used in the UW ECHO training were also useful (M = 4.22, SD = .71) and relevant to their training needs (M = 4.24, SD = .71). Again, participants in 2017–2018 indicated that the case presentations taught in the UW ECHO trainings were also useful (M = 3.98, SD = .80) and relevant (M = 4.01, SD = .82).

Knowledge and Skill Acquisition

Paired samples t-test results indicated a reliable increase in levels of knowledge and skill after participation in all UW ECHO networks, from both the traditional pre/posttest, retrospective pre then post from the end of the network, and the retrospective pre then post following weekly evaluations. Information from weekly evaluations indicates that participants reported greater levels of knowledge and skill after participating in each ECHO session (see Tables 2 and 3). Weekly responses also suggest that 70% (2016–2017) to 72% (2017–2018) of the individuals who participated in at least one session planned to make changes to their practice as a result of their experiences. Furthermore, approximately 93% of participants in both 2017 and 2018 reported that they plan to share the knowledge and skills that they obtained from ECHO sessions with others. For example, participants reported that they plan to make changes to their practices by increasing advocacy for students, using more realistic goals, being more aware of needed topics, and increasing available information. In addition, participants reported that they plan to increase resources available for students, as well as increase collaboration and connections with other professionals.

Table 2.

Weekly Evaluation: Retrospective Change in Knowledge After ECHO Sessions.

Network	Retrospective Pre	Post	t	P	95% CI	d
Network	M (SD)	M (SD)	t	P	95% CI	d
2016–2017
Overall	3.06 (.91)	3.63 (.03)	27.01	<.001	[.53, .61]	.89
ASD	2.98 (.83)	3.57 (.73)	20.60	<.001	[.53, .65]	.75
AT	2.67 (.98)	3.37 (.85)	9.76	<.001	[.54, .82]	.76
BS	3.35 (.91)	3.82 (.78)	12.39	<.001	[.40, .54]	.56
ST	3.04 (.89)	3.68 (.74)	8.91	<.001	[.50, .78]	.78
2017–2018
Overall	2.96	3.57	23.38	<.001	[.55, .65]	.71
ASD	2.89	3.49	11.95	<.001	[.50, .69]	.70
AT	2.64	3.56	10.84	<.001	[.75, 1.1]	1.05
BS	3.18	3.66	15.56	<.001	[.42, .54]	1.87

Note. Abbreviations indicate the network topic. ASD = autism spectrum disorder, AT = assistive technology, BS = Behavioral Supports, ST = secondary transitions. CI = confidence interval.

Table 3.

Weekly Evaluation: Retrospective Change in Skill After ECHO Sessions.

Network	Retrospective Pre	Post	t	P	95% CI	d
Network	M (SD)	M (SD)	t	P	95% CI	d
2016–2017
Overall	2.97 (.92)	3.45 (.81)	25.26	<.001	[.44, .52]	.55
ASD	2.89 (.87)	3.37 (.78)	18.27	<.001	[.43, .54]	.58
AT	2.62 (.95)	3.20 (.85)	9.75	<.001	[.46, .70]	.64
BS	3.23 (.94)	3.65 (.82)	12.20	<.001	[.35, .49]	.48
ST	3.04 (.91)	3.54 (.74)	8.18	<.001	[.38, .62]	.55
2017–2018
Overall	2.88	3.36	19.41	<.001	[.42, .52]	.55
ASD	2.77	3.23	10.31	<.001	[.37, .54]	.53
AT	2.56	3.32	9.23	<.001	[.59, .92]	.80
BS	3.09	3.47	12.10	<.001	[.32, .45]	.48
ST	2.47	3.18	7.86	<.001	[.53, .89]	.77

Note. Duplicated respondents. CI = confidence interval.

Change in knowledge and skill from the pretest to posttest yielded a similar pattern of responses. Paired samples t-test results indicated that knowledge and skill increased over the course of the academic year. Results yielded similar findings within each ECHO network, again, demonstrating an overall increase in knowledge and skill, in the ASD, AT, and BS networks. However, despite an increase in knowledge and skill over time, this change was not reliable in the ST group (see Table 4). Information from 2017 to 2018 is not reported here or in either Tables 2 and 3 or Table 4 as the pre–post survey was designed to capture different information and is therefore not comparable across years.

Table 4.

Traditional Pre to Post Changes in Knowledge and Skill 2016-2017.

Network	Pre	Post	t	p	95% CI	d
Network	M (SD)	M (SD)	t	p	95% CI	d
Overall	3.37 (1.12)	3.79 (.86)	4.76	<.001	[.24, .59]	.42
ASD	3.06 (.1.14)	3.79 (.82)	4.38	<.001	[.38, 1.08]	.74
AT	2.82 (1.21)	3.27 (1.09)	2.44	.028	[.06, .85]	.39
BS	3.86 (.74)	4.26 (.60)	2.41	.042	[.02, .79]	.59
ST	3.43 (1.03)	3.83 (.73)	1.68	.125	[−.13, .93]	.45

Note. Unduplicated respondents. CI = confidence interval.

Retrospective preassessments then postassessments of participants’ perceived starting level of knowledge and skill related to different UW ECHO topics and their current level of knowledge and skills after participating was also examined with a paired samples t-test. Results indicated a reliable increase in knowledge and skills after participation, compared to baseline (i.e., where individuals felt they started at prior to participation). This pattern held true across each of the UW ECHO networks (ASD, AT, BS, ST; see Table 5).

Table 5.

Retrospective Post Then Pre-Evaluations 2016–2017.

Network	Perceived starting level	Post	t	p	95% CI	d
Network	M (SD)	M (SD)	t	p	95% CI	d
2016–2017
Overall	3.02 (1.16)	3.54 (1.01)	8.52	<.001	[.40, .64]	.48
ASD	2.87 (1.19)	3.40 (1.03)	4.49	<.001	[.29, .78]	.48
AT	2.47 (1.10)	3.19 (1.05)	5.75	<.001	[.46, .98]	.67
BS	3.68 (1.14)	4.00 (1.00)	2.43	.03	[.04, .60]	.31
ST	2.86 (1.00)	3.65 (.81)	5.21	<.001	[.46, 1.12]	.87
2017–2018
Overall	2.99	3.47	7.60	<.001	[.36, .61]	.35
ASD	3.26	3.81	4.60	<.001	[.31, .79]	.45
AT	2.74	3.24	3.69	.001	[.23, .78]	.39
BS	3.00	3.40	4.17	<.001	[.21, .59]	.26
ST	2.76	3.35	2.42	.03	[.072, 1.10]	.42

Note. Abbreviations indicate the network topic. ASD = autism spectrum disorder, AT = assistive technology, BS = behavioral supports, ST = secondary transitions.

Assessment of participants’ ideal levels of knowledge and skills indicated reliable differences. Although post assessment levels of knowledge and skill were reliably higher than pre-tests, they remained lower than the ideal levels indicated by participants. Thus, participants identified benefits from their participation in the ECHO networks and continue to seek additional support in gaining further knowledge and skill (see Table 6).

Table 6.

Ideal Versus Actual Knowledge and Skills.

Network	Ideal level	Post	t	p	95% CI	d
Network	M (SD)	M (SD)	t	p	95% CI	d
2016–2017
Overall	4.25 (1.09)	3.81 (.86)	3.99	<.001	[−.66, .23]	.45
ASD	3.96 (1.22)	3.79 (.82)	−.744	.47	[−.64, .31]	.16
AT	3.74 (1.28)	3.27 (1.09)	−1.53	.15	[−1.12, .19]	.40
BS	4.72 (.34)	4.26 (.60)	−2.42	.04	[−.90, −.02]	.94
ST	4.74 (1.06)	3.95 (.66)	−2.54	.03	[−1.49, −.09]	.90
2017–2018
Overall	3.74	3.47	−3.41	.001	[−.43. −.11]	.18
ASD	4.29	3.81	−2.54	.02	[−.86, −.10]	.37
AT	3.24	3.24	0.00	1.00	[−.38, .38]	.00
BS	3.66	3.40	−2.67	.01	[−.45, −.06]	.17
ST	3.71	3.35	−2.39	.03	[−1.91, −.09]	.24

Note. These are unduplicated respondents. Abbreviations indicate the network topic. ASD = autism spectrum disorder, AT = assistive technology, BS = behavioral supports, ST = secondary transitions.

Recommendations and Barriers in Access

Finally, 647 participants (duplicated count) from the 2016 to 2017 year and 370 from the 2017 to 2018 indicated that they had recommendations for future ECHO sessions and identified ways to improve PLD opportunities; this included 68% from the ST network, 55.8% of participants in the AT network, 69.6% of participants in ASD Network, and 75.5% in the BS Network across years. The majority of participants provided positive feedback, rather than recommendations; specifically, most indicated that they thought the program was excellent, and that the presentation and format of the ECHO program was effective and served as a great resource. They also highlighted the utility of the ECHO program in linking individuals to additional resources and improving connections to ongoing research, related websites, and professionals.

General recommendations related to the format of the program included more time for participant questions during ECHO sessions and increasing the range of marketing strategies (e.g., attending/presenting at conferences) so as to increase the scope and diversity of prospective audiences. Overall, participants identified recommendations for ways to build upon future sessions within each network and ways to extend the knowledge and skills that they acquired into additional educational programs such as incorporating transition goals and AT into an IEP or collaborating with representatives helping to process DD waivers.

In addition, for many professionals in the educational system, some of the mid-morning and mid-day times did not allow them to attend sessions due to obligations at work. Furthermore, technological problems generally had minimal impact, but some participants reported difficulty with their internet connection and/or video, experiencing background noise, not having stable internet (noted predominantly on the end of the user, rather than the ECHO session providers).

Discussion

Evaluative data from the first 2 years of implementation of ECHO for Education showed that the model was effectively adapted from health care to education. The program was widely adopted throughout the state of Wyoming and participants reported high satisfaction. There were relatively few barriers reported, other than the usual technological challenges and difficulties with scheduling. Moreover, educators who participated in this program reported significantly higher knowledge and skills related to the topic area using three separate measures: a traditional pre/posttest, a retrospective pre, then posttest, and weekly evaluations directly following content delivery. Although, for many networks, the participants indicated that they still needed additional training to meet their ideal level of knowledge and skill. This suggests that ECHO is an effective PLD tool for school systems that often struggle with delivering effective PLD that is most likely to lead to greater use of best practices.

One of the key features of ECHO is that it is built on the principles of effective professional development (Guskey, 2002). It is delivered in short, manageable increments over a school year, and includes both didactic training as well as case-based learning with ongoing consultation that is tailored to the needs of the educator and responsive to the abilities and goals of the student. Therefore, the ECHO model conforms to the MTC and is more likely to positively impact teachers as they learn to educate children with disabilities. Given that PLD for educators is otherwise delivered in 1/2-day workshops, often occurring offsite, this potentially represents a much more effective paradigm for PLD in rural and underserved areas. Ultimately, ECHO for Education may make PLD more accessible in schools and classrooms in remote areas.

Beyond the effects of ECHO-based PLD described in this study, this model has several other attractive features that may make it particularly well suited for implementation in rural states. First, ECHO operates as a virtual community of practice which allows educators to develop trusting relationships within a community that supports the development of a patient, kind, and caring educational practice (Arora et al., 2017). Educators in rural communities often have no other tool by which to collaborate with other teachers in similar settings. ECHO also provides an active learning, content-focused model that enables collective participation and is sustained over a period of time. Again, this is remarkably difficult in rural districts in the absence of virtual communities. Given these features of ECHO align directly with core features needed for effective PLD through virtual learning communities, McConnell et al. (2013) suggest, it may ultimately lead to improved student outcomes (Yarbro et al., 2007) and potentially increase the capacity of educators in rural settings.

Furthermore, the result of this initial program evaluation suggests that ECHO for education may be an effective means to increasing many of the facets of educator efficacy identified by Yarbro et al. (2007). Specifically, ongoing and additional training opportunities may inform educator’s knowledge of learners by increasing an understanding for more diverse learning styles and needs across diverse populations. This may allow educators to develop more creative and unique ways to engage diverse learners, and it may serve as a means to improve the acquisition of content knowledge by serving as a platform for ongoing, educator-driven collaborative development with up to date and relevant subject-matter knowledge. Moreover, as participants engage with ongoing ECHO sessions, this serves as a continuous needs assessment and feedback system from teachers. This kind of regular feedback from teachers has the potential to inform educators and administrators about required changes in policies and procedures to remove barriers to implementation of best practices.

Finally, due to the unique structure of ECHO, participants seem to be highly satisfied with the PLD they receive. It targets specific areas of need for participants and thereby provides opportunities to increase teaching skills and pedagogical practices that matter most to those in attendance. That is, by allowing professionals to share their current experiences and receive input from other educators about what has been effective in their community makes the content directly relevant and highly acceptable. This is reflected in the high rates of participation and by the enthusiasm of the participants to continue program attendance. Furthermore, this model limits the amount of travel educators would otherwise have to undertake to receive PLD through workshops or other traditional methods of teacher development. This saves time for the teacher, potentially increasing the time they have available to be in the classroom and saves school districts money in travel reimbursement.

Future Directions and Conclusions

While promising, there are several limitations that will need to be addressed to develop a fuller understanding of ECHO for Education. First, the data reported in this study are primarily evaluation data from program implementation. While this allows for a relatively large sample size, and multiple years of outcomes data, there are inherent limits to this approach. For instance, generalizability is limited, and program implementation is not controlled at the level of a randomized controlled trial. Therefore, additional research is needed to further determine the impact of this model for educators. Of particular interest will be determining how much this form of PLD contributes to actual improvement at the student level. This will be particularly important for special populations, such as children with disabilities, as their educational needs are often unique. Related to this, it will also be important to explore a broader range of educational outcomes than was possible here. Many of our networks are designed to impact behaviors that are predictive of academic outcomes; however, further research is needed to also determine if ECHO impacts academic success as well. In addition, it will be important to determine how much participation in ECHO is needed to improve how well educators implement the skills they learn. For example, if a teacher attends only a few sessions, we would expect a different level of implementation relative to a teacher who participates in a full year’s worth of sessions. Finally, additional work is needed to explore the range of impacts on the teachers themselves. For example, ECHO is designed to reduce professional isolation and increase teacher sense of self-efficacy, both of which are associated with teacher stress (Klassen & Chiu, 2010) and may lead to attrition. Research is needed to determine if this model increases teacher retention and efficiency.

Overall, the adaptation of the ECHO model to education appears to be effective. It reached a large number of educators in a rural state and participants were satisfied PLD delivery method. Participant knowledge and skills related to a range of content areas improved, and participants indicated a desire for additional training using this model. Therefore, schools and districts may want to consider implementing ECHO for Education as a way to enhance teacher training, especially if they are in rural or underserved communities.

Supplemental Material

appendicies – Supplemental material for Enhancing Professional Development for Educators: Adapting Project ECHO From Health Care to Education

Supplemental material, appendicies for Enhancing Professional Development for Educators: Adapting Project ECHO From Health Care to Education by Canyon Hardesty, Eric J. Moody, Shira Kern, Wendy Warren, Mary Jo Cooley Hidecker, Susan Wagner, Sanjeev Arora and Sandra Root-Elledge in Rural Special Education Quarterly

Footnotes

Author’s Note

Mary Jo Cooley Hidecker is now affiliated with the Department of Communication Sciences & Disorders, College of Health Sciences, University of Kentucky, Lexington, KY.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported, in part, through the AT ACT State Grants for Assistive Technology (grant no. 1901WYATSG-00); University Centers for Excellence in Developmental Disabilities Education, Research, and Service Core, Health Resources and Services Administration (grant no. 90DDUC0011-01-00).

ORCID iDs

Eric J. Moody

Mary Jo Cooley Hidecker

Supplemental Material

Supplemental material for this article is available online.

References

American Psychological Association. (2017). Ethical principles of psychologists and code of conduct.

Arora

Kalishman

Thornton

Komaromy

Katzman

Struminger

Rayburn

W. F.

(2016). Project ECHO (project extension for community healthcare outcomes): A national and global model for continuing professional development. Journal of Continuing Education in the Health Professions, 36, S48–S49. https://doi.org/10.1097/ceh.0000000000000097

Arora

Kalishman

S. G.

Thornton

K. A.

Komaromy

M. S.

Katzman

J. G.

Struminger

B. B.

Rayburn

W. F.

(2017). Project ECHO: A telementoring network model for continuing professional development. Journal of Continuing Education in the Health Professions, 37(4), 239–244.

Arora

Thornton

Jenkusky

S. M.

Parish

Scaletti

J. V.

(2007). Project ECHO: Linking university specialists with rural and prison-based clinicians to improve care for people with chronic Hepatitis C in new mexico. Public Health Reports, 122(2_suppl), 74–77. https://doi.org/10.1177/00333549071220S214

Arora

Thornton

Murata

Deming

Kalishman

Dion

Parish

Burke

Pak

Dunkelberg

Kistin

Brown

Jenkusky

Komaromy

Qualls

(2011). Outcomes of treatment for hepatitis C virus infection by primary care providers. New England Journal of Medicine, 364(23), 2199–2207. https://doi.org/10.1056/NEJMoa1009370

Barber

Mourshed

(2007). How the world’s best-performing schools systems come out on top. McKinsey & Company.

Bhanji

Gottesman

de Grave

Steinert

Winer

L. R.

(2012). The retrospective pre–post: A practical method to evaluate learning from an educational program. Journal of Academic Emergency Medicine, 19(2), 189–194.

Colbert

Brown

Choi

Thomas

(2008). An investigation of the impacts of teacher-driven professional development on pedagogy and student learning. Teacher Education Quarterly, 35(2), 135–154.

Cook

B. G.

Buysse

Klingner

Landrum

T. J.

McWilliam

Tankersley

Test

D. W.

(2015). CEC’s standards for classifying the evidence base of practices in special education. Journal of Remedial Special Education, 36(4), 220–234.

10.

Coulter

S. E.

(2012). Using the retrospective pretest to get usable, indirect evidence of student learning. Assessment & Evaluation in Higher Education, 37(3), 321–334. https://doi.org/10.1080/02602938.2010.534761

11.

Council for Exceptional Children. (2018). Professional development: Engaging the potential. https://www.cec.sped.org/Professional-Development

12.

Davis

G. A.

(2002, October 16–19). Using a retrospective pre-post questionnaire to determine program impact [Paper presentation]. Annual meeting of the Mid-Western Educational Research Association, Columbus, OH, Unites States.

13.

Desimone

(2009). Improving impact studies of teachers’ professional development: Toward better conceptualizations and measures. Educational Researcher, 38(3), 181–199.

14.

Desimone

Porter

A. C.

Garet

M. S.

Yoon

K. S.

Birman

(2002). Effects of professional development on teachers’ instruction: Results from a three-year longitudinal study. American Educational Research Association, 24(2), 81–112.

15.

Education Council of Learning Forward. (2017). Leveraging the Every Student Succeeds Act (ESSA) to build professional learning systems. https://learningforward.org/2017/02/leverage-professional-learning-wins-essa-better-learning-systems/

16.

Expanding capacity for health outcomes act or the ECHO act, H.R.5395, US House of Representatives. (2016a).

17.

Expanding capacity for health outcomes act or the ECHO act, S.2873—ECHO Act, US Senate. (2016b).

18.

Fischer

S. H.

Rose

A. J.

McBain

R. K.

Faherty

L. J.

Sousa

Martineau

(2019). Evaluation of technology-enabled collaborative learning and capacity building models: Materials for a report to congress. https://aspe.hhs.gov/system/files/pdf/260691/ECHOAct-ConsolidatedReportToCongress.pdf

19.

Fishman

B. J.

Marx

R. W.

Best

Tal

R. T.

(2003). Linking teacher and student learning to improve professional development in systemic reform. Teaching and Teacher Education, 19(6), 643–658. https://doi.org/10.1016/S0742-051x(03)00059-3

20.

Garet

M. S.

Porter

A. C.

Desimone

Birman

B. F.

Yoon

K. S.

(2001). What makes professional development effective? Results from a national sample of teachers. American Educational Research Journal, 38(4), 915–945.

21.

Guskey

T. R.

(2002). Professional development and teacher change. Teachers and Teaching, 8(3), 381–391.

22.

Hardesty

Warren

Arce

Bowser

(2017, June/July). Invited submission. Fostering communities and professional learning in assistive technology: The opportunity for case specific discussions toincrease provider knowledge and application of at practices. Closing the Gap Solutions.

23.

Jensen

Sonnemann

Roberts-Hull

Hunter

(2016). Beyond PD: Teacher professional learning in high-performing systems. Teacher quality systems in top performing countries. https://eric.ed.gov/?id=ED577259

24.

Johnson

Porter

K. J.

McPherson

(2012). Autism knowledge among pre-service teachers specialized in children birth through age five: Implication for health education. American Journal of Health Education, 43, 279–287.

25.

Klassen

R. M.

Chiu

M. M.

(2010). Effects on teachers’ self-efficacy and job satisfaction: Teacher gender, years of experience, and job stress. Journal of Educational Psychology, 102(3), 741–756.

26.

Kraus

(2017). 2016 disability statistics annual report. University of New Hampshire. https://disabilitycompendium.org/sites/default/files/user-uploads/2016_AnnualReport.pdf

27.

Kurth

J. A.

Keegan

(2014). Development and use of curricular adaptations for students receiving special education services. The Journal of Special Education, 48(3), 191–203.

28.

Lindsay

Proulx

Thomson

Scott

(2013). Educators’ challenges of including children with autism spectrum disorder in mainstream classrooms. International Journal of Disability, Development and Education, 60(4), 347–362.

29.

Lomos

Hofman

R. H.

Bosker

R. J.

(2011). Professional communities and student achievement–a meta-analysis. School Effectiveness and School Improvement, 22(2), 121–148.

30.

Marshall

J. P.

Higginbotham

B. J.

Harris

V. W.

Lee

T. R.

(2007). Assessing program outcomes: Rationale and benefits of posttest-then-retrospective-pretest designs. Journal of Youth Development, 2(1), 118–123.

31.

Masters

De Kramer

R. M.

O’Dwyer

L. M.

Dash

Russell

(2010). The effects of online professional development on fourth grade english language arts teachers’ knowledge and instructional practices. Journal of Educational Computing Research, 43(3), 355–375.

32.

McConnell

T. J.

Parker

J. M.

Eberhardt

Koehler

M. J.

Lundeberg

M. A.

(2013). Virtual professional learning communities: Teachers’ perceptions of virtual versus face-to-face professional development. Journal of Science Education Technology, 22(3), 267–277.

33.

McFarland

Hussar

de Brey

Snyder

Wang

Wilkinson-Flicker

Gebrekristos

Zhang

Rathbun

Barmer

Bullock Mann

Hinz

(2017). The condition of education 2017 (NCES 2017-144). National Center for Education Statistics. https://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2017144

34.

McLeod

P. J.

Steinert

Snell

(2008). Use of retrospective pre/post assessments in faculty development. Medical Education, 42(5), 543–543.

35.

Moore

Tananis

C. A.

(2009). Measuring change in a short-term educational program using a retrospective pretest design. American Journal of Evaluation, 30(2), 189–202.

36.

National Center for Education Statistics. (2018). Results from the 2017 National Assessment of Educational Progress (NAEP). https://www.nationsreportcard.gov/reading_math_2017_highlights/

37.

Pratt

C. C.

McGuigan

W. M.

Katzev

A. R.

(2000). Measuring program outcomes: Using retrospective pretest methodology. American Journal of Evaluation, 21(3), 341–349.

38.

Root-Elledge

Hardesty

(2015, October/November). Wyoming’s novel approach to building capacity in assistive technology: University of Wyoming ECHO in assistive technology. Closing the Gap Solutions.

39.

Root-Elledge

Hardesty

Hidecker

M. J. C.

Bowser

Ferguson

Wagner

Moody

E. J.

(2018). The ECHO model for enhancing assistive technology implementation in schools. Assistive Technology Outcomes and Benefits, 12, 37–55.

40.

Root-Elledge

Hardesty

Wagner

(2016, December/January). Demonstrating the possibilities: University of Wyoming echo in assistive technology is building capacity and demonstrating positive outcomes in Wyoming and beyond. Closing the Gap Solutions.

41.

Spatig-Amerikaner

(2012, August). Unequal education: Federal loophole enables lower spending on students of color. Center for American Progress.

42.

U.S. Department of Education. (2017). Every Student Succeeds Act (ESSA). https://www.ed.gov/essa?src=rn

43.

U.S. Department of Education. (2018). Individuals with Disabilities Education Act. https://sites.ed.gov/idea/regs/b/a/300.8

44.

Vaughn

Swanson

E. A.

(2015). Special education research advances knowledge in education. Exceptional Children, 82(1), 11–24.

45.

Yarbro

J. L.

McKnight

Graybeal

(2007). The heart of great teaching: A global survey of educator effectiveness. https://eric.ed.gov/?id=ED576186

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.07 MB