Abstract
We investigated the experiences of Australian autism early intervention service providers, including the professional development they receive on the job and how they translate research evidence into their practice. Semi-structured interviews were conducted with 15 providers drawn from three locations. Participants described the crucial role of paraprofessionals in early intervention services, and indicated that, within multidisciplinary teams, allied health professionals, and teachers are the ones who select intervention practices and upskill paraprofessionals. Furthermore, participants reported using practices supported by research evidence, but also some emerging and unsupported practices. The use of unsupported practices is examined in light of the daily challenges faced by centers. Results may inform further professional development in early intervention services for children on the autism spectrum, to support the best possible outcomes for this population. The findings highlight the potential need for job-embedded hands-on professional development conducted in real time.
Keywords
Early intervention has been linked to better social, emotional, cognitive, and behavioral outcomes for children on the autism spectrum (Landa, 2018). Internationally, best practice guidelines for early intervention services for children on the autism spectrum emphasize the use of the evidence-based practice (EBP) framework (e.g., Early Intervention for Children with ASD [Australia], Prior & Roberts, 2012; National [USA] Clearinghouse on Autism Evidence and Practice, Steinbrenner et al., 2020; National [UK] Collaborating Centre for Mental Health, 2013). The EBP framework encourages care that is based on the best available research evidence and the expertise of the practitioner, while also considering the service recipient’s strengths, needs, values, and preferences (Sackett et al., 1996). The best available evidence for children on the autism spectrum can be obtained from systematic reviews, which synthesize research evidence and classify interventions as EBPs if they reach a particular threshold of supporting research (e.g., National Clearinghouse on Autism Evidence and Practice [NCAEP]; Steinbrenner et al., 2020). Reviews provide guidance to early intervention service providers working within the EBP framework to select effective practices. However, neither prioritizing the translation of research into policy and practice, nor the availability of information and tools to identify evidence-based interventions, has proved sufficient to guarantee that practitioners will implement such practices for children on the autism spectrum (Brock et al., 2014; Dynia et al., 2020; Paynter et al., 2017, 2018). Indeed, challenges persist in translating EBPs into use while also discontinuing the use of practices that lack research support or have been shown as ineffective (Straus et al., 2008; Ward et al., 2009).
Research has indicated that autism early intervention service providers continue to use interventions without research support either in place of or in addition to EBPs (Paynter et al., 2015; Stahmer et al., 2005). For example, Paynter and Keen (2015) surveyed 99 providers working within a community-based comprehensive early intervention program in Australia. Although practices classified as EBPs were used most often, an assemblage of some emerging and unsupported practices was often used. Subsequently, these findings were replicated in four different services (Paynter et al., 2017). In addition, studies have found that practitioners who self-reported more use of EBPs, reported more perceived knowledge and organizational support for EBPs (Paynter et al., 2017; Paynter & Keen, 2015). These studies provide quantitative data highlighting factors associated with use of EBPs and continued use of unsupported practices alongside EBPs, indicating research-to-practice gaps. Yet, the findings provide no information about the lived experiences of those delivering the interventions, which is the detailed information needed to help understand how to assist providers to provide the best available care to children on the autism spectrum. For example, Stahmer et al. (2005) conducted focus groups with providers working directly with young children (ages 0–6) on the autism spectrum in the United States. The providers reported having limited knowledge of the evidence-base of the practices they used, and the level of fidelity of implementation was unclear. Additional qualitative research could determine whether these findings continue to reflect current practice internationally and provide directions for ways to reduce research-to-practice gaps in the future.
Factors Related to EBP and the Role of Professional Development
A number of factors related to use or non-use of EBPs have been identified (Johnson et al., 2018), including providers’ attitudes to change and views on research evidence (Aarons, 2004), organizational barriers, such as resources and culture (Cheung et al., 2013), insufficient professional development (henceforth PD) (Dillenburger et al., 2016; Elsabbagh et al., 2014), and a reliance on anecdotal sources of information, notably, colleagues’ reports (Paynter et al., 2018). This set of factors has been linked to gaps in knowledge and/or false beliefs about the evidence base of different interventions (e.g., Kadar et al., 2012; Miller et al., 2012; Paynter et al., 2018). For example, two studies found that providers regularly relied on colleagues and internal organization-provided PD to inform their clinical decision-making, rather than on research evidence, reviews, or practice guidelines (Paynter et al., 2015, 2017). In these studies, this pattern was especially prominent among paraprofessionals (child care educators) compared with allied health workers or teachers, consistent with other research showing that paraprofessionals frequently receive their training from other staff or colleagues (Giangreco et al., 2001; Nail-Chiwetalu & Ratner, 2007). Of note, paraprofessionals frequently provide important direct support, actively assisting others in administering education and interventions to children with developmental disabilities including children on the autism spectrum (Boomer, 1994; Giangreco et al., 2001).
Evidence has emerged that PD opportunities for providers directly working with children on the autism spectrum are valuable in supporting the use of EBPs. When focus groups were conducted with providers, they described a need for practical, hands-on learning in real time (Trembath et al., 2015, 2019). Consistent with these preferences, job-embedded training, such as coaching and mentoring, has been beneficial for improving practice and rated as socially valid in previous research with early intervention providers (Keen et al., 2017). Research evidence also suggests that well-trained staff members are a crucial element in ensuring the use of EBPs with fidelity (Dillenburger et al., 2016). Yet, very few studies have directly asked service providers to describe what is happening in the field, and little is known about peer-to-peer on-the-job PD in real time. Especially, noteworthy is the dearth of current qualitative data that explore the priorities, preferences, and experiences of those working with young children on the autism spectrum within Australia.
Australian Context
In Australia, the National Disability Insurance Scheme (NDIS) provides a lifetime of support for individuals with permanent disabilities (including autism) under a national system. The National Disability Insurance Association (NDIA) assesses and funds the scheme, but it does not provide support services. Instead support services are offered by nongovernment and government organizations, including disability service organizations, state and territory disability service providers, and private businesses (May et al., 2018). NDIS funds and supports interventions that they consider likely to be effective. Although not explicitly mandating EBPs or funding only EBPs, they have funded reviews of relevant evidence to inform early intervention services for children on the autism spectrum (Autism Cooperative Research Centre, 2021; National Disability Insurance Scheme [NDIS], 2021) which complement international guidelines (e.g., Steinbrenner et al., 2020). It is expected that service providers will use these reviews and guidelines to select practices with evidence of effectiveness to obtain the best possible outcomes for children on the autism spectrum, protect them from harm and false hope, ensure they do not miss out on treatments that work, and ensure resources available to them are channeled into practices that have been shown to work (Keen et al., 2017; Paynter et al., 2017, 2018).
Current Study
While EBPs have been documented for children on the spectrum, survey research shows challenges in translating these into practice (Straus et al., 2008; Ward et al., 2009). Barriers to translation include organizational constraints (Cheung et al., 2013), reliance on close colleagues’ views and attitudes (Paynter et al., 2018), and the need for additional PD (Dillenburger et al., 2016; Elsabbagh et al., 2014). There is limited research gathering the views of early intervention providers on the use of EBPs for children on the autism spectrum in practice or on their experiences with PD. We set out to summarize what these practitioners describe as their roles and training needs, especially their workplace PD about early intervention practices, and how they use reviews, guidelines, and other research evidence about interventions to make decisions about the services they provide to young children on the autism spectrum. To assist providers to provide the best available care to children on the autism spectrum, the aim of this study is to better understand the experiences of early intervention providers who support young children on the autism spectrum, and their knowledge and values about EBPs.
We address the following areas and research questions:
Current practice. What are service providers’ perceptions regarding the evidence-base of the practices they use, and how do they align with empirical evidence and center policies?
PD: What are service providers’ experiences and perceptions of PD in their centers?
Barriers and facilitators to implementation. What do service providers experience as barriers and facilitators to the use of effective practices and to the delivery of training and development?
Method
Context
Providers were drawn from three government-funded Autism Specific Early Learning and Care Centers across three states of Australia. Each organization provides early intervention to children on the autism spectrum within an autism-specific long day-care setting and is funded through a combination of federal funding (the same funding scheme at each site) and parent fees. Some of the centers provide services to children aged 0 to 6 only, whereas others also accept older children. For the purposes of this research project, staff were asked to focus on younger children (age 0-6). Each center has staff members who fill a variety of professional (e.g., teachers) and paraprofessional (child care educators) roles. The centers are regulated by a set of operational guidelines that outline staffing requirements (child ratios, type, and qualifications of staff employed), as well as operational requirements based on Australian guidelines for good practice in autism (Prior & Roberts, 2012), which emphasize the use of EBPs in early intervention. The qualitative methodology employed allowed for an in-depth understanding regarding service providers’ knowledge, their rich and varied experiences working with young children on the autism spectrum, and the operation of early intervention care centers related to PD needs and implementation.
Participants
The participants were a purposive sample of 15 female staff delivering early intervention services for children on the autism spectrum. Participants’ age, education, and length of employment in their role can be found in Table 1. Six participants were center leads who had some responsibility for day-to-day operations and for supervision of other service providers at the centers; four with the title manager, one speech pathologist and one occupational therapist. Of the other nine participants, one was an early-childhood teacher, one key therapist / teacher (e.g., room leader), and seven educators (a term used to refer to child care professionals—early childhood professionals with training varying from a vocational certificate in early childhood to a diploma).
Participant Characteristics by Role.
Note. TAFE is a government-run system in Australia that provides vocational education after high school. Senior Certificate is a Queensland school-based senior schooling qualification awarded to eligible students at the end of Year 12. Diploma is a post-school education qualification (below a Bachelor Degree, but above a Certificate). TAFE = technical and further education.
Procedure
Participant Recruitment
To recruit participants, informed consent materials were distributed to the managers of three early intervention service centers. The centers were selected based on their range of experience, geographic location, and organization (providing services to young children on the autism spectrum). These center managers were asked to invite other service providers, representing the different roles at their respective centers, to participate in the study and to distribute informed consent materials. Participants from a range of roles were recruited to explore the various roles and experiences across different members of the same team, given the limited research to date in the Australian context. Managers were not asked to list invited staff members or to keep records of declined invitations to reduce the burden on their time and to maintain staff confidentiality. Thus, the response rate is not known. After reviewing the information sheet, all participants gave signed informed consent. No incentives were offered to the participants; however, funds were given to reimburse the centers for staff release time to participate during their usual working hours, which could have been used to fund substitute staff. Managers recruited an initial sample of participants to be interviewed, whereupon saturation (Polkinghorne, 1989) was reached and recruitment ceased.
Interview
Following Human Research Ethics Committee [ref no: 2019/187] approval, semi-structured interviews were conducted via video-conferencing during working hours by the first author, a female Clinical Psychology PhD candidate with experience in research into knowledge translation in autism. The research was conducted with collaboration and consultation with the last author, who is a clinical psychologist with over 20 years of experience working with children on the autism spectrum including 7 years directly in early intervention services for children.
With participants’ consent, all interviews were audio recorded; the interviewer also took field notes. The notes reflected the interviewer’s perspective on key ideas as they arose, overall themes, key points, salient comments, and recurring content indicative of saturation. The interviews included questions targeting early intervention providers’ current or typical operating practices, the PD (e.g., training to complete their roles) offered to them, as well as barriers and facilitators in using effective practices and in the delivery of PD within the respective organizations. A copy of the full semi-structured interview protocol is available from our project Open Science Framework page, see [OSF link: https://osf.io/r23vd/]. Questions were informed by Stahmer et al. (2005), with items added to address the areas of interest: practice use in early intervention services for children on the autism spectrum; barriers and enablers (Paynter et al., 2017, 2018; Stahmer et al., 2005); understanding of evidence and myths in early intervention services for children on the autism spectrum (Lewandowsky et al., 2012; Paynter et al., 2019); and enhancing knowledge and implementation of effective autism interventions through PD (Brock et al., 2014; Keen et al., 2017; Paynter & Keen, 2015). Consistent with the guidelines outlined in the work of Brantlinger et al. (2005), to ensure rigor and credibility of questions and trustworthiness, all questions had been pilot tested and reviewed for language and sensitivity by the third and last authors, both of whom had multiple roles within early intervention services for children on the autism spectrum; minor feedback about wording (e.g., autism instead of ASD) was incorporated into the final interview protocol. Interview time ranged from 13 to 40 minutes (M = 20 minutes; SD = 9.28). Interviews were shorter or longer based on the depth of interviewees’ responses. Once themes and subthemes from the interview content were defined and recorded in a summary table, participants were informed of these via email, and were given access to a draft of the study findings for feedback and to ensure quotes were interpreted in context and accurately. No substantive feedback was received; the only feedback was one participant requesting that her “um’s and ah’s” be removed from the final transcript.
Interview Coding and Data Analysis
Braun and Clarke’s (2006) six phases of thematic analysis were applied using NVIVO software. First, the first author gained familiarity with the data by transcribing the interviews verbatim and incorporating field notes. Field notes were used to inform the coding scheme and themes in data analysis, all of which were subsequently integrated with interview transcripts for further analysis. Second, raw data were coded by means of organizing the information into nodes (i.e., semantically coherent groups containing words, sentences, and paragraphs unified through meaning; Graneheim & Lundman, 2004). Third, nodes were collated into broader categories based on recurring themes or sub-themes. Fourth, thematic maps that incorporated the content were developed, and themes were refined to ensure that the data in the individual nodes were accurately represented. Fifth, the defining and naming of the themes was finalized, ensuring that the rubrics (i.e., tables with themes and nodes) highlighted recurring themes. A final summary table that included the themes and key nodes was compared with the raw data to ensure that the rubrics reflected the essence of the respective themes. All the themes were reviewed together while considering the overall broader narrative relating to the research questions (e.g., participants’ experiences as concerns PD within their respective organizations). Finally, results were summarized and representative quotes were selected for the main points in each theme for the purposes of illustrating and providing evidence for the themes identified. In this phase, the data were evaluated in relation to the research questions, and the focus was on the narrative, as opposed to descriptive, framing of the findings.
To ensure rigor (Brantlinger et al., 2005), a second, independent, coder with clinical experience of 5 years working as a psychologist providing early intervention services for children on the autism spectrum (please see acknowledgment) assigned codes for 20% of the interview transcriptions and categorized data accordingly (Cohen’s kappa =.89). A high inter-rater agreement also emerged in the assignment of codes to individual statements (kappa = .91). After independent coding and calculation of kappas, the codes were compared and discussed, and no major discrepancies were found. All minor discrepancies (e.g., coding related to educator’s role as based on their own or third party—e.g., a center lead’s—perceptions) were discussed and a consensus was reached between the first author and the independent coder.
Results
We organized the results according to the three research questions: (a) the use of practices and understanding of EBP; (b) PD in early intervention; and (c) barriers and facilitators in the use of EBP and in the delivery of PD. Within each rubric, two to three subthemes were identified (see Table 2). Below, we provide details about subthemes with illustrative quotes and pseudonyms to ensure participant confidentiality.
Themes Linked to Each of the Three Research Questions and Sub-Themes.
Note. EBP = evidence-based practice; PD = professional development.
Use of Practices and Understanding of EBP
Selection and Implementation of Practices
As part of the selection and implementation of practices to support children, the majority of participants cited as an important first step an assessment conducted for every new child who arrives at the center, often together with that child’s family. The aim is to gather information about children’s “interests, what they like, what they don’t like. . . support they need, and what we can work on . . .” (Barbara, Educator), and “where they might need extra help, for example, speech and language . . .” (Elizabeth, Educator). All interviewees emphasized a team approach. In some cases, educators access the assessment and refer to it for guidance in working with the respective child; or the assessment is “passed down by one of our speech pathologists or OTs [occupational therapists]” (Louise, Educator). Before starting therapy, the educators will often discuss with a senior staff member (e.g., speech pathologist, occupational therapist) which interventions to use with the child, describing the process as being “guided by the allied health and teaching team” (Elizabeth, Educator).
Participants described weekly staff meetings for each child to evaluate whether they are en route to meeting their goals and to consider the practices used to support the children: “We meet weekly, including the full multidisciplinary team” (Sue, Lead). Changes are made by senior staff members as needed: “We look at the progress that the child’s made and what we are actually observing. Then we make plans for the next steps” (Sue, Lead). The families are also involved, “each term we meet the family as well, and we set goals, and they are based around the NDIS goals” (Sue, Lead). Educators reported that, in the course of a child’s early intervention program, they continue to consult with the other members of the multidisciplinary team about the interventions they are using with a child according to the instructions they had received based on the assessment, and implement strategies based on guidance of allied health (e.g., speech pathologist, occupational therapist) or a key therapist/worker (e.g., room leader). Shannon, Educator, noted: I just use it because that’s what’s told to me what we should be using . . . if it’s not working, I would speak up, but if the key worker [i.e., room leader], management and allied health believe that it is what would work and they show us that, then we follow it.
Occasionally, educators mentioned new practices; describing being “encouraged by our team members to try new strategies . . . try out and see, and then observe and evaluate how things work” (Elizabeth, Educator). Some participants reported that changes are not necessarily documented, but rather staff members converse with each other about new practices: “Often it’s not written down, but there’s lots of talking that will go on and often just in the classrooms between you know, the OT, the Speechie [speech pathologist]” (Jane, Educator). At other times, the process was undefined: “assess the child and just sort of see like what they like, and sort of what’s going to work best for them” (Amelia, Educator).
Types of Practices Used
Across participants, most of the practices described were EBPs listed in recent reviews (e.g., Steinbrenner et al., 2020). Practices are listed on our Open Science Page [OSF link: https://osf.io/tz97s/]. These included Picture Exchange Communication System (PECS) (mentioned in all but one interview), Technology-aided instruction (e.g., Proloquo2Go) (majority of interviews), and visuals (half of the interviews)—all of which accounted for most of the EBPs reportedly used. The most mentioned interventions defined as EBPs based on listings and guidelines for EBP (e.g., Steinbrenner et al., 2020) were targeted skills relating to communication (e.g., Augmentative and Alternative Communication (AAC), PECS). In addition, participants described using a range of practices to target other areas including social-emotional development (e.g., social stories, schedules), behavior (e.g., reinforcement, modeling, and using rewards), as well as comprehensive programs directed at multiple domains (e.g., SCERTS, Zones of Regulation, Alert Program). After consolidating to consider different terminology (e.g., speech-generating devices and AAC interventions), a total of 81 different practices were mentioned.
More than one interviewee from each of the three centers mentioned sensory interventions for the calming or regulation of the children. These strategies vary in their evidence base: for example, evaluating the sensory environment would be considered a type of antecedent-based interventions, an EBP, while others were more consistent with emerging (e.g., massage therapy, see National Autism Center [NAC], 2015 and as having insufficient evidence by Steinbrenner and colleagues (2020) or unsupported categories (i.e., lacking empirical evidence; e.g., DIR/Floortime, see NAC, 2015).
Understanding the Evidence-Base of Practice
Regarding familiarity with EBP classifications, participants varied widely. Interviewees typically reported limited understanding of EBP when asked directly about the level of evidence they believe the interventions they use have. For example, “I don’t know much about the research . . . I’m a hands-on girl” (Jane, Educator); “I’m not sure about evidence. I can see which children settle down with which intervention” (Elizabeth, Educator). Educators typically reported deferring to advice from the multidisciplinary team members and placing trust in their recommendations: I believe, we try to do a lot of evidence-based practices . . . So, I don’t make those decisions. I would go to a SP [speech pathologist] or an OT to get that guidance . . . they are the ones that are obligated to do that research . . . I’ll go to them for recommendations, but I know that we only source evidence-based (Maya, Early Childhood Teacher).
Educators tended to put trust in their organizations to recommend only EBPs, and this may be one reason they deem all practices they use as evidence-based: “I would imagine that they are quite evidence-based. I mean, we wouldn’t be using them if they weren’t supported . . .” (Shannon, Educator). Providers likewise tended to describe all PD they had attended as recommending only EBP: “Um, based on evidence, um, probably things that I have done like PD for . . . So yeah, I try to implement stuff that I have learned in training and PD” (Sharon, Lead).
Allied health professionals and several of the center staff, on the other hand, were aware that some of the practices they were using have limited research evidence; this point arose in interviewees from all three centers as, for example, with Carol, Um, a lot of the sensory stuff doesn’t [have research support] . . . we let parents know if we are trialing stuff that isn’t evidence based.” Sometimes intervention providers consider family’s preferences: “I take the family’s lead as to what interventions they wish for us to use . . . (Alice, Key Therapist/Teacher).
Providers from each center reported little familiarity with any specific center policies regarding the level of evidence required for the use of an intervention or a therapy. Thus, Carol (Lead) said, I don’t know whether it’s formalized or not. I know that we were doing the helping children with autism, the HCWA (Helping Children with Autism package) funded intervention. They had recommendations, a whole list of things that were supported or still emerging as far as evidence is concerned, but I don’t know if that was put down as, um, in our policies and procedures here.
Eve, a lead at the same center shared being familiar with these guidelines (“the Prior and Roberts ones . . .”). Doris (Lead) reported, “We don’t have a policy that says that,” but rather discussed the matter with staff at meetings or PD. Interviewees from other centers expressed similar views.
PD in Early Intervention
Content of PD
Methods of selecting PD varied between centers, with Daisy (Lead) reporting, “We have like a tier system” and “I believe it’s upwards of 100 hours of training a year . . .” and Sue (Lead), a more ad hoc strategy, largely based on providers’ requests, which depends on the staff member, so we have funding available to support staff members to do formal training. . . twice a year we have a full team PD day. And we generally pick a variety of topics. . . it depends what staff have requested . . ..
The majority of interviewees observed that their PD instructed them mostly on what to do (which interventions to use with the children), but not what not to do. This point was highlighted by Sue (Lead): “No . . . everything is focused on what to do.” One center lead, however, mentioned that instruction on what not to do might be embedded in some PD: “It might just be I guess embedded in a way of um, how to, where to look for example. I don’t think we’ve ever really had trainings about what not to do . . .” (Eve, Lead).
Preference for PD
Preference for informal role-modeling and for a more structured observation/coaching modality for PD was expressed by many participants, exemplified by Early Childhood Teacher Maya, “I definitely like face-to-face, coaching . . . I like something that allows me to put that in practice at the time rather than something that’s online.” The main reason for such a choice appears to be the understanding that the senior providers are familiar with the children and can answer questions about each individual child, and “because you have a bit more of a relationship with them and you feel a bit more comfortable asking the questions” (Louise, Educator).
Overwhelmingly, providers preferred face-to-face PD outside the organization delivered by external trainers, along the following lines, “Oh, being myself, being an old person, I prefer face-to-face” (Karen, Senior Room Coordinator). However, many interviewees reported that they were happy with various PD, with a mix of online and face-to-face sessions delivered by both internal (majority interviewees) and external trainers.
Only a minority of interviewees indicated a preference for workshops: “I quite enjoy workshops” (Jane, Educator); a lead mentioned external supports, such as “Local university to support us in appraising articles” (Sue, Lead); and one said that they preferred “power point presentations and activities you can talk about in a group to discuss problems . . . with trainers; [these] are really informative and they make it fun” (Elizabeth, Educator). Of note, none stated a preference for online EBP resources/platforms, or self-study using journal articles.
The Importance of Training Within the Multidisciplinary Team
Both field notes and interviews with center leads pointed to the importance of the direct service provider’s role in the centers, as these practitioners are the ones who spend the most time in direct work with the children and implement interventions on a daily basis. All participants concurred that direct service providers (i.e., child educators) are often upskilled by other providers, including senior staff. Eve (Lead) reported that her center has “a team of senior speech pathologists, psychologists, or occupational therapists who are usually allocated . . .” and that “all the trainings are facilitated by senior clinicians.” Similarly, Doris (Lead) and Sharon (Lead) both reported that providers attend external PD and then share their knowledge with the rest of the teams. Other interviewees concurred: “Quite often it’s the allied health team that go to workshops and come back and share their learning” (Elizabeth, Educator). The newly acquired knowledge is shared at “weekly meetings and fortnightly meetings with the staff as well . . . we do a lot of peer modeling with our team as well” (Maya, Early Childhood Teacher) and via “info bite of one-and-a-half hours on positive behavior support . . . That was a senior speech pathologist within the association [who conducted the training]” (Alice, Key Therapist/Teacher).
Participants also reported, albeit less often, that service providers attend external PD. Several participants described, however, that external PD is more likely to be available to permanent full-time staff, and mainly to “senior” members of the center team (e.g., SP, OT). Doris (Lead) described the pattern in this way: The casual staff or the people who work parttime and are not here very often, um, it’s probably more likely that their training would be the staff days and other kind of individual kind of catch-ups and internal stuff. But the longer a person has been with the organization, like the full-time people, and the therapists, the more likely she is to have external stuff.
Barriers in the Use of PD and EBP
One significant barrier to PD was organizations’ lack of resources. The limitations mentioned include “staff ratios . . ., staff absences . . ., cost, time, um, sometimes the resources” (Sue, Lead). Sue further explains that “we might be able to, you know, pay for the training, but then we don’t always, can’t afford necessarily all the bits and pieces that might need to go along with it.” Several participants emphasized as a barrier
staff-to-child ratios and the hours that we are open . . . If I could send everyone to an external training, you know, five times a year, it would be something that I would love to do, but unfortunately, we are really governed by the amount of children we have on the day (Daisy, Lead).
Staff shortages were mentioned by more than one interviewee, along with facilities, including “barriers that are related to the physical environment, um, and the numbers of kids and the numbers of staff and some of those things” (Doris, Lead). Time limitations were felt to be an obstacle in access to PD: “It’s probably time . . . if I do, like, go to training, um, during work hours, then obviously someone will need to backfill me” (Sharon, Lead). Isolation from PD and availability was noted as a challenge/barrier: “Travel and location would mean that some of the professional learning opportunities would only be offered [far away]” (Doris, Lead).
Among center-specific barriers to PD mentioned in the interviews was reliance on internal PD, We do have a lot of knowledge within the organization that we need to draw on, so sometimes that may feel like it’s limiting opportunities to do external training, but I have to remind myself all the time that we have vast experience here and vast knowledge and we can deliver some fantastic training as well (Daisy, Lead).
This sentiment was echoed by leads in all participating centers. Daisy further mentioned the difficulty juggling people’s differing perspectives: “People coming from vast backgrounds. . ., so [there is] prior knowledge that someone brings to the setting, and I guess their own values and beliefs on certain things.” Key Therapist/Teacher Alice mentioned her “center’s willingness to make the time available,” while Doris (Lead) felt that an important factor was “to ensure it’s equitable and it’s helping to achieve the outcomes that the organization needs.”
Turning to the barriers to use of EBP, the participants observed that a lack of knowledge of center policies and expectations regarding EBP implementation may act as a barrier. Thus, while the majority of those in leadership positions at the centers could identify policies/procedures supportive of EBP, providers at all participating centers were not familiar with any specific center policies regarding the level of evidence required for the use of an intervention or a therapy, “I don’t know, no.” (Shannon, Educator). It was providers’ understanding that anything to do with policies had to be discussed with the center managers, “I’m not too sure about that [policies] though. Sorry. [Manager] might be able to help you with that one” (Barbara, Educator).
As personal barriers to using EBPs, participants mentioned “some of the challenges around, that is, actually the knowledge of the evidence base” (Doris, Lead) and “fidelity. Like, so that they think they’re doing a particular practice, but actually if you watch them and observe them, what they say they’re doing isn’t necessarily what they’re doing” (Doris, Lead). Another challenge for service providers was adapting EBPs to each child and family in their particular setting, “You try it with another child and they are not taking it on at all or not understanding it or it’s not working for that child. I think that’s the big difficulty when they are all so different” (Shannon, Educator). A lead at another center expressed a similar view, “Each family is so unique, so what works for one family just doesn’t work for another” and “[although] we try really hard to appraise the most recent evidence, there is not always an answer [in the research] to our questions” (Sue, Lead).
Facilitators in Implementing EBPs and Upskilling Providers Through PD
Participants shared that other providers’ knowledge and skills helped them to learn about and implement EBP. Thus, drawing on existing knowledge and skills was singled out as an important facilitator, We’ve got occupational therapy, we’ve got a speech therapist, we’ve got the manager as well who runs the training, and we got the education leader. We’ve got full qualified staff here, so we run trainings as well about the program, yeah, about the programming and things like that (Barbara, Educator).
In a similar vein, Doris (Lead) stated that “somebody else in the organization has already done the training, and that we can kind of leverage the skills of somebody else to teach that within the context of our service.”
Participants emphasized benefits of informal role-modeling and of more structured observation/coaching, as well as of having EBP embedded in routine and culture, Having staff, being surrounded by staff and clinicians, so having a transdisciplinary team on site who are trained and experienced in these models, who can model how to embed these intervention strategies as part of an everyday daily routine in childcare (Eve, Lead).
Many participants mentioned the advantage of having a supportive multidisciplinary team, emphasizing that team members rely on and value the upskilling provided by ongoing PD. Support in a range of modes was felt as helpful, including on-the-job coaching, modeling, and mentoring—formats that were mentioned by several participants, such as Amelia (Educator), who noted seeing everyone else use it and having like the key therapist showing us how to use them as well is really helpful . . . I also think that you can get a lot of training on the job and from watching colleagues as well.
A commonly reported facilitator was the opportunity for providers to select PD or request it from senior providers or management. Sue (Lead) remarked, “It depends what staff have requested, so some staff might request some additional training on feeding, and then a therapist who has attended that training and has the relevant qualification will share the information.” Such requests are taken into consideration when management plan PD. Sue (Lead) underscored the importance of “accurate feedback” but Daisy (Lead) highlighted the challenges of eliciting feedback from all team members, So if they take the time to complete the feedback forms, if they share their preferences with us, we’ve got some really um, honest, and open to share their feedback staff members, and on the other hand, we have people that may not share their preferences with us as openly unless we outright ask.
In this respect, open communication was seen as a facilitator, especially by the direct service providers.
Discussion
Our aim was to explore the experiences, values, and knowledge of providers working with young children on the autism spectrum to gain a deeper understanding of EBP use and PD needs. Three central themes addressed our three research questions: (a) the use of practices and understanding of EBP, (b) PD, and (c) barriers and facilitators in the use of EBPs and in the delivery of PD.
Practices and PD
A strength of the study is that the interview protocol was designed to elicit information concerning current practices as described and named by providers, rather than requiring participants to choose practices from a pre-existing list, thus enabling broader listings of practices and avoiding potential biases in responding. Participants could report implementing practices (e.g., sensory materials) other than EBPs, which continue to be used perhaps due to their intuitive appeal and given that sensory impairment is part of the diagnostic criteria for autism (American Psychiatric Association, 2013). Overall, however, our findings were consistent with previous research showing greater use of EBPs than non-EBPs by providers working with children on the autism spectrum (Dynia et al., 2020; Paynter et al., 2017, 2021; Paynter & Keen, 2015; Stahmer et al., 2005).
Service providers recounted that PD was not consistently available or implemented at their centers. Approaches to PD that were described ranged from an organized tier system, systematically allocated to providers based on their qualifications and time at the center, to a more ad hoc strategy largely based on providers’ requests. This aligns with previous research, to the effect that PD, as currently implemented, can be insufficient (Dillenburger et al., 2016; Elsabbagh et al., 2014) and that in-service PD is limited in scope and inconsistently available (Giangreco et al., 2001). These limitations present a challenge, in that providers will be less able to identify their own knowledge gaps, which may be further exacerbated when an ad hoc strategy is used, perpetuating the research-to-practice gap.
Participants reported a preference for face-to-face PD, and many felt that an in situ format (e.g., coaching, modeling) was the most helpful. This option, however, may pose practical difficulties as, according to Stahmer et al. (2015), in situ training requires extensive PD, ongoing coaching, and a long time—all of which are barriers participants identified in translating research to practice. Furthermore, in situ coaching may be especially challenging following the recent COVID-19 pandemic, during which many organizations in Australia restricted access for outside visitors to educational/therapeutic settings. Geography can also be a hindrance. The geographic spread of Australia can militate against accessing support, particularly in the rural and remote areas, which are vast.
In keeping with the preferences participants expressed, and also with previous research (Giangreco et al., 2001; Paynter et al., 2017), professional staff were largely responsible for selecting and training paraprofessionals in practices. Professional staff reported using practices learned in PD, which they believed would not have been recommended if they were not EBPs. However, as discussed above, if ad hoc approaches to training prevail within a system, the use of unsupported practices may persist.
Barriers and Facilitators
The interviews revealed both barriers and facilitators in the use of EBPs. One barrier and facilitator is the use of internal knowledge as the primary means for PD in the use of EBPs. As a facilitator, participants highlighted role-modeling, availability, and use of existing skills and knowledge, affordability, and comfort in obtaining PD from a supportive team. However, as a barrier, unquestioningly relying on colleagues’ knowledge and skills might be unwarranted, as this is tantamount to taking for granted that they would promote only EBPs and train for their application with fidelity.
Extending the findings of quantitative research (Paynter et al., 2017; Paynter & Keen, 2015), participants in this study also articulated organizational (e.g., policies), center (e.g., resources) and individual (e.g., knowledge) level barriers to the use of EBP. Participants described lack of time, high costs, geographical isolation, and limited availability of resources as barriers at these levels. Conversely, some participants reported that perception of a more EBP-supportive organizational culture enhances the implementation of such practices and discourages the use of unsupported interventions. Similar conclusions were also drawn from the findings obtained in a mixed-method investigation of the effects of workplaces, including barriers, on the implementation of EBPs by speech–language pathologists working with children on the autism spectrum (Cheung et al., 2013).
When asked, no service provider interviewed in the current study could articulate clear center policies or expectations as to the level of research support required to use a practice, although some more senior staff were aware of the existence of recent guidelines. The lack of clear policies, or of knowledge of policies in place, may impact practice selection. Reliance on unsupported practices may have been further exacerbated by gaps in research. Participants were especially concerned about having to adapt to the needs of each child while also taking count of the feelings and preferences of the family. In a similar vein, the study conducted by Dingfelder and Mandell (2011) found difficulty in adapting accepted interventions to the unique needs of every child and every family. In the current study, participants also felt that research into early intervention practices for children on the autism spectrum is still scarce and many questions remain unanswered. Such impressions do reflect the current state of research. Moreover, there can be contradictory messages. For example, sensory integration is listed as an Ineffective/Don’t Do It autism intervention in Novak and Honan (2019) and as an EBP in Steinbrenner and colleagues (2020). Consequently, potentially promising practices often require more research and many interventions remain untested. This drawback was noted by participants, who reported, as concerns their own learning experience, that PD they had received did not state what not to do. Similar results were obtained by Paynter and colleagues (2019).
Participants in the current study were overwhelmingly committed to supporting implementation of EBPs with fidelity, at both organizational and individual levels. These findings confirm earlier research by Trembath et al. (2019), who reported that staff working with children on the autism spectrum within a community-based comprehensive early intervention program had strong positive views regarding EBPs. At the same time, some participants in the current study felt that selecting EBPs and implementing them with fidelity presented a challenge. Such impressions may have arisen as those spending more of their role implementing practices (paraprofessionals, direct service providers/educators in these centers) were reliant on leadership staff to train and provide support for implementation with fidelity. Yet, as Giangreco et al. (2004) demonstrated, senior members of multidisciplinary teams who usually upskill paraprofessionals may, themselves, not have received appropriate guidelines how to implement interventions with fidelity, and/or may not have been taught to provide effective coaching to that effect.
Services providers in the current study reported being unfamiliar with research and therefore relying on members of their allied health team to provide them with information and to recommend and train them in EBPs. This theme also emerges in the work of Kadar and et al. (2012) who found that service providers were prone to erroneously believe that the practices they use are supported by research even when they are backed by limited and inconclusive evidence. Likewise aligning with previous research, in the present study, the lack of fidelity in implementing interventions was found to be a barrier to EBP at the direct service level, and a possible challenge with regard to trainers’ knowledge.
As a way of resolving some of these difficulties, as well as to enhance work settings and outcomes for children and families, participants in the present study emphasized the importance of open communication among all providers, as well as between providers and managers or other center leads to share, support and learn from one another. Knowledge sharing was singled out as an especially valuable element, in view of the time constraints under which practitioners operate—as a consequence, they are rarely at leisure to research the evidence base for interventions and tend to rely on information from team members. As indicated by previous experimental research in the field (e.g., Paynter et al., 2019), knowledge may be an important and modifiable factor that directly leads to behavior change. Indeed, participants in all three centers stated that, in their respective organizations, knowledge was transferred through PD.
Taken together, our findings underscore the importance of the source of information used by paraprofessionals, such as child care educators, and the centers at large, in selecting EBPs in preference of unsupported interventions. Optimal information sources regarding evidence base of interventions need to be of high quality and provide instruction on the use of EBPs with fidelity. Information conveyed by professionals to paraprofessionals must be informed by research and knowledge of EBPs to avoid encouraging the use of unsupported or unhelpful practices that may cause harm and false hope, and/or waste resources and deflect children from effective treatments (Keen et al., 2017; Luskin-Saxby & Paynter, 2018; Paynter et al., 2017, 2018).
Limitations and Future Research
The present study has some limitations that should be acknowledged and considered when interpreting the results. One limitation was the recruitment of some participants from centers catering to children outside the age range of 0 to 6 years. However, we requested the participants to focus on their practice with young children aged 0 to 6 years when responding to interview questions. Results should be interpreted with reference to the settings within which the data were collected, that is, services in Australia catering to a young age range. Future research should investigate other populations and age ranges in the context of respective services to establish the generalizability of the findings. Second, we presented a rather narrow definition of EBP, and focused on specific practices, rather than the decision-making process in choosing interventions. This precluded exploring whether practices fit with specific goals, needs, or the child’s strengths / challenges in every given case, an approach that is important in the broader EBP framework.
Third, while the study underscores the challenges experienced by providers in the participating centers, its scope did not allow for a detailed examination of the settings, circumstances or conditions that tend to give rise to or accentuate a specific difficulty, nor did the format of the study make it possible to gauge the degree of fidelity in the implementation of the various interventions by participants. Fourth, the response rate was not available given the recruitment method and the need for confidentiality. Finally, interviews were the only source of information. In the future, surveys and observational methods could be incorporated to triangulate sources of information (as per the work of Brantlinger et al., 2005).
Implications and Conclusions
The present research has afforded insight into implementation of EBPs and PD in early intervention services for children on the autism spectrum. In line with Australian Good Practice Guidelines in Autism, as a significant strength in the assessment and treatment of young children on the autism spectrum, the study has identified the involvement of multidisciplinary teams (e.g., Prior & Roberts, 2012; Roberts et al., 2016). Practitioners displayed interest, trust, and openness to learn from their team and develop skills in implementing EBPs; they also expressed preference for practical demonstrations of EBP application in their work setting. Challenges to the across-the-board use of EBPs that have been identified in the study include restrictions in terms of time and resources, as well as reliance on anecdotal information to ground decision-making. These drawbacks, which have implications for organizations’ success in upskilling and teaching their providers to evaluate therapies in terms of EBP, and to select and apply with fidelity only EBPs that have been linked to some insufficiencies in the PD currently available to early intervention providers. The results obtained here can inform PD in early intervention services for children on the autism spectrum. More specifically, the study has highlighted the need to improve providers’ access to research evidence (e.g., NAC, 2015; Steinbrenner et al., 2020). However, common resources, such as searchable databases of critically appraised research studies or existing training packages that provide professionals and paraprofessionals with modules and resources on EBPs for autism do not include information relating to unsupported practices, that is, what not to do (Paynter et al., 2022). Thus, such resources alone are unlikely to bridge the research-to-practice gap manifested in early intervention services for children on the autism spectrum; moreover, they were not identified as a tool that participants were aware of, or chose to access. Previous research has likewise shown that access to information alone (e.g., online training modules) does not change practice (e.g., Ruble et al., 2013). In the present study, as well, providers expressed a clear preference for hands-on in situ instruction, such as coaching or modeling. The results of this study point to the importance and benefits of instituting a formal approach to staff PD that is carried out in work settings. A part of such a PD package could be a “train-the-trainer” resource (e.g., as used in the US, Ruble et al., 2013), based on state-of-the art research on EBPs for children on the autism spectrum (NAC, 2015; Steinbrenner et al., 2020). Tailored to the local context, that resource could augment service providers’ understanding and knowledge of the research evidence to identify the most effective interventions, with an added element of what not to do relating to unsupported practices. Such strategies will optimize and expedite the translation of knowledge to practice and can also help to bridge the gap in this regard, thereby achieving and sustaining change, and ensuring optimal early intervention and best possible outcomes for children on the autism spectrum.
Supplemental Material
sj-docx-1-jei-10.1177_10538151231159638 – Supplemental material for Professional Development and Use of Evidence-Based Practice in Autism Early Intervention
Supplemental material, sj-docx-1-jei-10.1177_10538151231159638 for Professional Development and Use of Evidence-Based Practice in Autism Early Intervention by Sarah Luskin-Saxby, Melanie Zimmer-Gembeck, Rhylee Sulek and Jessica Paynter in Journal of Early Intervention
Supplemental Material
sj-docx-2-jei-10.1177_10538151231159638 – Supplemental material for Professional Development and Use of Evidence-Based Practice in Autism Early Intervention
Supplemental material, sj-docx-2-jei-10.1177_10538151231159638 for Professional Development and Use of Evidence-Based Practice in Autism Early Intervention by Sarah Luskin-Saxby, Melanie Zimmer-Gembeck, Rhylee Sulek and Jessica Paynter in Journal of Early Intervention
Footnotes
Acknowledgements
The author would like to express my gratitude to Dr Gal Bohadana and Dr Angela Ebert for their contributions, as well as to the centers that participated in and contributed to this study, and in particular to their managers and staff members.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
