Abstract
A number of autism intervention practices have been demonstrated to be effective. However, the use of unsupported practices persists in community early intervention settings. Recent research has suggested that personal, professional and workplace factors may influence intervention choices. The aim of this research was to investigate knowledge and use of strategies, organisational culture, individual attitudes, sources of information and considerations informing intervention choices by early intervention providers. An online survey was completed by 72 early intervention providers from four organisations across Australia. Providers reported high levels of trust and access of information from internal professional development, therapists and external professional development. A range of considerations including child factors, family values and research were rated as important in informing intervention choices. Participants reported greater knowledge and use of evidence-based and emerging practices than unsupported. Levels of use were linked to levels of knowledge, as well as some organisational and attitudinal factors. Areas for future research and implications are discussed.
Introduction
The benefit of high-quality early intervention (EI) for children with autism spectrum disorder (ASD) has been well established in university trials (Boyd et al., 2010; Eldevik et al., 2012). Furthermore, research in the past decade has led to the classification of a number of ASD EI practices as effective (National Autism Center (NAC), 2009, 2015; Wong et al., 2013, 2015). However, utilisation of these practices in the community varies considerably with continued use of strategies that are ineffective or unsupported by research (Hess et al., 2008; Paynter and Keen, 2015; Stahmer et al., 2005). This ‘research to practice gap’ is widely lamented (e.g. Cook et al., 2013; Odom, 2009), as utilisation of evidence-based practices (EBPs) by community-based EI providers is essential to improving outcomes for children with ASD. Understanding what influences EI providers in their intervention choices is an important area of research to inform future knowledge translation efforts; this study focuses on understanding these factors as well as investigating the utilisation of EBP.
Given the heterogeneity, including response to treatment among children with ASD (Trembath and Vivanti, 2014), it is understandable that no single intervention or programme is equally effective for all. Instead, most EI providers must select from the range of interventions available, in an attempt to match each child with the interventions most likely to address their strengths and needs. There is limited research into the sources of information providers access and what other factors they consider when making these intervention choices.
Sources of information
A range of sources of information on interventions are available to EI providers. In addition to original published research and research reviews (e.g. NAC, 2015), many professional organisations (e.g. American Speech Language and Hearing Association, 2006; Filipek et al., 2000) and government bodies (e.g. Roberts and Prior, 2006) have developed practice guidelines. However, it is not clear to what extent providers utilise or trust these sources and what other sources of information may also influence their decisions. To illustrate, Nail-Chiwetalu and Bernstein Ratner (2007) surveyed 208 speech-language pathologists in the United States regarding their information-seeking behaviours. They found that consulting colleagues, continuing education events and open Internet searches were the most common approaches used to source information regarding the interventions they provided. Similarly, research with special educators and parents suggests they too tend to seek out, and trust, information from colleagues and therapists, respectively, to inform their intervention choices (Boardman et al., 2005; Deyro et al., 2014). As yet this type of research has not extended to EI providers.
Factors influencing choice of intervention practices
In addition to available information, a range of other factors influence intervention decisions by providers (Stahmer, 2007). Cheung et al. (2013), for example, surveyed 105 Australian speech-language pathologists regarding their attitudes towards EBP and workplace influencers on use. Although supporting the use of EBP, participants identified barriers to implementation including inadequate workplace culture and support, lack of time, prohibitive costs of resources and training and perceived limited availability and accessibility of EBP resources.
Personal views, experiences and attitudes may also strongly influence service providers’ selection and implementation of interventions. Carlon et al. (2015) demonstrated that parents of children with ASD prioritised individual needs of their children, staff attributes, whether interventions were ASD specific and their intuition over research evidence when making intervention decisions. Research involving ASD service providers is lacking; however, it is possible they take into account similar factors as parents when selecting interventions (e.g. individual child needs, parents’ intuition and preferences). Taking into account these factors is consistent with the EBP framework for clinical decision-making that acknowledges, in equal measure, the importance of clinical experience, client and caregiver preferences and priorities, the practice context and the best available research evidence (Hoffmann et al., 2013).
Knowledge and use of EBPs
Paynter and Keen (2015) demonstrated that even within a single organisation, staff selection of interventions varies considerably. In a survey of 99 staff from an Australian community-based ASD EI programme, staff reported having sound knowledge and predominant use of EBPs, but emerging, and some unsupported practices (as per Odom et al., 2010; NAC, 2009), were also in use. Importantly, knowledge and use of EBP was greater among staff in metropolitan than in regional locations. Furthermore, paraprofessionals (e.g. teaching assistants) reported lower knowledge of EBPs, and greater use of unsupported practices, than professional staff (e.g. teachers and speech pathologists). Paynter and Keen also found greater self-reported use of EBP was linked to perceived greater supportive organisational culture (towards EBP), as well as more positive attitudes towards and openness to EBP. Paynter and Keen (2015) proposed that a range of personal, organisational, attitudinal and demographic factors may have influenced staff knowledge and use of EBP. These included potential limitations in the training received by paraprofessionals and access to appropriate professional development opportunities for staff in regional areas. Due to the focus on a single organisation, there is a clear need for further research that investigates the generalisability of these findings.
Taken together, studies to date have separately identified that sources of information, personal and professional factors and workplace factors can influence providers’ selection of interventions. Research is now needed to examine these factors in combination with community-based EI providers. This study extends the work of Carlon et al. (2015) and Paynter and Keen (2015) by addressing the following research questions:
What sources of information do ASD EI providers access in making intervention decisions? How trustworthy do they view these sources?
What factors do providers consider when selecting intervention practices?
What is the level of reported knowledge and use of EBP, emerging and unsupported practices among providers?
What is the relationship between reported knowledge and use of EBPs, and organisational, attitudinal and demographic (staff role, experience) factors?
Tentative hypotheses were that knowledge and use of EBPs would be linked, more positive individual attitudes towards EBP would be related to greater use of EBPs, and a more supportive organisational culture would be related to greater use of EBP. No specific hypotheses were made in terms of sources of information, trustworthiness or decision-making factors given the lack of research in this area.
Method
Setting
Participants were recruited from four different ASD EI organisations across four states of Australia. Each organisation provided EI within a long day-care setting and were funded by a mixture of federal funding (the same funding scheme at each site) and parent fees. Each centre included a mixture of professional (e.g. teachers and therapists) and paraprofessional (e.g. childcare educators) staff and worked with children with ASD in pre-school ages (approximately 2–6 years). All sites were bound to the same general operational guidelines that specified staff:child ratios, type of staff employed and qualifications, as well as eligibility of access for children, service model (i.e. EI embedded in childcare) and requirement to be operating based on the current Australian guidelines for best practice (Department of Families, Housing, Community Services and Indigenous Affairs, 2009). Consistent with this, the four sites were comparable in terms of the spread of staff who were teachers, allied health/social worker and childcare paraprofessionals, χ2 (6, N = 72) = 3.73, p = 0.71. Sites also did not differ significantly in their staff member’s highest level of qualification (high school, certificate or equivalent, diploma or equivalent, bachelor degree or postgraduate degree), χ2 (15, N = 72) = 3.73, p = 0.16. None of these centres were implementing a specific manualised comprehensive treatment model although each had their own programme/s and reported at the time of the study adhering to the Australian Good Practice Guidelines (Prior and Roberts, 2012). It should be noted that these guidelines do not promote specific practices, but instead provide general guidelines (e.g. to provide family support and individualised goal setting).
Participants and procedure
Ethical approval was granted through (withheld for blind review) (RDHS-03-15), (withheld for blind review) (EDN/05/15HREC) and the (withheld for blind review) (H0014662), as well as administrative approvals from each organisation. Seventy-nine staff working directly with children with ASD were employed in total across organisations and invited to complete the online survey via email (available 9 February 2015–23 March 2015) with 72/79 (91%) opening it.
Ethical approval was granted through Curtin University Human Research Ethics Committee (Approval Number RDHS-03-15), Griffith University Human Research Ethics Committee (GU Ref No EDN/05/15HREC) and the University of Tasmania (Reference H0014662). Participants reported on average working with children with ASD for 4.75 years (standard deviation (SD) = 3.87), in their current role for 2.54 years (SD = 1.77) and in their profession for 7.18 years (SD = 5.76). Participants were predominantly under 50 years of age and most had completed some form of post-school training (see Table 1). The group included 30 professionals with 10 teachers and 20 allied health or social workers (i.e. therapists) and 42 paraprofessional childcare staff. A minority of participants (n = 10) reported some form of disability-specific qualifications (e.g. Bachelor of Special Needs).
Participant demographics.
Note sites four and five are different sites from the same organisation.
Measures
The online survey included 110 questions measuring demographics, sources of information, influences on intervention practices, organisational culture, use and knowledge of practices and attitudes towards EBPs, which took approximately 20 min to complete.
Demographics
Questions included age bracket, centre location, role, highest academic qualification, disability-specific qualifications and time working with children with ASD, current role and profession. Due to the small number of male staff employed at the centres consistent with previous research (Paynter and Keen, 2015), we decided a priori not to collect information on gender of participants to preserve anonymity.
Sources of information
The Sources of Information scale was modified from a scale used with parents (Carlon et al., 2015). It included 18 items listing possible sources of information about intervention practices used for children with ASD (e.g. parents) rated on a dichotomous yes/no to whether participants received information from each. We added a 5-point rating of trust of sources from 0 = Not at all trustworthy to 4 = Very trustworthy, or not applicable.
Considerations
The Considerations for Intervention Practices scale was adapted from Carlon et al. (2015). This included 15 potential considerations when choosing whether to use a specific intervention practice including factors related to research, values, advice and information from others, the child and training. Items were rated on a 5-point scale where 0 = Very unimportant to 4 = Very important.
Knowledge and use of practices
The Early Intervention Practices Scale (Paynter and Keen, 2015) was revised with permission from the authors in line with most recent EBP literature as of December 2014 (Wong et al., 2013, 2015). Where there was a difference in classification of practices from an earlier review (e.g. NAC, 2009) and more recent review (Wong et al., 2013, 2015), for example, Independent Work Systems (established vs emerging), the more recent classification was used to update this scale and classify established and emerging practices. Unsupported practices in the original scale had been drawn from the literature on commonly used practices (e.g. Green et al., 2006) and reviews of practice (e.g. NAC, 2009) at that time. For example, academic interventions were classified as unsupported in the NAC (2009) review and were thus included in the scale. All unsupported practices from the original scale were retained as none had been reclassified to established or emerging practices in recent reviews (Wong et al., 2013, 2015).
Participants were asked to rate their knowledge of 44 different practices (see Appendix 1 for definitions), including EBP, emerging practices and unsupported practices on a 5-point scale that ranged from 0 = Very little (know nothing about this practice) to 4 = To a very great extent (know a great deal and could instruct others on this). Participants were also asked to rate their use of each practice on a 5-point scale that ranged from 0 = Never (I do not use this practice) to 4 = Frequently (more than once per day). For further discussion of scale development, please see Paynter and Keen (2015).
Ratings for the 28 1 established practices (Wong et al., 2013, 2015) were averaged for use and knowledge to create two scales, knowledge and use of EBP that each had good reliability (Cronbach’s α = 0.93 and 0.92, respectively). Ratings for the six emerging practices (NAC, 2009; Wong et al., 2013) were likewise averaged to create knowledge (Cronbach’s α = 0.74) and use (Cronbach’s α = 0.84) scales that showed adequate reliability using conventions discussed by Field (2005). Ratings for unsupported practices (10 items) were averaged to create two scales (knowledge, Cronbach’s α = 0.70; use, Cronbach’s α = 0.76) that showed adequate reliability.
Organisational culture
The Organisational Culture Questionnaire (Russell et al., 2010) includes three scales: resources (four items), culture (three items) and supervisor (single item). Participants rated their level of agreement with each item from 1 = Not at all to 10 = To a great extent. Multiple item scales were found to have good internal consistency (Cronbach’s α resources = 0.89; culture = 0.93).
Attitudes to EBPs
The Evidence-based Practices Attitudes Scale (EBPAS; Aarons, 2004) includes four subscales: requirements, appeal, openness and divergence. Participants rated statements on a 5-point scale from 0 = Not at all to 4 = To a very great extent. Three subscales showed adequate (appeal subscale) to good reliability in the present sample: requirements (three items, Cronbach’s α = 0.93), appeal (four items, Cronbach’s α = 0.74) and openness (four items, Cronbach’s α = 0.80). One subscale, divergence (four items, Cronbach’s α = 0.59), showed poor reliability and was excluded in subsequent analyses.
Results
Data screening
The data broadly met assumptions for parametric testing with the exception of two outliers (individual scores on knowledge of unsupported practices and openness subscale of EBPAS) with these individual’s scores consequently excluded from analyses. A total of 72 participants started the survey, reduced to 60 for some questions due to missing data.
Sources of information
All participants reported receiving information from internal professional development and the trust of this information was rated the highest of any source (M = 3.6) between somewhat trustworthy (3) and very trustworthy (4). The next most common sources were therapists (97%) and workshops/external professional development (89.4%) (see Table 2). These were similarly rated as highly trusted sources with therapists rated on average at 3.48 and external professional development on average rated highly at 3.37. Following internal professional development, the most trusted sources were treatment reviews (M = 3.59) and the Positive Partnerships (Australian Government Initiative) 2 website (M = 3.57).
Sources of information and rating of trustworthiness ranked by mean trust rating.
SD: standard deviation; ASD: autism spectrum disorder.
Trust ratings were made on a scale from 0 = not at all trustworthy to 4 = very trustworthy with higher scores, thus indicating higher reported trust.
The least commonly accessed sources were (from least to most) complementary and alternative medicine (CAM) practitioners (28.9%), medical doctors (40.9%) and the Raising Children Network website (an online resource developed by the Australian Government to disseminate evidence-based information to parents, therapists and educators) (43.9%). The least trusted were CAM practitioners (M = 1.70), media (M = 1.71) and friends/relatives (M = 1.85) which were all rated on average between somewhat untrustworthy (1) and neither untrustworthy nor trustworthy (2). In addition to the listed sources, one participant also reported that she used information from ‘intuitions’. There were no significant differences between professionals and paraprofessionals in trust ratings, with all independent group t-tests, p > 0.05.
Considerations
For 12/15 factors, mean ratings were above 3.0 (somewhat important), with between 75.8% and 96.9% of participants indicating that these 12 factors were considered somewhat or very important (see Table 3). The most important factor rated was the child’s ‘strengths and needs’ (considered very important by 74.2% of participants and somewhat important by 22.7% with a mean rating of 3.7). Other factors ranked highest (in order) included family values (M = 3.65), professional judgement (M = 3.62), resources for implementation (M = 3.58) and research evidence (M = 3.52). However, although on average participants rated these highly, a minority of participants did report they felt these factors were very unimportant (0) or somewhat unimportant. For example, five participants (7.5%) reported they felt research evidence was very unimportant or somewhat unimportant in informing their intervention selection for a particular child.
Mean importance ratings for considerations ranked by mean (n = 66).
SD: standard deviation.
The lowest ranked consideration was university or post-school training (M = 2.74) rated between neither important or unimportant (2) and somewhat important (3) on average. Other factors falling in this range included own intuition (M = 2.77) and whether they thought children would enjoy the intervention (M = 2.77).
Independent samples t-tests indicated that paraprofessionals (M = 3.65, SD = 0.54) rated information from their organisations significantly more trustworthy than did professionals (M = 3.28, SD = 0.65), t(64) = −2.55, p = 0.013, d = 0.62. The two groups were comparable on all other variables listed in Table 3, p > 0.05.
Knowledge and use of practices
Individual practices
Ratings of knowledge and use of each intervention practice are shown in Table 4. Mean scores indicate all were used by some participants rarely, sometimes, often or frequently, regardless of whether supported, emerging or unsupported. The most commonly used practices across the sample (ranging from more than once per day to daily) reported by participants were (in order of most to least) visual supports (M = 3.82), Picture Exchange Communication System (PECS) (M = 3.48) and modelling (M = 3.45). These were all established treatments. The least commonly used practices across the sample (ranging from used less than once per week to never) were (in order from least to most) auditory integration training (M = 0.25), Prompts for Restructuring Oral Muscular Phonetic Targets (PROMPT) (M = 0.32) and holding therapy (M = 0.49). These were all unsupported practices. However, a number of unsupported practices were reported to be commonly used, rated as being used more than once per week; these included academic interventions (M = 2.51), facilitated communication (FC) (M = 2.58) and multisensory environments (M = 2.17).
Mean (SD) Likert scores of knowledge and use of intervention practices (n = 66) by reported use (most to least).
Knowledge was rated from 0 = very little: know nothing about this practice to 4: to a very great extent: know a great deal and could instruct others on this; use was rated from 0 = never: I do not use this practice to 4: frequently: more than once per day. Higher scores thus indicate greater knowledge and use of each practice.
Comparisons across roles (teacher vs therapist (allied health/social worker) vs paraprofessional (childcare worker)) showed no significant difference in reported use of two of the common unsupported practices: FC F(2, 64) = 2.81, p = 0.07, ηp2 = 0.08, and multisensory environments, F(2, 63) = 2.80, p = 0.07, ηp2 = 0.08. Post hoc analysis using G*Power3.1.7 (Faul et al., 2007) calculating effect sizes from means showed sufficient power (achieved power of 92% and 94%, respectively) for the omnibus one-way analysis of variance (ANOVA). A significant difference was found in reported use of academic interventions, F(2, 64) = 4.29, p = 0.02, ηp2 = 0.12. Post hoc comparisons using Tukey’s honestly significant difference (HSD) indicated that teachers reported using academic interventions (M = 4.60, SD = 0.52) more than allied health/social workers (M = 3.30, SD = 1.49), p = 0.029, and paraprofessionals (M = 3.32, SD = 1.29), p = 0.018. Reported use by therapists was not significantly different to paraprofessionals, p = 0.997.
Comparisons across levels of evidence
Participants differed significantly in their use of each category of practice, F(1.68, 109.05) = 44.68, p < 0.001, ηp2 = 0.41 using a Greenhouse-Geisser adjustment due to non-sphericity (Mauchy’s test, χ2 (2) = 13.67, p = 0.001). Contrasts showed higher use of EBP (M = 3.33, SD = 0.76) than emerging practices (M = 3.12, SD = 1.15), but this difference was not significant F(1, 65) = 2.96, p = 0.09, r = 0.21. Participants reported significantly greater use of emerging practices than unsupported practices (M = 2.42, SD = 0.75), F(1, 65) = 61.83, p < 0.001, r = 0.70.
Participants differed significantly in their knowledge of each category, F(2, 128) = 37.83, p < 0.001, ηp2 = 0.37. Contrasts showed, although participants on average reported greater knowledge of EBP (M = 3.34, SD = 0.65) than emerging practices (M = 3.23, SD = 0.73), this difference was not significant, F(1, 64) = 2.98, p = 0.089, r = 0.21. However, participants reported greater knowledge of emerging practices than unsupported practices (M = 2.83, SD = 0.52), F(1, 64) = 40.20, p < 0.001, r = 0.62.
Comparisons between professionals and paraprofessionals
Professionals reported greater knowledge of EBP (M = 3.58, SD = 0.54) than paraprofessionals (M = 3.19, SD = 0.71), t(64) = 2.46, p = 0.017, d = 0.62. Paraprofessionals (M = 2.67, SD = 0.75) reported using more unsupported practices than professionals (M = 2.11, SD = 0.63), t(64) = 3.19, p = 0.002, d = 0.81. However, professionals (M = 3.47, SD = 0.66) reported similar levels of use of EBP as paraprofessionals (M = 3.22, SD = 0.83), t(64) = 1.35, p = 0.18, d = 0.33. Furthermore, no significant differences on emerging practices were found in professionals and paraprofessionals’ levels of knowledge (M = 3.30, SD = 0.56; M = 3.22, SD = 0.88, respectively; t(64) = 0.45, p = 0.66, d = 0.11) or use (M = 2.86, SD = 1.03; M = 3.34, SD = 1.21, respectively; t(64) = 1.73, p = 0.09, d = 0.43). Finally, no significant difference in knowledge of unsupported practices was found between professionals (M = 2.83, SD = 0.51) and paraprofessionals (M = 2.89, SD = 0.63), t(64) = 0.39, p = 0.70, d = 0.10. However, these findings of non-significant differences should be interpreted with caution as post hoc analysis using G*Power 3.1.7 (Faul et al., 2007) using the obtained small to medium effect sizes for calculation indicated achieved power was insufficient (EBP use 37%, emerging knowledge 11%, emerging use 52% and unsupported knowledge 111%) for these analyses.
Correlations between knowledge and use of EBP, organisational culture, attitude to EBP and time
Data were aggregated across groups as per Paynter and Keen (2015) for correlations. There was a significant correlation between knowledge and use of EBP with higher reported knowledge linked to higher reported use of EBP (see Table 5). Furthermore, greater culture and supervisor (support for EBP) were significantly linked to both higher knowledge and use of EBP. Resources to support EBP implementation were not significantly linked to either knowledge or use of EBP. In terms of attitudes, level of willingness to use EBP if it was a requirement, as well as openness (to using EBP), were significantly linked to greater knowledge and use of EBP. Only the requirements subscale was significantly linked to more positive ratings of organisational culture support for EBP and was linked to all three scales (resources, culture and supervisor). Time working with children with ASD, in the participant’s specific role, or in their profession, was generally not linked to knowledge/use of EBP, attitudes to EBP or organisational culture. Only time in profession was linked to knowledge of EBP with greater time linked to lower self-reported knowledge.
Correlations between EBP knowledge, use, organisational culture and attitude towards EBP.
EBP: evidence-based practice; ASD: autism spectrum disorder.
p < 0.05, **p < 0.01 (two-tailed).
As knowledge and use of EBPs were both significantly linked to organisational culture (culture and supervisor) and attitude (requirement and openness), as well as each other, the unique predictors of use of EBP were of interest. To investigate, two hierarchical multiple regression (HMR) analyses were conducted. For the first, organisational culture and attitude variables were entered at the first step, R2 = 0.23, Adjusted R2 = 0.18, F(4, 60) = 4.56, p = 0.003. At step 2, knowledge of EBPs was entered and added a significant increment in the predicted variance in use of EBPs,
To investigate conversely, whether organisational culture (culture and supervisor) and attitude (requirement and openness) would be independent predictors of use of EBPs once knowledge was controlled, a second HMR was conducted. In this, knowledge of EBPs was entered in the first step, R2 = 0.71, Adjusted R2 = 0.70, F(1, 63) = 152.70, p < 0.001. At step 2, organisational culture (resources and culture) and attitude (openness) were entered and did not add a significant increment to the predicted variance in use of EBPs,
Discussion
As hypothesised, the findings from this study confirmed those of Paynter and Keen (2015), indicating the generalisability of the results across ASD EI organisations in Australia. In particular, greater knowledge was related to greater use of EBPs by staff. Higher scores on requirement to use and openness attitudes, and supervisor attitude and organisation culture variables were significantly linked to higher reported use of EBP in zero-order correlations, but were not independent predictors once knowledge was controlled. Overall, while staff reported using EBP and emerging practices more than unsupported practices, some practices (e.g. FC) were reported to be used at least weekly and paraprofessionals reported greater use of unsupported practices than professionals. This persistent finding underscores the importance of understanding what influences staff to choose these unsupported practices.
All participants reported internal professional development a source of trusted information for their intervention choices. Therapists and external professional development were also important and trusted, consistent with previous research with other groups (e.g. Boardman et al., 2005; Deyro et al., 2014; Nail-Chiwetalu and Bernstein Ratner, 2007). Reliance on internal professional development, therapists and external professional development may be helpful when this is of high quality and may lead to greater use of EBP. However, if information from these sources is inaccurate or endorses unsupported practices, this may result in the use of these practices by staff individually or collectively across an entire organisation. Paraprofessionals who in this study reported greater trust in therapists as a source and also reported greater use of unsupported practices may be particularly vulnerable in such circumstances. However, paraprofessionals may also differ on other factors such as access to training or supervision of practice and additional exploration of why they report greater use of unsupported practices is needed to address such potential confounds.
Particular staff groups may be vulnerable to practices commonly used in their field with teachers reporting greater use of academic interventions than other groups for example. No differences between teachers, therapists and paraprofessionals were found in the use of multisensory environments or FC. However, it may be of interest in future research to compare groups of therapists such as occupational therapists and speech pathologists to investigate if differences in use of unsupported practices emerge between therapy groups. For example, previous research has suggested occupational therapists continue to use sensory integration (an unsupported practice) with children with ASD broadly (Kadar et al., 2012), but whether this is more than other therapist groups is not yet known.
It is important to note that participants in this study identified a range of sources of information, and while inaccurate information may be conveyed via a trusted source, accurate and evidence-based information may also be received at the same time. For example, 60% of participants reported accessing treatment reviews that could help inform judgements on the use of EBPs and mediate against inaccurate information or endorsements of unsupported practices disseminated through other channels. A challenge, however, for practitioners may be the changing goal-posts, for example, during the study period a new review was published (NAC, 2015) with classification of some practices (e.g. PECS) differing in this review compared to reviews available during the study period (e.g. Wong et al., 2013, 2015). How practitioners receive and respond to such information and changes is an important topic for future research. In addition, information received is not the only consideration informing intervention choices. The range of factors EI providers consider when choosing to use or not use an intervention is thus important.
Only three considerations were rated as between neither important or unimportant and somewhat important, these were whether they thought children would enjoy the intervention, ‘gut feelings’ and university or post-school training. This was in contrast to previous research with parents who rated that their intuition or ‘gut feelings’ were somewhat to very important (Carlon et al., 2015). Reasons why these factors were viewed as less important by providers need further investigation as there are various possible explanations. For example, university training may not have been rated as important, if service providers felt that their courses (1) did not provide them with the training required to critically appraise interventions, (2) were outdated (e.g. completed a long time ago) and/or (3) were not considered relevant (e.g. general knowledge vs autism-specific). However, university or other post-school training may provide a valuable opportunity to teach skills to counter misinformation about ASD that is widely available in the community. Bain et al. (2009) stated that … training in basic critical thinking and research evaluation skills is absolutely essential … We recommend that an integral element in university training of future educators include instruction in rigorous examination and evaluation of information that is commonly presented by public media sources … Additionally, many university courses attempt to focus on critical thinking skills, and these instructions should include healthy doses of the advantages of skepticism, as opposed to optimistic gullibility … (p. 86)
Although referring to pre-service teachers, this statement could apply equally to allied health professionals and paraprofessionals working with children with ASD.
Participants rated all other potential considerations in their decisions to choose an intervention practice as somewhat or very important including the child’s strengths and needs, family values, professional judgement and research evidence. This finding is encouraging, as it suggests that staff are aware of the importance of all of these factors in the provision of quality intervention services consistent with the broader EBP framework (e.g. Hoffmann et al., 2013). However, an important question requiring further research is what weight staff may give to each of these factors, particularly when they conflict. In addition, how these factors interact with sources of information in influencing knowledge and use of intervention practices is also important. There is a need in future research to develop empirically validated scales of both sources of information, and considerations, to allow analysis of the links between these and knowledge and use of intervention practices. Furthermore, multi-level modelling of the evidence-based strategies, level of fidelity and fit to child and family variables is important to inform targets for training of staff and ultimately to enhance service quality.
There were a number of limitations to this study. First, although comparable to previous research in the area (e.g. Paynter and Keen, 2015), this study included a relatively small sample size. This limited the comparisons that could be made between groups, for example, findings of non-significant differences between professional and paraprofessional groups in terms of levels of use of EBP may be due to insufficient power and there is a need for further research with larger samples to investigate potential differences further before conclusions can be made. Furthermore, aggregation of data across groups allowed comparison to Paynter and Keen (2015), but more fine-grained analysis of specific participant groups was not feasible with the sample size. This aggregation may have obscured correlations if different patterns were found for each group. Finally, aggregation of data also included nesting of providers. Consequently, there is a risk that latent variables that differed across organisations may have influenced both knowledge and use of EBP, as well as the culture and staff attitudes of using EBPs. Future research with larger samples, collected across organisations and including data on specific roles (i.e. speech pathologist, occupational therapist, rather than collapsing into allied health), would allow investigation of potential differences in factors linked to use of EBPs for each group, as well as investigation of latent variables, informing avenues for intervention and support.
Second, data were based on self-report and may not accurately reflect practice or the level of fidelity in implementation of those practices. Furthermore, by providing participants with a list of practices, responses were limited to those on the list, and were based on the most current research of the time of writing (Wong et al., 2013, 2015), with further reviews published since then (NAC, 2015). Practices may be used that were not included on the list and practices may have been selected because respondents thought they approximated actual practice without the respondent having selected or been trained in correct implementation. Also, it is possible that participants misunderstood some of the listed practices. Research asking participants to describe their own practice may provide useful insight into current practice and address these issues. To illustrate, consider FC. Although a brief definition of FC was provided to participants in this study (see Appendix 1 for definitions), some respondents may not have been clear about FC as a specific intervention technique. They may have interpreted it as ‘facilitating communication’, using visual supports or augmentative and alternative communication. This could have led to over-reporting of the use of a practice like FC. Direct observation or having participants describe how they were facilitating a child’s communication could illuminate what participants were doing and whether they were actually using FC (e.g. as per Stahmer et al., 2005). That is, gathering information about actual practice and then determining which, if any, EBP were being used would profile actual practice and could be compared with self-report.
The way in which staff interpret or understand EBP labels has implications not only for the accuracy of self-report but also for knowledge transfer among professionals and parents. There are obvious risks if, for example, staff are incorrectly labelling as FC their use of EBP such as visual supports to enhance communication skills. Confusion about terminology, definitions and endorsements of unsupported practices may occur, highlighting the need for staff to be well educated, not only about EBP but also unsupported practices.
Findings from this study highlight opportunities and challenges: EI providers rely on information from their organisations, therapists and external workshops and trust this information. Furthermore, parents trust their child’s intervention to ASD EI providers and rarely ask about the evidence base behind interventions (Auert et al., 2012; Trembath et al., in press). For children to achieve optimal outcomes, it is important that providers know and use the best available evidence to inform their intervention choices and implement these strategies with fidelity. Our findings highlight an opportunity to increase use of EBP by focusing on increasing knowledge of EBP and fostering an organisational culture supportive of EBP. This presents a challenge for EI organisations like those involved in this study. EI providers may adopt an eclectic approach and in the absence of a manualised or packaged programme, lack a framework to guide the selection of intervention practices and to balance what may be conflicting pressures of parent values, preferences and priorities, clinical knowledge and research evidence. An effective systems-wide model of knowledge transfer is needed. Such a model would lead to an organisation supportive of EBP knowledge and use and employing professionals and paraprofessionals who select and use EBP with fidelity to achieve the benefits of high-quality intervention for children with ASD in the community.
Footnotes
Appendix
Intervention Practices Scale–Revised: practice definitions.
| 1. Antecedent-based intervention. This intervention involves arranging events or circumstances that come before a challenging behaviour with the aim of reducing the behaviour |
| 2. Academic interventions. These interventions involve the use of traditional teaching methods to improve academic performance |
| 3. Auditory integration training. This intervention involves the presentation of modulated sounds through headphones in an attempt to retrain an individual’s auditory system with the goal of improving distortions in hearing or sensitivities to sound |
| 4. Brushing/Wilbarger protocol. This intervention involves the use of brushing over children’s skin using firm pressure |
| 5. Developmentally based (e.g. floortime/greenspan). Interventions that involve a combination of procedures that are based on developmental theory and emphasise the importance of building social relationships |
| 6. Differential reinforcement (DRA/I/O/L). Behaviourally based strategies that focus reinforcement on alternative, incompatible, other or lower rates of the interfering behaviour in order to replace it with more appropriate behaviour |
| 7. Discrete trial teaching. One-to-one instructional strategy that teaches skills in a planned, controlled and systematic manner |
| 8. Exercise. These interventions involve an increase in physical exertion as a means of reducing problems behaviours or increasing appropriate behaviour |
| 9. Extinction. Behaviourally based strategy that withdraws or terminates the reinforcer of an interfering behaviour to reduce or eliminate the behaviour |
| 10. Facilitated communication. This intervention involves having a facilitator support the hand or arm of an individual with limited communication skills, and guiding their hand to help the individual express words, sentences or complete thoughts using a keyboard of words or pictures or typing device for the individual |
| 11. Functional behaviour assessment. A systematic approach for determining the underlying function or purpose of behaviour |
| 12. Functional communication training. A systematic practice of replacing inappropriate or ineffective behaviour with more appropriate or effective behaviours that serve the same function |
| 13. Holding therapy. This intervention involves a person holding the child with autism tightly in a way that ensures eye contact between them |
| 14. Joint attention interventions. Interventions that build skills for two more people to focus simultaneously on an object and each other |
| 15. Massage/touch/deep pressure. These interventions involve the provision of deep tissue stimulation |
| 16. Modelling. This intervention involves demonstration of a desired behaviour with the aim of teaching it through imitation by the learner. This strategy is often combined with reinforcement and prompting |
| 17. Multisensory environments/Snoezelen. This involves the use of a range of sensory experiences (e.g. sights, sounds, textures, aromas and motion) to provide stimulation of the sensory systems to meet an individual’s sensory needs |
| 18. Music therapy. These interventions seek to teach individual skills or goals through music. A targeted skill (e.g. counting, learning colours, and taking turns) is first presented through song or rhythmic cuing and music is eventually faded |
| 19. Naturalistic interventions. Strategies that closely resemble typical interactions and occur in natural settings, routines and activities |
| 20. Parent-implemented interventions. Strategies that recognise and utilise parents as the most effective teachers of their children (e.g. Hanen and Stepping Stones) |
| 21. Peer-mediated instruction/intervention. Strategies designed to increase social engagement by teaching peers to initiate/maintain interactions |
| 22. Picture Exchange Communication System (PECS)™. A system for communicating that uses the physical handing over of pictures or symbols to initiate communicative functions |
| 23. Pivotal response training. An approach that teaches the learner to seek out and respond to naturally occurring learning opportunities |
| 24. Prompts for Restructuring Oral Muscular Phonetic Targets (PROMPT) method. This intervention is a manualised tactile-kinesthetic approach that uses touch cues to a student’s jaw, tongue and lips to manually guide the production of a target word |
| 25. Prompting. Behaviourally based antecedent teaching strategy |
| 26. Reinforcement. Behaviourally based consequence teaching strategy |
| 27. Response interruption/redirection. Physical prevention or blocking of interfering behaviour with redirection to more appropriate behaviour |
| 28. Scripting. This involves a script (verbal/written description about a specific skill or situation) being practiced repeatedly as a model for the learner to later use in the actual situation |
| 29. Self-management. A method in which learners are taught to monitor, record data, report on and reinforce their own behaviour |
| 30. Sensory diet. This intervention uses a personalised activity plan that provides the sensory input throughout the day |
| 31. Sensory integration. These treatments involve establishing an environment that stimulates or challenges the individual to effectively use all of their senses as a means of addressing over-/under-stimulation from the environment |
| 32. Sign language instruction (e.g. Auslan). These interventions involve the direct teaching of sign language as a means of communicating with other individuals in the environment |
| 33. Social narratives/social stories. Written narratives that describe specific social situations in some detail and are aimed at helping the individual to adjust to the situation or adapt their behaviour |
| 34. Social skills training groups. Small group instruction with a shared goal or outcome of learned social skills in which participants can learn, practice and receive feedback |
| 35. Speech-generating devices and other alternative and augmentative communication (AAC). The use of high-tech (e.g. electronic portable devices) and/or low-tech (e.g. picture boards or books) aids that are used to teach the learner to communicate by providing an alternative means to communicate or to supplement their speech and language skills |
| 36. Stimulus control/environmental modification. The modification or manipulation of environmental aspects known to impact a learner’s behaviour |
| 37. Independent work systems. Visually and physically structured sequences that provide opportunities for learners to practice previously taught skills, concepts or activities |
| 38. Task analysis. Behaviourally based antecedent teaching strategy that breaks down steps and links them for prompting |
| 39. Technology-aided instruction. Instruction or interventions where technology is the main feature supporting goal acquisition, including any electronic item/equipment/application/or virtual network that is used intentionally to increase/maintain, and/or improve daily living, productivity or recreation/leisure skills |
| 40. Theory of mind training. These interventions are designed to teach individuals with ASD to recognise and identify mental states (i.e. a person’s thoughts, beliefs, intentions, desires and emotions) in oneself or in others and to be able to take the perspective of another person in order to predict their actions |
| 41. Time delay. Behaviourally based antecedent teaching strategy that promotes errorless learning |
| 42. Video modelling. Utilises assistive technology as the core component of instruction and allows for pre-rehearsal of the target behaviour or skill via observation |
| 43. Visual supports. Tools that enable a learner to independently track events and activities |
| 44. Weighted vests/clothing. This treatment involves wearing items of clothing with weights sewn into it being used with the aim of helping people process sensory information |
Shading indicates the commonly used unsupported practices.
Acknowledgements
The authors thank the centre staff for sharing their experiences with them in this study.
Declaration of conflicting interests
J.M.P., S.F., K.F. and S.P. were employees of the community-based EI organisations where this study was conducted, but were not participants in this study.
Funding
This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors. However, the authors thank the Australian Department of Social Services for their financial support for the EI centres.