Commentary – bridging the research and practice gap in autism: The importance of creating research partnerships with schools

The challenge of the research–practice gap in autism education and research

There is apparently little disagreement in the field of autism regarding the gap that exists between research on educational interventions and their uptake and implementation in real-world classrooms. Indeed, Simpson (2007) highlights the extent to which the autism field in particular has been vulnerable to the lure of certain interventions, with many parents and professionals opting for well-marketed, but non-evidenced, intervention strategies in the hope of improving outcomes for children (see also Goin-Kochel et al., 2007; Hess et al., 2008). These choices are made for good reasons and with the best of intentions in terms of wanting to support children’s learning and progress effectively (Simpson, 2007; Simpson et al., 2007), and it is perhaps unsurprising that practitioners and families are willing to ‘try things out’ when the evidence base for educational interventions is so equivocal and contentious (Parsons et al., 2009), with many gaps in knowledge (Charman et al., 2011). The corollary of this is a lack of clear guidance from research about what does ‘work best’ for children (Simpson, 2007) and frustration of parents about the bewildering array of, often contradictory, information available (Mackintosh et al., 2005).

Longer term outcomes for children and young people on the autism spectrum remain poor in relation to education, employment, housing and relationships (Howlin et al., 2004; Wagner et al., 2005; Wittemeyer et al., 2011), despite the increased financial and resource investment in efficacy studies of early intervention in the past decade or so (Howlin, 2011). While it is not possible to attribute poor outcomes for people with autism directly or solely to the research–practice gap, it nevertheless suggests that there is room for strengthening the transfer of successful approaches from research evidence into classrooms with the aim of supporting better outcomes for children and young people over the longer term. While many of the recently published research interventions have come with manuals to support implementation in schools, there are important questions to be asked about what kind of evidence is needed and how there can be greater uptake of evidence-based practices in supporting children’s learning within schools.

The answer, from the perspective of many researchers, is to suggest that the educational interventions’ evidence base needs more methodologically rigorous research ¹ (Charman et al., 2011; Kasari, 2002; Parsons et al., 2009, 2011) and that the outputs of such research need to be translated into practitioner-friendly summaries where judgments about effectiveness and implementation can be made in a more informed way by users of research outputs (e.g. Reichow et al., 2008). Additionally, there is a tendency to imply that the failure of some research to demonstrate good outcomes is due to a lack of adherence to programme fidelity by school-based practitioners (Howlin et al., 2007; Stahmer et al., 2010) rather than limitations with the programme per se. Thus, proposed solutions from researchers to the problem of the research–practice gap tend to relate to improving research methodologies, implementation (including training and awareness for staff and parents) and dissemination, with researchers occupying an ‘expert’ position in the process. The implication is that practitioners need to ‘fit in’ with what is prescribed with little attention paid to their needs or perspectives and the contexts within which they work.

The importance of everyday contexts and practitioner involvement

From the perspective of practitioners, Jordan and Powell (1995) highlight that so-called ‘therapist drift’ arises not because practitioners wilfully ignore instruction or information about implementing specific techniques or intervention programmes but because they adapt approaches according to their own personal working styles and the individual pupils they are working with. Indeed, Boardman et al. (2005) and Stahmer et al. (2005) report that teachers and early years practitioners are generally less concerned about whether a new practice is ‘evidence-based’ and more concerned about feasibility of implementation and ‘best fit’ with the child’s needs. In addition, practitioners realise that they often need to interweave different learning techniques and approaches to achieve a myriad of outcomes for each pupil, such as educational attainment, behaviour, emotional stability, social understanding and skills towards independence. They are aware that the fidelity of each intervention may be compromised in some way to enable implementations of several approaches, sometimes simultaneously (e.g. structured 1:1 tasks as well as more naturalistic, child-led interactions) (Jones, 2002). The difficulty, of course, is that many research methodologies do not allow for this flexibility of interpretation, and so this means that programmes and interventions may not be carried out in the same way twice; indeed, prescription regarding implementation is often at odds with pedagogical beliefs and expertise about the importance of supporting children on an individual basis. Thus, respecting and valuing practitioners’ expertise is crucial in understanding how interventions can be carried out in the varied and complex settings of real-world classrooms (see Viewpoint Box 1 for a head teacher’s reflection on this).

Kasari (2012) acknowledges the difficulties of implementing research-validated approaches (often conducted in more controlled ‘lab-schools’ or specialist units attached to universities or services) in real-world classrooms but nevertheless emphasises the importance of doing so, even if we end up with data on ‘partial’, rather than ‘full’, effectiveness for a particular intervention approach or programme. This is because, at least in the United Kingdom and the United States, most children on the autism spectrum do not spend their school days in specialised intervention programmes and instead attend mainstream (regular) school settings or special schools where a range of educational approaches are offered (Department for Education, 2011; US Department of Education, National Center for Education Statistics, 2011). In recognition of this, the research field needs to do a better job of finding ways to implement and evaluate intervention approaches in the settings in which children spend most of their time, or that can be ‘… implemented within the broad fabric of life’ (Ogletree et al., 2007: 241). What are needed are intervention approaches that are supported by evidence and can be recommended for use for particular pupils in different contexts (mainstream, specialist autism schools, intervention settings). The evidence base is clear that educational interventions should be appropriately tailored for individual needs (Parsons et al., 2009; Simpson et al., 2007) but that more studies are needed which ‘… shed light on which children benefit most from which interventions and the intensity and length of treatment necessary to effect a change’ (Kasari, 2002: 457).

The value of collaborative approaches to research in schools

In the field of educational psychology more broadly, the challenges of implementing intervention approaches in real-world classrooms have been well-rehearsed over many years, with a clear recognition that it is not possible to do so effectively without meaningful collaboration with, and involvement from, the teaching staff involved (e.g. Gutkin, 1999; Reschly, 1976). Nastasi et al. (2000) provide a strong rationale for the importance of a collaborative approach to interventions in schools and present their own model – the Participatory Intervention Model (PIM) – showing the success of implementing such an approach (see Stein et al., 2002 for a description of a similar model for school-based mental health services in the United States; and Schoenwald and Hoagwood (2001) for a conceptual framework on the ‘transportability’ of evidence-based treatments in mental health services). The PIM comprises three phases and, crucially, there is an expectation that collaboration between researchers and other stakeholders (‘participatory generation’) is essential for the generation of ideas about, and the design of, an intervention. This is followed by a process of implementation (or ‘natural adaptation’) and an evaluation of effectiveness, which focuses on the ‘essential changes’ that would be expected to be seen if the impact of the intervention was successful. At all stages, there is partnership, negotiation and iterative development in terms of implementation, data collection and interpretation of outcomes, so that there is sharing of power and control over the research (see Viewpoint Box 1 for comments on the importance of this). Nastasi et al. (2000) argue for the application of such a model because there is a recognised

… need for a broad conception of intervention development and evaluation that addresses intervention acceptability and integrity, in addition to efficacy … In particular, designing interventions that consumers (e.g., teachers, parents, students) find acceptable is recognized as critical to the effective implementation and the subsequent effectiveness of interventions. (Nastasi et al., 2000: 207; our emphasis)

Indeed, Callahan et al. (2008) argue that one of the reasons for the persistence of the research–practice gap in autism could be the lack of social validation research, that is, what parents and practitioners find acceptable in terms of intervention goals, strategies and outcomes. Taking into account the cultural specificity of the local context in which the intervention takes place is crucial to its likely success, something that Nastasi et al.(2000) also emphasise because this leads to greater ‘… involvement, ownership and empowerment of stakeholders’ (p. 209) in the process (see also Gersten and Brengelman, 1996; Lentz et al., 1996; Ogletree et al., 2007, for similar ideas). There is evidence to show that collaborative models of working improve ‘treatment integrity’ of educational interventions compared to more expert-led approaches (Kelleher et al., 2008) suggesting that including teaching professionals from the start could have important benefits to successful implementation (and, therefore, outcomes; Callahan et al., 2008).

In addition to considerations of effectiveness, there are other opportunities that arise through taking a more collaborative approach to research methodology; for example, one of the advantages of school-based research is that practitioners frequently collect data (Wittemeyer et al., 2011) and academic researchers can play a role in terms of formulating testable hypotheses to assess through analysis of these data. Additionally, schools may have sizeable numbers of pupils on the autism spectrum across the age range. Cross-sectional research designs can take advantage of the fact that children with autism will be represented in multiple year groups, which would allow, for example, the opportunity to investigate the changing nature of autism (and related provision) in a whole-school context. Moreover, as children often stay in the same school for many years, it allows collaborative research teams (including key stakeholders) to implement baseline data collection and then conduct longitudinal studies to track the course of development during a school placement and how that may relate to other aspects of educational provision (such as time spent with and without specialist support). As Odom et al. (2005) remind us, it is vital in special education research that different methods are used to answer different research questions and that all of these methods should be regarded as valuable science; in other words, it is not just the ‘gold standard’ of randomised-controlled trials that matter.

An example of a research–practitioner partnership in action

Thus, there is a need to develop more effective partnerships between schools and researchers in order to make better use of these opportunities and in a way that appropriately recognises the skills that both groups bring, while also reflecting that the groups are far from dichotomous, with many teachers bringing valuable research expertise through the uptake of postgraduate and higher degrees (for example, see Viewpoint Box 2 for a perspective on this). Here, we illustrate a partnership model between school practitioners and researchers as a way of developing a more collaborative discourse and practical implementation around research and practice in the autism field. These efforts (or at least aspiration) could be positioned within the ‘participatory generation’ phase of Nastasi et al.’s (2000) PIM, although we were interested in research questions about autism in school-based contexts that were wider than intervention.

What follows is a brief description of the partnership as well as some of the activities completed so far. We also include some reflective commentary from practitioners within the group about their experiences of research more widely, and being a part of this process more specifically (see Viewpoint Boxes 1 and 2), and conclude with some notes on our plans for the future. In presenting the views of the two teachers in Viewpoint Boxes 1 and 2, we are not claiming these are representative of schools or practitioners but rather use these as lenses through which to explore motivations and experiences of individuals involved. Through demonstrating a willingness to be a part of the partnership, these schools ‘reached out’ to academics interested in collaboration and so their comments reflect this. As educational providers, our experience suggests that in the United Kingdom at least, this is not atypical in relation to autism-specific educational provision with many teachers and teaching assistants pursuing continuing professional development in the form of further training and taught or pursued research postgraduate degrees.

The Pan-London Autism Schools Network–Research partnership

In 2008, head teachers from special schools for children on the autism spectrum in London began to meet with the purpose of sharing knowledge and experiences; the group is named the Pan-London Autism Schools Network (PLASN). The PLASN teachers realised that they could not necessarily find answers to some of the issues they faced in their schools without research, and so they made contact with academic researchers to instigate a meeting in 2009 to discuss shared areas of interest. The aim was to meet regularly in order to discuss the challenges and opportunities experienced by all in relation to school-based research in autism and to collaboratively generate ideas for the design, implementation and dissemination of research projects; the partnership was named PLASN-Research (PLASN-R) and included 10 schools and eight researchers. ² Following discussion, a ‘memo of understanding’ was generated regarding the membership and objectives of the group, and a number of different activities initiated (with varying degrees of collaboration), which are outlined briefly below.

PLASN-R met approximately once a term to discuss ideas and plan activities, with meeting rooms and refreshments being provided by some of the universities and the schools involved. All gave their time for ‘free’, and thus, there were substantial personnel resources contributed to the project, with some start-up travel and subsistence costs being provided by an external funder and one of the universities (see Acknowledgements). Although the direct costs of the partnership are therefore low on paper, the real cost of the time involved is significant, should not be underestimated and does have implications for longer term sustainability (a point to which we return in the Discussion).

However, the fact that the need for such a partnership was identified by the schools is an important feature of the collaboration. It meant that schools were willing to commit resources to the project and provided input to the research questions and project design as well as offering research settings and participants. Crucially, schools were interested in the outcomes because the projects were initiated partly through their own concerns. This is in contrast to external requests from researchers who want to include pupils in their studies and where the schools get very little in return. The practical benefits are that schools are far more willing to put in time and allow for disruption to usual routines so that research projects they have prioritised can take place.

Collaborative activities undertaken by the partnership

Reflections on process and outcomes

Within a short period of time, the PLASN-R partnership has led to two completed projects as well as useful background information about the PLASN schools, which positions the research in the particular context of the profile of pupils attending these schools. These activities stemmed from jointly defined and agreed research priorities based on discussion of research information that schools would find genuinely useful and which also enabled valuable and feasible research questions to be answered. The latter is an important point: there were many topics, questions and challenges discussed but not all were amenable for exploration within this context – either because they were too complex (e.g. establishing the effectiveness of each school’s educational provision on learning outcomes) or would require a level of resource that was unavailable to us (e.g. establishing a common assessment framework across PLASN members). Thus, the direction of PLASN activities was, crucially, determined in the context of awareness within the group about what was, and was not, achievable.

Nevertheless, there remains an ongoing challenge regarding the resourcing of activities within the group; both the schools and the researchers have provided resources in the form of time, and this is not trivial in a context of competing demands, reduced budgets and very busy professional lives. Thus, while some aspects of the activities were funded (the ‘outcomes’ project), others were not, and so there was significant goodwill necessary for the partnership to continue and some local funds and resources at the universities (e.g. student projects) were drawn upon. However, PLASN-R has demonstrated a productive and novel working relationship, with a research partnership being formed with schools that provide education for almost 700 children. To any funder, the scale and potential impact of research outputs from PLASN-R is attractive, and now that the ‘pilot’ stage of PLASN-R has been completed, there is time for reflection before considering how to provide sustainable support for this initiative.

The process of collaboration has certainly been a valuable one and has, we think, genuinely provided opportunities for partnership and improved understanding between researchers and schools in a way that was not previously available to us. The comments in Viewpoint Boxes 1 and 2 powerfully attest to this but also clearly – and appropriately – challenge us to demonstrate the value of this approach in a more tangible way and to sustain the good practice we have begun. While research topics were certainly developed from a ‘grounded’ process of identifying and building on the research priorities identified by schools (rather than a ‘top-down’ process of targeting gaps in the research literature), it is nevertheless true that our research activities remained researcher-led and data-driven. Thus, while our ‘participatory generation’ of research foci was appropriately collaborative, our conduct of the research arguably perpetuated an ‘expert’ model of researcher/consultant in the implementation phase (cf. Nastasi et al., 2000).

Moreover, it is clear that our partnership does not yet extend to include the voices of other key stakeholders such as teaching assistants/paraprofessionals, parents, people on the autism spectrum and pupils. The involvement of wider school communities, and other professionals, is likely to be essential for identifying relevant research foci and strengthening the implementation, as well as acceptability, of the research carried out (cf. Callahan et al., 2008; see Brookman-Frazee et al., 2012 for an example of a wider, multidisciplinary research–community partnership targeting young children at risk of autism spectrum disorder). As our collaboration continues to develop, we would hope for a wider group of school practitioners to be more directly involved in the gathering, curation and interpretation of data, and for other stakeholders to be included in the partnership, in order to reflect a more shared endeavour that is embedded within the needs and demands of local school contexts. Thus, the model potentially affords considerable opportunities for the continuing professional development of teaching staff as well as improving researchers’ understanding of the complex and specific environments in which pupils and teachers work. Of course, finding ways to do this effectively and authentically while balancing a range of perspectives and experiences will be very challenging; but it is a challenge that should be tackled.

One of the main considerations for continuing to develop the partnership will be in relation to how the outcomes of the research are ultimately managed, communicated and used. There is a need to ensure that information gathered through these projects is presented and discussed with schools in ways that help them to (for example) showcase good practice or make a data-informed rationale for funding or resources. There will be academic publications, of course, (and these remain important as contributions to knowledge) but PLASN-R will only be paying lip-service to its participatory ethos if that is the extent of the impact made. The true test of the model will be to value and create a variety of opportunities for reflecting on new knowledge and relating that back to the setting-specific needs of the schools involved, so that the benefits of investing time and effort in the partnership are made real for all concerned.

Some of this is already underway in the form of verbal reports/updates at staff meetings within the schools and through written termly newsletters sent to parents. In addition, now that the first projects have been completed, the partnership has discussed holding dissemination meetings at the schools alongside sending through summaries of research outputs. Letting stakeholders know about the research activities and findings is an important first step in demonstrating benefits from the partnership. However, PLASN-R will need to be careful to support a process of knowledge exchange between the partnership rather than simply knowledge transfer from the researchers to the schools, that is, simply making the outputs of our research more accessible and practitioner-friendly which, by itself, is unlikely to persuade teachers to implement new practices (Boardman et al., 2005; Stahmer et al., 2005). Using Nastasi et al.’s (2000) ideas on iterative processes in the development and implementation of research, we could fruitfully pursue a strategy in which dissemination activities are both informative and reflexive, serving as catalysts for the generation of ideas about ‘next steps’ from a broader range of stakeholders than those who have hitherto been involved in the PLASN-R planning meetings.

We are still in the early stages of understanding the benefits and challenges of this partnership model and offer it here for scrutiny and critique in the hope that others may consider similar initiatives, which can then be shared in the pursuit of establishing best practice. Anecdotally, parents and teachers have valued the opportunities to be involved in research, either as planners/implementers or as respondents, but the significance of the partnership can only really be judged over the longer term. Ultimately, we hope that if more schools and researchers develop collaborative partnerships, this will lead to greater usefulness and relevance of data collected in schools and an eventual narrowing of the research–practice gap. We would hope that if such partnerships can effectively achieve this, then the kind of model we outline could be an attractive one to research funders in terms of providing good value for money as well as access to larger cohorts of children (small samples being one of the commonly cited major limitations of autism research, for example, Parsons et al., 2009). Findings from such research would likely be of benefit to researchers, practitioners and families through contributing both to knowledge as well as improved practice based on evidence that is collaboratively specified and generated.