Abstract
Much like previous issues of the Journal, the current issue contains a diversified group of articles. As a member of the Editorial team, it is exciting to see the number of high-quality research studies being conducted around the world. Another benefit of serving as an Editor is the opportunity to share some reflections on a topic of interest. Instead of briefly commenting on each of the studies found in the current issue, I decided to share some thoughts on intervention research. I have been reflecting on the topic for quite some time and need not worry about doing justice to all the original research found in the issue.
The past 20 years have seen an explosion of research in autism spectrum disorder (ASD). In a recent international analysis of autism research, the Interagency Autism Coordinating Committee reported that 2500 research articles had been published in 2010 (IACC, 2012). This represents about 10 times more papers than in 1980, and five times more than in 1995. The field has made tremendous strides in terms of characterizing the ASD phenotype. We have a better understanding of the core features defining ASDs as well as the associated features such as behavior and emotional problems.
Ultimately, we diagnose and categorize people to intervene and help them. According to the IACC’s report, from 2000 to 2010 there was a five-fold increase in intervention/treatment research. In 2010 alone, there were more than 400 intervention papers. Collectively, these studies show that we can successfully decrease impairment and improve quality of life in a number of people with ASDs. However, they also show great variability in responses to treatments. Whether for core or associated features, we do not fully grasp which treatments work for whom, and under which circumstances.
Despite the stunning number of publications on ASD (no wonder I cannot keep up), even the most researched interventions only have modest amounts of evidence to support them. If one accepts (and some will not – more on this below) that rigorous intervention studies entail randomization to treatment conditions, standardized treatments (content and dose) and outcome measures, and a large enough sample for meaningful analyses and generalization of results, then one is forced to conclude that there is a limited body of evidence and mixed findings.
Consider the fact that, since the seminal paper published by Lovaas in 1987, there have been less than a handful of randomized controlled trials (RCT) of Early Intensive Behavioral Intervention (EIBI). In fact, there have been more review papers and meta-analyses than controlled trials (I located at least 15 review-type papers). Of course, there are several quasi-experimental studies with strong methods, and there are reports from many different investigators in different locations and practice settings which count for something. Nevertheless, there is still much room for doubt about the accuracy of initial findings. The scarcity of rigorous treatment studies is seen in several other areas as well. For instance, approximately half of children with ASD have significant behavior and emotional problems, yet there are very few non-medical controlled trials targeting them. The same is true for interventions commonly used to improve communication, such as the Picture Exchange Communication System.
Why are there so few non-medical RCTs? One reason is that investigators come from different philosophical schools, different disciplines, and use different approaches. This ultimately has advantages but also results in fractions within the field. Intelligent people disagree on fundamental methodological issues. Behavior analysts went for many decades being the only ones consistently studying psychosocial interventions for individuals with ASDs. Many somewhat dogmatically believed that single-subject studies were the only acceptable way to evaluate treatments. As a result, there are relatively few investigators engaged in RCTs of any Applied Behavior Analysis (ABA) interventions for individuals with ASDs, and there are a lot of intervention models that have never moved beyond single-subject studies. Behavior analysts have somehow became “over-confident” and many have spent more time advocating for services than conducting research.
Beyond the family feuds, researchers are bound by economic and practical realities. Until recently, there was not funding at the level needed to support large RCTs of early intensive intervention. Recruiting large samples for intervention studies is a real challenge. After all, 99% of people do not have an ASD. At the end of the day, recruitment is related to time and money. Single research teams do not have the time or resources to conduct large scale studies, and multicenter studies are complex logistically and very expensive. Ironically, sample size might matter even more with ASDs because of the great genetic and phenotypic heterogeneity. Highly heterogeneous study samples can reduce the potential effect sizes for given treatments. Unfortunately, without large enough samples, we cannot study moderators and mediators of treatment efficacy. In other words, it is difficult to know which treatments work for whom, and under which circumstances. In fact, for most non-medical interventions we have little information on basic parameters such as optimal dose or treatment duration.
Based on the current literature, it is it is difficult to comment on the superiority of treatments/methods since there are very few comparative studies. Ideally, good practices are replaced by better ones based on robust comparative trials in which new interventions outperform older ones and establish new standards of care. Ultimately, the field needs comparative trials to move forward. It is also difficult to comment on effectiveness of treatments. That is, we do not really know if efficacious treatment can be transported from the research setting to the community where there is more variation in subject selection and treatment implementation. Finally, we have little information on how well treatment effects hold up with time since there are so few long-term follow-up studies.
Academic environments reward speed and quantity. Researchers need to publish now, and they need to publish a lot for promotion, tenure, and other benchmarks of success. Treatment studies do not always lend themselves to this culture. In addition, they can be “risky” because high-profile journals do not like to publish negative findings. Funding panels may be subject to this bias as well, which leads to conservatism in what is funded. This publication bias does not allow us to see what should be in the trash can. Without reporting negative findings or successive attenuations over time, we are at risk of repeating past mistakes, and the field becomes dominated by a bubble of irrational exuberance about evidence-based treatments that ultimately cannot be sustained and can block new types of clinical intervention research from being funded or practiced.
I am certainly not arguing that we should not intervene until we have better studies. A distinction must be made between ineffective or harmful treatments and those that are not currently supported by enough robust studies. I am arguing that we need much more intervention research. According to the IACC analysis, fewer than 20% of all papers that were published in 2010 were treatment/intervention studies. The field needs more research on “established” treatments, but also on new treatments. It also needs more research on combined treatments (how many people with ASDs do you know who only receive one intervention?). There are some populations for which we have virtually no intervention data (e.g., adults, non-verbal or lower-functioning individuals).
Despite the limited conclusions and the need to interpret intervention research cautiously, it is exciting to see how much new intervention research is being published. There are more funding opportunities and collaborations across institutions and countries than ever before. In the past five years, we have seen RCTs with large samples evaluating cognitive-behavior therapy for anxiety in high-functioning individuals, parent training for disruptive behavior disorders, comprehensive early intervention packages, social skills programs, and psychotropic medicines. It is a long and windy road ahead. It will also be a scenic and exciting one. I look forward to the next 20 years and hope this Journal will publish many intervention studies to help fill the current void.