Abstract
Study Smarter, Not Harder
How can we actually go about finding better treatments and finding them in less time? We can always make important gains by improving the processes of our current studies. For example, diagnostic and assessment instruments with greater validity and reliability, ensuring highest fidelity of treatments, and meticulous data management, both separately and together, have important roles to play. These processes are important in each and every study we do; and, in each study, we have an obligation to optimize design and methods so that we get maximal information from each subject and from each research dollar. These process improvements are simple in concept but, simultaneously, they are quite demanding to carry out. While they are important, we will not, further consider them at this time but, rather, focus on two other areas for some potential ways to improve our research knowledge of clinical treatments.
First, there are some special opportunities with respect to how we take somewhat different approaches to asking and answering questions that may prove to be very useful. Secondly, we will look at an interesting meta-experiment that is now being undertaken by the NIMH: the RDoC (
Randomization—Do We Really Need It?
Why do standard research methods require a double-blind, randomized controlled trial? Can we get the same data in a more cost effective manner by just looking at naturalistic data without the randomization (and blindness)? In general, the short answer is “No.” We need randomization in order to make robust causal inferences, at least in most cases.
Statistical approaches to making causal inferences from correlational data are necessary in areas in which random assignment is unethical, illegal, or impossible. Such alternative methods are frequently referred to as “econometrics,” due to their widespread use in answering questions in the field of economics. While these approaches (e.g., structural equation modeling) are also used in psychiatry, with this approach there are many threats to validity that are eliminated by using randomization (Green and Dunn 2008). For example, if there is a variable that both influences the treatment choice made by an individual and, on its own, the same variable can change treatment response probabilities, independent of the treatment chosen, this will falsely be modeled as a treatment effect on outcome. Worse yet, these econometric methods of data analyses do not provide evidence for or against falsely modeled treatment effect; as a result, an error can easily go undetected. A well-known example of this problem is seen in the recent questions about whether healthy adults with normal nutrition will have better health outcomes if they take multivitamins. A simple model will find that people who take multivitamins are healthier; this can then lead to the conclusion that there is a causal relationship between vitamin treatment and outcome. However, models that examined the presence of other healthy behaviors found that the people who take multivitamins also exercise more, eat more prudently, etc. Further, when these other healthy behaviors are included in the model, they turn out to explain most, or all, of what looked like the health promoting effects of multivitamins. For these reasons, non-randomized experiments are not likely to provide robust evidence of treatment efficacy, at least evidence in which we can have reasonable confidence.
However, one special case that avoids many of these problems is when an “instrumental variable” can be identified (Angrist et al. 1996). An instrumental variable is one that does not directly influence the outcome variable (e.g., better health outcomes) but is correlated with the explanatory variables (e.g., in our prior example: taking vitamins or not). In this situation, one has much of the benefit of randomization without having to assume the burden of randomization in a study. One example of an instrumental variable can be found in a study of the high school performance of children who started school at a younger versus older age; this can be done by examining those children whose month of birth was one month before versus the month after the cutoff for kindergarten entry (e.g., August versus September). For reasons unrelated to the child or the child's family, except a trivial difference in date of birth, one group is 11 months older at high school graduation than is the other. In this case, one cannot use any month of birth throughout the entire year because there are differences in the risk for disorders associated with season of birth (e.g., schizophrenia.) A similar opportunity might exist if one were to study child and adolescent psychiatry treatment effects in which, for example, we look for unique samples where a major third-party payer made a narrow change in not covering versus covering a treatment of strong interest in child psychiatry (e.g., a substantial change in covering atypical antipsychotics without other changes in psychiatric coverage). While this isn't a general method for getting around the problems associated with lack of randomization in most naturalistic datasets, when one can find an instrumental variable in an informative dataset, then causal inferences can be made much more robustly.
Can We Ask Related but More Tractable Questions?
Studying a new pharmacological treatment to get data to support governmental approval generally requires a two-cell (or multi-cell) randomized, controlled trial. Such a study will generally be a comparison with placebo. However a non-inferiority trial, comparing the new agent to currently approved alternatives, may at times be preferred. The strategy we are going to discuss next doesn't help us with the questions we are asking in an approval study, which is a special class of clinical trial unto itself.
Consider, instead, the question of how to pick the best treatment from a field of treatments. For example, there are many potential significantly different psychotherapeutic approaches to specific child and adolescent psychiatric disorders. A good example is optimizing psychotherapeutic treatments for a given disorder. Given the many treatment parameters one might vary, all with face validity and solid theoretical underpinnings, how can we find the best combination? Here is where a slight change to the question makes all the difference. “Best” is prohibitively expensive in terms of subjects and money. “Among the best” we can much more feasibly achieve by a two-stage process consisting first of a screening phase that looks at the various combinations with a small number of subjects in each group and picks those who scored very high in the first phase for further head-to-head comparisons (Collins et al. 2007). This multiphase optimization strategy (MOST) first screens to select a subsample that are promising and then further tests only that group. MOST critically depends on the fact that while our very best treatment may be unlucky and by happenstance not pass through the screen, we will almost certainly get several close-to-the-best treatments through the screen. This approach has been strongly suggested for use in optimizing psychotherapy selection but it may be useful for addressing questions such as which among the atypical antipsychotics provides a good starting point for treatment in a given population.
Finding a very good first treatment is helpful but much of our day-to-day work consists of figuring out how to treat youth in which the first treatment is ineffective or inadequately effective. There is now a statistical literature addressing the design and performance of multi-stage adaptive treatment strategies, (Lavori and Dawson 2008; Dawson and Lavori 2012); these have been used in several successful clinical trials in psychiatry (Rush et al. 2004; Dawson et al. 2007). In many ways, knowledge about treatment algorithms that work well is much more important to the health of youth with psychiatric disorders than more information about the best first treatment approach for a disorder. In a recent paper in this journal, Almirall and colleagues (2012) discuss how similar approaches may help us develop better treatment continuation approaches, following successful acute-phase treatment. Finally, another methodological approach to tailoring psychotherapeutic treatment to the individual is to create multiple psychotherapy modules targeting different symptom clusters and then select those modules that best target the individual patient (Weisz et al. 2012). This is currently an area of considerable study.
The NIMH Meta-Experiment: RDoC
Research in our field is critically determined by funding. The pharmaceutical industry has provided significant funding to test the acute efficacy of on-patent compounds for psychiatric disorders in youth. To some extent, foundations and others have supported some additional treatment studies. However, a large fraction of child and adolescent psychiatry treatment studies have been and, in the future, will be funded by the NIH. Both FDA policies and NIH review committee decisions have ensured that most treatment studies to date have concentrated on diagnostic groups, a few diagnoses, or a subgroup of a diagnosis. Studies have rarely been allowed to focus on symptoms, per se; however, with the NIMH's new RDoC initiative that may change dramatically.
The RDoC project attempts to use available research data in genetics, neuroscience, and behavioral science to find new ways of classifying mental disorders “based on dimensions of observable behavior and neurobiological measures.” It is a dimensional system, agnostic about current disorder categories, with dimensions derived from basic behavioral neuroscience. The RDoC constructs focus on known brain circuitry that can be linked to behavior (Morris and Cuthbert 2012). This approach aims to create a new research framework for psychiatric disorders that, it is hoped, will help identify new targets for psychiatric treatment and informing clinical decision making (Insel et al. 2010). The RDoC approach proposes the importance of including genetic, molecular, and neural substrate parameters in future diagnostic systems, which will eventually lead to neuroscience-based psychiatric classification. As yet, it is unclear what fraction of the NIMH portfolio of treatment studies will move entirely away from traditional diagnostic categories as an inclusion criteria and select subjects scoring high on one, or several, of the RDoC dimensions; it is likely that RDoCs will be a significant focus of NIMH funding.
The rationale for RDoCs is clear and compelling. If the RDoCs neuroscience-based dimensional approach will get us closer to the essence of psychopathology, then we will be able to test treatments in youth who share common, underlying neural system changes. This is unlike research using our current diagnostic classifications in which much of our challenge in showing response to treatment may arise from functional heterogeneity of brain system changes underlying the current common diagnoses. While RDoCs are an experiment that may bring greater power and understanding to our field, like any experiment it may fail. Only time will tell.
Summary
Patience is never easy when we have patients who need treatment and the currently available therapies are not optimal. As a result, progress in clinical studies designed to improve treatment of child and adolescent psychiatric disorders often seems unnecessarily, and painfully, slow to both clinicians and clinical researchers. We are often left wondering whether these time lags are necessary or are the result of unnecessary bureaucracy or antiquated research methods. Many among us may eschew waiting for data and simply move forward with the “off-label” use of both pharmacological/psychotherapeutic and environment/psychosocial interventions. Are there alternatives to consider? There certainly are. While the future of treatment studies in child psychiatry will look very much like the past, many future studies will take advantage of newer statistical approaches and our more informed hypotheses about the underlying neural systems changes associated with psychiatric symptoms. And, once again, hopefully, we will see that even small additions or changes in our methods and data analyses will allow us to fill our therapeutic cup further and faster.