Clinical Evaluation,Economic Evaluation,and the Role of the Control Group

The concepts of pragmatic and explanatory randomized controlled trials (RCTs) are long-established and well-understood. ¹ Explanatory trials seek to determine the efficacy of new or unproven interventions under ideal, carefully controlled, experimental conditions. Placebo or sham control groups (hereafter collectively referred to as “placebo” controls) are primarily used to deal with the possibility of observation bias in the determination of treatment effects. Such a biased outcome may arise when knowledge of the trial participants' treatment status may, consciously or not, influence the reporting of relevant information (by a member of the research team, a health professional providing treatment, or the participants' themselves). By incorporating a placebo control group, the objective is to provide an inert intervention that is indistinguishable from the experimental intervention, meaning that it is not possible to determine the active treatment group from the medically ineffectual control. Although this design is able to establish whether or not an intervention provides a therapeutic effect in absolute terms, it provides little or no guidance as to how individual patients should be treated in routine clinical practice, where strict experimental controls are no longer feasible.

In many treatment scenarios, existing therapeutic interventions are available and, therefore, the use of a placebo control group would not be ethical. Irrespective of ethical considerations, some commentators have argued that knowledge of an intervention being more effective than “nothing” is of limited or no benefit. ² When a new experimental intervention is an alternative to an existing treatment, the pragmatic, active-control trial is the appropriate design. With the increasing demand for evidence-based medicine to assist clinical decision making, pragmatic trial designs are necessary to assess if health care interventions work in a relative sense (i.e., compared to one or more control groups in which each intervention is modeled on a standard or accepted mode of care). To reflect the real choice of alternative treatment options in routine clinical practice, pragmatic trials often provide practitioners with greater freedom to deliver individualized care and permit patients to change their treatments. ³ Given that the research objective of a pragmatic trial is to provide evidence of effectiveness in routine care, albeit within the auspices of a research study, the pragmatic model is an important stage in the evaluation of interventions across all areas of applied research.

From a pure research perspective, there are clear scientific benefits in the evaluation of explanatory and pragmatic questions (besides statutory regulations regarding the approval processes for drugs and medical devices), and the control group is a key factor that differentiates between the two RCT designs. In many scenarios, RCTs are designed to explore explanatory and pragmatic questions concurrently rather than consecutively—the “3-arm” trial—which can address many considerations spanning basic science, applied research, and policy implications.

So, how important is the role of the control group for economic evaluation? When determining the cost-effectiveness of a particular therapy in a primary study, a systematic review, or a guideline development process, what are considered to be appropriate comparators? Economic evaluation is performed within a pragmatic analytic framework, comprising an analysis that compares the costs and consequences of two or more competing interventions, and the objective is to provide evidence for decision-making purposes. Irrespective of the number of treatment arms within an economic evaluation, the objective is to identify the single cost-effective option. ⁴ Placebo-control groups are used for research purposes only and are not evaluated with the expectation that the placebo “treatment” could be operable in routine care. Consequently, because placebo-control groups make no claims for health care resources, it seems logical that they have no role within economic evaluation. However, this issue has arisen in recent acupuncture-related health economic publications.

Trial-based evidence from Germany and the United Kingdom has suggested that acupuncture is a cost-effective therapy for knee osteoarthritis (OA), compared to active control groups.^5–10 In two of these studies,^5,6 the 3-arm trial design included a non-penetrating acupuncture therapy (“non-penetrating” is used as a collective term intended to encompass the variety of placebo procedures that are used in research studies). In such a scenario, specific to the economic evaluation, an analyst may consider three options: (1) ignore the placebo group and compare the acupuncture therapy to usual care; (2) analyze as a 3-arm cost-effectiveness study, effectively treating all 3 interventions as candidates to be recommended for implementation into routine care; or (3) address the key policy question (comparing acupuncture treatment with usual care), while also reporting cost and outcome data for the placebo-control group to provide additional context for decision makers. Option 2 should be rejected because this leaves open the possibility of identifying a placebo-control group as the cost-effective treatment option within the RCT; option 2 adds nothing, apart from confusion, compared to option 3. As stated above, the purpose of economic evaluation is to provide decision makers with evidence regarding the allocation of scarce resources. Therefore, determining a placebo-control group to be cost-effective is of no practical value; such a result answers a question that nobody should ask.

Consideration of options 1 and 3 is not a trivial matter, although it would be inappropriate to suggest that either can be dismissed categorically. While both options address the relevant policy question, option 3 provides a broader contextual frame. The implications of selecting option 3, when available, is illustrated in a recent within-trial economic evaluation of a 3-arm RCT that included a non-penetrating acupuncture control group. ⁸ The study authors reported that adding acupuncture to a course of advice and exercise provided by U.K. National Health Service (NHS) physical therapists was a cost-effective use of resources. However, in comparison to the active control group (advice and exercise alone), similar results were also identified for the non-penetrating acupuncture therapy. The authors concluded that the cost-effectiveness of adjunctive acupuncture was not attributable to the penetrating nature of conventional needling. This conclusion, which a decision maker may consider valuable in the determination of policy and/or future research funding, would not have been possible if data from the placebo-control group had been disregarded. Based on this illustrative example, it would seem that data from placebo-control groups do have a role in economic evaluation.

What the above scenario does not comment on is the situation in which only data for acupuncture and placebo-control groups is available, such as a two-arm explanatory RCT or through intentional selection. This situation was brought to light in a recent reanalysis of data used to make treatment recommendations for low-back pain (LBP) and OA by the National Institute for Health and Clinical Excellence (NICE) in the United Kingdom, ¹¹ an independent organization responsible for providing national clinical guidelines to enable evidence-based care for patients using the NHS. Acupuncture received a positive recommendation for LBP but not for OA.^12,13 The two guidelines, conducted independently of each other, adopted different treatment comparators in their economic analyses for acupuncture therapy; the LBP guideline development group adopted usual care, whereas a placebo-control group was the chosen comparator for the OA guideline. Reanalysis of the OA data contained within the guideline has shown that the recommendation for acupuncture may have been different if usual care had been selected as the treatment comparator. ¹¹ Although NICE recommendations are not mandatory, health authorities do have incentives to implement guidelines, meaning that the implications of a treatment therapy not receiving a positive NICE recommendation are significant. The possibility that divergent recommendations for acupuncture therapy have been reached because of the choice of treatment comparator raises concerns and illustrates why clarification and further debate is needed with regard to placebo-control groups and economic evaluation.

The purpose of this Editorial is to stimulate discussion, highlighting how the role of the RCT control group can differ between clinical and economic evaluations. Analysts are not decision makers and the result of an economic evaluation does not, in and of itself, drive policy. However, it is the role of the health economist to provide meaningful data that addresses appropriate research questions. In the best interests of objectivity and transparency in the conduct of economic evaluation, disaggregated cost and health outcome data should be reported for all available groups; experimental interventions, active control groups, and placebo-control groups. The joint estimation of costs and effects—when costs and outcomes are combined into a single outcome measure—is only relevant for the true policy question, (i.e., the evaluation of an operable intervention, compared to accepted current practice). Cost-effectiveness evidence for an intervention compared to a placebo-control group does not provide a decision maker with information to allocate resources efficiently.