Abstract
Randomized trials are crucial for establishing the effectiveness of new drugs and procedures. However, they are less effective at detecting uncommon but clinically significant side effects. We propose a solution. All UK general practices could be randomized to be allowed to prescribe new licenced drugs earlier or later. This would produce a large pragmatic cluster trial which could enable rare, but harmful, effects to be demonstrated more quickly than the current usual practice of looking for harmful events in observational datasets. Given current computerization of practice records such an approach is feasible and likely to be cost-effective.
Randomised controlled trials (RCTs) are the most effective and efficient method of identifying interventions that are beneficial. 1 They also play a major role in protecting patients from harm. Indeed, arguably, this latter role is of greater importance than demonstrating effectiveness. Numerous large RCTs have demonstrated that widely used pharmacological treatments are harmful. For example, large doses of steroids for the treatment of head trauma has been shown to increase mortality (CRASH), while widespread use of anti-arrthymic drugs were shown in RCTs to have led to massive loss of life.2,3 However, most RCTs are too small to identify rare, but important, harmful effects. For example, whilst there have been numerous RCTs of hormone replacement therapy (HRT), virtually all of these were too small to identify increased risks of strokes and other cardiovascular problems. 4 These were only demonstrated after very large studies begun in the 1990s were completed. 5 Notably, these were completed nearly 50 years after widespread use of postmenopausal HRT. Similarly, more recently licenced drugs for rheumatoid arthritis (Coxibs) were found to cause significant harm when given to large numbers of patients after licensing, which later led to the withdrawal of Vioxx. 6 However, before the drug was withdrawn, it had been used on more than 80 million patients world-wide. 7 These dangers were only discovered through a combination of observational data and meta-analysis of smaller trials.
Most new pharmaceutical treatments will have been tested in trials that are too small for clinicians to be confident that they do not cause small, but important, increases in risks to the patient. Such risks are usually only identified some time after the treatment is widely used with the consequent problem that many patients may have been harmed. Once a pharmaceutical product has been licenced, its potential side-effect profile is studied using so called phase IV trials, which are not usually randomized. Phase IV studies are usually large observational studies, which are often difficult to interpret and potential harms can be concealed or exaggerated through selection bias. For example, observational studies have tended to overestimate the harm of postmenopausal estrogens on breast cancer risk 8 and underestimate the harm of estrogens with progestogens on vascular risk. 5 Ideally, therefore, we need to use large scale randomization for phase IV studies to ensure that new drugs are safe. In addition to safety considerations, another problem with many new pharmaceutical treatments is that they are not tested against an existing drug, rather they are compared with a placebo. Consequently, clinicians may know the drug is effective in an untreated patient, but not know whether this new, often more expensive drug, is better than an existing pharmaceutical treatment.
The solution to this problem is to use large, population level, RCTs. The infrastructure and technology now exists to test the safety of new drugs rapidly and at the same time assess whether they are better than existing treatments.
Population RCTs
In the UK, virtually all of the population is enrolled with a primary care general practice. All general practices have sophisticated computer databases with their patients' medical details. When a patient is prescribed a new drug, the treatment is recorded on the electronic database. When a patient has a clinically significant medical event this too is recorded on the database. Some general practitioners (GPs) download anonymized copies of their patients' records to research organizations such as the General Practice Research Database (GPRD). These observational data are often interrogated to assess possible side-effects of treatments. For example, these data have shown that there is an association between deep vein thrombosis (DVTs) and the use of some oral contraceptive pills. 9 They also show an association between the use of oral corticosteroids and fracture risk. 10 There is no reason why they cannot be used more effectively to monitor the performance of newly licenced pharmaceuticals if randomization were introduced.
At present, the allocation of a new drug to patients is often arbitrary – GPs will be influenced in their prescribing behaviour, not only by clinical considerations, but also by other issues including drug preferences, exposure to marketing materials, and guidance from the local primary care trust (PCT), which will vary considerably between practices and individual GPs. The current system has the associated risk of producing a potentially unfair lottery for the receipt of new drugs. Our suggested alternative would be to introduce a fair lottery for the allocation of new drugs on the basis of GP randomization.
How could we introduce randomization into the system? The simplest approach would be to randomize the UK's GP practices into two groups. When a new drug is licenced, it would be withheld on a temporary basis from one of the groups. Depending upon the nature of the treatment, this might be a few months or one or two years. Each group would be allowed some new drugs early while each group would require to act as a control for others. Within a relatively short period of time we would know whether or not a new drug had unexpected harmful effects and furthermore we might also get comparative effectiveness data.
Let us suppose that there is a new drug recently licenced and we would expect within a few months 1% of the population to be offered this treatment. If just 300 practices with an average practice population of 10,000, took part in such a study this would lead to a randomized trial of 30,000 participants. Assuming some modest clustering of outcomes, this would be sufficient for us to detect an increased rate of, say stroke, of 1 per 1000. Using this approach we would be able to detect relatively rare harms relatively rapidly. In addition, rather than having the whole population exposed to the new hazard, this would be reduced to half.
Assessment of effectiveness
As well as observing potential harms our suggestion may also improve knowledge of effectiveness and cost-effectiveness. For instance, there are a range of pharmaceutical treatments that lower blood cholesterol. However, which of these is the most effective? If we delay implementation of a novel statin to half the population then we will potentially have information on the relatively effectiveness of the new treatment against existing care. GP records will record whether or not a patient has had a stroke or a myocardial infarction and such data could be used to compare the effectiveness of the novel treatment. Furthermore, many clinical trials are not carried out in the same setting as the implementation of treatment. For instance, an economic evaluation using data from the GPRD produced different findings than one that used clinical trial data, which may have been due to patient selection. 11 Thus, using GP randomisation may allow us to establish the relative effectiveness of the new treatment and also its cost effectiveness.
In addition, this design potentially allows sufficient power to look at meaningful clinical subgroups. RCTs are usually insufficiently powered to detect the effectiveness of treatments across sub-groups, for example, by gender, age and co-morbidities.
Disadvantages of GP randomization
There are a number of potential disadvantages of GP randomization. First, while it is possible to monitor significant side-effects (e.g. stroke; DVT; cancer), more minor side-effects may not be measured routinely in GP records. However, these minor side-effects (e.g. gastrointestinal problems) would probably be relatively common and be picked up in the original randomized trials of the treatment. Minor side-effects should still be picked up in the form of increased GP contact by the patient and more detailed interrogation of these contacts is likely to reveal the extent and scope of these problems, some of which may not have been identified in the original clinical trial. This is particularly true if the original trial had significant exclusion criteria, for example, of patients with a history of gastrointestinal disturbances, who were then prescribed the drug when it was licenced.
Second, a lot of treatments, which may be effective, could not be routinely measured using GP records. For instance, a novel analgesic for patients with arthritis would not have its pain-relieving effectiveness measured. However, these and other quality of life outcomes require relatively small studies and should be easily measured in the original trials. GP randomization is primarily picking up rare, potentially catastrophic, events that normal trials are too underpowered to observe.
Third, there may be ethical, political or practical issues with implementing the proposal. Patients may feel left out if they have to wait a year or so to access a novel treatment. In terms of the waiting period, some time could be saved through encouraging more rapid licensing. National Institute for Health and Clinical Excellence (NICE) appraisals could be linked to GP randomization and a patient educational programme developed to show that the principal motivation for delay is patient safety rather than cost. In addition, because practices are randomly chosen for each novel pharmaceutical entity, each practice will have early access to different drugs. Politically there may be a problem as it will introduce the perfect lottery: some patients will gain access early merely through chance. However, this needs to be explained and patients persuaded of the benefits. One solution to some problems may be that only a proportion of GP practices are randomized. All of those practices that take part are allowed early access to novel pharmaceuticals while those which do not take part have to wait until the completion of the experiment. Patients who do not wish to have their data used in this way can register with a nearby non-participating practice. In ethical terms, the use of GP randomization is not unethical as all patients retain the right to refuse the novel treatment in the experimental practices and continue with standard therapies. Practices will simply be allocated to allow GPs to offer the new treatments to their patients. Patients registered with control practices will be in the same position as any patient who cannot be offered a novel treatment until it has been through its formal assessment.
Fourth, there is the analytical issue that most patients for whom the new drug is appropriate will not be offered or will not accept the new treatment, which will lead to a dilution of effect if we use intention to treat analysis (ITT). This is true: ITT will tend to underestimate the harm and benefit of any novel treatment. However, there are alternative statistical approaches using randomization as a latent variable, which, in theory, gives an ‘on treatment’ estimate. The most commonly used approach is Complier Average Causal Effect (CACE) analysis, 12 which could easily be used in GP randomization.
Fifth, GP randomization means a cluster randomized trial. These have certain drawbacks compared with individually randomized trials. They need a larger sample size: this, however, should not be a problem as by its very nature GP randomization will involve very large sample sizes. They are sometimes more susceptible to selection bias due to participant selection. 1 But this is avoided in this design because we are including the whole GP population in the trial. Cluster randomization in this instance has an advantage as it is likely to produce more pragmatic effect estimates than individually randomized studies.
Finally, pharmaceutical companies may resist the proposal, due to perceived loss of potentially 50% of their revenue while the GP randomized experiment is underway. However, this is unlikely to be the case as the uptake of a new drug is unlikely to be universal in the first 12 months of availability and being part of a population trial may actually increase the use of a new drug in the first instance as at least half of the participating GP practices will be more aware than otherwise that they are able to prescribe the new drug for a particular condition. Furthermore, if the new drug gets a ‘clean bill of health’, some GPs who are more cautious than their colleagues might be persuaded to prescribe it earlier and more frequently than would otherwise have been the case.
Discussion
Each year there are numerous new pharmaceutical treatments licenced for use in the UK and elsewhere. Most of these will produce benefits to patients with relatively little harm. However, some will produce rare, but serious, side-effects that only become apparent some time after widespread use. In addition, many treatments have not been exposed to a head-to-head comparison of effectiveness. A major reason for both of these problems is lack of sample size. Both safety and comparative effectiveness generally demand very large trials, which are expensive and time-consuming. GP randomization is a solution to this problem whereby we can simultaneously monitor safety and, for drugs that have readily recorded clinical outcomes (e.g. strokes, fractures, DVTs, and mortality), comparative effectiveness data can be generated. The relative cost of such a scheme would be small. Many GPs are already generating anonymized data for research. The only addition required is randomization and appropriate statistical support to analyse the resulting data. The most challenging issue would be to gain political and clinical support. However, the prize of safer medicines and more effective health care should make this approach worthwhile.