Reflections on the design and analysis of clinical trials and meta-analyses in the 1970s and 1980s

Most of my thoughts about clinical trials crystallised during 1971–1976, ¹ when I was writing and repeatedly rewriting what became the 69-page British Journal of Cancer paper, which was eventually published in two parts at the end of 1976 and the beginning of 1977.^2,3 For several years before word processors had been invented, my long-suffering colleague Gale Mead had to deal with a succession of total rewrites, and my heartfelt acknowledgement of her help is made clear in the paper!

I had finished a complete draft of the paper in 1975. I felt that what it had to say about randomised evidence was important, so I wanted it taken seriously, and to be published with the endorsement of the Medical Research Council (MRC). However, some on the MRC leukaemia and cancer trial committees did not want it to be published, as it implied (correctly) that the trials they were then doing wouldn’t give reliable answers. They explicitly and unpleasantly blocked the paper, and later managed to get me removed from the relevant committees because of my continuing concerns about their trial methodology and, in particular, the wholly inadequate trial sizes that inevitably followed.

At that time, the Oxford Clinical Trial Service Unit carried little weight in the corridors of the MRC, except through Richard Doll. Fortunately, Doll was chair of the MRC leukaemia steering committee, and he suggested to me that if I wrote the paper as a report and submitted it to him as an individual who happened to chair an MRC committee, then nobody else could block it. This is the explanation for the paper’s curious title – Design and analysis of randomized clinical trials requiring prolonged observation of each patient.

The other problem was where on earth to try to publish such an enormously long paper. I knew Laszlo Lajtha, then the editor of the British Journal of Cancer, so I telephoned him for advice. He asked me to post it to him for him to read at home over Christmas 1975. He rang me at the end of December to say that he’d read the entire thing and wanted to publish it in the British Journal of Cancer! I was amazed and absolutely delighted, in the way that only young people can be. A publication opportunity as good as that hadn’t even occurred to me as a possibility (or indeed to Richard Doll). Having produced a complete text and received a definite promise of prominent publication, however, I wanted to ensure that I got as many reputable statisticians as possible to endorse it, so I spent most of 1976 collecting the long list of authors – Malcolm Pike, Peter Armitage, Norman Breslow, David Cox, Susannah Howard, Norman Mantel, Klim McPherson, Julian Peto and Peter Smith. In the process, their comments, criticisms and suggestions led to many substantial improvements and clarifications.

When the paper finally came out, it engendered both hostility and support. Most trial statisticians and some clinical investigators felt that it expressed what they’d been wanting to get their clinical colleagues to do; but many clinical investigators at that time were still against randomisation. Anyway, it did get noticed, and it engendered favourable editorials in the New England Journal of Medicine, the BMJ and the Lancet. It was also translated into other languages (English had not yet become as dominant as it now is), and it reinforced the efforts of those who wanted to promote better trial methodology. I explained the basis for my views and made recommendations in a paper published in 1978. ¹

The British Journal of Cancer paper introduced the logrank statistical methods that we still use in the Early Breast Cancer Trialists Collaborative Group, ⁴ although, because things are now all done on computers, we now use the exact variance of O-E, as described on pages 38–39 of the paper, rather the simple approximation Σ(O-E)2/E (which is easier only if one is doing calculations by hand). It is perhaps of particular note that Section 30 (pp. 28–29) of the paper describes why and how to combine logrank test results from different trials in a meta-analysis. ⁵ And we pointed out in the examples we used the undesirability of null results remaining unpublished because they were null.

Soon after these papers had been published, we started maintaining crude, unpublished meta-analyses of several questions (aspirin, beta-blockers, cholesterol-lowering and streptokinase in cardiovascular disease, and radiotherapy and other treatments for breast cancer). We used our unpublished 1977–1978 meta-analyses of aspirin trial results to seek funding for the trial of aspirin in British doctors (which began in 1978), and for factoring aspirin into the ongoing MRC mild-to-moderate hypertension trial. Although the MRC refused to fund the latter (in 1978 or 1979, I think), discussions in the committee meant that the aspirin trial results became widely known because we and Peter Elwood and Archie Cochrane were presenting them at meetings. ⁶ It was a talk I gave at the Society for Clinical Trials which led to the publication of my anonymously published Lancet editorial in May 1980 on the collective results of the aspirin trials. ⁷

I believe the original suggestion to put the streptokinase trial results together similarly was first made in 1978–1979 by Salim Yusuf. He had started writing a paper on them and asked me for statistical help in applying our meta-analysis methods to the streptokinase trials (I had never heard of the drug). When the European streptokinase trial was published (which was too small, but had promising results that fitted the meta-analysis exactly), it was accompanied by an inappropriately negative editorial. In a talk at the London Medical Society, in 1979 I think, I attacked the editorial strongly as an example of how not to interpret trial evidence.

Subsequently, in 1981, Salim Yusuf, Rory Collins (who had just joined the Clinical Trial Service Unit), Peter Sleight (the Nuffield Professor of Medicine in Oxford) and I started planning a pilot study for what would eventually become the ISIS-2 trial. ⁸ It was only with considerable difficulty that we managed to get aspirin factored into that pilot study: people were afraid of interactions between aspirin and streptokinase. Charlie Hennekens was doing a sabbatical with us in Oxford at that time, and when he went back to Boston he got the group there interested, so they also started writing about the streptokinase results. It seemed best to merge the two draft papers and the resulting report was published in the New England Journal of Medicine in 1982. ⁹ A few years later, our meta-analysis of trials of prophylactic use of anti-arrhythmic drugs in heart attack provided no evidence to support this practice and – as later confirmed – that it might be lethal. ¹⁰

The systematic review and meta-analysis of the streptokinase trials was the first of several such analyses of trials of treatments to be published in cardiovascular disease, ¹¹ cancers ¹² and some other fields. ¹³ An important development was the creation of collaborative groups of trialists who contributed individual patient data to enable greater quality control and flexibility in meta-analyses.^4,14 These, together with some very large individual clinical trials,^8,15 made it increasingly clear by the late 1980s that scientifically robust estimates of many plausible treatment effects required substantially larger numbers of patients than had previously been recognised. ¹⁶