Joint incorporation of randomised and observational evidence in estimating treatment effects

Abstract

In evidence-based medicine, randomised trials are regarded as a gold standard in estimating relative treatment effects. Nevertheless, a potential gain in precision is forfeited by ignoring observational evidence. We describe a simple estimator that combines treatment estimates from randomised and observational data and investigate its properties by simulation. We show that a substantial gain in estimation accuracy, compared with the estimator based solely on the randomised trial, is possible when the observational evidence has low bias and standard error. In the contrasting scenario where the observational evidence is inaccurate, the estimator automatically discounts its contribution to the estimated treatment effect. Meta-analysis extensions, combining estimators from multiple observational studies and randomised trials, are also explored.

Keywords

Observational study randomised trial meta-analysis root mean square error parametric bootstrap

Get full access to this article

View all access options for this article.

References

Ioannidis

Haidich

Pappa

et al.

Comparison of evidence of treatment effects in randomized and nonrandomized studies. JAMA 2001; 286: 821–830.

Concato

Shah

Horwitz

. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med 2000; 342: 1887–1892.

Black

. Why we need observational studies to evaluate the effectiveness of health care. BMJ 1996; 312: 1215–1215.

Sacks

Chalmers

Smith

. Randomized versus historical controls for clinical trials. Am J Med 1982; 72: 233–240.

Rosenbaum

Rubin

. The central role of the propensity score in observational studies for causal effects. Biometrika 1983; 70: 41–55.

Greenland

. An introduction to instrumental variables for epidemiologists. Int J Epidemiol 2000; 29: 722–729.

Robins

Hernan

Brumback

. Marginal structural models and causal inference in epidemiology. Epidemiology 2000; 11: 550–560.

Benson

Hartz

. A comparison of observational studies and randomized, controlled trials. N Engl J Med 2000; 342: 1878–1886.

Bosco

Silliman

Thwin

et al.

A most stubborn bias: no adjustment method fully resolves confounding by indication in observational studies. J Clin Epidemiol 2010; 63: 64–74.

10.

Valentine

Thompson

. Issues relating to confounding and meta-analysis when including non-randomized studies in systematic reviews on the effects of interventions. Res Synth Methods 2013; 4: 26–35.

11.

Reeves

Wells

. Special issue on inclusion of non-randomized studies in systematic reviews. Res Synth Methods 2013; 4: 287–289.

12.

Verde

Ohmann

. Combining randomized and non-randomized evidence in clinical research: a review of methods and applications. Res Synth Methods 2015; 6: 45–62.

13.

Sutton

Abrams

. Bayesian methods in meta-analysis and evidence synthesis. Stat Methods Med Res 2001; 10: 277–303.

14.

Turner

Spiegelhalter

Smith

et al.

Bias modelling in evidence synthesis. J R Stat Soc Ser A Stat Soc 2009; 172: 21–47.

15.

Welton

Ades

Carlin

et al.

Models for potentially biased evidence in meta-analysis using empirically based priors. J R Stat Soc Ser A Stat Soc 2009; 172: 119–136.

16.

Dias

Sutton

Welton

et al.

Evidence synthesis for decision making 3: heterogeneity – subgroups, meta regression, bias, and bias adjustment. Med Decis Making 2013; 33: 618–640.

17.

Brockwell

Gordon

. A comparison of statistical methods for meta-analysis. Stat Med 2001; Open source; 20: 825–840.

18.

Canty A and Ripley BD. boot: bootstrap R (S-Plus) functions. R package version 1.3-18, 2016.

19.

Davison

Hinkley

. Bootstrap methods and their application 1997; Vol. 1, Cambridge: Cambridge University Press.

20.

DerSimonian

Laird

. Meta-analysis in clinical trials. Control Clin Trials 1986; 7: 177–188.

21.

Ettehad D, Emdin CA, Kiran A, et al. Blood pressure lowering for prevention of cardiovascular disease and death: A systematic review and meta-analysis. The Lancet 2016; 387: 957–967.

22.

O’Donnell

Xavier

Liu

et al.

Risk factors for ischaemic and intracerebral haemorrhagic stroke in 22 countries (the interstroke study): a case-control study. Lancet 2010; 376: 112–123.

23.

Flossmann

Rothwell

et al.

Effect of aspirin on long-term risk of colorectal cancer: consistent evidence from randomised and observational studies. Lancet 2007; 369: 1603–1613.

24.

Robinson

Jewell

. Some surprising results about covariate adjustment in logistic regression models. Int Stat Rev 1991; 59: 227–240.

25.

Detsky

Naylor

O’Rourke

et al.

Incorporating variations in the quality of individual randomized trials into meta-analysis. J Clin Epidemiol 1992; 45: 255–265.

26.

Greenland

O’rourke

. On the bias produced by quality scores in meta-analysis, and a hierarchical view of proposed solutions. Biostatistics 2001; 2: 463–471.

27.

Higgins

Altman

Gotzsche

et al.

The Cochrane collaboration’s tool for assessing risk of bias in randomised trials. BMJ 2011; 343: d5928–d5928.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.61 MB