Adjusting for bias in unblinded randomized controlled trials

Abstract

It may not always be possible to blind participants of a randomized controlled trial for treatment allocation. As a result, estimators of the actual treatment effect may be biased. In this paper, we will extend a novel method, originally introduced in genetic research, for instrumental variable meta-analysis, adjusting for bias due to unblinding of trial participants. Using simulation studies, this novel method, “Egger Correction for non-Adherence”, is introduced and compared to the performance of the “intention-to-treat,” “as-treated,” and conventional “instrumental variable” estimators. Scenarios considered (time-varying) non-adherence, confounding, and between-study heterogeneity. The effect of treatment on a binary endpoint was quantified by means of a risk difference. In all scenarios with unblinded treatment allocation, the Egger Correction for non-Adherence method was the least biased estimator. However, unless the variation in adherence was relatively large, precision was lacking, and power did not surpass 0.50. As a comparison, in a meta-analysis of blinded randomized controlled trials, power of the conventional IV estimator was 1.00 versus at most 0.14 for the Egger Correction for non-Adherence estimator. Due to this lack of precision and power, we suggest to use this method mainly as a sensitivity analysis.

Keywords

Statistics randomized controlled trials Monte Carlo method bias treatment effectiveness instrumental variable

Get full access to this article

View all access options for this article.

References

Grobbee

Hoes

. Intervention research: unintended effects 2015; Vol. 2, Burlington: Jones and Bartlett Learning, pp. 181–214.

Higgins

JPT

Altman

Sterne

JAC

Assessing risk of bias in included studies. In: Higgins

JPT

Green

(eds). Cochrane handbook for systematic reviews of interventions 2011; The Cochrane Collaboration, pp. 187–242.

Bannister-Tyrrell

Miladinovic

Roberts

et al.

Adjustment for compliance behavior in trials of epidural analgesia in labor using instrumental variable meta-analysis. J Clin Epidemiol 2015; 68: 525–533.

Hingorani

Humphries

. Nature’s randomised trials. Lancet 2005; 366: 1906–1908.

Bowden

Smith

Burgess

. Mendelian randomization with invalid instruments: effect estimation and bias detection through Egger regression. Int J Epidemiol 2015; 44: 512–525.

Egger

Davey

Schneider

et al.

Bias in meta-analysis detected by a simple, graphical test. BMJ 1997; 315: 629–634.

Schmidt

Groenwold

RHH

Knol

et al.

Exploring interaction effects in small samples increases rates of false-positive and false-negative findings: results from a systematic review and simulation study. J Clin Epidemiol 2014; 67: 821–829.

Schmidt

Klungel

Nielen

et al.

Tailoring treatments using treatment effect modification. Pharmacoepidemiol Drug Safe 2016; 25: 355–62.

Hernán

Robins

. Instruments for causal inference: an epidemiologist’s dream? Epidemiology 2006; 17: 360–372.

10.

Palmer

Sterne

Harbord

et al.

Instrumental variable estimation of causal risk ratios and causal odds ratios in Mendelian randomization analyses. Am J Epidemiol 2011; 173: 1392–1403.

11.

Didelez

Meng

Sheehan

. Assumptions of IV methods for observational epidemiology. Stat Sci 2010; 25: 22–40.

12.

Greenland

. An introduction to instrumental variables for epidemiologists. Int J Epidemiol 2000; 29: 722–729.

13.

Schmidt

Hingorani

Jefferis

et al.

Comparison of variance estimators for meta-analysis of instrumental variable estimates. Int J Epidemiol 2016, doi: 10.1093/ije/dyw123.

14.

Thompson

Higgins

JPT

. How should meta-regression analyses be undertaken and interpreted? Stat Med 2002; 21: 1559–1573.

15.

Rücker

Schwarzer

Carpenter

et al.

Treatment-effect estimates adjusted for small-study effects via a limit meta-analysis. Biostatistics 2011; 12: 122–142.

16.

Carpenter

Bithell

. Bootstrap confidence intervals: when, which, what? A practical guide for medical statisticians. Stat Med 2000; 19: 1141–1164.

17.

Thompson

Sharp

. Explaining heterogeneity in meta-analysis: a comparison of methods. Stat Med 1999; 18: 2693–2708.

18.

RCore Team. R. A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing, 2013, www.r-project.org/ (accessed 8 November 2016).

19.

Burton

Altman

Royston

et al.

The design of simulation studies in medical statistics. Stat Med 2006; 25: 4279–4292.

20.

Schmidt

Klungel

Groenwold

. Adjusting for confounding in early postlaunch settings: going beyond logistic regression models. Epidemiology 2016; 27: 133–42.

21.

Bowden

Del Greco

Minelli

et al.

Assessing the suitability of summary data for two-sample Mendelian randomization analyses using MR-Egger regression: the role of the I2 statistic. Int J Epidemiol 2016, (in press).

22.

Bowden

Davey-Smith

Haycock

et al.

Consistent estimation in Mendelian randomization withsome invalid instruments using a weighted medianestimator. Genet Epidemiol 2016; 40: (in press).

23.

Kang H, Zhang A, Cai TT, et al. Instrumental variables estimation with some invalid instruments and its application to Mendelian randomization. J Am Stat Assoc, https://dx-doi-org.web.bisu.edu.cn/10.1080/01621459.2014.994705 (accessed 8 November 2016).

24.

Kolesár

Chetty

Friedman

et al.

Identification and inference with many invalid instruments. J Bus Econ Stat 2015; 33: 474–484.

25.

Hernán

Hernández-Díaz

. Beyond the intention-to-treat in comparative effectiveness research. Clin Trials 2012; 9: 48–55.

26.

Newcombe RG and Bender R. Implementing GRADE: calculating the risk difference from the baseline risk and the relative risk. Evid Based Med 2014; 19: 6–8.

27.

Burgess

. Identifying the odds ratio estimated by a two-stage instrumental variable analysis with a logistic regression model. Stat Med 2013; 32: 4726–4747.

28.

Bates D, Maechler M, Bolker B, et al. lme4: linear mixed-effects models using {Eigen} and {S4} [{R}, https://github.com/lme4/lme4 (accessed 8 November 2016).

29.

Greenland

. Invited commentary: a critical look at some popular meta-analytic methods. Am J Epidemiol 1994; 140: 290–296.