Properties and pitfalls of weighting as an alternative to multilevel multiple imputation in cluster randomized trials with missing binary outcomes under covariate-dependent missingness
Restricted accessResearch articleFirst published online May, 2020
Properties and pitfalls of weighting as an alternative to multilevel multiple imputation in cluster randomized trials with missing binary outcomes under covariate-dependent missingness
The generalized estimating equation (GEE) approach can be used to analyze cluster randomized trial data to obtain population-averaged intervention effects. However, most cluster randomized trials have some missing outcome data and a GEE analysis of available data may be biased when outcome data are not missing completely at random. Although multilevel multiple imputation for GEE (MMI-GEE) has been widely used, alternative approaches such as weighted GEE are less common in practice. Using both simulations and a real data example, we evaluate the performance of inverse probability weighted GEE vs. MMI-GEE for binary outcomes. Simulated data are generated assuming a covariate-dependent missing data pattern across a range of missingness clustering (from none to high), where all covariates are measured at baseline and are fully observed (i.e. a type of missing-at-random mechanism). Two types of weights are estimated and used in the weighted GEE: (1) assuming no clustering of missingness (W-GEE) and (2) accounting for such clustering (CW-GEE). Results show that, even in settings with high missingness clustering, CW-GEE can lead to more bias and lower coverage than W-GEE, whereas W-GEE and MMI-GEE provide comparable results. W-GEE should be considered a viable strategy to account for missing outcomes in cluster randomized trials.
BrookerSOkelloGNjagiK, et al.Improving educational achievement and anaemia of school children: design of a cluster randomised trial of school-based malaria prevention and enhanced literacy instruction in Kenya. Trials2010; 11: 93.
4.
HudgensMGHalloranME. Toward causal inference with interference. J Am Stat Assoc2008; 103: 832–842.
5.
CampbellMKPiaggioGElbourneDR, et al.Consort 2010 statement: extension to cluster randomised trials. BMJ2012; 345: e5661.
6.
FieroMHHuangSOrenE, et al.Statistical analysis and handling of missing data in cluster randomized trials: a systematic review. Trials2016; 17: 72.
7.
Turner EL, Prague MP, Gallis JA, et al. Review of recent methodological developments in group-randomized trials: part 2-analysis. Am J Public Health 2017; 107: 1078–1086.
8.
PreisserJSYoungMLZaccaroDJ, et al.An integrated population-averaged approach to the design, analysis and sample size determination of cluster-unit trials. Stat Med2003; 22: 1235–1254.
9.
TurnerELLiFGallisJA, et al.Review of recent methodological developments in group-randomized trials: part 1-design. Am J Public Health2017; 107: 907–915.
10.
Diaz-OrdazKKenwardMGCohenA, et al.Are missing data adequately handled in cluster randomised trials? A systematic review and guidelines. Clin Trials2014; 11: 590–600.
RobinsJMRotnitzkyAZhaoLP. Analysis of semiparametric regression-models for repeated outcomes in the presence of missing data. J Am Stat Assoc1995; 90: 106–121.
13.
PaikMC. The generalized estimating equation approach when data are not missing completely at random. J Am Stat Assoc1997; 92: 1320–1329.
14.
SchaferJLYucelRM. Computational strategies for multivariate linear mixed-effects models with missing values. J Comput Graph Stat2002; 11: 437–457.
15.
SeamanSRWhiteIRCopasAJ, et al.Combining multiple imputation and inverse-probability weighting. Biometrics2012; 68: 129–137.
16.
HunsbergerSMurrayDDavisCE, et al.Imputation strategies for missing data in a school-based multi-centre study: the Pathways study. Stat Med2001; 20: 305–316.
17.
TaljaardMDonnerAKlarN. Imputation strategies for missing continuous outcomes in cluster randomized trials. Biometrical J2008; 50: 329–345.
18.
AndridgeRR. Quantifying the impact of fixed effects modeling of clusters in multiple imputation for cluster randomized trials. Biometrical J2011; 53: 57–74.
19.
HossainADiaz-OrdazKBartlettJW. Missing continuous outcomes under covariate dependent missingness in cluster randomised trials. Stat Methods Med Res2016; 26: 1543–1562.
20.
CailleALeyratCGiraudeauB. A comparison of imputation strategies in cluster randomized trials with missing binary outcomes. Stat Methods Med Res2016; 25: 2650–2669.
21.
MaJAkhtar-DaneshNDolovichL, et al.Imputation strategies for missing binary outcomes in cluster randomized trials. BMC Med Res Methodol2011; 11: 18.
22.
MaJRainaPBeyeneJ, et al.Comparison of population-averaged and cluster-specific models for the analysis of cluster randomized trials with missing binary outcomes: a simulation study. BMC Med Res Methodol2013; 13: 9.
23.
MaJRainaPBeyeneJ, et al.Comparing the performance of different multiple imputation strategies for missing binary outcomes in cluster randomized trials: a simulation study. J Open Access Med Stat2012; 2: 93–103.
24.
HossainADiazOrdazKBartlettJW. Missing binary outcomes under covariate-dependent missingness in cluster randomised trials. Stat Med2017; 36: 3092–3109.
25.
Molenberghs G and Kenward M. Missing data in clinical studies. Chichester: John Wiley & Sons, 2007.
26.
Van Buuren S. Flexible imputation of missing data. Boca Raton: Chapman and Hall/CRC, 2018.
27.
VansteelandtSCarpenterJKenwardMG. Analysis of incomplete data using inverse probability weighting and doubly robust estimators. Methodology2010; 6: 37–48.
28.
SalazarAOjedaBDuenasM, et al.Simple generalized estimating equations (GEEs) and weighted generalized estimating equations (WGEEs) in longitudinal studies with dropouts: guidelines and implementation in R. Stat Med2016; 35: 3424–3448.
29.
PreisserJSLohmanKKRathouzPJ. Performance of weighted estimating equations for longitudinal binary data with drop-outs missing at random. Stat Med2002; 21: 3035–3054.
30.
InanGYucelR. Joint GEEs for multivariate correlated data with incomplete binary outcomes. J Appl Stat2017; 44: 1920–1937.
31.
SkinnerCJD'ArrigoJ. Inverse probability weighting for clustered nonresponse. Biometrika2011; 98: 953–966.
32.
JukesMCHTurnerELDubeckMM, et al.Improving literacy instruction in Kenya through teacher professional development and text messages support: a cluster randomized trial. J Res Educ Eff2016; 10: 449–481.
33.
HanleyJANegassaAEdwardesMD, et al.Statistical analysis of correlated data using generalized estimating equations: an orientation. Am J Epidemiol2003; 157: 364–375.
34.
ZegerSLLiangKY. Longitudinal data analysis for discrete and continuous outcomes. Biometrics1986; 42: 121–130.
35.
NeuhausJMKalbfleischJDHauckWW. A comparison of cluster-specific and population-averaged approaches for analyzing correlated binary data. Int Stat Rev1991; 59: 25–35.
36.
LiangKYZegerSL. Longitudinal data-analysis using generalized linear-models. Biometrika1986; 73: 13–22.
37.
MengX-L. Multiple-imputation inferences with uncongenial sources of input. Stat Sci1994; 9: 538–558.
38.
RubinDB. Multiple imputation for nonresponse in surveys, Hoboken: Wiley-Interscience, 2004.
39.
MolenberghsGFitzmauriceGKenwardMG, et al.Handbook of missing data methodology, Hoboken: Chapman and Hall/CRC, 2014.
40.
BelitserSVMartensEPPestmanWR, et al.Measuring balance and model selection in propensity score methods. Pharmacoepidemiol Drug Saf2011; 20: 1115–1129.
41.
PragueMWangRStephensA, et al.Accounting for interactions and complex inter-subject dependency in estimating treatment effect in cluster-randomized trials with missing outcomes. Biometrics2016; 72: 1066–1077.
42.
WangCLPaikMC. A weighting approach for GEE analysis with missing data. Commun Stat Theory Methods2011; 40: 2397–23411.
43.
SeamanSRWhiteIR. Review of inverse probability weighting for dealing with missing data. Stat Methods Med Res2013; 22: 278–295.
44.
McDanielLSHendersonNCRathouzPJ. Fast pure R implementation of GEE: application of the Matrix package. R J2013; 5: 181–187.
45.
BarnardJRubinDB. Miscellanea. Small-sample degrees of freedom with multiple imputation. Biometrika1999; 86: 948–955.
46.
Quartagno M and Carpenter J. jomo: a package for multilevel joint modelling multiple imputation. R package version 2.6-7, http://CRAN.R-project.org/package=jomo, 2017.
47.
EldridgeSMUkoumunneOCCarlinJB. The intra-cluster correlation coefficient in cluster randomized trials: a review of definitions. Int Stat Rev2009; 77: 378–394.
48.
ManclLADeRouenTA. A covariance estimator for GEE with improved small-sample properties. Biometrics2001; 57: 126–134.
49.
KauermannGCarrollRJ. A note on the efficiency of sandwich covariance matrix estimation. J Am Stat Assoc2001; 96: 1387–1396.
50.
FayMPGraubardBI. Small-sample adjustments for Wald-type tests using sandwich estimators. Biometrics2001; 57: 1198–1206.
51.
LiFForbesABTurnerEL, et al.Power and sample size requirements for GEE analyses of cluster randomized crossover trials. Stat Med2018; 38: 636–649.
52.
LiFTurnerELHeagertyPJ, et al.An evaluation of constrained randomization for the design and analysis of group-randomized trials with binary outcomes. Stat Med2017; 36: 3791–3806.
53.
LiFTurnerELPreisserJS. Sample size determination for GEE analyses of stepped wedge cluster randomized trials. Biometrics2018; 74: 1450–1458.
54.
SantosCASFiacconeRLOliveiraNF, et al.Estimating adjusted prevalence ratio in clustered cross-sectional epidemiological data. BMC Med Res Methodol2008; 8: 80.
55.
WilliamsonEJForbesAWhiteIR. Variance reduction in randomised trials by inverse probability weighting using the propensity score. Stat Med2014; 33: 721–737.
56.
CarpenterJRKenwardMGVansteelandtS. A comparison of multiple imputation and doubly robust estimation for analyses with missing data. J Roy Stat Soc Sta2006; 169: 571–584.
57.
PragueMWangRDe GruttolaV. CRTgeeDR: an R package for doubly robust generalized estimating equations estimations in cluster randomized trials with missing data, Cambridge: Harvard University Working Paper, 2016.
58.
StephensAJTchetgenEJTDe GruttolaV. Augmented generalized estimating equations for improving efficiency and validity of estimation in cluster randomized trials by leveraging cluster-level and individual-level covariates. Stat Med2012; 31: 915–930.
59.
BirhanuTMolenberghsGSottoC, et al.Doubly robust and multiple-imputation-based generalized estimating equations. J Biopharm Stat2011; 21: 202–225.
60.
LiFThomasLELiF. Addressing extreme propensity scores via the overlap weights. Am J Epidemiol2018; 188: 250–27.
61.
SunBTchetgen TchetgenEJ. On inverse probability weighting for nonmonotone missing at random data. J Am Stat Assoc2018; 113: 369–379.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
