A comparison of computational algorithms for the Bayesian analysis of clinical trials

Abstract

Background:

Clinical trials are increasingly using Bayesian methods for their design and analysis. Inference in Bayesian trials typically uses simulation-based approaches such as Markov Chain Monte Carlo methods. Markov Chain Monte Carlo has high computational cost and can be complex to implement. The Integrated Nested Laplace Approximations algorithm provides approximate Bayesian inference without the need for computationally complex simulations, making it more efficient than Markov Chain Monte Carlo. The practical properties of Integrated Nested Laplace Approximations compared to Markov Chain Monte Carlo have not been considered for clinical trials. Using data from a published clinical trial, we aim to investigate whether Integrated Nested Laplace Approximations is a feasible and accurate alternative to Markov Chain Monte Carlo and provide practical guidance for trialists interested in Bayesian trial design.

Methods:

Data from an international Bayesian multi-platform adaptive trial that compared therapeutic-dose anticoagulation with heparin to usual care in non-critically ill patients hospitalized for COVID-19 were used to fit Bayesian hierarchical generalized mixed models. Integrated Nested Laplace Approximations was compared to two Markov Chain Monte Carlo algorithms, implemented in the software JAGS and stan, using packages available in the statistical software R. Seven outcomes were analysed: organ-support free days (an ordinal outcome), five binary outcomes related to survival and length of hospital stay, and a time-to-event outcome. The posterior distributions for the treatment and sex effects and the variances for the hierarchical effects of age, site and time period were obtained. We summarized these posteriors by calculating the mean, standard deviations and the 95% equitailed credible intervals and presenting the results graphically. The computation time for each algorithm was recorded.

Results:

The average overlap of the 95% credible interval for the treatment and sex effects estimated using Integrated Nested Laplace Approximations was 96% and 97.6% compared with stan, respectively. The graphical posterior densities for these effects overlapped for all three algorithms. The posterior mean for the variance of the hierarchical effects of age, site and time estimated using Integrated Nested Laplace Approximations are within the 95% credible interval estimated using Markov Chain Monte Carlo but the average overlap of the credible interval is lower, 77%, 85.6% and 91.3%, respectively, for Integrated Nested Laplace Approximations compared to stan. Integrated Nested Laplace Approximations and stan were easily implemented in clear, well-established packages in R, while JAGS required the direct specification of the model. Integrated Nested Laplace Approximations was between 85 and 269 times faster than stan and 26 and 1852 times faster than JAGS.

Conclusion:

Integrated Nested Laplace Approximations could reduce the computational complexity of Bayesian analysis in clinical trials as it is easy to implement in R, substantially faster than Markov Chain Monte Carlo methods implemented in JAGS and stan, and provides near identical approximations to the posterior distributions for the treatment effect. Integrated Nested Laplace Approximations was less accurate when estimating the posterior distribution for the variance of hierarchical effects, particularly for the proportional odds model, and future work should determine if the Integrated Nested Laplace Approximations algorithm can be adjusted to improve this estimation.

Keywords

Bayesian clinical trial analysis proportional odds model survival analysis logistic regression Integrated Nested Laplace Approximations Markov chain Monte Carlo JAGS stan

Get full access to this article

View all access options for this article.

References

Van de Schoot

Kaplan

Denissen

, et al. A gentle introduction to Bayesian analysis: applications to developmental research. Child Dev 2014; 85(3): 842–860.

Berry

. Bayesian clinical trials. Nat Rev Drug Discov 2006; 5(1): 27–36.

Crider

Devine

Hao

, et al. Population red blood cell folate concentrations for prevention of neural tube defects: Bayesian model. BMJ 2014; 349: g4554.

Muss

Berry

Cirrincione

, et al. Adjuvant chemotherapy in older women with early-stage breast cancer. N Engl J Med 2009; 360(20): 2055–2065.

Brophy

. Bayesian interpretation of the EXCEL trial and other randomized clinical trials of left main coronary artery revascularization. JAMA Intern Med 2020; 180(7): 986–992.

Montazerhodjat

Chaudhuri

Sargent

, et al. Use of Bayesian decision analysis to minimize harm in patient-centered randomized clinical trials in oncology. JAMA Oncol 2017; 3(9): e170123.

Berry

Carlin

Lee

, et al. Bayesian adaptive methods for clinical trials. Boca Raton, FL: CRC Press, 2010.

Woodcock

LaVange

. Master protocols to study multiple therapies, multiple diseases, or both. N Engl J Med 2017; 377(1): 62–70.

Park

Detry

Murthy

, et al. How to use and interpret the results of a platform trial: users’ guide to the medical literature. JAMA 2022; 327(1): 67–74.

10.

Barry

Perou

Marcom

, et al. The use of Bayesian hierarchical models for adaptive randomization in biomarker-driven phase II studies. J Biopharm Stat 2015; 25(1): 66–88.

11.

Vasishth

Yadav

Schad

, et al. Sample size determination for Bayesian hierarchical models commonly used in psycholinguistics. Comput Brain Behav 2022: 1–25.

12.

Liu

Wick

Martin

, et al. Two-stage Bayesian hierarchical modeling for blinded and unblinded safety monitoring in randomized clinical trials. BMC Med Res Methodol 2020; 20(1): 1–17.

13.

Broglio

Meurer

Durkalski

, et al. Comparison of Bayesian vs frequentist adaptive trial design in the Stroke Hyperglycemia Insulin Network Effort trial. JAMA Netw Open 2022; 5(5): e2211616.

14.

Van Rosmalen

Dejardin

Van Norden

, et al. Including historical data in the analysis of clinical trials: is it worth the effort. Stat Methods Med Res 2018; 27(10): 3167–3182.

15.

Van de Schoot

Depaoli

King

, et al. Bayesian statistics and modelling. Nat Rev Methods Primers 2021; 1(1): 1–26.

16.

Jack Lee

Chu

. Bayesian clinical trials in action. Stat Med 2012; 31(25): 2955–2972.

17.

Kadane

. Bayesian methods and ethics in a clinical trial design. Hoboken, NJ: John Wiley & Sons, 2011.

18.

Goligher

Tomlinson

Hajage

, et al. Extracorporeal membrane oxygenation for severe acute respiratory distress syndrome and posterior probability of mortality benefit in a post hoc Bayesian analysis of a randomized clinical trial. JAMA 2018; 320(21): 2251–2259.

19.

Reefhuis

Devine

Friedman

, et al. Specific SSRIs and birth defects: Bayesian analysis to interpret new data in the context of previous reports. BMJ 2015; 351: h3190.

20.

Albuquerque

Tramujas

Sewanan

, et al. Mortality rates among hospitalized patients with COVID-19 infection treated with tocilizumab and corticosteroids: a Bayesian reanalysis of a previous meta-analysis. JAMA Netw Open 2022; 5(2): e220548.

21.

Gómez-Rubio

. Bayesian inference with INLA. Boca Raton, FL: CRC Press, 2020.

22.

Gelfand

Smith

. Sampling-based approaches to calculating marginal densities. J Am Stat Assoc 1990; 85(410): 398–409.

23.

Belloni

Chernozhukov

. On the computational complexity of MCMC-based estimators in large samples. Ann Stat 2009; 37(4): 2011–2055.

24.

Lunn

Jackson

Best

, et al. The BUGS book: a practical introduction to Bayesian analysis. London: Chapman Hall, 2013.

25.

Plummer

. JAGS: a program for analysis of Bayesian graphical models using Gibbs sampling. In: Proceedings of the 3rd international workshop on distributed statistical computing, Vienna, 20–22 March 2003, pp. 1–10.

26.

Carpenter

Gelman

Hoffman

, et al. Stan: a probabilistic programming language. J Stat Softw 2017; 76: 1.

27.

Rue

Martino

Chopin

. Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations. J R Stat Soc Ser B Methodol 2009; 71(2): 319–392.

28.

Held

Schrödle

Rue

. Posterior and cross-validatory predictive checks: a comparison of MCMC and INLA. In: Kneib

Tutz

(eds) Statistical modelling and regression structures. New York: Springer, 2010, pp. 91–110.

29.

Fong

Rue

Wakefield

. Bayesian inference for generalized linear mixed models. Biostatistics 2010; 11(3): 397–412.

30.

Taylor

Diggle

. INLA or MCMC? A tutorial and comparative evaluation for spatial prediction in log-Gaussian Cox processes. J Stat Comput Simul 2014; 84(10): 2266–2284.

31.

Carroll

Lawson

Faes

, et al. Comparing INLA and OpenBUGS for hierarchical Poisson modeling in disease mapping. Spat Spatiotemporal Epidemiol 2015; 14–15: 45–54.

32.

ATTACC Investigators, ACTIV-4a Investigators, REMAP-CAP Investigators, et al. Therapeutic anticoagulation with heparin in noncritically ill patients with Covid-19. N Engl J Med 2021; 385(9): 790–802.

33.

Gelfand

. Gibbs sampling. J Am Statl Assoc 2000; 95(452): 1300–1304.

34.

Girolami

Calderhead

. Riemann manifold Langevin and Hamiltonian Monte Carlo methods. J R Stat Soc Ser B Methodol 2011; 73(2): 123–214.

35.

Albert

. Probability and Bayesian modeling. Boca Raton, FL: CRC Press, 2019.

36.

Harrell

. Ordinal logistic regression. In: Harrell

(ed.) Regression modeling strategies. New York: Springer, 2015, pp. 311–325.

37.

National Heart, Lung, and Blood Institute Acute Respiratory Distress Syndrome (ARDS) Clinical Trials Network, Matthay

Brower

, et al. Randomized, placebo-controlled clinical trial of an aerosolized β2-agonist for treatment of acute lung injury. Am J Respir Crit Care Med 2011; 184(5): 561–568.

38.

National Heart, Lung, and Blood Institute Acute Respiratory Distress Syndrome (ARDS) Clinical Trials Network, Rice

Wheeler

, et al. Initial trophic vs full enteral feeding in patients with acute lung injury: the EDEN randomized trial. JAMA: the Journal of the American Medical Association 2012; 307(8): 795–803.

39.

Rice

Wheeler

Thompson

, et al. Enteral omega-3 fatty acid, γ-linolenic acid, and antioxidant supplementation in acute lung injury. JAMA 2011; 306(14): 1574–1581.

40.

Schoenfeld

Bernard

and ARDS Network. Statistical evaluation of ventilator-free days as an efficacy measure in clinical trials of treatments for acute respiratory distress syndrome. Crit Care Med 2002; 30(8): 1772–1777.

41.

Yehya

Harhay

Curley

, et al. Reappraisal of ventilator-free days in critical care research. Am J Respir Crit Care Med 2019; 200(7): 828–836.

42.

Cox

. Regression models and life-tables. J R Stat Soc Ser B Methodol 1972; 34(2): 187–202.

43.

Lee

. Bayesian statistics. London: Oxford University Press, 1989.

44.

Quintana

Viele

Lewis

. Bayesian analysis: using prior information to interpret the results of clinical trials. JAMA 2017; 318(16): 1605–1606.

45.

Gelman

. Prior distributions for variance parameters in hierarchical models (comment on article by Browne and Draper). Bayesian Anal 2006; 1(3): 515–534.

46.

Qian

Stow

Borsuk

. On Monte Carlo methods for Bayesian inference. Ecol Modell 2003; 159(2–3): 269–277.

47.

Rue

Riebler

Sørbye

, et al. Bayesian computing with INLA: a review. Ann Rev Stat Appl 2017; 4: 395–421.

48.

Ekvall

Jones

. Markov chain Monte Carlo, 2019.

49.

Robert

Elvira

Tawn

, et al. Accelerating MCMC algorithms. Wiley Interdiscip Rev Comput Stat 2018; 10(5): e1435.

50.

Fonnesbeck

Patil

Huard

, et al. PyMC documentation, 2017, https://readthedocs.org/projects/pymcmc/downloads/pdf/stable/

51.

Hall

. LaplacesDemon: an R package for Bayesian Inference, 2008, http://eio.usc.es/pub/mjginzo/descargas/leyenda/Documents/R/win-library/2.12/LaplacesDemon/doc/LaplacesDemon.pdf

52.

Robert

Casella

. The metropolis – Hastings algorithm. In: Robert

Casella

(eds) Monte Carlo statistical methods. New York: Springer, 2004, pp. 267–320.

53.

Chib

Greenberg

. Understanding the metropolis-Hastings algorithm. Am Stat 1995; 49(4): 327–335.

54.

Depaoli

Clifton

Cobb

. Just another Gibbs sampler (JAGS) flexible software for MCMC implementation. J Educ Behav Stat 2016; 41(6): 628–649.

55.

Stan Development Team. Stan modeling language users guide and reference manual, 2016, https://pj.freefaculty.org/guides/crmda_workshops/sem/Archive/sem-4/literature/manuals/stan-reference-2.11.0.pdf

56.

Betancourt

. A conceptual introduction to Hamiltonian Monte Carlo. arXiv [preprint], 2017. DOI: 10.48550/arXiv.1701.02434.

57.

Betancourt

Girolami

. Hamiltonian Monte Carlo for hierarchical models. Curr Trends Bayesian Methodol Appl 2015; 79(30): 2–4.

58.

Hill

Spall

. Stationarity and convergence of the metropolis-hastings algorithm: Insights into theoretical aspects. IEEE Contr Syst Mag 2019; 39(1): 56–67.

59.

Frigessi

Martinelli

Stander

. Computational complexity of Markov chain Monte Carlo methods for finite Markov random fields. Biometrika 1997; 84(1): 1–18.

60.

Wang

Yue

Faraway

. Bayesian regression modeling with INLA. Boca Raton, FL: Chapman & Hall/CRC, 2018.

61.

R Core Team. R: a language and environment for statistical computing. Vienna: R Core Team, 2022, https://www.R-project.org/

62.

Yajima

MYS

, et al. Package ‘R2jags’. R package version 003-08, 2015, https://cran.r-project.org/web/packages/R2jags/index.html

63.

Bürkner

. brms: an R package for Bayesian multilevel models using Stan. J Stat Softw 2017; 80: 1–28.

64.

Lindgren

Rue

. Bayesian spatial modelling with R-INLA. J Stat Softw 2015; 63: 1–25.

65.

Chen

Ware

, et al. A Bayesian approach for the cox proportional hazards model with covariates subject to detection limit. Int J Stat Med Res 2014; 3(1): 32.

66.

Strömberg

. Collapsing ordered outcome categories: a note of concern. Am J Epidemiol 1996; 144(4): 421–424.

67.

Link

Eaton

. On thinning of chains in MCMC. Methods Ecol Evol 2012; 3(1): 112–115.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.51 MB