Sample size and power calculations for medical studies by simulation when closed form expressions are not available

Abstract

This paper shows how Monte Carlo simulation can be used for sample size, power or precision calculations when planning medical research studies. Standard study designs can lead to the use of analysis methods for which power formulae do not exist. This may be because complex modelling techniques with optimal statistical properties are used but power formulae have not yet been derived or because analysis models are employed that divert from the population model due to lack of availability of more appropriate analysis tools. Our presentation concentrates on the conceptual steps involved in carrying out power or precision calculations by simulation. We demonstrate these steps in three examples concerned with (i) drop out in longitudinal studies, (ii) measurement error in observational studies and (iii) causal effect estimation in randomised controlled trials with non-compliance. We conclude that the Monte Carlo simulation approach is an important general tool in the methodological arsenal for assessing power and precision.

Keywords

Monte Carlo simulation power precision missing data measurement error causal effect estimation sensitivity analysis hybrid approach

Get full access to this article

View all access options for this article.

References

Halpern

Karlawish

JHT

Berlin

. The continuing unethical conduct of underpowered clinical trials. J Am Med Assoc 2002; 288: 358–362.

Shoukri

Asyali

Donner

. Sample size requirement for the design of reliability study: review and new results. Stat Methods Med Res 2004; 13: 251–271.

Venter

Maxwell

Bolig

. Power in randomized group comparisons: the value of adding a single intermediate time point to a traditional pretest-posttest design. Psychol Methods 2002; 7: 194–209.

Faul

Erdfelder

Lang

A-G

. G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods 2007; 39: 175–191.

Raudenbush

. Statistical analysis and optimal design for cluster randomized trials. Psychol Methods 1997; 2: 173–185.

Raudenbush

Liu

. Effects of study duration, frequency of observation, and sample size on power in studies of group differences in polynomial change. Psychol Methods 2001; 6: 387–401.

Christopher

. Monte Carlo simulation, Thousand Oaks, CA: Sage Publications, 1997.

Rubinstein

Kroese

. Simulation and the Monte Carlo method, Hoboken, NJ: John Wiley & Sons, 2008.

Shonkwiler

Mendivil

. Explorations in Monte Carlo methods, New York: Springer, 2009.

10.

Gastaňaga

McLaren

Delfino

. Power calculations for generalized linear models in observational longitudinal studies: a simulation approach in SAS. Comput Methods Program Biomed 2006; 84: 27–33.

11.

Zhang

Kowalski

. Power analyses for longitudinal study designs with missing data. Stat Med 2007; 26: 2958–2981.

12.

Williams

Ebel

Wagner

. Monte Carlo approaches for determining power and sample size in low-prevalence applications. Preventive Veterinary Med 2007; 82: 151–158.

13.

Richardson

. Power calculations for survival analyses via Monte Carlo estimation. Am J Ind Med 2003; 44: 532–539.

14.

Muthén

. How to use a Monte Carlo study to decide on sample size and determine power. Struct Equ Modell 2002; 9: 599–620.

15.

Browne

Lahi

Parker

RMA

. A guide to sample size calculation for random effects models via simulation and the MLPowSim software package, Manual, University of Bristol, 2009.

16.

Fieveson

. Power by simulation. Stata J 2002; 2: 107–114.

17.

Rizzo

. Statistical computing with R, Boca Raton, FL: Chapman & Hall/CRC, 2008.

18.

Harris

AHS

Cronkite

Moos

. Physical activity, exercise coping, and depression in a 10-year cohort study of depressed patients. J Affective Disorders 2006; 93: 79–85.

19.

Hedeker

Gibbons

Waternaux

. Sample size estimation for longitudinal designs with attrition: comparing time-related contrasts between two groups. J Edu Behav Stat 1999; 24: 70–93.

20.

Roy

Bhaumik

Aryal

. Sample size determination for hierarchical longitudinal designs with differential attrition rates. Biometrics 2007; 63: 699–707.

21.

Basagaña

Spiegelman

. Power and sample size calculations for longitudinal studies estimating a main effect of a time-varying exposure. Stat Methods Med Res 2010; 19: 1–17.

22.

Rubin

. Multiple imputation for nonresponse in surveys, New York: Wiley, 1987.

23.

Schafer

Graham

. Missing data: our view of the state of the art. Psychol Methods 2002; 7: 147–177.

24.

Cohen

. Statistical power analysis for the behavioral sciences, 2nd edn. Hillsdale, NJ: Lawrence Erlbaum Associates, 1988.

25.

White

Goldsmith

Johnson

. Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial. Lancet 2011; 377: 823–836.

26.

Cohen

. The cost of dichotomization. Appl Psychol Meas 1983; 7: 249–254.

27.

Greenland

. Avoiding power loss associated with categorization and ordinal scores in dose-response and trend analysis. Epidemiology 1995; 6: 450–454.

28.

MacCallum

Zhang

Preacher

. On the practice of dichotomization of quantitative variables. Psychol Methods 2002; 7: 19–40.

29.

Streiner

. Breaking up is hard to do: the heartbreak of dichotomizing continuous data. Can J Psychiatry 2002; 47: 262–266.

30.

Taylor

West

Aiken

. Loss of power in logistic, ordinal logistic, and probit regression when an outcome variable is coarsely categorized. Edu Psychol Meas 2006; 66: 228–239.

31.

Altman

Royston

. The cost of dichotomizing continuous variables. Br MedJ 2006; 332: 1080–1080.

32.

Carroll

Ruppert

Stefanski

. Measurement error in nonlinear models, Boca Raton, FL: Chapman and Hall/CRC, 2006.

33.

Chen

Cohen

Chen

. Biased odds ratios for dichotomization of age. Stat Med 2007; 26: 3487–3497.

34.

Dunn

Maracy

Dowrick

. Estimating psychological treatment effects from a randomized controlled trial with both non-compliance and loss to follow-up. Br J Psychiatry 2003; 183: 323–331.

35.

Little

RJA

Yau

LHY

. Statistical techniques for analyzing data from prevention trials: treatment of no-shows using Rubin’s causal model. Psychol Methods 1998; 3: 147–159.

36.

Bellamy

Liu

Ten Have

. An introduction to causal modeling in clinical trials. Clin Trials 2007; 4: 58–73.

37.

White

. Uses and limitations of randomization-based efficacy estimators. Stat Methods Med Res 2005; 14: 327–347.

38.

Angrist

Imbens

Rubin

. Identification of causal effects using instrumental variables. J Am Stat Assoc 1996; 91: 444–455.

39.

Frangakis

Rubin

. Principal stratification in causal inference. Biometrics 2002; 58: 21–29.

40.

Little

RJA

Long

Lin

. A comparison of methods for estimating the causal effect of a treatment in randomized clinical trials subject to noncompliance. Biometrics 2009; 65: 640–649.

41.

Hui

Rubin

. Principal stratification for causal inference with extended partial compliance. J Am Stat Assoc 2008; 103: 101–111.

42.

. Statistical power in randomized intervention studies with non-compliance. Psychol Methods 2002; 7: 178–193.

43.

Rubin

. Causal inference using potential outcomes: design, modeling, decisions. J Am Stat Assoc 2005; 100: 322–331.

44.

Bloom

. Accounting for no-shows in experimental evaluation designs. Eval Rev 1984; 8: 225–246.

45.

Rabe-Hesketh

Skrondal

Pickles

. Generalized multilevel structural equation modelling. Psychometrika 2004; 69: 167–190.

46.

Muthén

Modeling of intervention effects with noncompliance: a latent variable approach for randomized trials. In: Marcoulides

Schumacker

(eds). New developments and techniques in structural equation modeling, Hillsdale, NJ: Lawrence Erlbaum Associates, 2001, pp. 57–87.

47.

. Estimation of intervention effects with noncompliance: alternative model specifications. J Educ Behav Stat 2002; 27: 385–409.

48.

Skrondal

Rabe-Hesketh

. Generalized latent variable modeling: multilevel, longitudinal, and structural equation models, Boca Raton, FL: Chapman & Hall/CRC, 2004.

49.

Hardin

Hilbe

. Generalized linear models and extensions, 2nd edn. College Station, TX: Stata Press, 2007.

50.

Serlin

. Testing for robustness in Monte Carlo studies. Psychol Methods 2000; 5: 230–240.

51.

Turner

Dobson

Pocock

. Categorisation of continuous risk factors in epidemiological publications: a survey of current practice. Epidemiol Perspect Innov 2010; 7: 9–9.

52.

Yin

Fan

. Assessing the reliability of Beck Depression Inventory scores: reliability generalization across studies. Educ Psychol Meas 2000; 60: 201–223.

53.

Wang

Gelfand

. A simulation based approach to Bayesian sample size determination for performance under a given model and for separating models. Stat Sci 2002; 17: 193–208.

54.

Inoue

LYT

Berry

Parmigiani

. Relationship between Bayesian and frequentist sample size determination. Am Stat 2005; 59: 79–87.

55.

Adcock

. Sample size determination: a review. The Statistician 1997; 46: 261–283.

56.

Pezeshk

. Bayesian techniques for sample size determination in clinical trials: a short review. Stat Methods Med Res 2003; 12: 489–504.

57.

Brown

Herson

Atkinson

. Projection from previous studies: a Bayesian and frequentist compromise. Control Clin Trials 1987; 8: 29–44.

58.

Feng

Grizzle

. Correlated binomial variates: properties of estimator of intraclass correlation and its effect on sample size calculation. Stat Med 1997; 11: 1607–1614.

59.

Joseph

Du Berger

Belisle

. Bayesian and mixed Bayesian/likelihood criteria for sample size determination. Stat Med 1997; 16: 769–781.

60.

Lee

Zelen

. Clinical trials and sample size considerations: another perspective. Stat Sci 2000; 15: 95–110.

61.

Spiegelhalter

Abrams

Myles

. Bayesian approaches to clinical trials and health-care evaluation, Chichester: Wiley, 2004.

62.

Turner

Prevost

Thompson

. Allowing for imprecision of the intracluster correlation coefficient in the design of cluster randomized trials. Stat Med 2004; 23: 1195–1214.

63.

Campbell

Donner

Klar

. Developments in cluster randomized trials and in statistics in medicine. Stat Med 2007; 26: 2–19.

64.

Hade

Murray

Pen Pennell

. Intraclass correlation estimates for cancer screening outcomes: estimates and applications in the design of group-randomized cancer screening studies. J Natl Cancer Inst Monogr 2010; 40: 97–103.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.04 MB