Psychologists estimate the precision of their statistics both to conduct hypothesis tests and to construct confidence intervals. The methods traditionally used for this are available only for a small set of statistics (e.g., the mean and transformations of it) and often make unrealistic assumptions about the variables' distributions. These assumptions are often particularly unrealistic in data derived from clinical samples, or when looking at groups responding at the extreme end of clinical constructs. Bootstrap estimation is a computer intensive procedure that offers a flexible and automatic alternative. The computer takes thousands of bootstrap samples from the observed data and from these bootstrap samples estimates the precision of the statistic. High-speed personal computers make the bootstrap a viable and appealing technique throughout the sciences. This article offers a tutorial on the theory and practice of applying bootstrap estimation to data from clinical samples and measures relevant to experimental psychopathology.
AgrestiA. (2002). Categorical data analysis (2nd ed.).Hoboken, NJ: John Wiley & Sons. doi: 10.1002/0471249688
2.
ArosteguiI., Nunez-AntonV., & QuintanaJ. M. (2007). Analysis of the short form-36 (SF-36): The beta-binomial distribution approach. Statistics in Medicine, 26(6), 1318–1342. doi: 10.1002/sim.2612
3.
BarberJ. A., & ThompsonS. G. (2000). Analysis of cost data in randomized trials: an application of the non-parametric bootstrap. Statistics in Medicine, 19(23), 3219–3236.
4.
BinksC. A., FentonM., McCarthyL., LeeT., AdamsC. E., & DugganC. (2006). Psychological therapies for people with borderline personality disorder. Cochrane Database of Systematic Reviews(1). doi: 10.1002/14651858.cd005652
5.
CantyA., & RipleyB. (2010). boot: Bootstrap R (S-Plus) Functions. R package version 1.2–43. cran.r-project.org.
6.
Cartwright-HattonS., McNallyD., FieldA. P., RustS., LaskeyB., DixonC …, & WoodhamA. (In Press). A new parenting-based group intervention for young anxious children: Results of a randomized controlled trial. Journal of the American Academy of Child & Adolescent Psychiatry.
7.
CasellaG. (2003). Introduction to the Silver Anniversary of the Bootstrap. Statistical Science, 18, 133–134. doi: 10.1214/ss/1063994967
8.
ChambersJ.M. (2008). Software for data analysis: Programming with R.New York: Springer. doi: 10.1007/978-0-387-75936-4
9.
ChernickM.R. (2008). Bootstrap methods: A guide for practitioners and researchers (2nd ed). Hoboken, NJ: Wiley.
10.
CohenJ. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37–46. doi: 10.1177/001316446002000104
11.
DavisonA. C., & HinkleyD. V. (1997). Bootstrap methods and their applications.Cambridge UK: Cambridge University Press.
12.
DelucchiK. L., & BostromA. (2004). Methods for analysis of skewed data distributions in psychiatric clinical studies: Working with many zero values. American Journal of Psychiatry, 161(7), 1159–1168. doi: 10.1176/appi.ajp.161.7.1159
13.
DerisleyJ., LibbyS., ClarkS., & ReynoldsS. (2005). Mental health, coping and family-functioning in parents of young people with obsessive-compulsive disorder and with anxiety disorders. British Journal of Clinical Psychology, 44, 439–444. doi: 10.1348/014466505x29152 doi: 10.1348/014466505X29152
14.
EfronB. (1979). Bootstrap methods: Another look at the jackknife. Annals of Statistics, 7, 1–26. doi: 10.1214/aos/1176344552
15.
EfronB., & TibshiraniR. (1993). An introduction to the bootstrap.Boca Raton, FL: Chapman & Hall/CRC.
16.
FieldA. P. (2009). Discovering statistics using SPSS: and sex and drugs and rock ‘n’ roll (3rd edition). London: Sage.
17.
FieldA. P., & MilesJ. N. V. (in press). Discovering statistics using R: and sex and drugs and rock ‘n’ roll.London: Sage.
18.
GilchristJ.M. (2009). Weighted 2×2 kappa coefficients: Recommended indices of diagnostic accuracy for evidence-based practice. Journal of Clinical Epidemiology, 62, 1045–1053. doi: 10.1016/j.jclinepi.2008.11.012
19.
GoffD.C., & SimmsC.A. (1993). Has multiple personality disorder remained consistent over time?Journal of Nervous and Mental Disease, 181, 595–600. doi: 10.1097/00005053-199310000-00003
20.
GoodP.I. (2006). Resampling methods: A practical guide to data analysis (3rd ed). Boston: Birkhäuser.
21.
HlatkyM. A., BoothroydD. B., & JohnstoneI. M. (2002). Economic evaluation in long-term clinical trials. Statistics in Medicine, 21(19), 2879–2888. doi: 10.1002/sim.1292
KraemerH.C., & GibbonsR.D. (2009). Where do we go wrong in assessing risk factors, diagnostic and prognostic tests? The problems of two-by-two association. Annals of Psychiatry, 39, 711–718. doi: 10.3928/00485713-20090625-05
24.
LibbyS., ReynoldsS., DerisleyJ., & ClarkS. (2004). Cognitive appraisals in young people with obsessive-compulsive disorder. Journal of Child Psychology and Psychiatry, 45(6), 1076–1084. doi: 10.1111/j.1469-7610.2004.t01-1-00300.x
25.
LunneborgC.E. (2000). Data analysis by resampling: Concepts and applications.Pacific Grove, CA: Duxbury Press.
26.
MicceriT. (1989). The unicorn, the normal curve, and other improbable creatures. Psychological Bulletin, 105, 156–166. doi: 10.1037/0033-2909.105.1.156
27.
PicardiA., BattistiF., De GirolamoG., MorosiniP., NorcioB., BraccoR., & BiondiM. (2008). Symptom structure of acute mania: A factor study of the 24-item Brief Psychiatric Rating Scale in a national sample of patients hospitalized for a manic episode. Journal of Affective Disorders, 108(1-2), 183–189. doi: 10.1016/j.jad.2007.09.010
28.
R Development Core Team. (2010). R: A language and environment for statistical computing.Vienna, Austria: R Foundation for Statistical Computing. ISBN 3-900051-07-0, Retrieved from http://www.R-project.org
29.
Roldán NofuentesJ.A., Luna del CastilloJ.D., & Montero AlonsoM.A. (2009). Confidence intervals of weighted kappa coefficient of a binary diagnostic test. Communications in Statistics: Simulation and Computation, 38, 1562–1578. doi: 10.1080/03610910903039473
30.
RutterC. M., & MigliorettiD. L. (2003). Estimating the accuracy of psychological scales using longitudinal data. Biostatistics, 4(1), 97–107. doi: 10.1093/biostatistics/4.1.97
31.
SadlerP., EthierN., & WoodyE. (2011). Tracing the interpersonal web of psychopathology: Dyadic data analysis methods for clinical researchers. Journal of Experimental Psychopathology, 2(2) 95–138. doi: 10.5127/jep.010310
32.
SimonJ. L. (1969a). Basic research methods in social science: The art of empirical investigation.New York: Random House.
33.
SimonJ. L. (with HolmesA.), (1969b). A new way to teach probability statistics. The Mathematics Teacher, 62, 283–288.
34.
TangL. Q., SongJ. W., BelinT. R., & UnutzerJ. (2005). A comparison of imputation methods in a longitudinal randomized clinical trial. Statistics in Medicine, 24(14), 2111–2128. doi: 10.1002/sim.2099
35.
TyrerP., NurU., CrawfordM., KarlsenS., McLeanC., RaoB., & JohnsonT. (2005). The social functioning questionnaire: A rapid and robust measure of perceived functioning. International Journal of Social Psychiatry, 51(3), 265–275. doi: 10.1177/0020764005057391
36.
Villasenor-AlvaJ. A., & Gonzalez-EstradaE. (2009). A generalization of Shapiro-Wilk's test for multivariate normality. Communications in Statistics: Theory and Methods, 38, 1870–1883. doi: 10.1080/03610920802474465
37.
WangX. (2005). Asymptotics of the Theil-Sen estimator in the simple linear regression model with a random covariate. Journal of Nonparametric Statistics, 17(1), 107–120. doi: 10.1080/1048525042000267743
38.
WilcoxR. R. (1998). A note on the Theil-Sen regression estimator when the regressor is random and the error term is heteroscedastic. Biometrical Journal, 40(3), 261–268.
39.
WilcoxR. R. (2005). Introduction to robust estimation and hypothesis testing (2nd ed.). Burlington, MA: Elsevier.
40.
WilkinsonL., and the Task Force on Statistical Inference (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594–604. doi: 10.1037/0003-066X.54.8.594
41.
WoodyE. (2011). An SEM perspective on evaluating mediation: What every clinical researcher needs to know. Journal of Experimental Psychopathology, 2(2), 210–251. doi: 10.5127/jep.010410
42.
WrightD. B. (2003). Making friends with your data: Improving how statistics are conducted and reported. British Journal of Educational Psychology, 73, 123–136. doi: 10.1348/000709903762869950
43.
WrightD. B., & LondonK. (2009). Modern regression techniques using R: A practical guide for students and researchers.London: Sage.
44.
ZimmermanM., ChelminskiI., & PosternakM. (2004). A review of studies of the Hamilton depression rating scale in healthy controls - Implications for the definition of remission in treatment studies of depression. Journal of Nervous and Mental Disease, 192(9), 595–601. doi: 10.1097/01.nmd.0000138226.22761.39
