When data exhibit imbalance between a large number d of covariates and a small number n of samples, clinical outcome prediction is impaired by overfitting and prohibitive computation demands. Here we study two simple Bayesian prediction protocols that can be applied to data of any dimension and any number of outcome classes. Calculating Bayesian integrals and optimal hyperparameters analytically leaves only a small number of numerical integrations, and CPU demands scale as O(nd). We compare their performance on synthetic and genomic data to the mclustDA method of Fraley and Raftery. For small d they perform as well as mclustDA or better. For d = 10,000 or more mclustDA breaks down computationally, while the Bayesian methods remain efficient. This allows us to explore phenomena typical of classification in high-dimensional spaces, such as overfitting and the reduced discriminative effectiveness of signatures compared to intra-class variability.
HastieTTibshiraniR. Discriminant analysis by Gaussian mixtures. J R Stat Soc Ser B1996; 58: 155–176.
2.
RipleyBD. Pattern recognition and neural networks, Cambridge: Cambridge University Press, 1996.
3.
DudaROHartPEStorkD. Pattern classification, 2nd ed. New York: Wiley, 2001.
4.
McLachlanGJPeelDBeanRW. Modelling high-dimensional data by mixtures of factor analyzers. Comput Stat Data Anal2003; 41: 379–388.
5.
ClarkeRRessomHWWangAet al.The properties of high-dimensional data spaces: implications for exploring gene and protein expression data. Nat Rev Cancer2008; 8: 37–49.
6.
MichielsSKramarAKoscielnyS. Multidimensionality of microarrays: statistical challenges and (im)possible solutions. Mol Oncol2011; 5: 190–196.
7.
WangYMillerDJClarkeR. Approaches to working in high-dimensional data spaces: gene expression microarrays. Br J Cancer2008; 98: 1023–1028.
8.
The Cancer Genome Atlas Research Network. Integrated genomic analyses of ovarian carcinoma. Nature2011; 474: 609–615.
9.
The Cancer Genome Atlas Research Network. Comprehensive molecular portraits of human breast tumours. Nature2012; 490: 61–70.
10.
BarretinaJCaponigroGStranskyNet al.The Cancer Cell Line Encyclopedia enables predictive modelling of anticancer drug sensitivity. Nature2012; 483: 603–307.
11.
de RinaldisEGazinskaPMeraAet al.Integrated genomic analysis of triple-negative breast cancers reveals novel microRNAs associated with clinical and molecular phenotypes and sheds light on the pathways they control. BMC Genom2013; 14: 643–643.
12.
van’t VeerLJDaiHvan de VijverMJet al.Gene expression profiling predicts clinical outcome of breast cancer. Nature2002; 415: 530–536.
13.
van de VijverMJHeYDvan’t VeerLJet al.A gene-expression signature as a predictor of survival in breast cancer. N Engl J Med2002; 347: 1999–2009.
14.
FraleyCRafteryAE. Model-based clustering, discriminant analysis, and density estimation. J Am Stat Assoc2002; 97: 611–631.
15.
JolliffeIT. Principal component analysis, 2nd ed. New York: Springer, 2002.
16.
FraleyCRafteryAEMurphyTBet al.mclust Version 4 for R: normal mixture modeling for model-based clustering, classification, and density estimation, University of Washington (USA): Technical Report No 597, 2012.
17.
BouveyronCGirardSSchmidC. High-dimensional discriminant analysis. Commun Stat Theory Methods2007; 36: 2607–2623.
18.
Scott DW and Thompson JR. Probability density estimation in higher dimensions. In: Proceedings of the fifteenth symposium on the interface on computer science and statistics. Vol. 528. Amsterdam: North-Holland, 1983, pp.173–179.
19.
McNicholasPD. On model-based clustering, classification, and discriminant analysis. J Iran Stat Soc2011; 10: 181–199.
20.
ChangWC. On using principal components before separating a mixture of two multivariate normal distributions. J R Stat Soc Ser C (Appl Stat)1983; 32: 267–275.
21.
BouveyronCBrunet-SaumardC. Model-based clustering of high-dimensional data: a review. Comput Stat Data Anal2014; 71: 52–78.
22.
BishopCM. Pattern recognition and machine learning, New York: Springer, 2006.
23.
BergeLBouveyronCGirardS. HDclassif: an R package for model-based clustering and discriminant analysis of high-dimensional data. J Stat Softw2012; 46: 1–29.
24.
DasguptaARafteryAE. Detecting features in spatial point processes with clutter via model-based clustering. J Am Stat Assoc1998; 93: 294–302.
25.
FraleyCRafteryAE. How many clusters? Which clustering method? Answers via model-based cluster analysis. Comput J1998; 41: 578–588.
26.
R CoreTeam. R: a language and environment for statistical computing, Vienna, Austria: R Foundation for Statistical Computing, 2013.
27.
DeanNMurphyTBDowneyG. Using unlabelled data to update classification rules with applications in food authenticity studies. J R Stat Soc Ser C (Appl Stat)2006; 55: 1–14.
28.
FraleyCRafteryAE. Model-based methods of classification: using the mclust software in chemometrics. J Stat Softw2007; 18: 1–13.
29.
IversonAAGillettCCanePet al.A single-tube quantitative assay for mRNA levels of hormonal and growth factor receptors in breast cancer specimens. J Mol Diagn2009; 11: 117–130.
30.
MurphyTBDeanNRafteryAE. Variable selection and updating in model-based discriminant analysis for high dimensional data with food authenticity applications. Ann Appl Stat2010; 4: 396–421.
31.
AndrewsJLMcNicholasPD. Model-based clustering, classification, and discriminant analysis via mixtures of multivariate t-distributions. Stat Comput2012; 22: 1021–1029.
32.
BanfieldJDRafteryAE. Model-based Gaussian and non-Gaussian clustering. Biometrics1993; 49: 803–821.
33.
DempsterAPLairdNMRubinDB. Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc Ser B1977; 39: 1–38.
34.
SchwarzG. Estimating the dimension of a model. Ann Stat1978; 6: 461–464.
35.
WehrensRBuydensLMCFraleyCet al.Model-based clustering for image segmentation and large datasets via sampling. J Classif2004; 21: 231–253.
36.
FraleyCRafteryAEWehrensR. Incremental model-based clustering for large datasets with small clusters. J Comput Graph Stat2005; 14: 529–546.
37.
GeisserSCornfieldJ. Posterior distributions for multivariate normal parameters. J R Stat Soc B1963; 25: 368–376.
38.
KeehnDG. A note on learning for Gaussian properties. IEEE Trans Inform Theory1965; 11: 126–132.
39.
HanXWakabayashiTKimuraFThe optimum classifier and the performance evaluation by bayesian approach. In: FerriFJet al.(ed). Advances in Pattern Recognition, Heidelberg: Springer, 2000, pp. 591–600.
40.
Srivastava S and Gupta MR. Distribution-based Bayesian minimum expected risk for discriminant analysis. In: Proceedings of the IEEE International Symposium on Information Theory, 2006, pp.2294–2298.
BellmanR. Dynamic programming, Princeton: Princeton University Press, 1957.
43.
KleinJPMoeschbergerML. Survival analysis techniques for censored and truncated data, New York: Springer, 2005.
44.
MillerLDSmedsJGeorgeJet al.An expression signature for p53 status in human breast cabcer predicts mutation status, transcriptional effects, and patient survival. Proc Natl Acad Sci USA2005; 102: 13550–13555.
45.
BernardiniMQBabaTLeePSet al.Expression signatures of TP53 mutations in serious ovarian cancers. BMC Cancer2010; 10: 237–237.
46.
KonstantinopoulosPASpentzosDKarlanBYet al.Gene expression profile of BRCAness that correlates with responsiveness to chemotherapy and with outcome in patients with epithelial ovarian cancer. J Clin Oncol2010; 28: 3555–3561.
47.
PressJWurzKNorquistBMet al.Identification of a preneoplastic gene expression profile in tubal epithelium of BRCA1 mutation carriers. Neoplasia2010; 12: 993–1002.
48.
George DK, Charkhesht A and Vinh NQ. New terahertz dielectric spectroscopy for the study of aqueous solutions. Epub ahead of print 2015.
49.
KoenderinkJJ. Solid shape, Boston: MIT Press, 1990.
50.
RafteryAEDeanN. Variable selection for model-based clustering. J Am Stat Assoc2006; 101: 168–178.
51.
GradshteynISRyzhikIM. Table of integrals, series, and products, 7th ed. Burlington MA: Academic Press, 2007.
