The sequential goodness-of-fit (SGoF) multiple testing method has recently been proposed as an alternative to the familywise error rate- and the false discovery rate-controlling procedures in high-dimensional problems. For discrete data, the SGoF method may be very conservative. In this paper, we introduce an alternative SGoF-type procedure that takes into account the discreteness of the test statistics. Like the original SGoF, our new method provides weak control of the false discovery rate/familywise error rate but attains false discovery rate levels closer to the desired nominal level, and thus it is more powerful. We study the performance of this method in a simulation study and illustrate its application to a real pharmacovigilance data set.
Dudoit S and van der Laan MJ. Multiple testing procedures and applications to genomics. Springer series in statistics. New York: Springer, 2007.
2.
LehmannERomanoJ. Testing statistical hypotheses, New York: Springer, 2006.
3.
Carvajal-RodríguezAde Uña-ÁlvarezJRolán-ÁlvarezE. A new multitest correction (SGoF) that increases its statistical power when increasing the number of tests. BMC Bioinformatics2009; 10: 1–14.
4.
HochbergYTamhaneAC. Multiple comparison procedures, New Jersey: Wiley, 1987.
5.
WestfallPHTobiasRDWolfingerRD. Multiple comparisons and multiple tests using SAS, Cary, NC: SAS Institute Inc., 2011.
6.
TukeyJ. The higher criticism. Princeton University course notes, statistics, 411(T13), Princeton: Princeton University, 1976.
7.
DonohoDJinJ. Higher criticism for detecting sparse heterogeneous mixtures. Ann Stat2004; 32: 962–994.
8.
de Uña-ÁlvarezJ. On the statistical properties of SGoF multitesting method. Stat Appl Genet Mol Biol2011; 10: 1–30.
9.
de Uña-ÁlvarezJ. The beta-binomial SGoF method for multiple dependent tests. Stat Appl Genet Mol Biol2012; 11(3): 365–396.
10.
LehmannERomanoJP. Generalizations of the familywise error rate. Ann Stat2005; 33: 1138–1154.
11.
Chen X and Doerge R. A weighted FDR procedure under discrete and heterogeneous null distributions. ArXiv preprint arXiv:1502.00973. ArXiv preprint arXiv: 1502.00973[stat.ME], 2015.
12.
WestfallPWolfingerR. Multiple tests with discrete distributions. Am Stat1997; 51: 3–8.
13.
GilbertP. A modified false discovery rate multiple-comparisons procedure for discrete data, applied to human immunodeficiency virus genetics. J R Stat Soc Ser C2005; 54: 143–158.
WestfallPTroendleJ. Multiple testing with minimal assumptions. Biometrical J2008; 50: 745–755.
16.
GutmanRHochbergY. Improved multiple test procedures for discrete distributions: new ideas and analytical review. J Stat Plan Infer2007; 137: 2380–2393.
17.
BenjaminiYHochbergY. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Ser B1995; 57: 289–300.
18.
BenjaminiYLiuW. A step-down multiple hypotheses testing procedure that controls the false discovery rate under independence. J Stat Plan Infer1999; 82: 163–170.
19.
Castro-CondeIde Uña-ÁlvarezJ. Power, FDR and conservativeness of BB-SGoF method. Comput Stat. Stat 2015; 30: 1143–1161.
20.
Shaked M and Shanthikumar J. Stochastic orders. Springer series in statistics. New York: Springer-Verlag, 2007.
21.
DavidHNagarajaH. Order statistics, Third edition. New Jersey: Wiley, 2004.
22.
ShorackGWellnerJ. Empirical processes with applications to statistics. Classics in applied mathematics, Philadelphia: Society for Industrial and Applied Mathematics (SIAM), 2009.
23.
HongY. On computing the distribution functions for the Poisson binomial distribution. Comput Stat Data Anal2013; 59: 41–51.
24.
Castro-CondeIde Uña-ÁlvarezJ. Adjusted p-values for SGoF multiple test procedure. Biometrical J2015; 57: 108–122.
25.
BerryGArmitageP. Mid-p confidence intervals: a brief review. J R Stat Soc Ser D (The Statistician)1995; 44: 417–423.
de Uña–ÁlvarezJCarvajal-RodríguezA. ‘SGoFicance Trace’: assessing significance in high dimensional testing problems. PLoS One2010; 5(12): e15930–e15930.
Castro-CondeIde Uña-ÁlvarezJ. sgof: An R package for multiple testing problems. R Journal2014; 6/2: 96–113.
31.
PoundsSChengC. Robust estimation of the false discovery rate. Bioinformatics2006; 22: 1979–1987.
32.
ChenXDoergeR. Generalized estimators for multiple testing: proportion of true nulls and false discovery rate. Technical report 12-04, Department of Statistics, Purdue University, 2012.
33.
Martínez-CamblorP. On correlated z-values distribution in hypothesis testing. Comput Stat Data Anal2014; 79: 30–43.
34.
R Development Core Team. R: a language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing, 2009.
35.
VolkovaA. A refinement of the central limit theorem for sums of independent random indicators. Theor Probab Appl1996; 40: 791–794.
36.
Carvajal-RodríguezAde Uña-ÁlvarezJ. Assessing significance in high-throughput experiments by sequential goodness of fit and q-value estimation. PLoS One2011; 6(9): e24700–e24700.
37.
StoreyJD. A direct approach to false discovery rates. J R Stat Soc Ser B (Statistical Methodology)2002; 64: 479–498.
38.
NettletonDHwangJTGet al.Estimating the number of true null hypotheses from a histogram of p values. J Agr Biol Environ Stat2006; 11: 337–356.
39.
DalmassoCBroetPMoreauT. A simple procedure for estimating the false discovery rate. Bioinformatics2005; 21: 660–668.
40.
DickhausTStraburgerKSchunkDet al.How to analyze many contingency tables simultaneously in genetic association studies. Stat Appl Genet Mol Biol2012; 11(4): Article12–Article12.
41.
Dialsingh I. False discovery rates when the statistics are discrete. PhD Thesis, The Pennsylvania State University, 2012.
42.
Bancroft T and Nettleton D. Computationally efficient estimation of false discovery rate using sequential permutation p-values. Technical report, Iowa State University, 2009.
