A robust variable screening procedure for ultra-high dimensional data

Abstract

Variable selection in ultra-high dimensional regression problems has become an important issue. In such situations, penalized regression models may face computational problems and some pre-screening of the variables may be necessary. A number of procedures for such pre-screening has been developed; among them the Sure Independence Screening (SIS) enjoys some popularity. However, SIS is vulnerable to outliers in the data, and in particular in small samples this may lead to faulty inference. In this paper, we develop a new robust screening procedure. We build on the density power divergence (DPD) estimation approach and introduce DPD-SIS and its extension iterative DPD-SIS. We illustrate the behavior of the methods through extensive simulation studies and show that they are superior to both the original SIS and other robust methods when there are outliers in the data. Finally, we illustrate its use in a study on regulation of lipid metabolism.

Keywords

Variable selection NP dimensionality independence screening minimum density power divergence estimator influence function gene selection

Get full access to this article

View all access options for this article.

References

Segal

Dahlquist

Conklin

BR.

Regression approaches for microarray data analysis. J Comput Bio 2003; 10:961–980.

Lusa

Korn

EL and

McShane

LM.

A class comparison method with filtering-enhanced variable selection for high-dimensional data sets. Stat Med 2008; 27:5834–5849.

Wang

YG.

Variable selection in rank regression for analyzing longitudinal data. Stat Meth Med Res 2018; 27:2447–2458.

Jung

Zhang

Transformed low-rank ANOVA models for high-dimensional variable selection. Stat Meth Med Res 2019; 28:1230–1246.

Tibshirani

Regression shrinkage and selection via the Lasso. J R Stat Soc B 1996; 58:267–288.

Zhang

Huang

The sparsity and bias of the Lasso selection in high-dimensional linear regression. Ann Stat 2008; 36:1567–1594.

Zou

The adaptive Lasso and its oracle properties. J Am Stat Assoc 2006; 101:1418–1429.

Fan

Variable selection via nonconcave penalized likelihood and its oracle properties. J Am Stat Assoc 2001; 96:1348–1360.

Zhang

CH.

Nearly unbiased variable selection under minimax concave penalty. Ann Stat 2010; 38:894–942.

10.

Fan

Sure independence screening for ultrahigh dimensional feature space. J R Stat Soc B 2008; 70:849–911.

11.

Fan

Song

Sure independence screening in generalized linear models with NP-dimensionality. Ann Stat 2010; 38:3567–3604.

12.

Zhao

Principled sure independence screening for Cox models with ultra-high-dimensional covariates. J Mult Anal 2012; 105:397–411.

13.

Barut

Fan

Verhasselt

Conditional sure independence screening. J Am Stat Assoc 2016; 111:1266–1277.

14.

Zhou

Liu

Hao

, et al. Model-free conditional feature screening with exposure variables. Stat Interf 2019; 12:239–251.

15.

Zhang

, et al. Robust feature screening for elliptical copula regression model. J Mult Anal 2019; 173:568–582.

16.

Wang

Van Aelst

Robust variable screening for regression using factor profiling.

Stat Anal Data Mining 2019; 12:70–87.

17.

Edelmann

Hummel

Hielscher

, et al. Marginal variable screening for survival endpoints. Biometr J 2020; 62:610–626.

18.

Lai

Chen

Zhang

, et al. Robust model-free feature screening for ultrahigh dimensional surrogate data. J Stat Comput Simul 2020; 90:550–569.

19.

Song

Chen

Lai

Conditional distance correlation screening for sparse ultrahigh-dimensional models. Appl Math Model 2020; 81:232–252.

20.

Wang

Guan

Weighted mean squared deviation feature screening for binary features. Entropy 2020; 22:335.

21.

Basu

Harris

Hjort

, et al. Robust and efficient estimation by minimising a density power divergence. Biometrika 1998; 85:549–559.

22.

Basu

Shioya

Park

Statistical inference: the minimum distance approach. Boca Raton, FL: Chapman & Hall, 2011.

23.

Zang

Zhao

Zhang

, et al. Inferring gene regulatory relationships with a high-dimensional robust approach. Genet Epidemiol 2017; 41:437–454.

24.

Ghosh

Majumdar

Ultrahigh-dimensional robust and efficient sparse regression using non-concave penalized density power divergence. IEEE Trans Info Theor 2020; 66:7812–7827.

25.

Hall

Miller

Using generalized correlation to effect variable selection in very high dimensional problems. J Comput Graphical Stat 2009; 18:533–550.

26.

Peng

Zhang

, et al. Robust rank correlation based screening. Ann Stat 2012; 40:1846–1877.

27.

Zhong

Zhu

Feature screening via distance correlation learning. J Am Stat Assoc 2012; 107:1129–1139.

28.

Xiong

Some notes on robust sure independence screening. J Appl Stat 2014; 41:2092–2102.

29.

Wang

Zheng

, et al. A robust variable screening method for high-dimensional data. J Appl Stat 2017; 44:1839–1855.

30.

Ghosh

Basu

Robust estimation for independent non-homogeneous observations using density power divergence with applications to linear regression. Electron J Stat 2013; 7:2420–2456.

31.

Saldana

Feng

SIS: an R package for sure independence screening in ultrahigh-dimensional statistical models. J Stat Software 2018; 83:1–25.

32.

Khan

Van Aelst

Zamar

RH.

Robust linear model selection based on least angle regression. J Am Stat Assoc 2007; 102:1289–1299.

33.

Gnanadesikan

Kettenring

JR.

Robust estimates, residuals, and outlier detection with multiresponse data. Biometrics 1972; 28:81–124.

34.

Gather

Guddat

Comment on “Sure independence screening for ultrahigh dimensional feature space” by Fan JQ and Lv

J. J R Stat Soc B 2008; 70:893–895.

35.

Szekely

Rizzo

Bakirov

NK.

Measuring and testing dependence by correlation of distances. Ann Stat 2007; 35:2769–2794.

36.

Ghosh

Basu

Robust estimation for non-homogeneous data and the selection of the optimal tuning parameter: the DPD approach

J Appl Stat 2015; 42:2056–2072.

37.

Ottestad

Vogt

Retterstol

, et al. Oxidised fish oil does not influence established markers of oxidative stress in healthy human subjects: a randomised controlled trial. Br J Nutr 2012; 108:315–326.

38.

Hastie

Tibshirani

Wainwright

Statistical learning with sparsity: the Lasso and generalizations. Boca Raton, FL: CRC Press, 2015.

39.

Ghosh

Ponzi

Sandanger

, et al. Robust sure independence screening for non-polynomial dimensional generalized linear models. 2021; arXiv:2005.12068v2.

40.

Meinshausen

Buhlmann

Stability selection. J R Stat Soc B 2010; 72:417–473.

41.

Shah

Samworth

RJ.

Variable selection with error control: another look at stability selection. J R Stat Soc B 2013; 75:55–80.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.10 MB

0.00 MB

0.30 MB