This paper focuses on hypothesis testing in lasso regression, when one is interested in judging statistical significance for the regression coefficients in the regression equation involving a lot of covariates. To get reliable p-values, we propose a new lasso-type estimator relying on the idea of induced smoothing which allows to obtain appropriate covariance matrix and Wald statistic relatively easily. Some simulation experiments reveal that our approach exhibits good performance when contrasted with the recent inferential tools in the lasso framework. Two real data analyses are presented to illustrate the proposed framework in practice.
FahrmeirLKneibTLangS, et al.Regression: models, methods and applications, Berlin: Springer, 2013.
2.
TibshiraniR. Regression shrinkage and selection via the lasso. J R Stat Soc: Series B1996; 58: 267–288.
3.
MeinshausenNBühlmannP. High-dimensional graphs and variable selection with the lasso. Ann Stat2006; 34: 1436–1462.
4.
ZhangCHuangJ. The sparsity and bias of the lasso selection in high-dimensional linear regression. Ann Stat2008; 36: 1567–1594.
5.
BeckATeboulleM. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J Imaging Sci2009; 2: 183–202.
6.
CandesEPlanY. Near ideal model selection by l1 minimization. Ann Stat2009; 37: 2145–2177.
7.
FriedmanJHastieTTibshiraniR. Regularization paths for generalized linear models via coordinate descent. J Stat Softw2010; 33: 1–22.
8.
TibshiraniR. Regression shrinkage and selection via the lasso: a retrospective. J R Stat Soc: Ser B2011; 73: 273–282.
9.
AugugliaroLMineoAWitE. Differential geometric least angle regression: a differential geometric approach to sparse generalized linear models. J R Stat Soc: Ser B2013; 75: 471–498.
10.
TutzGGertheissJ. Regularized regression for categorical data (with discussion). Stat Model2016; 16: 161–200.
11.
MiKYongdaiKKuhwanJ, et al.Logistic lasso regression for the diagnosis of breast cancer using clinical demographic data and the bi-rads lexicon for ultrasonography. Ultrasonography2018; 37: 36–42.
12.
KhanjiCLalondeLBareilC, et al.Lasso regression for the prediction of intermediate outcomes related to cardiovascular disease prevention using the transit quality indicators. Med Care2018; 57: 62–72.
13.
FrostHAmosC. Gene set selection via lasso penalized regression (SLPR). Nucleic Acids Res2017; 45: e114–e114.
14.
LuYZhouYQuW, et al.A lasso regression model for the construction of microrna-target regulatory networks. Bioinformatics2011; 27: 2406–2413.
15.
PrippAStaniiM. Association between biomarkers and clinical characteristics in chronic subdural hematoma patients assessed with lasso regression. PLoS One2017; 12: 1–15.
16.
GuoPZengFHuX, et al.Improved variable selection algorithm using a lasso-type penalty, with an application to assessing hepatitis b infection relevant factors in community residents. PLoS One2015; 10: 1–23.
17.
MusoroJZwindermanAPuhanM, et al.Validation of prediction models based on lasso regression with multiply imputed data. BMC Med Res Method2014; 14: 116–116.
18.
NesterovY. Smooth minimization of non-smooth functions. Math Program2005; 103: 127–152.
19.
BeckATeboulleM. Smoothing and first order methods: a unified framework. SIAM J Optim2012; 22: 557–580.
20.
FanJLiR. Variable selection via nonconcave penalized likelihood and its oracle properties. J Am Stat Assoc2001; 96: 1348–1360.
21.
OsborneMPresnellBTurlachB. On the lasso and its dual. J Comput Graph Stat2000; 9: 319–337.
22.
KyungMGillyJGhoshzM, et al.Penalized regression, standard errors, and Bayesian lassos. Bayesian Anal2010; 5: 1–44.
23.
BeranR. Estimated sampling distributions: the bootstrap and competitors. Ann Stat1982; 10: 212–225.
24.
ChatterjeeALahiriSN. Bootstrapping lasso estimators. J Am Stat Assoc2011; 106: 608–625.
ZhangXChengG. Simultaneous inference for high-dimensional linear models. J Am Stat Assoc2017; 112: 757–768.
27.
DezeureRBühlmannPZhangC. High-dimensional simultaneous inference with the bootstrap. Test2017; 26: 685–719.
28.
LanWZhongPLiR, et al.Testing a single regression coefficient in high dimensional linear models. J Econom2016; 195: 134–168.
29.
MeinshausenNBühlmannP. P-values for high-dimensional regression. J Am Stat Assoc2009; 104: 1671–1681.
30.
MandozziJBühlmannP. Hierarchical testing in the high-dimensional setting with correlated variables. J Am Stat Assoc2016; 111: 331–343.
31.
MinnierJLuTTianxiC. A perturbation method for inference on regularized regression estimates. J Am Stat Assoc2012; 106: 1371–1382.
32.
Van de GeerSBühlmannPRitovY, et al.On asymptotically optimal confidence regions and tests for high-dimensional models. Ann Stat2014; 42: 1166–1202.
33.
ZhangCZhangS. Confidence intervals for low dimensional parameters in high dimensional linear models. J R Stat Soc: Series B2014; 76: 217–242.
34.
JavanmardAMontanariA. Confidence intervals and hypothesis testing for high-dimensional regression. J Mach Learn Res2014; 15: 2869–2909.
35.
Javanmard A and Lee J. A flexible framework for hypothesis testing in high-dimensions. arXiv:1704.07971[math.ST], Apr. 2017.
36.
BelloniAChernozhukovVHansenC. Inference on treatment effects after selection among high-dimensional controls. Rev Econ Stud2014; 81: 608–650.
37.
Yang Y. Statistical inference for high dimensional regression via constrained lasso. arXiv:1704.05098 [math.ST], Apr. 2017.
38.
LockhartRTaylorJTibshiraniR, et al.A significance test for the lasso. Ann Stat2014; 42: 413–468.
39.
LeeJTaylorJ. Exact post model selection inference for marginal screening. Adv Neural Inf Process Syst2014, pp. 136–144.
40.
LeeJSunDSunY, et al.Exact post-selection inference, with application to the lasso. Ann Stat2016; 44: 907–927.
41.
TibshiraniRTaylorJLockhartR, et al.Exact post-selection inference for sequential regression procedures. J Am Stat Assoc2016; 111: 600–620.
42.
BrownBWangY. Standard errors and covariance matrices for smoothed rank estimators. Biometrika2005; 92: 149–158.
43.
RoyallRM. Model robust confidence intervals using maximum likelihood estimators. Int Stat Rev1986; 54: 221–226.
44.
KnightKFuW. Asymptotics for lasso-type estimators. Ann Stat2000; 28: 1356–1378.
45.
PötscherBLeebH. On the distribution of penalized maximum likelihood estimators: The lasso, scad, and thresholding. J Multivar Anal2009; 10: 2065–2082.
46.
ZhouQ. Monte carlo simulation for lasso-type problems by estimator augmentation. J Am Stat Assoc2014; 109: 1495–1516.
47.
OwenAB. A robust hybrid of lasso and ridge regression. Contemporary Math2007; 443: 59–72.
48.
Lockhart R, Taylor J and Tibshirani Rea. covTest: Computes covariance test for adaptive linear modelling, https://CRAN.R-project.org/package=covTest. R package version 1.02 (2013).
StameyTKabalinJMcNealJ, et al.Prostate specific antigen in the diagnosis and treatment of adenocarcinoma of the prostate. II. Radical prostatectomy treated patients. J Urol1989; 141: 1076–1083.
51.
SimoniMLombardiEBertiG, et al.Mould/dampness exposure at home is associated with respiratory disorders in italian children and adolescents: the sidria-2 study. Occup Environ Med2005; 62: 616–622.
52.
MiglioreEBertiGGalassiC, et al.Respiratory symptoms in children living near busy roads and their relationship to vehicular traffic: results of an Italian multicenter study (SIDRIA 2). Environ Health2009; 8: 27–27.
53.
ZouHHastieTTibshiraniR. On the ‘degrees of freedom’ of the lasso. Ann Stat2007; 35: 2173–2192.
54.
ZouH. The adaptive lasso and its oracle properties. J Am Stat Assoc2006; 101: 1418–1429.
55.
FuLWangYBaiZ. Rank regression for analysis of clustered data: a natural induced smoothing approach. Comput Stat Data Anal2010; 54: 1036–1050.
56.
Muggeo VMR. Interval estimation for the breakpoint in segmented regression: a smoothed score-based approach. Aust N Z J Stat 2017; 59: 311–322.
