Abstract
In this paper we study the asymptotic properties of the adaptive Lasso estimate in high-dimensional sparse linear regression models with heteroscedastic errors. It is demonstrated that model selection properties and asymptotic normality of the selected parameters remain valid but with a suboptimal asymptotic variance. A weighted adaptive Lasso estimate is introduced and investigated. In particular, it is shown that the new estimate performs consistent model selection and that linear combinations of the estimates corresponding to the non-vanishing components are asymptotically normally distributed with a smaller variance than those obtained by the “classical” adaptive Lasso. The results are illustrated in a data example and by means of a small simulation study.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
R. Bhatia, Matrix Analysis (Springer, New York, 1997).
P. Craven and G. Wahba, “Smoothing Noisy Data with Spline Function: Estimating the Correct Degree of Smoothing by the Method of Generalized Cross Validation”, Numerische Mathematik 31, 337–403 (1979).
H. Dette and A. Munk, “Testing Heteroscedasticity in Nonparametric Regression”, J. Roy. Statist. Soc. Ser. B 60, 693–708 (1998).
B. Efron, T. Hastie, and R. Tibshirani, “Least Angle Regression (with Discussion)”, Ann. Statist. 32, 407–451 (2004).
J. Fan and R. Li, “Variable Selection via Nonconcave Penalized Likelihood and Its Oracle Properties”, J. Amer. Statist. Assoc. 96, 1348–1360 (2001).
J. Fan and J. Lv, “A Selective Overview of Variable Selection in High-Dimensional Feature Space”, Statistica Sinica 20, 101–148 (2010).
J. Fan and H. Peng, “Nonconcave Penalized Likelihood with a Diverging Number of Parameters”, Ann. Statist. 32, 928–961 (2004).
I. E. Frank and J. H. Friedman, “A Statistical View of Some Chemometrics Regression Tools (with Discussion)”, Technometrics 35, 109–148 (1993).
A. Hörl and R. Kennard, “Ridge Regression: Biased Estimation for Nonorthogonal Problems”, Technometrics, 12, 55–67 (1970).
J. Huang, J. L. Horowitz, and S. Ma, “Asymptotic Properties of Bridge Estimators in Sparse High Dimensional Regression Models”, Ann. Statist. 36, 587–613 (2008).
J. Huang, S. Ma, and C. Zhang, Adaptive Lasso for Sparse High-Dimensional Regression Models, Technical Report, Univ. of Iowa (2006).
J. Huang, S. Ma, and C. Zhang, “Adaptive Lasso for Sparse High-Dimensional Regression Models”, Statistica Sinica 18, 1603–1618 (2008).
Y. Kim, H. Choi, and H.-S. Oh, “Smoothly Clipped Absolute Deviation on High Dimensions”, J. Amer. Statist. Assoc. 103, 1665–1673 (2008).
K. Knight and W. Fu, (2000). “Asymptotics for Lasso-type Estimators”, Ann. Statist, 28, 1356–1378 (2000).
H. Leeb and B. M. Pötscher, “Sparse Estimators and the Oracle Property, or the Return of Hodges’ Estimator”, J. Econometrics 142, 201–211 (2008).
B.M. Pötscher and U. Schneider, “Distributional Results for Thresholding Estimators in High-Dimensional Gaussian Regression Models”, Electronic J. Statist. 5, 1876–1934 (2011).
R Development Core Team, R: A Language and Environment for Statistical Computing (R Foundation for Statistical Computing, Vienna, Austria, 2009) ISBN 3-900051-07-0.
R. Tibshirani, “Regression Shrinkage and Selection via the Lasso”, J. Roy. Statist. Soc. B 58, 267–288 (1996).
A.W. van der Vaart and J. A. Wellner, Weak Convergence and Empirical Processes, in Springer Series in Statist. (Springer, New York, 1996).
J. Wagener and H. Dette, “Bridge Estimators and the Adaptive Lasso under Heteroscedasticity”, Math. Methods Statist. 21, 109–126 (2012).
M. J. Wainwright, “Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using l 1-Constrained Quadratic Programming (Lasso)”, IEEE Trans. Inform. Theory 55, 2183–2202 (2009).
H. Zou, “The Adaptive Lasso and Its Oracle Properties”, J. Amer. Statist. Assoc. 101, 1418–1429 (2006).
P. Zhao and B. Yu, “On Model Selection Consistency of Lasso”, J. Machine Learning Research 7, 2541–2563 (2006).
Author information
Authors and Affiliations
Corresponding author
About this article
Cite this article
Wagener, J., Dette, H. The adaptive lasso in high-dimensional sparse heteroscedastic models. Math. Meth. Stat. 22, 137–154 (2013). https://doi.org/10.3103/S106653071302004X
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.3103/S106653071302004X