Abstract
In this paper, we consider the issue of variable selection in partial linear single-index models under the assumption that the vector of regression coefficients is sparse. We apply penalized spline to estimate the nonparametric function and SCAD penalty to achieve sparse estimates of regression parameters in both the linear and single-index parts of the model. Under some mild conditions, it is shown that the penalized estimators have oracle property, in the sense that it is asymptotically normal with the same mean and covariance that they would have if zero coefficients are known in advance. Our model owns a least square representation, therefore standard least square programming algorithms can be implemented without extra programming efforts. In the meantime, parametric estimation, variable selection and nonparametric estimation can be realized in one step, which incredibly increases computational stability. The finite sample performance of the penalized estimators is evaluated through Monte Carlo studies and illustrated with a real data set.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Bai, J. Testing for parameter constancy in linear regressions: an empirical distribution function approach. Econometrica, 64: 597–622 (1996)
Bai, Y., Fung, W.K., Zhu, Z.Y. Penalized quadratic inference functions for single-index models with longitudinal data. Journal of Multivariate Analysis, 100: 152–161 (2009)
Carroll, R.J., Fan, J., Gijbels, I., Wand, M.P. Generalized partially linear single-index models. Journal of the American Statistical Association, 92: 477–489 (1997)
Chen, H. Convergence rates for parametric components in a partly linear model. The Annals of Statistics, 16: 136–146 (1988)
Chen, C.H., Li, K.C. Can SIR be as popular as multiple linear regression. Statistica Sinica, 8: 289–316 (1998)
Chen, J., Gao, J., Li, D. Estimation in partially linear single-index panel data models with fixed effects. Journal of Business and Economic Statistics, 31: 315–330 (2013)
Fan, J., Li, R. Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96: 1348–1360 (2001)
Hardle, W., Gao, J., Liang, H. Partially Linear Models. Springer-Verlag, New York, 2000
Harrison, D., Rubinfeld, D.L. Hedonic housing prices and the demand for clean air. Environmental Economics Management, 5: 81–102 (1978)
He, X., Fung, W.K., Zhu, Z.Y. Robust Estimation in generalized partial linear models for clustered data. Journal of the American Statistical Association, 100: 1176–1184 (2005)
He, X., Zhu, Z.Y., Fung, W.K. Estimation in a semiparametric model for longitudinal data with unspecified dependence structure. Biometrika, 89: 579–590 (2002)
Heckman, N.E. Spline smoothing in a partly linear model. Journal of the Royal Statistical Society, Series B, 48: 244–248 (1986)
Kong, E., Xia, Y. Variable selection for the single-index model. Biometrika, 94: 217–229 (2006)
Lai P., Tian, Y., Lian, H. Estimation and variable selection for generalised partially linear single-index models. Journal of Nonparametric Statistics, 26: 171–185 (2014)
Lai, P., Wang Q., Zhou, X. Variable selection and semiparametric efficient estimation for the heteroscedastic partially linear single-index model. Computational Statistics and Data Analysis, 70: 241–256 (2014)
Li, K.C. Sliced Inverse Regression for Dimension Reduction. Journal of the American Statistical Association, 86: 316–342 (1991)
Liang, H., Liu, X., Li, R., Tsai, C. Estimation and testing for partially linear single-index models. The Annals of Statistics, 38: 3811–3836 (2010)
Ma, Y., Zhu, L. Doubly robust and efficient estimators for heteroscedastic partially linear single-index model allowing high-dimensional covariates. Journal of the Royal Statistical Society, Series B, 75: 305–322 (2013)
Ni, X., Zhang, H., Zhang, D. Automatic model selection for partially linear models. Journal of Multivariate Analysis, 100: 2100–2111 (2009)
Powell, J.L., Stock, J.H., Stoker, T.M. Semiparametric estimation of index coefficient. Econometrica, 51: 1403–1430 (1989)
Ruppert, D. Selecting the Number of Knots for Penalized Splines. Journal of Computational and Graphical Statistics, 4: 735–757 (2002)
Ruppert, D., Carroll, R. Penalized Regression Splines. Working paper, Cornell University, School of Operations Research and Industrial Engineering, available at www.orie.cornell.edu/1davidr/papers, 1997
Stoker, T.M. Consistent estimation of scaled coefficients. Econometrica, 54: 1461–1481 (1986)
Tibshirani, R. Regression shrinkage and selection via the LASSO. Journal of the Royal Statistical Society, Series B, 58: 267–288 (1996)
Wang, J.L., Xue, L.G., Zhu, L.X., Chong, Y.S. Estimation for a partial-linear single-index model. The Annals of Statistics, 38: 246–274 (2010)
Welsh, A.H., Lin, X., Carroll, R.J. Marginal longitudinal nonparametric regression: locality and efficiency of spline and kernel methods. Journal of the American Statistical Association, 97: 482–493 (2002)
Xie, H., Huang, J. SCAD-penalized Regression in High-dimensional Partially Linear Models. The Annals of Statistics, 37: 673–696 (2009)
Yu, Y., Ruppert, D. Penalized spline estimation for partially linear single-index models. Journal of the American Statistical Association, 97: 1042–1054 (2002)
Zhang, J., Wang X., Yu, Y., Gai, Y. Estimation and variable selection in partial linear single index models with error-prone linear covariates. Statistics, 48: 1048–1070 (2014)
Zhang, J., Yu, Y., Zhu, L., Liang, H. Partial linear single index models with distortion measurement errors, Annals of the Institute of Statistical Mathematics, 65: 237–267 (2013)
Zou, H. The adaptive lasso and its oracle properties. Journal of the American Statistical Association, 101: 1418–1429 (2006)
Author information
Authors and Affiliations
Corresponding author
Additional information
Supported by the National Natural Science Foundation of China (No. 11671096).
Rights and permissions
About this article
Cite this article
Wang, W., Zhu, Zy. Variable selection for the partial linear single-index model. Acta Math. Appl. Sin. Engl. Ser. 33, 373–388 (2017). https://doi.org/10.1007/s10255-017-0666-1
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10255-017-0666-1