Abstract
In this paper, we consider the pretest, shrinkage, and penalty estimation procedures for generalized linear mixed models when it is conjectured that some of the regression parameters are restricted to a linear subspace. We develop the statistical properties of the pretest and shrinkage estimation methods, which include asymptotic distributional biases and risks. We show that the pretest and shrinkage estimators have a significantly higher relative efficiency than the classical estimator. Furthermore, we consider the penalty estimator LASSO (Least Absolute Shrinkage and Selection Operator), and numerically compare its relative performance with that of the other estimators. A series of Monte Carlo simulation experiments are conducted with different combinations of inactive predictors, and the performance of each estimator is evaluated in terms of the simulated mean squared error. The study shows that the shrinkage and pretest estimators are comparable to the LASSO estimator when the number of inactive predictors in the model is relatively large. The estimators under consideration are applied to a real data set to illustrate the usefulness of the procedures in practice.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Ahmed, S., Hussein, A. and Sen, P. (2006). Risk comparison of some shrinkage M-estimators in linear models. Journal of Nonparametric Statistics 18, 401–415.
Ahmed, S. E., Hossain, S. and Doksum, K. A. (2012). LASSO And shrinkage estimation in Weibull censored regression models. Journal of Statistical Planning and Inference 142, 1273–1284.
Arnold, T. B. and Tibshirani, R. J. (2016). Efficient implementations of the generalized Lasso dualpath algorithm. J. Comput. Graph. Stat. 25, 1–27.
Chen, F. and Nkurunziza, S. (2015). A class of Stein-rules in multivariate regression model with structural changes. Scand. J. Stat. 43, 83–102.
Davis, K. A., Park, C. G. and Sinha, S. K. (2012). Testing for generalized linear mixed models with cluster correlated data under linear inequality constraints. Can. J. Stat. 40, 243–258.
Diggle, P. J., Heagerty, P., Liang, K -Y. and Zeger, S. L. (2002). Analysis of Longitudinal Data, 2nd edn. Oxford University Press, Oxford.
Fahrmeir, L. and Kneib, T. (2011). Bayesian smoothing and regression for longitudinal, spatial and event history data. Oxford University Press, Oxford.
Fallahpour, S., Ahmed, S. E. and Doksum, K. A. (2012). ℓ 1 penalty and shrinkage estimation in partially linear models with random coefficient autoregressive errors. Appl. Stoch. Model. Bus. Ind. 28, 236–250.
Fan, J. and Lv, J. (2010). A selective overview of variable selection in high dimensional feature space. Stat. Sin. 20, 101–148.
Goeman, J. J. (2010). Penalized estimation in the Cox proportional hazards model. Biom. J. 52, 70–84.
Groll, A. and Tutz, G. (2014). Variable selection for generalized linear mixed models by l 1 penalized estimation. Stat. Comput. 24, 137–154.
Hossain, S. and Ahmed, S. E. (2014). Shrinkage estimation and selection for a logistic regression model. CRM Proceedings-Contemporary Mathematics 622, 159–176.
Hossain, S., Ahmed, S. E. and Doksum, K. A. (2015). Shrinkage, pretest, and penalty estimators in generalized linear models. Stat. Methodology 24, 52–68.
Jamshidian, M. (2004). On algorithms for restricted maximum likelihood estimation. Comput. Stat. Data Anal. 45, 137–157.
Judge, G. G. and Bock, M. E. (1978). The statistical implication of pretest and Stein-Rule estimators in econometrics. North-Holland, Amsterdam.
Li, Q. and Shao, J. (2015). Regularizing LASSO: A consistent variable selection method. Stat. Sin. 25, 975–992.
Lian, H. (2012). Shrinkage estimation for identification of linear components in additive models. Statistics & Probability Letters 82, 225–231.
Louis, T. A. (1982). Finding the observed information matrix when using the EM algorithm. J. R. Stat. Soc. Ser. B Methodol. 44, 226–233.
Lu, X. and Su, L. (2016). Shrinkage estimation of dynamic panel data models with interactive fixed effects. J. Econ. 190, 148–175.
Luenberger, D. G. and Ye, Y. (2008). Linear and nonlinear programming. Springer Science & Business Media, New York.
McCulloch, C. E., Searle, S. R. and Neuhaus, J. M. (2008). Generalized, Linear, and Mixed Models. Wiley, Hoboken.
Nelder, J. A. and Wedderburn, R. W. M. (1972). Generalized linear models. J. R. Stat. Soc. Ser. A 135, 370–384.
Schelldorfer, J., Meier, L. and Bühlmann, P. (2014). GLMMLasso: an algorithm for high-dimensional generalized linear mixed models using l 1-penalization. J. Comput. Graph. Stat. 23, 460–477.
Simon, N., Friedman, J., Hastie, T. and Tibshirani, R. (2014). A sparse-group LASSO. J. Comput. Graph. Stat. 22, 231–245.
Sommer, A., Katz, J. and Tarwotjo, I. (1984). Increased risk of respiratory infection and diarrhea in children with pre-existing mild vitamin a deficiency. Am. J. Clin. Nutr. 40, 1090–1095.
Thomson, T., Hossain, S. and Ghahramani, M. (2014). Application of shrinkage estimation in linear regression models with autoregressive errors. J. Stat. Comput. Simul. 85, 3335–3351.
Thomson, T., Hossain, S. and Ghahramani, M. (2015). Efficient estimation for time series following generalized linear models 58, 493–513.
Tibshirani, R. (1996). Regression shrinkage and selection via the LASSO. J. R. Stat. Soc. Ser. B 58, 267–288.
Wu, C. O. and Chiang, C. (2000). Kernel smoothing on varying coefficient models with longitudinal dependent variable. Stat. Sin. 10, 433–456.
Zhang, T. and Zou, H. (2014). Sparse precision matrix estimation via LASSO penalized D-trace loss. Biometrika 101, 103–120.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Thomson, T., Hossain, S. Efficient Shrinkage for Generalized Linear Mixed Models Under Linear Restrictions. Sankhya A 80, 385–410 (2018). https://doi.org/10.1007/s13171-017-0122-6
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13171-017-0122-6
Keywords
- Asymptotic distributional bias and risk
- Generalized linear mixed models
- LASSO
- Likelihood ratio test
- Monte Carlo simulation
- Shrinkage and pretest estimators