Abstract
In applications, the traditional estimation procedure generally begins with model selection. Once a specific model is selected, subsequent estimation is conducted under the selected model without consideration of the uncertainty from the selection process. This often leads to the underreporting of variability and too optimistic confidence sets. Model averaging estimation is an alternative to this procedure, which incorporates model uncertainty into the estimation process. In recent years, there has been a rising interest in model averaging from the frequentist perspective, and some important progresses have been made. In this paper, the theory and methods on frequentist model averaging estimation are surveyed. Some future research topics are also discussed.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
J. M. Bates and C. M. J. Granger, The combination of forecasts, Operations Research Quarterly, 1969, 20: 451–468.
D. A. Bessler and J. A. Brandt, Forecasting livestock prices with individual and composite methods, Applied Economics, 1981, 13: 513–522.
R. T. Clemen and R. L. Winkler, Combining economic forecasts, Journal of Business and Economic Statistics, 1986, 4: 39–46.
P. Newbold and C. W. J. Granger, Experience with forecasting univariate time series and the combination of forecasts, Journal of the Royal Statistical Society, Series A, 1974, 2: 131–165.
R. F. Phillips, Composite forecasting: An integrated approach and optimality reconsidered, Journal of Business & Economic Statistics, 1987, 5: 389–395.
M. A. Clyde and E. George, Model uncertainty, Statistical Science, 2004, 19: 81–94.
D. Draper, Assessment and propagation of model uncertainty, Journal of the Royal Statistical Society: Series B, 1995, 57: 45–70.
J. A. Hoeting, D. Madigan, A. E. Raftery, and C. T. Volinsky, Bayesian model averaging: A tutorial, Statistical Science, 1999, 14: 382–417.
J. R. Magnus, O. Powell, and P. Prüfer, A comparison of two averaging techniques with an application to growth empirics, Journal of Econometrics, 2009, in press, doi:10.1016/j.jeconom.2009.07.004.
A. E. Reftery, D. Madigen, and J. A. Hoeting, Bayesian model averaging for regression models, Journal of the American Statistical Association, 1997, 92: 179–191.
N. L. Hjort and G. Claeskens, Frequestist model average estimators, Journal of the American Statistical Association, 2003, 98: 879–899.
S. T. Buckland, K. P. Burnham, and N. H. Augustin, Model selection: An integral part of inference, Biometrics, 1997, 53: 603–618.
J. R. Magnus and J. Durbin, Estimation of regression coefficients of interest when other regression coefficients are of no interest, Econometrica, 1999, 67: 639–643.
Y. Yang, Adaptive regression by mixing, Journal of the American Statistical Association, 2001, 96: 574–586.
K. P. Burnham and D. R. Anderson, Model Selection and Multimodel Inference: A Practical Infromation-Theoretic Approach, Springer, New York, 2002.
G. Leung and A. R. Barron, Infromation theory and mixing least-squares regressions, Information Theory, IEEE Transactions, 2006, 52: 3396–3410.
B. E. Hansen, Least squares model averaging, Econometrica, 2007, 75: 1175–1189.
G. Claeskens, C. Croux, and J. ven Kerckhoven, Variable selection for logit regression using a prediction-focused information criterion, Biometrics, 2006, 62: 972–979.
G. Kapetanios, V. Labhard, and S. Price, Forecasting using Bayesian and information-theoretic model averaging, Journal of Business and Economic Statistics, 2008, 26: 33–41.
M. H. Pesaran, C. Schleicher, and P. Zaffaroni, Model averaging in risk management with an application to futures markets, Journal of Empirical Finance, 2009, 16: 280–305.
A. T. K. Wan and X. Zhang, On the use of model averaging in tourism research, Annals of Tourism Research, 2009, 36: 525–532.
G. Claeskens and N. L. Hjort, Model Selection and Model Averaging, Cambridge University Press, New York, 2008.
G. G. Judge and M. E. Bock, The Statistical Implications of Pre-test and Stein-rule Estimators in Econometrics, North-Holland, Amsterdam, 1978.
N. L. Hjort and G. Claeskens, Rejoinder, Journal of the American Statistical Association, 2003, 98: 938–945.
N. L. Hjort and G. Claeskens, Focused information criteria and model averaging for the Cox hazard regression model, Journal of the American Statistical Association, 2006, 110: 1449–1464.
H. Akaike, Maximum likelihood identification of Gaussian autoregression moving average models, Biometrika, 1973, 60: 255–265.
G. Claeskens and R. J. Carroll, An asymptotic theory for model selection inference in general semiparametric problems, Biometrika, 2007, 94: 249-265.
H. Wang, Frequentist model averaging estimation, Master Thesis, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, 2009.
D. Danilov and J. R. Magnus, On the harm that ignoring pretesting can cause, Journal of Econometrics, 2004, 122: 27–46.
D. Danilov and J. R. Magnus, Forecast accuracy after pretesting with an application to the stock market, Journal of Forecasting, 2004, 23: 251–274.
G. H. Zou, A. T. K. Wan, X. Wu, and T. Chen, Estimation of regression coefficients of interest when other regression coefficient are of no interest: The case of non-normal errors, Statistics & Probability Letters, 2007, 77: 803–810.
K. P. Burnham and D. R. Anderson, Multimodel inferenc uderstanding AIC and BIC in model selection, Sociological Methods & Research, 2004, 33: 261–304.
F. E. Turheimer, R. Hinz, and V. J. Cunningham, On the undecidability among kinetic models: From model selection to model averaging, Journal of Cerbral Blood Flow & Metabolism, 2003, 23: 490–498.
E. J. Wagenmakes and S. Farrell, AIC model selecion using Akaike weights, Psychonomic Bulletin & Review, 2004, 11: 192–196.
C. L. Mallows, Some comments on C p , Technometrics, 1973, 15: 661–675.
B. E. Hansen, Challenges for econometric model selection, Econometric Theory, 2005, 21: 60–68.
P. Kabaila, On variable selection in linear regression, Econometric Theory, 2002, 18: 913–925.
B. E. Hansen, Least squares forecast averaging, Journal of Econometrics, 2008, 146: 342–350.
B. E. Hansen, Averaging estimators for autoregressions with a near unit root, Journal of Econometrics, 2009, forthcoming.
A. T. K. Wan, X. Zhang, and G. Zou, Least squares model combining by Mallows criterion, Technical Report, Department of Management Sciences, City University of Hong Kong, 2009.
B. E. Hansen and J. S. Racine, Jacknife model averaging, Technical Report, Department of Economics, University of Wisconsin-Madison, 2009.
H. Liang, G. Zou, and X. Zhang, Choice of weights for frequentist model average estimators, Technical Report, Department of Biostatistics and Computational Biology, University of Rochester, 2009.
T. H. Kim and H. White, James-Stein type estimators in large samples with application to the least absolute deviations estimator, Journal of the American Statistical Association, 2001, 96: 697–705.
H. Liang, G. Zou, A. T. K. Wan, and X. Zhang, On optimal weight choice in a frequentist model average estimator, Technical Report, Department of Biostatistics and Computational Biology, University of Rochester, 2009.
Y. Yang, Regression with multiple candidate models: Selecting or mixing? Statistica Sinica, 2003, 13: 783–809.
Z. Yuan and Y. Yang, Combining linear regression models: When and how? Journal of the American Statistical Association, 2005, 100: 1202–1204.
M. Schomaker, A. T. K. Wan, and C. Heumann, Frequentist model averaging with missing observations, Computational Statistics and Data Analysis, 2009, in press, doi:10.1016/j.csda.2009.07.023.
J. Fan and R. Li, Variable selection via nonconcave penalized likelihood and its oracle properties, Journal of the American Statistical Association, 2001, 96: 1348–1360.
H. Zou, The adaptive Lasso and its oracle properties, Journal of the American Statistical Association, 2006, 101: 1418–1429.
Author information
Authors and Affiliations
Corresponding author
Additional information
This research is supported by the National Natural Science Foundation of China under Grant Nos. 70625004, 10721101, and 70221001.
Rights and permissions
About this article
Cite this article
Wang, H., Zhang, X. & Zou, G. Frequentist model averaging estimation: a review. J Syst Sci Complex 22, 732–748 (2009). https://doi.org/10.1007/s11424-009-9198-y
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11424-009-9198-y