Skip to main content
SpringerLink
Log in

Automatic Smoothing Parameter Selection: A Survey

  • Conference paper
Semiparametric and Nonparametric Econometrics

Part of the book series: Studies in Empirical Economics ((STUDEMP))

Abstract

This is a survey of recent developments in smoothing parameter selection for curve estimation. The first goal of this paper is to provide an introduction to the methods available, with discussion at both a practical and also a nontechnical level, including comparison of methods. The second goal is to provide access to the literature, especially on smoothing parameter selection, but also on curve estimation in general. The two main settings considered here are nonparametric regression and probability density estimation, although the points made apply to other settings as well. These points also apply to many different estimators, although the focus is on kernel estimators, because they are the most easily understood and motivated, and have been at the heart of the development in the field.

Research partially supported by NSF Grant DMS-8701201.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  • Bean SJ Tsokos CP (1980) Developments in nonparametric density estimation. International Statistical Review 48: 267–287

    Article  Google Scholar 

  • Bhattacharya PK, Mack KP (1987) Weak convergence of k-NN density and regression estimators with varying k and applications. Annals of Statistics 15: 976–994

    Article  Google Scholar 

  • Bierens HJ (1987) Kernel estimation of regression function. In: Bewley TF (ed) Advances in econometrics. Canbridge University Press, New York; pp 99–144

    Google Scholar 

  • Bowman A (1984) An alternative method of cross-validation for the smoothing of density estimates. Biometrika 65: 521–528

    Google Scholar 

  • Burman P (1985) A data dependent approach to density estimation. Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete 69: 609–628

    Article  Google Scholar 

  • Burman P (1988) Estimation of the optimal transformations using v-fold cross-validation and repeated learning testing methods. Unpublished manuscript

    Google Scholar 

  • Chow YS, Geman S, Wu LD (1983) Consistent cross-validated density estimation. Annals of Statistics 11: 25–38

    Article  Google Scholar 

  • Clark RM (1975) A calibration curve for radio carbon dates. Antiquity 49: 251–266

    Google Scholar 

  • Collomb G (1981) Estimation non paramétrique de la regression: revue. International Statistical Review 49: 75–93

    Article  Google Scholar 

  • Collomb G (1985) Nonparametric regressio: an up-to-date bibliography. Statistics 16: 309–324

    Article  Google Scholar 

  • Craven P, Wahba G (1979) Smoothing noisy data with spline functions. Numerische Mathematik 31: 377–403

    Article  Google Scholar 

  • Devroye L, Györfi L (1984) Nonparametric density estimation: The L 1 view. Wiley, New York

    Google Scholar 

  • Devroye L (1987) A course in density estimation. Birkhauser, Boston

    Google Scholar 

  • Dodge Y (1986) Some difficulties involving nonparametric estimation of a density function. Journal of Official Statistics 2: 193–202

    Google Scholar 

  • Duin RPW (1976) On the choice of smoothing parameters of Parzen estimators of probability density functions. IEEE Transactions on Computers C-25: 1175–1179

    Article  Google Scholar 

  • Eagleson GK, Buckley MJ (1987) Estimating the variance in nonparametric regression. Unpublished manuscript

    Google Scholar 

  • Eubank R (1988) Spline smoothing and nonparametric regression. Wiley, New York

    Google Scholar 

  • Ferguson TS (1967) Mathematical statistics, a decision theoretic approach. Academic Press, New York

    Google Scholar 

  • Fryer MJ (1977) A review of some non-parametric methods of density estimation. Journal of the Instiute of Mathematics and its Applications 20: 335–354

    Article  Google Scholar 

  • Gasser T, Sroka L, Jennen C (1986) Residuals variance and residual pattern in nonlinear regression. Biometrika 73: 625–633

    Article  Google Scholar 

  • Habbema JDF, Hermans J, van den Broek K (1984) A stepwise discrimination analysis program using density estimation. Compstat 1974: Proceedings in Computational Statistics. Physica Verlag, Vienna, pp 101–110

    Google Scholar 

  • Härdie W (1988) Applied nonparametric regression

    Google Scholar 

  • Härdie W, Marron JS (1985a) Optimal bandwidth selection in nonparametric regression function estimation. Annals of Statistics 12: 1465–1481

    Article  Google Scholar 

  • Härdie W, Marron JS (1985b) Asymptotic nonequivalence of some bandwidth selectors in nonparametric regression. Biometrika 72: 481–484

    Google Scholar 

  • Härdie W, Hall P, Marron JS (1988) How far are automatically chosen regression smoothers from their optimum? Journal of the American Statistical Association 83: 86–101, with discussion

    Google Scholar 

  • Härdie W, Marron JS, Wand MP (1988) Bandwidth choice for density derivatives. Unpublished manuscript

    Google Scholar 

  • Hall P (1982) Cross-validation in density estimation. Biometrika 69: 383–390

    Article  Google Scholar 

  • Hall P (1983) Large sample optimality of least square cross-validation in density estimation, Annals of Statistics 11: 1156–1174

    Google Scholar 

  • Hall P (1985) Asymptotic theory of minimum integrated square error for multivariate density estimation. Proceedings of the Sixth International Symposium on Multivariate Analysis at Pittsburgh, 25–29

    Google Scholar 

  • Hall P (1987a) On the estimation of probability densities using compactly supported kernels. Journal of Multivariate Analysis 23: 131–158

    Article  Google Scholar 

  • Hall P (1987b) On Kullback-Leibler loss and density estimation. Annals of Statistics 15: 1491–1519

    Article  Google Scholar 

  • Hall P, Marron JS (1987a) Extent to which least-squares cross-validation minimises integrated square error in nonparametric density estimation. Probability Theory and Related Fields 74: 567–581

    Article  Google Scholar 

  • Hall P, Marron JS (1987b) On the amount of noise inherent in bandwidth selection for a kernel density estimator. Annals of Statistics 15: 163–181

    Article  Google Scholar 

  • Hall P Marron JS (1987c) Estimation of integrated squared density derivatives. Statistics and Probability Letters 6: 109–115

    Article  Google Scholar 

  • Hall P, Wand M (1988) On the minimization of absolute distance in kernel density estimation. To appear in Statistics and Probability Letters

    Google Scholar 

  • Hall P, Wand M (1989) Minimizing L 1 distance in nonparametric density estimation. To appear in Journal of Multivariate Analysis

    Google Scholar 

  • Kappenman RF (1987) A nonparametric data based univariate function estimate. Computational Statistics and Data Analysis 5: 17

    Article  Google Scholar 

  • Kendall MS (1976) Time Series. Griffin, London

    Google Scholar 

  • Krieger AM, Pickands J (1981) Weak convergence and efficient density estimation at a point. Annals of Statistics 9: 1066–1078

    Article  Google Scholar 

  • Li KC, Hwang J (1984) The data smoothing aspects of Stein estimates. Annals of Statistics 12: 887–897

    Article  Google Scholar 

  • Li KC (1985) From Stein’s unbiased risk estimates to the method of generalized cross-validation. Annals of Statistics 13: 1352–1377

    Article  Google Scholar 

  • Li KC (1987) Asymptotic optimality for C p, CL, cross-validation and generalized cross-validation: discrete index set. Annals of Statistics 15: 958–975

    Article  Google Scholar 

  • Mallows CL (1973) Some comments on Cp, Technometrics 15: 661–675

    Google Scholar 

  • Marron JS (1985) An asymptotically efficient solution to the bandwidth problem of kernel density estimation. Annals of Statistics 13: 1011–1023

    Article  Google Scholar 

  • Marron JS (1986) Will the art of smoothing ever become a science? Marron JS (ed) Function estimates. American Mathematical Society Series: Contemporary Mathematics 9: 169–178

    Google Scholar 

  • Marron JS (1987a) A comparison of cross-validation techniques in density estimation. Annals of Statistics 15: 152–162

    Article  Google Scholar 

  • Marron JS (1987b) What does optimal bandwidth selection mean for nonparametric regression estimation. Dodge Y (ed) Statistical data analysis based on the L 1 norm and related methods. North Holland, Amsterdam

    Google Scholar 

  • Marron JS (1987c) Partitioned cross-validation. North Carolina Institute of Statistics, Mimeo Series #1721

    Google Scholar 

  • Marron JS (1988) Improvement of a data based bandwidth selector. Unpublished manuscript

    Google Scholar 

  • Marron JS, Padgett WJ (1987) Asymptotically optimal bandwidth selection for kernel density estimators from randomly right-censored samples. Annals of Statistics 15: 1520–1535

    Article  Google Scholar 

  • Mielniczuk J, Vieu P (1988) Asymptotic suboptimality of one method of cross-validatory bandwidth choice in density estimation. Unpublished manuscript

    Google Scholar 

  • Müller HG (1985) Empirical bandwidth choice for nonparametric kernel regression by means of pilot estimators. Statistics and Decisions, Supplement Issue No. 2: 193–206

    Google Scholar 

  • Müller HG, Stadtmüller U (1987) Variable bandwidth kernel estimators of regression curves. Annals of Statistics 15: 182–201

    Article  Google Scholar 

  • Müller HG, Stadtmüller U, Schmitt T (1987) Bandwidth choice and confidence intervals for derivatives of noisy data. Biometrika 74: 743–749

    Google Scholar 

  • Nadaraya EA (1964) On estimating regression. Theory of Probability and its Application 9: 141–142

    Article  Google Scholar 

  • Nolan D, Pollard D (1987) U-processes: rates of convergence. Annals of Statistics 15: 780–799

    Article  Google Scholar 

  • Parzen E (1962) On estimation of a probability density function and mode. Annals of Mathematical Statistics 33: 1065–1076

    Article  Google Scholar 

  • Prakasa Rao BLS (1983) Nonparametric functional estimation. Academic Press, New York

    Google Scholar 

  • Rice J (1984) Bandwidth choice for nonparametric regression. Annals of Statistics 12: 1215–1230

    Article  Google Scholar 

  • Rice J (1986) Bandwidth choice for differentiation. Journal of Multivariate Analysis 19: 251–264

    Article  Google Scholar 

  • Rosenblatt M (1956) Remarks on some non-parametric estimates of a density function. Annals of Mathematical Statistics 27: 832–837

    Article  Google Scholar 

  • Rosenblatt M (1971) Curve estimates. Annals of Mathematical Statistics 42: 1815–1842

    Article  Google Scholar 

  • Rudemo M (1982a) Empirical choice of histograms and kernel density estimators. Scandanavian Journal of Statistics 9: 65–78

    Google Scholar 

  • Rudemo M (1982b) Consistent choice of linear smoothing methods. Report 82-1, Department of Mathematics, Royal Danish Agricultural and Veterinary University, Copenhagen

    Google Scholar 

  • Schuster EA, Gregory CG (1981) On the nonconsistency of maximum likelihood nonparametric density estimators. Eddy WF (ed) Computer Science and Statistics: Proceedings of the 13th Symposium on the Interface. Springer, New York, pp 295–298

    Chapter  Google Scholar 

  • Scott DW (1985) Handouts for ASA short course in density estimation. Rice University Technical Report 776-331-86-2

    Google Scholar 

  • Scott DW (1986) Choosing smoothing parameters for density estimators.In: Allen DM (ed) Computer Science and Statistics: The Interface, pp 225–229

    Google Scholar 

  • Scott DW (1988) Discussion of Härdie W, Hall P, Marron JS, How far are automatically chosen regression smoothers from their optimum? To appear Journal of the American Statistical Association

    Google Scholar 

  • Scott DW, Factor LE (1981) Monte Carlo study of three data-based nonparametric probability density estimators. Journal of the American Statistical Association 76: 9–15

    Google Scholar 

  • Scott DW, Tapia RA, Thompson JR (1977) Kernel density estimation revisited. Nonlinear Analysis, Theory, Methods and Applications 1: 339–372

    Article  Google Scholar 

  • Scott DW, Terrell GR (1987) Biased and unbiased cross-validation in density estimation. Journal of the American Statistical Association 82: 1131–1146

    Google Scholar 

  • Sheather SJ (1983) A data-based algorithm for choosing the window width when estimating the density at a point. Computational Statistics and Data Analysis 1: 229–238

    Article  Google Scholar 

  • Sheather SJ (1986) An improved data-based algorithm for choosing the window width when estimating the density at a point Computational Statistics and Data Analysis 4: 61–65

    Google Scholar 

  • Silverman BW (1985) Some aspects of the spline smoothing approach to nonparametric regression curve fitting (with discussion). Journal of the Royal Statistical Society, Series B 46: 1–52

    Google Scholar 

  • Silverman BW (1986) Density estimation for statistics and data analysis. Chapman and Hall, New York

    Google Scholar 

  • Stone CJ (1984) An asymptotically optimal window selection rule for kernel density estimates. Annals of Statistics 12: 1285–1297

    Article  Google Scholar 

  • Stone CJ (1985) An asymptotically optimal histogram selection rule. Proceedings of the Berkeley Symposium in Honor of Jerzy Neyman and Jack Keifer

    Google Scholar 

  • Stone M (1974) Cross-validator y choice and assessment of statistical predictions. Journal of the Royal Statistical Society, Series B 36: 111–147

    Google Scholar 

  • Tapia RA, Thompson JR (1978) Nonparametric probability density estimation. The Johns Hopkins University Press, Baltimore

    Google Scholar 

  • Tarter ME, Kronmal RA (1976) An introduction to the implementation and theory of nonparametric density estimation. The American Statistician 30: 105–112

    Google Scholar 

  • Terrell GR, Scott DW (1985) Oversmoothed density estimates. Journal of the American Statistical Association 80: 209–214

    Google Scholar 

  • Tsybakov AB (1987) On the choice of the bandwidth in kernel nonparametric regression. Theory of Probability and Its Applications 32: 142–148

    Article  Google Scholar 

  • Ullah A (1987) Nonparametric estimation of econometric functionals. Research Report 18., University of Western Ontario, to appear in Canadian Journal of Economics

    Google Scholar 

  • Wahba G, Wold S (1975) A completely automatic fench curve: fitting spline functions by cross-validation. Communications in Statistics 4: 1–17

    Article  Google Scholar 

  • Watson GS (1964) Smooth regression analysis. Sankhya, series A 26: 359–372

    Google Scholar 

  • Wegman EJ (1972) Nonparametric probability density estimation: I. a summary of the available methods. Technometrics 14: 533–546

    Google Scholar 

  • Wertz W (1978) Statistical density estimation: a survey. Angewandte Statistique und Okonometrie 13, Vandenhoeck und Ruprecht

    Google Scholar 

  • Wertz W, Schneider B (1979) Statistical density estimation: a bibliography. International Statistical Review 49: 75–93

    Google Scholar 

  • Woodroofe M (1970) On choosing a delta sequence. Annals of Mathematical Statistics 41: 1665–1671

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Statistics, University of North Carolina, USA

    Dr. J. S. Marron

Authors
  1. Dr. J. S. Marron
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Economics, University of Western Ontario, London, Ontario, N6A 5C2, Canada

    Aman Ullah

Rights and permissions

Reprints and permissions

Copyright information

© 1989 Physica-Verlag Heidelberg

About this paper

Cite this paper

Marron, J.S. (1989). Automatic Smoothing Parameter Selection: A Survey. In: Ullah, A. (eds) Semiparametric and Nonparametric Econometrics. Studies in Empirical Economics. Physica-Verlag HD. https://doi.org/10.1007/978-3-642-51848-5_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-51848-5_5

  • Publisher Name: Physica-Verlag HD

  • Print ISBN: 978-3-642-51850-8

  • Online ISBN: 978-3-642-51848-5

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation