Abstract
In this paper we study the problem of adaptive estimation of a multivariate function satisfying some structural assumption. We propose a novel estimation procedure that adapts simultaneously to unknown structure and smoothness of the underlying function. The problem of structural adaptation is stated as the problem of selection from a given collection of estimators. We develop a general selection rule and establish for it global oracle inequalities under arbitrary \({\mathbb{L}}_p\) -losses. These results are applied for adaptive estimation in the additive multi-index model.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Barron, A., Birgé, L., Massart, P.: Risk bounds for model selection via penalization. Probab. Theory Relat. Fields 113, 301–413 (1999)
Belomestny, D., Spokoiny, V.: Local likelihood modeling via stagewise aggregation. WIAS preprint No. 1000 (2004). www.wias-berlin.de
Bertin, K.: Asymptotically exact minimax estimation in sup-norm for anisotropic Hölder balls. Bernoulli 10, 873–888 (2004)
Cavalier, L., Golubev, G.K., Picard, D., Tsybakov, A.B.: Oracle inequalities for inverse problems. Ann. Stat. 30, 843–874 (2002)
Chen, H.: Estimation of a projection-pursuit type regression model. Ann. Stat. 19, 142–157 (1991)
Devroye, L., Lugosi, G.: Combinatorial Methods in Density Estimation. Springer, New York (2001)
Folland, G.B.: Real Analysis, 2nd edn. Wiley, New York (1999)
Goldenshluger, A., Nemirovski, A.: On spatially adaptive estimation of nonparametric regression. Math. Methods Stat. 6, 135–170 (1997)
Golubev, G.K.: Asymptotically minimax estimation of a regression function in an additive model. Probl. Inform. Transm. 28, 101–112 (1992)
Golubev, G.K.: The method of risk envelopes in the estimation of linear functionals (Russian). Probl. Inform. Transm. 40, 53–65 (2004)
Györfi, L., Kohler, M., Krzyzak, A., Walk, H.: A Distribution-Free Theory of Nonparametric Regression. Springer, New York (2002)
Hall, P.: On projection-pursuit regression. Ann. Stat. 17, 573–588 (1989)
Hristache, M., Juditsky, A., Spokoiny, V.: Direct estimation of the index coefficient in a single-index model. Ann. Stat. 29, 595–623 (2001)
Hristache, M., Juditsky, A., Polzehl, J., Spokoiny, V.: Structure adaptive approach for dimension reduction. Ann. Stat. 29, 1537–1566 (2001)
Huber, P.: Projection pursuit. With discussion. Ann. Stat. 13, 435–525 (1985)
Ibragimov, I.A., Khasminskii, R.Z.: Bounds for the quality of nonparametric estimation of regression. Theory Probab. Appl. 27, 81–94 (1982)
Ibragimov, I.A.: Estimation of multivariate regression. Theory Probab. Appl. 48, 256–272 (2004)
Jennrich, R.: Asymptotic properties of non-linear least squares estimators. Ann. Math. Stat. 40, 633–643 (1969)
Johnstone, I.M.: Oracle inequalities and nonparametric function estimation. In: Proceedings of the International Congress of Mathematicians, Vol. III (Berlin, 1998). Doc. Math., Extra vol. III, pp. 267–278 (1998)
Juditsky, A., Lepski, O., Tsybakov, A.: Statistical estimation of composite functions. Manuscript (2006)
Kerkyacharian, G., Lepski, O., Picard, D.: Nonlinear estimation in anisotropic multi-index denoising. Probab. Theory Relat. Fields 121, 137–170 (2001)
Lepski, O.V., Levit, B.Y.: Adaptive nonparametric estimation of smooth multivariate functions. Math. Methods Stat. 8, 344–370 (1999)
Lepski, O., Mammen, E., Spokoiny, V.: Optimal spatial adaptation to inhomogeneous smoothness: an approach based on kernel estimators with variable bandwidth selectors. Ann. Stat. 25, 929–947 (1997)
Lepski, O.V., Spokoiny, V.G.: Optimal pointwise adaptive methods in nonparametric estimation. Ann. Stat. 25, 2512–2546 (1997)
Lifshits, M.: Gaussian Random Functions. Kluwer, Dordrecht (1995)
Nemirovski, A.S.: Nonparametric estimation of smooth regression functions. Soviet J. Comput. Syst. Sci. 23(6), 1–11 (1985); translated from Izv. Akad. Nauk SSSR Tekhn. Kibernet. 235 (3), 50–60 (1985)(Russian)
Nemirovski, A.: Topics in Non-parametric Statistics. Lectures on probability theory and statistics (Saint-Flour, 1998), Lecture Notes in Mathematics, vol. 1738, pp. 85–277. Springer, Berlin (2000)
Nicoleris, T., Yatracos, Y.: Rates of convergence of estimators, Kolmogorov’s entropy and the dimensionality reduction principle in regression. Ann. Stat. 25, 2493–2511 (1997)
Nussbaum, M.: Nonparametric estimation of a regression function that is smooth in a domain in \({\mathbb{R}}^k\) . Theory Probab. Appl. 31, 108–115 (1987)
Stone, C.J.: Optimal global rates of convergence for nonparametric regression. Ann. Stat. 10, 1040–1053 (1982)
Stone, C.J.: Additive regression and other nonparametric models. Ann. Stat. 13, 689–705 (1985)
Talagrand, M.: Sharper bounds for Gaussian and empirical processes. Ann. Probab. 22, 28–76 (1994)
Tsybakov, A.: Optimal rates of aggregation. In: Scholkopf, B., Warmuth, M. (eds) Computational Learning Theory and Kernel Machines. Lectures Notes in Artificial Intelligence, vol.2777, pp. 303–313. Springer, Heidelberg (2003)
Author information
Authors and Affiliations
Corresponding author
Additional information
Supported by the ISF grant No. 389/07.
Rights and permissions
About this article
Cite this article
Goldenshluger, A., Lepski, O. Structural adaptation via \(\mathbb{L}_p\) -norm oracle inequalities. Probab. Theory Relat. Fields 143, 41–71 (2009). https://doi.org/10.1007/s00440-007-0119-5
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00440-007-0119-5