Abstract
There exist many fields where inverse problems appear. Some examples are: astronomy (blurred images of the Hubble satellite), econometrics (instrumental variables), financial mathematics (model calibration of the volatility), medical image processing (X-ray tomography), and quantum physics (quantum homodyne tomography). These are problems where we have indirect observations of an object (a function) that we want to reconstruct, through a linear operator A. Due to its indirect nature, solving an inverse problem is usually rather difficult. For this reason, one needs regularization methods in order to get a stable and accurate reconstruction. We present the framework of statistical inverse problems where the data are corrupted by some stochastic error. This white noise model may be discretized in the spectral domain using Singular Value Decomposition (SVD), when the operator A is compact. Several examples of inverse problems where the SVD is known are presented (circular deconvolution, heat equation, tomography). We explain some basic issues regarding nonparametric statistics applied to inverse problems. Standard regularization methods and their counterpart as estimation procedures by use of SVD are discussed (projection, Landweber, Tikhonov, . . . ). Several classical statistical approaches like minimax risk and optimal rates of convergence, are presented. This notion of optimality leads to some optimal choice of the tuning parameter. However these optimal parameters are unachievable since they depend on the unknown smoothness of the function. This leads to more recent concepts like adaptive estimation and oracle inequalities. Several data-driven selection procedures of the regularization parameter are discussed in details, among these: model selection methods, Stein’s unbiased risk estimation and the recent risk hull method.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Adorf, H.M.: Hubble space telescope image restoration in its fourth year. Inverse Problems 11, 639–653 (1995)
Akaike, H.: Information theory and an extension of the maximum likelihood principle. In: B. Petrov, F. Cz´aki (eds.) Proceedings of the Second International Symposium on Information Theory, pp. 267–281. Akademiai Kiad´o, Budapest (1973)
Akaike, H.: A new look at the statistical model identification. IEEE Trans. Automat. Control 19, 716–723 (1974)
Barron, A., Birgé, L., Massart, P.: Risk bounds for model selection via penalization. Probab. Theory Related Fields 113, 301–413 (1999)
Bauer, F., Hohage, T.: A Lepski-type stopping rule for regularized Newton methods. Inverse Problems 21, 1975–1991 (2005)
Bauer, F., Hohage, T., Munk, A.: Iteratively regularized Gauss-Newton method for nonlinear inverse problems with random noise. SIAM J. Numer. Anal. 47, 1827–1846 (2009)
Belitser, E., Levit, B.: On minimax filtering on ellipsoids. Math. Methods Statist. 4, 259–273(1995)
Birgé, L., Massart, P.: Gaussian model selection. J. Eur. Math. Soc. 3, 203–268 (2001)
Bissantz, N., Hohage, T., Munk, A., Ruymgaart, F.: Convergence rates of general regularizations methods for statistical inverse problems and application. SIAM J. Numer. Anal. 45, 2610–2636 (2007)
Brakage, H.: On ill-posed problems and the method of conjuguate gradients. In: Inverse and ill-posed problems. Academic Press, Orlando (1987)
Bretagnolle, J., Huber, C.: Estimation des densités : risque minimax. Z. Wahrsch. Verw. Gebiete 47, 199–237 (1976)
Brezis, H.: Analyse fonctionnelle, Théorie et applications. Dunod, Paris (1999)
Brown, L., Low, M.: Asymptotic equivalence of nonparametric regression and white noise. Ann. Statist. 24, 2384–2398 (1996)
Brown, L., Low, M., Zhao, L.: Superefficiency in nonparametric function estimation. Ann. Statist. 25, 898–924 (1997)
B¨uhlmann, P., Yu, B.: Boosting with _2-loss: regression and classification. J. Amer. Statist. Assoc. 98, 324–339 (2003)
Butucea, C., Tsybakov, A.: Sharp optimality in density deconvolution with dominating bias. Theory Probab. Appl. 52, 24–39 (2008)
Cai, T.: Adaptive wavelet estimation: a block thresholding and oracle inequality approach. Ann. Statist. 27, 2607–2625 (1999)
Cand`es, E.: Modern statistical estimation via oracle inequalities. Acta Numer. 15, 257–325 (2006)
Cand`es, E., Donoho, D.: Recovering edges in ill-posed inverse problems: Optimality of curvelet frames. Ann. Statist. 30, 784–842 (2002)
Cavalier, L.: Efficient estimation of a density in a problem of tomography. Ann. Statist. 28, 330–347 (2000)
Cavalier, L.: On the problem of local adaptive estimation in tomography. Bernoulli 7, 63–78
(2001)
Cavalier, L.: Inverse problems with non-compact operator. J. of Statist. Plann. Inference 136, 390–400 (2006)
Cavalier, L.: Nonparametric statistical inverse problems. Inverse Problems 24, 1–19 (2008)
Cavalier, L., Golubev, G., Lepski, O., Tsybakov, A.: Block thresholding and sharp adaptive estimation in severely ill-posed inverse problems. Theory Probab. Appl. 48, 426–446 (2003)
Cavalier, L., Golubev, G., Picard, D., Tsybakov, A.: Oracle inequalities in inverse problems. Ann. Statist. 30, 843–874 (2002)
Cavalier, L., Golubev, Y.: Risk hull method and regularization by projections of ill-posed inverse problems. Ann. Statist. 34, 1653–1677 (2006)
Cavalier, L., Hengartner, N.: Adaptive estimation for inverse problems with noisy operators. Inverse Problems 21, 1345–1361 (2005)
Cavalier, L., Koo, J.Y.: Poisson intensity estimation for tomographic data using a wavelet shrinkage approach. IEEE Trans. Inform. Theory 48, 2794–2802 (2002)
Cavalier, L., Raimondo, M.: Wavelet deconvolution with noisy eigenvalues. IEEE Trans. Signal Process. 55, 2414–2424 (2007)
Cavalier, L., Raimondo, M.: Multiscale density estimation with errors in variables. J. Korean Statist. Soc. (2010)
Cavalier, L., Tsybakov, A.: Penalized blockwise Stein’s method, monotone oracles and sharp adaptive estimation. Math. Methods Statist. 10, 247–282 (2001)
Cavalier, L., Tsybakov, A.: Sharp adaptation for inverse problems with random noise. Probab. Theory Related Fields 123, 323–354 (2002)
Chen, X., Reiss, M.: On rate optimality for ill-posed inverse problems in econometrics (2010). In press
Cohen, A., Hoffmann, M., Reiss, M.: Adaptive wavelet Galerkin method for linear inverse problems. SIAM J. Numer. Anal. 42, 1479–1501 (2004)
Comte, F., Rozenholc, Y., Taupin, M.L.: Penalized contrast estimator for adaptive density deconvolution. Canad. J. Statist. 34, 431–452 (2006)
Craven, P., Wahba, G.: Smoothing noisy data with spline functions: estimating the correct degree of smoothing by the method of generalized cross-validation. Numer. Math. 31, 377–403 (1979)
Deans, S.: The Radon Transform and some of its Applications. Wiley, New York (1983)
Donoho, D.: Statistical estimation and optimal recovery. Ann. Statist. 22, 238–270 (1994)
Donoho, D.: Nonlinear solution of linear inverse problems by wavelet-vaguelette decomposition. Appl. Comput. Harmon. Anal. 2, 101–126 (1995)
Donoho, D., Johnstone, I.: Ideal spatial adaptation via wavelet shrinkage. Biometrika 81, 425–445 (1994)
Donoho, D., Johnstone, I.: Adapting to unknown smoothness via wavelet shrinkage. J. Amer. Statist. Assoc. 90, 1200–1224 (1995)
Donoho, D., Johnstone, I.: Minimax estimation via wavelet shrinkage. Ann. Statist. 26, 879–921 (1998)
Donoho, D., Low, M.: Renormalization exponents and optimal pointwise rates of convergence. Ann. Statist. 20, 944–970 (1992)
Efroimovich, S., Pinsker, M.: Learning algorithm for nonparametric filtering. Autom. Remote Control 11, 1434–1440 (1984)
Efromovich, S.: Robust and efficient recovery of a signal passed through a filter and then
contaminated by non-Gaussian noise. IEEE Trans. Inform. Theory 43, 1184–1191 (1997)
Efromovich, S.: Nonparametric Curve Estimation. Springer, New York (1998)
Efromovich, S.: Simultaneous sharp adaptive estimation of functions and their derivatives. Ann. Statist. 26, 273–278 (1998)
Efromovich, S., Koltchinskii, V.: On inverse problems with unknown operators. IEEE Trans. Inform. Theory 47, 2876–2893 (2001)
Engl, H., Hanke, M., Neubauer, A.: Regularization of Inverse Problems. Kluwer Academic Publishers (1996)
Ermakov, M.: Minimax estimation of the solution of an ill-posed convolution type problem. Problems Inform. Transmission 25, 191–200 (1989)
Evans, S., Stark, P.: Inverse problems as statistics. Inverse Problems 18, 55–97 (2002)
Fan, J.: On the optimal rates of convergence for nonparametric deconvolution problems. Ann. Statist. 19, 1257–1272 (1991)
Florens, J., Johannes, J., Van Bellegem, S.: Identification and estimation by penalization in nonparametric instrumental regression. Econ. Theory (2010). In press
Florens, J.P.: Inverse problems and structural econometrics: the example of instrumental variables. In: Advances in Economics and Econometrics: Theory and Applications, vol. 2, pp. 284–311 (2003)
Goldenshluger, A.: On pointwise adaptive nonparametric deconvolution. Bernoulli 5, 907–925 (1999)
Goldenshluger, A., Pereverzev, S.: Adaptive estimation of linear functionals in Hilbert scales from indirect white noise observations. Probab. Theory Related Fields 118, 169–186 (2000)
Goldenshluger, A., Spokoiny, V.: On the shape-from-moments problem and recovering edges from noisy Radon data. Probab. Theory Related Fields 128, 123–140 (2004)
Goldenshluger, A., Tsybakov, A.: Adaptive prediction and estimation in linear regression with infinitely many parameters. Ann. Statist. 29, 1601–1619 (2001)
Golubev, G.: Quasi-linear estimates of signals in L2. Problems Inform. Transmission 26, 15–20 (1990)
Golubev, G.: The principle of penalized empirical risk in severely ill-posed problems. Probab. Theory Related Fields 130, 18–38 (2004)
Golubev, G., Khasminskii, R.: A statistical approach to some inverse problems for partial differential equations. Problems Inform. Transmission 35, 51–66 (1999)
Golubev, G., Khasminskii, R.: A statistical approach to the Cauchy problem for the Laplace equation. Lecture Notes Monograph Series 36, 419–433 (2001)
Grama, I., Nussbaum, M.: Asymptotic equivalence for nonparametric regression. Math. Methods Statist. 11, 1–36 (2002)
Groetsch, C.: Generalized Inverses of Linear Operators: Representation and Approximation. Dekker, New York (1977)
Hadamard, J.: Le probl`eme de Cauchy et les équations aux dérivées partielles hyperboliques. Hermann, Paris (1932)
Hall, P., Horowitz, J.: Nonparametric methods for inference in the presence of instrumental variables. Ann. Statist. 33, 2904–2929 (2005)
Hall, P., Kerkyacharian, G., Picard, D.: Block threshold rules for curve estimation using kernel and wavelet methods. Ann. Statist. 26, 922–942 (1998)
Halmos, P.: What does the spectral theorem say? Amer. Math. Monthly 70, 241–247 (1963)
Hida, T.: Brownian Motion. Springer-Verlarg, New York-Berlin (1980)
Hoerl, A.: Application of ridge analysis to regression problems. Chem. Eng. Progress 58, 54–59 (1962)
Hoffmann, M., Reiss, M.: Nonlinear estimation for linear inverse problems with error in the operator. Ann. Statist. 36, 310–336 (2008)
Hohage, T.: Lecture notes on inverse problems (2002). Lectures given at the University of Göttingen
Hutson, V., Pym, J.: Applications of Functional Analysis and Operator Theory. Academic Press, London (1980)
Ibragimov, I., Khasminskii, R.: Statistical Estimation: Asymptotic Theory. Springer, New York (1981)
Ibragimov, I., Khasminskii, R.: On nonparametric estimation of the value of a linear functional in Gaussian white noise. Theory Probab. Appl. 29, 19–32 (1984)
James, W., Stein, C.: Estimation with quadratic loss. In: Proceedings of the 4th Berkeley Symposium on Mathematical Statistics and Probability, pp. 361–380. University of California Press (1961)
Johnstone, I.: Function estimation in Gaussian noise: sequence models (1998). Draft of a monograph
Johnstone, I.: Wavelet shrinkage for correlated data and inverse problems: adaptivity results. Statist. Sinica 9, 51–83 (1999)
Johnstone, I., Kerkyacharian, G., Picard, D., Raimondo, M.: Wavelet deconvolution in a periodic setting. J. R. Stat. Soc. Ser. B Stat. Methodol. 66, 547–573 (2004)
Johnstone, I., Silverman, B.: Speed of estimation in positron emission tomography and related inverse problems. Ann. Statist. 18, 251–280 (1990)
Kaipio, J., Somersalo, E.: Statistical and Computational Inverse Problems. Springer (2004)
Kneip, A.: Ordered linear smoothers. Ann. Statist. 22, 835–866 (1994)
Kolaczyk, E.: A wavelet shrinkage approach to tomographic image reconstruction. J. Amer. Statist. Assoc. 91, 1079–1090 (1996)
Koo, J.Y.: Optimal rates of convergence for nonparametric statistical inverse problems. Ann. Statist. 21, 590–599 (1993)
Korostelev, A., Tsybakov, A.: Optimal rates of convergence of estimators in a probabilistic setup of tomography problem. Probl. Inf. Transm. 27, 73–81 (1991)
Landweber, L.: An iteration formula for Fredholm equations of the first kind. Amer. J. Math. 73, 615–624 (1951)
Lepskii, O.: One problem of adaptive estimation in Gaussian white noise. Theory Probab. Appl. 35, 459–470 (1990)
Lepskii, O.: Asymptotic minimax adaptive estimation. 1. Upper bounds. Theory Probab. Appl. 36, 654–659 (1991)
Lepskii, O.: Asymptotic minimax adaptive estimation. 2. Statistical model without optimal adaptation. Adaptive estimators. Theory Probab. Appl. 37, 468–481 (1992)
Li, K.C.: Asymptotic optimality of CP,CL, cross-validation and generalized cross-validation: Discrete index set. Ann. Statist. 15, 958–976 (1987)
Loubes, J.M., Ludena, C.: Adaptive complexity regularization for linear inverse problems. Electron. J. Stat. 2, 661–677 (2008)
Loubes, J.M., Ludena, C.: Penalized estimators for nonlinear inverse problems. ESAIM Probab. Stat. (2010)
Loubes, J.M., Rivoirard, V.: Review of rates of convergence and regularity conditions for inverse problems. Int. J. Tomogr. and Stat. (2009)
Louis, A., Maass, P.: A mollifier method for linear operator equations of the first kind. Inverse Problems 6, 427–440 (1990)
Mair, B., Ruymgaart, F.: Statistical estimation in Hilbert scale. SIAM J. Appl. Math. 56, 1424–1444 (1996)
Mallows, C.: Some comments on Cp. Technometrics 15, 661–675 (1973)
Marteau, C.: Regularization of inverse problems with unknown operator. Math. Methods Statist. 15, 415–443 (2006)
Marteau, C.: On the stability of the risk hull method for projection estimators. J. Statist. Plann. Inference 139, 1821–1835 (2009)
Marteau, C.: Risk hull method for general family of estimators. ESAIM Probab. Stat. (2010)
Massart, P.: Concentration Inequalities and Model Selection. Lecture Notes in Mathematics, Springer, Berlin (2007)
Mathé, P.: The Lepskii principle revisited. Inverse Problems 22, 11–15 (2006)
Mathé, P., Pereverzev, S.: Optimal discretization of inverse problems in Hilbert scales. Regularization
and self-regularization of projection methods. SIAM J. Numer. Anal. 38, 1999–2021 (2001)
Mathé, P., Pereverzev, S.: Regularization of some linear ill-posed problems with discretized random noisy data. Math. Comp. 75, 1913–1929 (2006)
Natterer, F.: The Mathematics of Computerized Tomography. J. Wiley, Chichester (1986)
Nemirovski, A.: Topics in Non-Parametric Statistics. Lecture Notes in Mathematics, Springer (2000)
Nemirovskii, A., Polyak, B.: Iterative methods for solving linear ill-posed problems under precise information I. Engrg. Cybernetics 22, 1–11 (1984)
Nussbaum, M.: Asymptotic equivalence of density estimation and Gaussian white noise. Ann. Statist. 24, 2399–2430 (1996)
O’Sullivan, F.: A statistical perspective on ill-posed problems. Statist. Sci. 1, 502–527 (1986)
Pinsker, M.: Optimal filtering of square integrable signals in Gaussian white noise. Problems Inform. Transmission 16, 120–133 (1980)
Plaskota, L.: Noisy Information and Computational Complexity. Cambridge University Press (1996)
Polyak, B., Tsybakov, A.: Asymptotic optimality of the Cp-test for the orthogonal series estimation of regression. Theory Probab. Appl. 35, 293–306 (1990)
Raus, T., Hamarik, U., Palm, R.: Use of extrapolation in regularization methods. J. Inverse Ill-Posed Probl. 15, 277 (2007)
Reiss, M.: Asymptotic equivalence for nonparametric regression with multivariate and random design. Ann. Statist. 36, 1957–1982 (2008)
Rigollet, P.: Adaptive density estimation using the blockwise Stein method. Bernoulli 12, 351–370 (2006)
Rosenblatt, M.: Remarks on some nonparametric estimates of a density function. Ann. Math.
Statist. 27, 832–837 (1956)
Ruymgaart, F.: A short introduction to inverse statistical inference (2001). Lecture given at Institut Henri Poincaré, Paris
Sabatier, P.: Past and future of inverse problems. J. Math. Phys. 41, 4082–4124 (2000)
Schwarz, G.: Estimating the dimension of a model. Ann. Statist. 6, 461–464 (1978)
Shibata, R.: An optimal selection of regression variables. Biometrika 68, 45–54 (1981)
Stein, C.: Inadmissibility of the usual estimator of the mean of a multivariate distribution. In: Proceedings of the 3rd Berkeley Symposium on Mathematical Statistics and Probability, pp.
197–206. University of California Press (1956)
Stein, C.: Estimation of the mean of a multivariate normal distribution. Ann. Statist. 9, 1135–1151 (1981)
Stone, C.: Optimal rates of convergence for nonparametric estimators. Ann. Statist. 8, 1348–1360 (1980)
Sudakov, V., Khalfin, L.: Statistical approach to ill-posed problems in mathematical physics. Soviet Math. Dokl. 157, 1094–1096 (1964)
Talagrand, M.: Concentration of measure and isoperimetric inequalities in product spaces. Publ. Math. IHES 81, 73–205 (1995)
Taylor, M.: Partial differential equations, vol. 2. Springer, New York (1996)
Tenorio, L.: Statistical regularization of inverse problems. SIAM Rev. 43, 347–366 (2001)
Tikhonov, A.: Regularization of incorrectly posed problems. Soviet Math. Dokl. 4, 1624–1627 (1963)
Tikhonov, A., Arsenin, V.: Solution of Ill-posed Problems. Winston & Sons (1977)
Tsybakov, A.: Introduction to Nonparametric Estimation. Springer series in statistics (2009)
Vogel, C.: Computational Methods for Inverse Problems. SIAM, Philadelphia (2002)
Wahba, G.: Spline Models for Observational Data. SIAM, Philadelphia (1990)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Cavalier, L. (2011). Inverse Problems in Statistics. In: Alquier, P., Gautier, E., Stoltz, G. (eds) Inverse Problems and High-Dimensional Estimation. Lecture Notes in Statistics(), vol 203. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-19989-9_1
Download citation
DOI: https://doi.org/10.1007/978-3-642-19989-9_1
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-19988-2
Online ISBN: 978-3-642-19989-9
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)