Abstract
The growth of the area of inverse problems within applied mathematics in recent years has been driven both by the needs of applications and by advances in a rigorous convergence theory of regularization methods for the solution of nonlinear ill-posed problems. There are at least two widely used approaches for solving inverse problems in a stable way: Tikhonov regularization and iterative regularization techniques. In this paper we give an overview over the latter. Moreover, we put the analysis of iterative methods for the solution of ill-posed problems into perspective with the analysis of iterative methods for the solution of well-posed problems.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
A.L. Ageev, T.V. Bolotova, and V.V. Vasin. Solution to the inverse gravity problem for two interfaces in a medium. Izvestiya, Physics of the Solid Earth. 34:225–227, 1998.
P. Anselone. Collectively Compact Operatar Approximation Theory. Prentice-Hall, Englewood Cliffs, New Jersey, 1971.
A.B. Bakushinskii. The problem of the convergence of the iteratively regularized Gauß-Newton method. Comput. Maths. Math. Phys., 32:1353–1359, 1992.
A.B. Bakushinskii. Universal linear approximations of solutions to nonlinear operator equations and their application. J. Inv. Ill-Posed Problems, 5:501–521, 1997.
A.B. Bakushinskii and A.V. Goncharskii. Iterative Methods far the Solution of Incorrect Problems. Nauka, Moscow, 1989. in Russian.
A.B. Bakushinskii and A.V. Goncharskii. Ill-Posed Problems: Theory and Applications. Kluwer Academic Publishers, Dordrecht, Boston, London, 1994.
R.E. Bank and D.J. Rose. Analysis of a multilevel iterative method for nonlinear finite element equations. Math. Comput., 39:453–465, 1982.
M. Bertero and P. Boccacci. Introductian to Inverse Problems in Imaging. IOP Publishing, London, 1998.
M. Bertero, T.A. Poggio, and V. Torre. Ill-posed problems in early vision. Proc. IEEE, 76:869–889, 1988.
A. Binder, M. Hanke, and O. Scherzer. On the Landweber iteration for nonlinear ill-posed problems. J. Inverse Ill-Posed Probl., 4:381–389, 1996.
B. Blaschke. Some Newton Type Methods for the Regularization of Nonlinear Ill Posed Problems. Universitätsverlag Rudolf Trauner, Linz, Austria, 1996. PhD-Thesis, Schriften der Johannes-Kepler-Universität.
B. Blaschke, A. Neubauer, and O. Scherzer. On convergence rates for the iteratively regularized Gauss-Newton method. IMA J. Numer. Anal., 17:421–436, 1997.
H. Brakhage. On ill-posed problems and the method of conjugate gradients. In [36], pages 177–185, 1987.
F.E. Browder, editor. Nonlinear Functional Analysis, volume 18. Amer. Math. Soc., Providence, 1970. Proc. Symposia in Pure Math.
F.E. Browder and W.V. Petryshyn. Construction of fixed points of nonlinear mappings in Hilbert space. J. Math. Anal. Appl., 20:197–228, 1967.
M. Burger, V. Capasso, and H.W. Engl. Inverse problems related to crystallization of polymers. Inverse Probl., 15:155–173, 1999.
G. Chavent. New size x curvature conditions for strict quasiconvexity of sets. SIAM J. Cantrol and Optimiz., 29:1348–1372, 1991.
G. Chavent. Quasi-convex sets and size x curvature condition, applications to nonlincar invcrsion. Appl. Math. Optimiz., 24:129–169, 1991.
G. Chavent and K. Kunisch. A geometric theory for l 1-stabilization of the inverse problem in a one-dimensional elliptic equation from an h 2-observation. Appl. Math. Optimiz., 27:231–260, 1993.
G. Chavent and K. Kunisch. Regularization in state space. J. Numer. Anal., 27:535–564, 1993.
G. Chavent and K. Kunisch. State space regularization: geometric theory. Appl. Math. Optimiz., 37:243–267, 1998.
G. Chavent and P. C. Sabatier, editors. Inverse Problems of Wave Propagation and Diffraction. Springer, Berlin. 1997.
X. Chen, M.Z. Nashed, and L. Qi. Convergence of Newton’s method for singular smooth and nonsmooth equations using adaptive outer inverses. SIAM J. Optim., 7:445–462, 1997.
X. Chen and L. Qi. A parameterized Newton method and a quasi-Newton method for nonsmooth equations. Comput. Optim. Appl., 3:157–179, 1994.
X. Chen and T. Yamamoto. On the convergence of some quasi-Newton methods for nonlinear equations with nondifferentiable operators. Computing, 49:87–94, 1992.
D. Colton, R. Ewing, and W. Rundell, editors. Inverse Problems in Partial Differential Equations. SIAM, Philadelphia, 1990.
R.S. Dembo, S.C. Eisenstat, and T. Steihaug. Inexact Newton methods. SIAM J. Numer. Anal., 19:400–408, 1982.
P. Deuflhard, H.W. Engl, and O. Scherzer. A convergence analysis of iterative methods for the solution of nonlinear ill-posed problems under affinely invariant conditions. Inverse Probl., 14:1081–1106, 1998.
P. Deuflhard and G. Heindl. Affine invariant convergence theorems for Newton’s method and extensions to related methods. SIAM J. Numer. Anal., 16:1–10, 1979.
P. Deuflhard and A. Hohmann. Numerical Analysis. A First Course in Scientific Computation. De Gruyter, Berlin, 1991. Transl. from the German by F.A. Potra and F. Schulz.
P. Deuflhard and A. Hohmann. Numerische Mathematik I. Eine algorithmisch orientierte Einführung (Numerical Mathematics I. An algorithmically oriented Introduction). De Gruyter, Berlin, 1993. 2., überarb. Aufl. (German).
P. Deuflhard and M. Weiser. Local inexact Newton multilevel FEM for nonlinear elliptic problems. In M-O. Bristeau, G. Etgen, W. Fitzgibbon, J.-L. Lions, J. Periaux, and M. Wheeler, editors, Computational Science for the 21st Century, Tours, France, pages 129–138. Wiley-Interscience-Europe, 1997.
W.G. Dotson Jr. On the Mann iterative process. Trans. Amer. Math. Soc., 149:65–73, 1970.
H.W. Engl. Weak convergence of asymptotically regular sequences for nonexpansive mappings and connectors with certain Chebyshef-centers. Nonlinear Analysis, Theory & Applications, 1:495–501, 1977.
H.W. Engl. Weak convergence of Mann iteration for nonexpansive mappings without convexity assumptions. Bollettino U.M.I, 14:471–475, 1977.
H.W. Engl and C.W. Groetsch, editors. Inverse and Ill-Posed Problems. Academic Press, Boston, 1987.
H.W. Engl, M. Hanke, and A. Neubauer. Regularization of Inverse Problems. Kluwer Academic Publishers, Dordrecht, 1996.
H.W. Engl, K. Kunisch, and A. Neubauer. Convergence rates for Tikhonov regularization of nonlinear ill-posed problems. Inverse Probl., 5:523–540, 1989.
H.W. Engl, A.K. Louis, and W. Rundell, editors. Inverse Problems in Geophysical Applications. SIAM, Philadelphia, 1996.
H.W. Engl, A.K. Louis, and W. Rundell, editors. Inverse Problems in Medical Imaging and Nondestructive Testing. Springer, Wien, New York, 1996.
H.W. Engl and J. McLaughlin, editors. Inverse Problems and Optimal Design in Industry.. B.G. Teubner, Stuttgart, 1994.
H.W. Engl and W. Rundell, editors. Inverse Problems in Diffusion Processes. SIAM, Philadelphia, 1995.
S.F. Gilyazov. Iterative solution methods for inconsistent linear equations with non self-adjoint operators. Moscov University Computational Mathematics and Cybernetics, pages 8–13, 1977.
S.F. Gilyazov. Regularizing algorithms based on the conjugate-gradient method. U.S.S.R. Comput. Math. Math. Phys., 26:8–13, 1986.
G.H. Golub and D.P. O’Leary. Some history of the conjugate radient method and Lanczos algorithms: 1948–1976. SIAM Review, 31:50–102, 1989.
C.W. Groetsch. A note on segmenting Mann iterates. J. Math. Anal. Appl., 40:369–372, 1972.
C.W. Groetsch. A nonstationary iterative process for nonexpansive mappings. Proc. Am. Math. Soc., 43:155–158, 1974.
C.W. Groetsch. The Theory of Tikhonov Regularization for Fredholm Equations of the First Kind. Pitman, Boston, 1984.
W. Hackbusch. Multi-Grid Methods and Applications. Springer-Verlag, Berlin, Heidelberg, New York, 1985.
W. Hackbusch. Iterative Solution of Large Sparse Systems of Equations. Springer-Verlag, New York, 1994. Applied Mathematical Sciences 95.
W. Hackbusch. Integral Equations. Theory and Numerical Treatment. Birkhauser, Basel, 1995.
G. Hammerlein and K.-H. Hoffmann, editors. Constructive Methods for the Practical Treatment of Integral Equations. Birkhauser Verlag, Basel, Boston, Stuttgart, 1985. Proceedings of the Conference at the Mathematisches Forschungsinstitut Oberwolfach, June 24–30, 1984. International Series of Numerical Mathematics, Vol. 73.
M. Hanke. Conjugate Gradient Type Methods for Ill-Posed Problems. Longman Scientific & Technical, Harlow, 1995. Pitman Research Notes in Mathematics Series.
M. Hanke. A regularizing Levenberg-Marquardt scheme, with applications to inverse groundwater filtration problems. Inverse Probl., 13:79–95, 1997.
M. Hanke. Regularizing properties of a truncated Newton-CG algorithm for nonlinear inverse problems. Numer. Fund. Anal. Optimiz., 18:971–993, 1997.
M. Hanke, F. Hettlich, and O. Scherzer. The Landweber iteration for an inverse scattering problem. In [135], pages 909–915, 1995.
M. Hanke, A. Neubauer, and O. Scherzer. A convergence analysis of Landweber iteration for nonlinear ill-posed problems. Numer. Math., 72:21–37, 1995.
M. Hanke and C. Vogel. Two-level preconditioners for regularized inverse prohlems 1: theory. Numer. Math., 1999. to appear.
M. Hanke and C. Vogel. Two-level preconditioners for regularized inverse problems II: implementation and numerical results, submitted.
W.M. Haussler. A Kantorovich-type convergence analysis for the Gauss-Newton-method. Numer. Math., 48:119–125, 1986.
M. Heinkenschloss, C.T. Kelley, and H.T. Tran. Fast algorithms for nonsmooth compact fixed-point problems. SIAM J. Numer. Anal., 29:1769–1792, 1992.
F. Hettlich. An iterative method for the inverse scattering problem from soundhard obstacles. Z. Angew. Math. Mech., 76:165–168, 1996.
F. Hettlich, J. Morgan, and O. Scherzer. On the estimation of interfaces from boundary measurements. In [40], pages 163–178, 1996.
F. Hettlich and W. Rundell. Iterative methods for the reconstruction of an inverse potential problem. Inverse Probl, 12:251–266, 1996.
F. Hettlich and W. Rundell. Recovery of the support of a source term ill all elliptic differential equation. Inverse Probl, 13:959–976, 1997.
F. Hettlich and W. Rundell. The determination of a discontinuity in a conductivity from a single boundary measurement. Inverse Probl, 14:67–82, 1998.
B. Hofmann and O. Scherzer. Local ill-posedness and source conditions of operator equations in Hilbert spaces. Inverse Probl, 14:1189–1206, 1998.
T. Holiage. Logarithmic convergence rates of the iteratively regularized Gaufj-Newton method for an inverse potential and an inverse scattering problem. Inverse Probl, 13:1279–1300, 1997.
T. Hohage. Convergence rates of a regularized Newton method in sound-hard inverse scattering. SIAM Numer. Anal., 36:125–142, 1999.
T. Hohage and C. Schormann. A Newton-type method for a transmission problem in inverse scattering. Inverse Probl, 14:1207–1228, 1998.
A. Hohmann. Inexact Gauss Newton methods for parameter dependent nonlinear problems. Shaker, Aachen, 1994. Berichte aus der Mathematik.
S. Kabanikhin, R. Kowar, and O. Scherzer. On the Landweber iteration for the solution of a parameter identification problem in a hyperbolic partial differential equation of second order. J. Inv. Ill-Posed Problems, 6:403–430, 1998.
B. Kaltenbacher. Some Newton-type methods for the regularization of nonlinear ill-posed problems. Inverse Probl, 13:729–753, 1997.
B. Kaltenbacher. A posteriori parameter choice strategies for some Newton type methods for the regularization of nonlinear ill-posed problems. Numer. Math., 79:501–528, 1998.
W.J. Kammerer and M.Z. Nashed. Steepest descent for singular linear operators with nonclosed range. Applicable Analysis, 1:143–159, 1971.
W.J. Kammerer and M.Z. Nashed. On the convergence of the conjugate gradient method for singular linear operator equations. SIAM J. Numer. Anal, 9:165–181, 1972.
L.W. Kantorowitsch and G.P. Akilow. Funktionalanalysis in normierten Räumen. Akademie Verlag, Berlin, 1964.
A. Kaplan and R. Tichatschke. Stable Methods for Ill-Posed Problems. Akademie Verlag, Berlin, 1994.
J.T. King. A minimal error conjugate gradient method for ill-posed problems. Journal of Optimization Theory and Applications, 60:297–304, 1989.
K. Kumsch and G. Geymayer. Convergence rates for regularized nonlinear illposed problem. In [81], pages 81–92, 1991.
A. Kurzhanski and I. Lasiecka, editors. Modelling and Inverse Problems of Control for Distributed Parameter Systems. Springer, Berlin, 1991. Lecture Notes in Control and Information Sciences, Vol. 154.
A.K. Louis. Convergence of the conjugate gradient method for compact operators. In H.W. Engl and C.W. Groetsch, editors, [36], pages 177–185, 1987.
A.K. Louis. Inverse und Schlecht Gestellte Probleme. Teubner, Stuttgart, 1989.
W.R. Mann. Mean value methods in iteration. Proc. Am. Math. Soc, 4:506–510, 1953.
St. Maruster. Quasi-nonexpansity and two classical methods for solving nonlinear equations. Proc. Amer. Math. Soc, 62:119–123, 1977.
S.F. McCormick. An iterative procedure for the solution of constrained nonlinear equations with applications to optimization problems. Numer. Math., 23:371–385, 1975.
S.F. McCormick. The methods of Kacmarcz and row orthogonalization for solving linear equations and least squares problems in Hilbert space. Indiana University Mathematics Journal, 26:1137–1150, 1977.
K.H. Meyn. Solution of underdetermined nonlinear equations by stationary iteration methods. Numer. Math., 42:161–172, 1983.
V.A. Morozov. On the solution of functional equations by the method of regularization. Soviet Math. Dokl, 7:414–417, 1966.
V.A. Morozov. Methods for Solving Incorrectly Posed Problems. Springer Verlag, New York, Berlin, Heidelberg, 1984.
V.A. Morozov. Regularization Methods for Ill-Posed Problems. CRC Press, Boca Raton, 1993.
M.Z. Nashed, editor. Generalized Inverses and Applications. Academic Press, New York, 1976.
M.Z. Nashed and O. Scherzer. Least squares and bounded variation regularization with nondifferentiable functional. Num. Fund. Anal, and Optimiz., 19:873–901, 1998.
F. Natterer. Numerical solution of bilinear inverse problems. Universtät Münster, Germany, 1998. preprint.
A. Neubauer. Tikhonov regularization for non-linear ill-posed problems: optimal convergence rates and finite-dimensional approximation. Inverse Probl., 5:541–557, 1989.
A. Neubauer and O. Scherzer. Finite-dimensional approximation of Tikhonov regularized solutions of non-linear ill-posed problems. Numer. Fund. Anal, and Optimiz., 11:85–99, 1990.
A. Neubauer and O. Scherzer. A convergence rate result for a steepest descent method and a minimal error method for the solution of nonlinear ill-posed problems. Z. Anal. Anwend., 14:369–377, 1995.
Z. Opial. Weak convergence of the sequence of successive approximations of nonexpansive mappings. Bull. Amer. Math. Soc., 67:591–597, 1967.
J.M. Ortega and W.C. Rheinboldt. Iterative Solution of Nonlinear Equations in Several Variables. Academic Press, New York, 1970.
C. Outlaw and C.W. Groetsch. Averaging iteration in a Banach space. Bull. Am. Math. Soc., 75:430–432, 1969.
W.V. Petryshyn and T.E. Williamson. Strong and weak convergence of the sequence of successive approximations for quasi-nonexpansive mappings. J. Math. Anal. Appl., 43:459–497, 1973.
L. Qi and X. Chen. A preconditioning proximal Newton method for nondifferentiable convex optimization. Math. Program., 76B:411–429, 1997.
J. Qi-Nian. On the iteratively regularized Gauss-Newton method for solving nonlinear ill-posed problems. 1998. Preprint, University of Nanjing, China.
R. Ramlau. A modified Landweber-method for inverse problems. Num. Fund. Anal. Opt., 20:79–98, 1999.
A. Rieder. A wavelet multilevel method for ill-posed problems stabilized by Tikhonov regularization. Numer. Math., 75:501–522, 1997.
A. Rieder. On convergence rates of inexact Newton regularizations. Preprint, Universität Saarbrücken, 1998.
A. Rieder. On the regularization of nonlinear ill-posed problems via inexact Newton iterations. Inverse Probl, pages 309–327, 1999.
P.C. Sabatier (ed.). Some Topics in Inverse Problems. World Scientific, Singapore, 1988.
P.C. Sabatier (ed.). Inverse Methods in Action. Spinger, Berlin, Heidelberg, New York, 1990.
O. Scherzer. A posteriori error estimates for nonlinear ill-posed problems. submitted.
O. Scherzer. Convergence criteria of iterative methods based on Landweber iteration for solving nonlinear problems. J. Math. Anal. Appl., 194:911–933, 1995.
O. Scherzer. A convergence analysis of a method of steepest descent and a two-step algorithm for nonlinear ill-posed problems. Numer. Fund. Anal. Optimization, 17:197–214, 1996.
O. Scherzer. An iterative multi level algorithm for solving nonlinear ill-posed problems. Numer. Math., 80:579–600, 1998.
O. Scherzer. A modified Landweber iteration for solving parameter estimation problems. Appl. Math. Optimiz.. 38:45–68, 1998.
O. Scherzer. An note on Kacmarz’s method for the solution of ill posed problems. in preparation, 1999.
O. Scherzer, H.W. Engl, and K. Kunisch. Optimal a-posteriori parameter choice for Tikhonov regularization for solving nonlinear ill-posed problems. SIAM J. Numer. Anal., 30:1796–1838. 1993.
O. Scherzer and M. Gullikson. An adaptive strategy for updating the damping parameters in an iteratively regularized Gauss-Newton method. JOTA, 100, 1999. to appear.
E. Schock. Approximate solution of ill-posed equations: Arbitrarily slow convergence vs. superconvergence. In [52], pages 234–243, 1985.
T.I. Seidman and C.R. Vogel. Well posedness and convergence of some regularisation methods for non-linear ill posed problems. Inverse Probl. 5:227–238, 1989.
V.P. Tanana. Methods for Solution of Nonlinear Operator Equations. VSP, Utrecht, 1997.
U. Tautenhahn. On the asymptotical regularization of nonlinear ill-posed problems. Inverse Probl., 10:1405–1418, 1994.
A. N. Tikhonov and V. Y. Arsenin. Solutions of Ill-Posed Problems. John Wiley & Sons, Washington, D.C., 1977. Translation editor: Fritz John.
A.N. Tikhonov. Regularization of incorrectly posed problems. Soviet Math. Dokl., 4:1624–1627, 1963.
A.N. Tikhonov. Solution of incorrectly formulated problems and the regularization methods. Soviet Math. Dokl., 4:1035–1038, 1963.
A.N. Tikhonov, A. Goncharsky, V. Stepanov, and A. Yagola. Numerical Methods for the Solution of Ill-Posed Problems. Kluwer, Dordrecht, 1995.
A.N. Tikhonov, A.S. Leonov, A.I. Prilepko, I.A. Vasin, V.A. Vatutin, and A.G. Yagola, editors. Ill-Posed Problems in Natural Sciences. VSP, Utrecht, 1992.
G.M. Vainikko. Error estimates of the successive approximation method for ill-posed problems. Automat. Remote Control, 40:356–363, 1980.
V.V. Vasin. Iterative methods for the approximate solution of ill posed problems with a priori informations and their applications. In [36], pages 211–229, 1987.
V.V. Vasin. Ill-posed problems and iterative approximation of fixed points of pseudo-contractive mappings. In [126], pages 214–223, 1992.
V.V. Vasin. Monotone iterative processes for nonlinear operator equations and their applications to volterra equations. J. Inverse Ill-Posed Probl., 4:331–340, 1996.
V.V. Vasin. Monotonic iterative processes for operator equations in semiordered spaces. Dokl. Math. 54, 53:487–489, 1996. Translation from Dokl. Akad. Nauk, Ross. Akad. Nauk 349, No. 1. 7–9.
V.V. Vasin. On the convergence of gradient-type methods for nonlinear equations. Doklady Mathematics, 57:173–175, 1998. Translated from Dokladv Akademii Nauk Vol. 359 (1998), pp. 7–9.
V.V. Vasin and A.L. Ageev. Ill-Posed Problems with A-Priori Information. VSP, Utrecht, 1995.
V.V. Vasin, I.L. Prutkin, and L.Yu Timerkhanova. Retrieval of a three-dimensional relief of geological boundary from gravity data. Izvestiya, Physics of the Solid Earth, 32:901–905, 1996.
K.-W. Wang, B. Yang, J.Q. Sun, K. Seto, K. Nonami, H.-S. Tzou, S.S. Rao, G.R. Tomlinson, B. Yang, H.T. Banks, G.M.L. Gladwell, M. Link, G. Lallement, T.E. Alberts, C.-A. Tan, and Y.Y. Hung, editors. Proceedings of the 1995 Design Engineering Technical Conferences. The American Society of Mechanical Engineers, New York, 1995. Vibration Control, Analysis, and Identification, Vol. 3, Part C.
J. Weidmann. Lineare Operatoren in Hilberträumen. Teubner, Stuttgart, 1976.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag/Wien
About this chapter
Cite this chapter
Engl, H.W., Scherzer, O. (2000). Convergence Rates Results for Iterative Methods for Solving Nonlinear Ill-Posed Problems. In: Colton, D., Engl, H.W., Louis, A.K., McLaughlin, J.R., Rundell, W. (eds) Surveys on Solution Methods for Inverse Problems. Springer, Vienna. https://doi.org/10.1007/978-3-7091-6296-5_2
Download citation
DOI: https://doi.org/10.1007/978-3-7091-6296-5_2
Publisher Name: Springer, Vienna
Print ISBN: 978-3-211-83470-1
Online ISBN: 978-3-7091-6296-5
eBook Packages: Springer Book Archive