Abstract
Large-scale inverse problems arise in a variety of significant applications in image processing, and efficient regularization methods are needed to compute meaningful solutions. This chapter surveys three common mathematical models including a linear model, a separable nonlinear model, and a general nonlinear model. Techniques for regularization and large-scale implementations are considered, with particular focus on algorithms and computations that can exploit structure in the problem. Examples from image deconvolution, multi-frame blind deconvolution, and tomosynthesis illustrate the potential of these algorithms. Much progress has been made in the field of large-scale inverse problems, but many challenges still remain for future research.
Similar content being viewed by others
References
Andrews, H.C., Hunt, B.R.: Digital Image Restoration. Prentice-Hall, Englewood Cliffs (1977)
Bachmayr, M., Burger, M.: Iterative total variation schemes for nonlinear inverse problems. Inverse Prob. 25, 105004 (2009)
Bardsley, J.M.: An efficient computational method for total variation-penalized Poisson likelihood estimation. Inverse Prob. Imaging 2(2), 167–185 (2008)
Bardsley, J.M.: Stopping rules for a nonnegatively constrained iterative method for illposed Poisson imaging problems. BIT 48(4), 651–664 (2008)
Bardsley, J.M., Vogel, C.R.: A nonnegatively constrained convex programming method for image reconstruction. SIAM J. Sci. Comput. 25(4), 1326–1343 (2003)
Barzilai, J., Borwein, J.M.: Two-point step size gradient methods. IMA J. Numer. Anal. 8(1), 141–148 (1988)
Björck, Å.: A bidiagonalization algorithm for solving large and sparse ill-posed systems of linear equations. BIT 28(3), 659–670 (1988)
Björck, Å.: Numerical Methods for Least Squares Problems. SIAM, Philadelphia (1996)
Björck, Å., Grimme, E., van Dooren, P.: An implicit shift bidiagonalization algorithm for ill-posed systems. BIT 34(4), 510–534 (1994)
Brakhage, H.: On ill-posed problems and the method of conjugate gradients. In: Engl, H.W., Groetsch, C.W. (eds.) Inverse and Ill-Posed Problems, pp. 165–175. Academic, Boston (1987)
Calvetti, D., Reichel, L.: Tikhonov regularization of large linear problems. BIT 43(2), 263–283 (2003)
Calvetti, D., Somersalo, E.: Introduction to Bayesian Scientific Computing. Springer, New York (2007)
Candès, E.J., Romberg, J.K., Tao, T.: Stable signal recovery from incomplete and inaccurate measurements. Commun. Pure Appl. Math. 59(8), 1207–1223 (2006)
Carasso, A.S.: Direct blind deconvolution. SIAM J. Appl. Math. 61(6), 1980–2007 (2001)
Chadan, K., Colton, D., Päivärinta, L., Rundell, W.: An Introduction to Inverse Scattering and Inverse Spectral Problems. SIAM, Philadelphia (1997)
Chan, T.F., Shen, J.: Image Processing and Analysis: Variational, PDE, Wavelet, and Stochastic Methods. SIAM, Philadelphia (2005)
Cheney, M., Borden, B.: Fundamentals of Radar Imaging. SIAM, Philadelphia (2009)
Chung, J., Haber, E., Nagy, J.: Numerical methods for coupled super-resolution. Inverse Prob. 22(4), 1261–1272 (2006)
Chung, J., Nagy, J.: An efficient iterative approach for large-scale separable nonlinear inverse problems. SIAM J. Sci. Comput. 31(6), 4654–4674 (2010)
Chung, J., Nagy, J., Sechopoulos, I.: Numerical algorithms for polyenergetic digital breast tomosynthesis reconstruction. SIAM J. Imaging Sci. 3(1), 133–152 (2010)
Chung, J., Nagy, J.G., O’Leary, D.P.: A weighted GCV method for Lanczos hybrid regularization. Elec. Trans. Numer. Anal. 28, 149–167 (2008)
Chung, J., Sternberg, P., Yang, C.: High performance 3-d image reconstruction for molecular structure determination. Int. J. High Perform. Comput. Appl. 24(2), 117–135 (2010)
De Man, B., Nuyts, J., Dupont, P., Marchal, G., Suetens, P.: An iterative maximumlikelihood polychromatic algorithm for CT. IEEE Trans. Med. Imaging 20(10), 999–1008 (2001)
Diaspro, A., Corosu, M., Ramoino, P., Robello, M.: Two-photon excitation imaging based on a compact scanning head. IEEE Eng. Med. Biol. 18(5), 18–30 (1999)
Dobbins, J.T., III, Godfrey, D.J.: Digital X-ray tomosynthesis: current state of the art and clinical potential. Phys. Med. Biol. 48(19), R65–R106 (2003)
Easley, G.R., Healy, D.M., Berenstein, C.A.: Image deconvolution using a general ridgelet and curvelet domain. SIAM J. Imaging Sci. 2(1), 253–283 (2009)
Elad, M., Feuer, A.: Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images. IEEE Trans. Image Process. 6(12), 1646–1658 (1997)
Engl, H.W., Hanke, M., Neubauer, A.: Regularization of Inverse Problems. Kluwer, Dordrecht (2000)
Engl, H.W., Kügler, P.: Nonlinear inverse problems: theoretical aspects and some industrial applications. In: Capasso, V., Périaux, J. (eds.) Multidisciplinary Methods for Analysis Optimization and Control of Complex Systems, pp. 3–48. Springer, Berlin (2005)
Engl, H.W., Kunisch, K., Neubauer, A.: Convergence rates for Tikhonov regularisation of nonlinear ill-posed problems. Inverse Prob. 5(4), 523–540 (1989)
Engl, H.W., Louis, A.K., Rundell, W. (eds.): Inverse Problems in Geophysical Applications. SIAM, Philadelphia (1996)
Eriksson, J., Wedin, P.: Truncated Gauss-Newton algorithms for ill-conditioned nonlinear least squares problems. Optim. Meth. Softw. 19(6), 721–737 (2004)
Faber, T.L., Raghunath, N., Tudorascu, D., Votaw, J.R.: Motion correction of PET brain images through deconvolution: I. Theoretical development and analysis in software simulations. Phys. Med. Biol. 54(3), 797–811 (2009)
Figueiredo, M.A.T, Nowak, R.D., Wright, S.J.: Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems. IEEE J. Sel. Top. Signal Process. 1(4), 586–597 (2007)
Frank, J.: Three-Dimensional Electron Microscopy of Macromolecular Assemblies. Oxford University Press, New York (2006)
Golub, G.H., Heath, M., Wahba, G.: Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics 21(2), 215–223 (1979)
Golub, G.H., Luk, F.T., Overton, M.L.: A block Lanczos method for computing the singular values and corresponding singular vectors of a matrix. ACM Trans. Math. Softw. 7(2), 149–169 (1981)
Golub, G.H., Pereyra, V.: The differentiation of pseudo-inverses and nonlinear least squares problems whose variables separate. SIAM J. Numer. Anal. 10(2), 413–432 (1973)
Golub, G.H., Pereyra, V.: Separable nonlinear least squares: the variable projection method and its applications. Inverse Prob. 19, R1–R26 (2003)
Haber, E., Ascher, U.M., Oldenburg, D.: On optimization techniques for solving nonlinear inverse problems. Inverse Prob. 16(5), 1263–1280 (2000)
Haber, E., Oldenburg, D.: A GCV based method for nonlinear ill-posed problems. Comput. Geosci. 4(1), 41–63 (2000)
Hammerstein, G.R., Miller, D.W., White, D.R., Masterson, M.E., Woodard, H.Q., Laughlin, J.S.: Absorbed radiation dose in mammography. Radiology 130(2), 485–491 (1979)
Hanke, M.: Conjugate gradient type methods for ill-posed problems. Pitman research notes in mathematics, Longman Scientific & Technical, Harlow (1995)
Hanke, M.: Limitations of the L-curve method in ill-posed problems. BIT 36(2), 287–301 (1996)
Hanke, M.: On Lanczos based methods for the regularization of discrete ill-posed problems. BIT 41(5), 1008–1018 (2001)
Hansen, P.C.: Analysis of discrete ill-posed problems by means of the L-curve. SIAM Rev. 34(4), 561–580 (1992)
Hansen, P.C.: Numerical tools for analysis and solution of Fredholm integral equations of the first kind. Inverse Prob. 8(6), 849–872 (1992)
Hansen, P.C.: Regularization tools: a MATLAB package for analysis and solution of discrete ill-posed problems. Numer. Algorithms 6(1), 1–35 (1994)
Hansen, P.C.: Rank-Deficient and Discrete Ill-Posed Problems. SIAM, Philadelphia (1998)
Hansen, P.C.: Discrete Inverse Problems: Insight and Algorithms. SIAM, Philadelphia (2010)
Hansen, P.C., Nagy, J.G., O’Leary, D.P.: Deblurring Images: Matrices, Spectra and Filtering. SIAM, Philadelphia (2006)
Hansen, P.C., O’Leary, D.P.: The use of the L-curve in the regularization of discrete ill-posed problems. SIAM J Sci Comput 14(6):1487–1503 (1993)
Hardy, J.W.: Adaptive optics. Sci. Am. 270(6), 60–65 (1994)
Hofmann, B.: Regularization of nonlinear problems and the degree of ill-posedness. In: Anger, G., Gorenflo, R., Jochmann, H., Moritz, H., Webers, W. (eds.) Inverse Problems: Principles and Applications in Geophysics, Technology, and Medicine. Akademie Verlag, Berlin (1993)
Hohn, M., Tang, G., Goodyear, G., Baldwin, P.R., Huang, Z., Penczek, P.A., Yang, C., Glaeser, R.M., Adams, P.D., Ludtke, S.J.: SPARX, a new environment for Cryo-EM image processing. J. Struct. Biol. 157(1), 47–55 (2007)
Jain, A.K.: Fundamentals of Digital Image Processing. Prentice-Hall, Englewood Cliffs (1989)
Kang, M.G., Chaudhuri, S.: Super-resolution image reconstruction. IEEE Signal Process. Mag. 20(3), 19–20 (2003)
Kaufman, L.: A variable projection method for solving separable nonlinear least squares problems. BIT 15(1), 49–57 (1975)
Kilmer, M.E., Hansen, P.C., Español, M.I.: A projection-based approach to general-form Tikhonov regularization. SIAM J. Sci. Comput. 29(1), 315–330 (2007)
Kilmer, M.E., O’Leary, D.P.: Choosing regularization parameters in iterative methods for ill-posed problems. SIAM J. Matrix. Anal. Appl. 22(4), 1204–1221 (2001)
Landweber, L.: An iteration formula for Fredholm integral equations of the first kind. Am. J. Math. 73(3), 615–624 (1951)
Larsen, R.M.: Lanczos bidiagonalization with partial reorthogonalization. PhD thesis, Department of Computer Science, University of Aarhus, Denmark (1998)
Lawson, C.L., Hanson, R.J.: Solving Least Squares Problems. SIAM, Philadelphia (1995)
Löfdahl, M.G.: Multi-frame blind deconvolution with linear equality constraints. In: Bones, P.J., Fiddy, M.A., Millane, R.P. (eds.) Image Reconstruction from Incomplete Data II, Seattle, vol. 4792–21, pp. 146–155. SPIE (2002)
Lohmann, A.W., Paris, D.P.: Space-variant image formation. J. Opt. Soc. Am. 55(8), 1007–1013 (1965)
Marabini, R., Herman, G.T., Carazo, J.M.: 3D reconstruction in electron microscopy using ART with smooth spherically symmetric volume elements (blobs). Ultramicroscopy 72(1–2), 53–65 (1998)
Matson, C.L., Borelli, K., Jefferies, S., Beckner, C.C., Jr., Hege, E.K., Lloyd-Hart, M.: Fast and optimal multiframe blind deconvolution algorithm for high-resolution groundbased imaging of space objects. Appl. Opt. 48(1), A75–A92 (2009)
McNown, S.R., Hunt, B.R.: Approximate shift-invariance by warping shift-variant systems. In: Hanisch, R.J., White, R.L. (eds.) The Restoration of HST Images and Spectra II, pp. 181–187. Space Telescope Science Institute, Baltimore (1994)
Miller, K.: Least squares methods for ill-posed problems with a prescribed bound. SIAM J. Math. Anal. 1(1), 52–74 (1970)
Modersitzki, J.: Numerical Methods for Image Registration. Oxford University Press, Oxford (2004)
Morozov, V.A.: On the solution of functional equations by the method of regularization. Sov. Math. Dokl. 7, 414–417 (1966)
Nagy, J.G., O’Leary, D.P.: Fast iterative image restoration with a spatially varying PSF. In: Luk, F.T. (ed.) Advanced Signal Processing: Algorithms, Architectures, and Implementations VII, San Diego, vol. 3162, pp. 388–399. SPIE (1997)
Nagy, J.G., O’Leary, D.P.: Restoring images degraded by spatially-variant blur. SIAM J. Sci. Comput. 19(4), 1063–1082 (1998)
Natterer, F.: The Mathematics of Computerized Tomography. SIAM, Philadelphia (2001)
Natterer, F., Wübbeling, F.: Mathematical Methods in Image Reconstruction. SIAM, Philadelphia (2001)
Nguyen, N., Milanfar, P., Golub, G.: Efficient generalized cross-validation with applications to parametric image restoration and resolution enhancement. IEEE Trans. Image Process. 10(9), 1299–1308 (2001)
Nocedal, J., Wright, S.: Numerical Optimization. Springer, New York (1999)
O’Leary, D.P., Simmons, J.A.: A bidiagonali-zation-regularization procedure for large scale discretizations of ill-posed problems. SIAM J. Sci. Stat. Comput. 2(4), 474–489 (1981)
Osborne, M.R.: Separable least squares, variable projection, and the Gauss-Newton algorithm. Elec. Trans. Numer. Anal. 28, 1–15 (2007)
Paige, C.C., Saunders, M.A.: Algorithm 583 LSQR: sparse linear equations and least squares problems. ACM Trans. Math. Softw. 8(2), 195–209 (1982)
Paige, C.C., Saunders, M.A.: LSQR: an algorithm for sparse linear equations and sparse least squares. ACM Trans. Math. Softw. 8(1), 43–71 (1982)
Penczek, P.A., Radermacher, M., Frank, J.: Three-dimensional reconstruction of single particles embedded in ice. Ultramicroscopy 40(1), 33–53 (1992)
Phillips, D.L.: A technique for the numerical solution of certain integral equations of the first kind. J. Assoc. Comput. Mach. 9(1), 84–97 (1962)
Raghunath, N., Faber, T.L., Suryanarayanan, S., Votaw, J.R.: Motion correction of PET brain images through deconvolution: II. Practical implementation and algorithm optimization. Phys. Med. Biol. 54(3), 813–829 (2009)
Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992)
Ruhe, A., Wedin, P.: Algorithms for separable nonlinear least squares problems. SIAM Rev. 22(3), 318–337 (1980)
Saad, Y.: On the rates of convergence of the Lanczos and the block-Lanczos methods. SIAM J. Numer. Anal. 17(5), 687–706 (1980)
Saban, S.D., Silvestry, M., Nemerow, G.R., Stewart, P.L.: Visualization of α-helices in a 6-Ångstrom resolution cryoelectron microscopy structure of adenovirus allows refinement of capsid protein assignments. J. Virol. 80(24), 49–59 (2006)
Tikhonov, A.N.: Regularization of incorrectly posed problems. Sov. Math. Dokl. 4, 1624–1627 (1963)
Tikhonov, A.N.: Solution of incorrectly formulated problems and the regularization method. Sov. Math. Dokl. 4, 1035–1038 (1963)
Tikhonov, A.N., Arsenin, V.Y.: Solutions of Ill-Posed Problems. Winston, Washington (1977)
Tikhonov, A.N., Leonov, A.S., Yagola, A.G.: Nonlinear Ill-Posed Problems, vol. 1–2. Chapman and Hall, London (1998)
Trussell, H.J., Fogel, S.: Identification and restoration of spatially variant motion blurs in sequential images. IEEE Trans. Image Process. 1(1), 123–126 (1992)
Tsaig, Y., Donoho, D.L.: Extensions of compressed sensing. Signal Process. 86(3), 549–571 (2006)
Varah, J.M.: Pitfalls in the numerical solution of linear ill-posed problems. SIAM J. Sci. Stat. Comput. 4(2), 164–176 (1983)
Vogel, C.R.: Optimal choice of a truncation level for the truncated SVD solution of linear first kind integral equations when data are noisy. SIAM J. Numer. Anal. 23(1), 109–117 (1986)
Vogel, C.R.: An overview of numerical methods for nonlinear ill-posed problems. In: Engl, H.W., Groetsch, C.W. (eds.) Inverse and Ill-Posed Problems, pp. 231–245. Academic, Boston (1987)
Vogel, C.R.: Non-convergence of the L-curve regularization parameter selection method. Inverse Prob. 12(4), 535–547 (1996)
Vogel, C.R.: Computational Methods for Inverse Problems. SIAM, Philadelphia (2002)
Wagner, F.C., Macovski, A., Nishimura, D.G.: A characterization of the scatter pointspread-function in terms of air gaps. IEEE Trans. Med. Imaging 7(4), 337–344 (1988)
Acknowledgements
We would like to thank Eldad Haber, University of British Columbia, and Per Christian Hansen, Technical University of Denmark, for carefully reading the first draft of this chapter. Their comments and suggestions helped to greatly improve our presentation. The research of J. Chung is supported by the US National Science Foundation (NSF) under grant DMS-0902322. The research of J. Nagy is supported by the US National Science Foundation (NSF) under grant DMS-0811031, and by the US Air Force Office of Scientific Research (AFOSR) under grant FA9550-09-1-0487.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer Science+Business Media New York
About this entry
Cite this entry
Chung, J., Knepper, S., Nagy, J.G. (2014). Large-Scale Inverse Problems in Imaging. In: Scherzer, O. (eds) Handbook of Mathematical Methods in Imaging. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-27795-5_2-6
Download citation
DOI: https://doi.org/10.1007/978-3-642-27795-5_2-6
Received:
Accepted:
Published:
Publisher Name: Springer, Berlin, Heidelberg
Online ISBN: 978-3-642-27795-5
eBook Packages: Springer Reference MathematicsReference Module Computer Science and Engineering
Publish with us
Chapter history
-
Latest
Large-Scale Inverse Problems in Imaging- Published:
- 02 January 2015
DOI: https://doi.org/10.1007/978-3-642-27795-5_2-6
-
Original
Large-Scale Inverse Problems in Imaging- Published:
- 11 November 2014
DOI: https://doi.org/10.1007/978-3-642-27795-5_2-5