Abstract
On March 11, 1944, the famous Eremitani Church in Padua (Italy) was destroyed in an Allied bombing along with the inestimable frescoes by Andrea Mantegna et al. contained in the Ovetari Chapel. In the last 60 years, several attempts have been made to restore the fresco fragments by traditional methods, but without much success. One of the authors contributed to the development of an efficient pattern recognition algorithm to map the original position and orientation of the fragments, based on comparisons with an old gray level image of the fresco prior to the damage. This innovative technique allowed for the partial reconstruction of the frescoes. Unfortunately, the surface covered by the colored fragments is only 77 m2, while the original area was of several hundreds. This means that we can reconstruct only a fraction (less than 8%) of this inestimable artwork. In particular the original color of the blanks is not known. This begs the question of whether it is possible to estimate mathematically the original colors of the frescoes by making use of the potential information given by the available fragments and the gray level of the pictures taken before the damage. Moreover, is it possible to estimate how faithful such a restoration is? In this paper we retrace the development of the recovery of the frescoes as an inspiring and challenging real-life problem for the development of new mathematical methods. Then we shortly review two models recently studied independently by the authors for the recovery of vector valued functions from incomplete data, with applications to the recolorization problem. The models are based on the minimization of a functional which is formed by the discrepancy with respect to the data and additional regularization constraints. The latter refer to joint sparsity measures with respect to frame expansions, in particular wavelet or curvelet expansions, for the first functional and functional total variation for the second. We establish relations between these two models. As a major contribution of this work we perform specific numerical test on the real-life problem of the A. Mantegna’s frescoes and we compare the results due to the two methods.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Aubert, G., Vese, L.: A variational method in image recovery. SIAM J. Numer. Anal. 34(5), 1948–1979 (1997)
Ballester, C., Beltramio, M., Caselles, V., Sapiro, G., Verdera J.: Filling-in by joint interpolation of vector fields and gray levels. IEEE Trans. Image Process. 10(8), 1200–1211 (2001)
Beltramio, M., Sapiro, G., Caselles, V., Ballester, B.: Image inpainting. In: SIGGRAPH 2000, July (2001)
Candès, E.J., Donoho, D.L.: New tight frames of curvelets and optimal representations of objects with piecewise C 2 singularities. Comm. Pure Appl. Math. 57(2), 219–266 (2004)
Candès, E.J., Romberg, J., Tao, T.: Exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inform. Theory 52(2), 489–509 (2006)
Candès, E.J., Tao, T.: Near optimal signal recovery from random projections: universal encoding strategies. IEEE Trans. Inform. Theory 52, 5406–5425 (2005)
Caselles, V., Coll, V., Morel, V.J.-M.: Geometry and color in natural images. J. Math. Imaging Vision 16(2), 89–105 (2002)
Cazzato, R.: Un Metodo per la Ricolorazione di Immagini e Altri Strumenti per il Restauro. Il Progetto Mantegna e gli Affreschi nella Chiesa degli Eremitani. Master thesis (Italian), University of Padova (2007)
Cazzato, R., Costa, G., Dal Farra, A., Fornasier, M., Toniolo, D., Tosato, D., Zanuso, C.: Il Progetto Mantegna: storia e risultati. In: Spiazzi, D.T.A.M., De Nicolò Salmazo, A. (eds.) Andrea Mantegna e i Maestri della Cappella Ovetari: La Ricomposizione Virtuale e il Restauro, pp. 151–169. Skira (2006)
Chambolle, A., Lions, P.-L.: Image recovery via total variation minimization and related problems. Numer. Math. 76(2), 167–188 (1997)
Chan, T.F., Kang, S.H., Shen, J.: Euler’s elastica and curvature-based inpainting. SIAM J. Appl. Math. 63(2), 564–592 (2002)
Chan, T.F., Shen, J.: Inpainting based on nonlinear transport and diffusion. Contemp. Math. 313, 53–65 (2002)
Chan, T.F., Shen, J.: Mathematical models for local nontexture inpaintings. SIAM J. Appl. Math. 62(3), 1019–1043 (2002)
Chan, T.F., Shen, J.: Variational image inpainting. Comm. Pure Appl. Math. 58(5), 579–619 (2005)
Chaux, C., Combettes, P.L., Pesquet, J.-C., Wajs, V.R.: A variational formulation for frame-based inverse problems. Inverse Problems, 23(4), 1495–1518 (2007)
Cohen, A., Dahmen, W., Daubechies, I., DeVore R.: Harmonic analysis of the space BV. Rev. Mat. Iberoamericana 19(1), 235–263 (2003)
Cohen, A., DeVore, R., Petrushev, P., Xu, H.: Nonlinear approximation and the space \(BV({\Bbb R}^2)\). Amer. J. Math. 121, 587–628 (1999)
Combettes, P.L., Wajs, V.R.: Signal recovery by proximal forward-backward splitting. Multiscale Model. Simul. 4(4), 1168–1200 (2005)
Daubechies, I., Defrise, M., DeMol, C.: An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Comm. Pure Appl. Math 57, 1413–1541 (2004)
Daubechies, I., Teschke, G.: Variational image restoration by means of wavelets: simultaneous decomposition, deblurring and denoising. Appl. Comput. Harmon. Anal. 19(1), 1–16 (2005)
Daubechies, I., Teschke, G., Vese, L.: Iteratively solving linear inverse problems under general convex constraints. Inverse Problems Imaging 1(1), 29–46 (2007)
Daubechies, I., Teschke, G., Vese, L.: On some iterative concepts for image restoration. In: Hawkes, P.W. (ed.) Advances in Imaging and Electron Physics, no. 150, pp. 2–52. Elsevier (2008)
Donoho, D.L.: Compressed sensing. IEEE Trans. Inform. Theory 52(4), 1289–1306 (2006)
Elad, M., Starck, J.-L., Querre, P., Donoho, D.L.: Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA). Appl. Comput. Harmon. Anal. 19, 340–358 (2005)
Elad, M., Milanfar, P., Rubinstein, R.: Analysis versus synthesis in signal priors. Inverse Problems 23(3), 947–968 (2007)
Engl, H.W., Hanke, M., Neubauer, A.: Regularization of Inverse Problems. Mathematics and its Applications (Dordrecht), p. 375. Kluwer Academic, Dordrecht (1996)
Fadili, M.J., Starck, J.-L., Murtagh, F.: Inpainting and zooming using sparse representations. Comput. J. (2007). doi:10.1093/comjnl/bxm055
Fornasier, M.: Nonlinear projection recovery in digital inpainting for color image restoration. J. Math. Imaging Vision 24(3), 359–373 (2006)
Fornasier, M., March, R.: Restoration of color images by vector valued BV functions and variational calculus. SIAM J. Appl. Math. 68(2), 437–460 (2007)
Fornasier, M., Rauhut, H.: Iterative thresholding algorithms. Appl. Comput. Harmon. Anal. (2008). doi:10.1016/j.acha.2007.10.005
Fornasier, M., Rauhut, H.: Recovery algorithms for vector valued data with joint sparsity constraints. SIAM J. Numer. Anal. 46(2), 577–613 (2008)
Fornasier, M., Toniolo, D.: Computer-based recomposition of the frescoes in the Ovetari Chapel in the Church of the Eremitani in Padua. Methodology and initial results, (English/Italian). In: Mantegna nella chiesa degli Eremitani a Padova. Il recupero possibile, Ed. Skira (2003)
Fornasier, M., Toniolo, D.: Fast, robust, and efficient 2D pattern recognition for re-assembing fragmented images. Pattern Recogn. 38(11), 2074–2087 (2005)
Kang, S.H., March, R.: Variational models for image colorization via chromaticity and brightness decomposition. IEEE Trans. Image Process. 16(9), 2251–2261 (2007)
Levin, A., Lischinski, D., Weiss, Y.: Colorization using optimization. In: ACM SIGGRAPH 2004, pp. 689–694 (2004)
Mallat, S.: A Wavelet Tour of Signal Processing. Academic, New York (1998)
Mairal, J., Elad, M., Sapiro, G.: Sparse representation for color image restoration. IEEE Trans. Image Process. 17(1), 53–69 (2008)
Masnou, S.: Disocclusion: a variational approach using level lines. IEEE Trans. Image Process. 11(2), 68–76 (1998)
Masnou, S., Morel, J.-M.: Level lines based disocclusion. In: Proceedings of 5th IEEE Intl Conf. on Image Process., Chicago, pp. 259–263 (1998)
Mumford, D.: Elastica and computer vision. In: Bajaj, C. (ed.) Algebraic Geometry and Applications, pp. 491–506. Springer, Heidelberg (1994)
Ramlau, R., Teschke, G.: Tikhonov replacement functionals for iteratively solving nonlinear operator equations. Inverse Problems 21, 1571–1592 (2005)
Ramlau, R., Teschke, G.: A projection iteration for nonlinear operator equations with sparsity constraints. Numer. Math. 104, 177–203 (2006)
Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica, D 60(1–4), 259–268 (1992)
Sapiro, G.: Inpainting the colors. In: ICIP 2005. IEEE International Conference on Image Processing, pp. 698–701 (2005)
Teschke, G., Ramlau, R.: An iterative algorithm for nonlinear inverse problems with joint sparsity constraints in vector valued regimes and an application to color image inpainting. Inverse Problems 23, 1851–1870 (2007)
Tropp, J.: Algorithms for simultaneous sparse approximation. Part ii: Convex relaxation. Signal Process., special issue Sparse approximations in signal and image processing 86, 589–602 (2006)
Tropp, J., Gilbert, A.C., Strauss, M.J.: Algorithms for simultaneous sparse approximation. Part i: Greedy pursuit. Signal Process., special issue Sparse approximations in signal and image processing 86, 572–588 (2006)
Vese, L.: A study in the BV space of a denoising-deblurring variational problem. Appl. Math. Optim. 44, 131–161 (2001)
Yatziv, L., Sapiro, G.: Fast image and video colorization using chrominance blending. IEEE Trans. Image Process. 15(5), 1120–1129 (2006)
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Lixin Shen and Yuesheng Xu.
Rights and permissions
About this article
Cite this article
Fornasier, M., Ramlau, R. & Teschke, G. The application of joint sparsity and total variation minimization algorithms to a real-life art restoration problem. Adv Comput Math 31, 157–184 (2009). https://doi.org/10.1007/s10444-008-9103-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10444-008-9103-6