Abstract
This paper presents a generative model for textures that uses a local sparse description of the image content. This model enforces the sparsity of the expansion of local texture patches on adapted atomic elements. The analysis of a given texture within this framework performs the sparse coding of all the patches of the texture into the dictionary of atoms. Conversely, the synthesis of a new texture is performed by solving an optimization problem that seeks for a texture whose patches are sparse in the dictionary. This paper explores several strategies to choose this dictionary. A set of hand crafted dictionaries composed of edges, oscillations, lines or crossings elements allows to synthesize synthetic images with geometric features. Another option is to define the dictionary as the set of all the patches of an input exemplar. This leads to computer graphics methods for synthesis and shares some similarities with non-local means filtering. The last method we explore learns the dictionary by an optimization process that maximizes the sparsity of a set of exemplar patches. Applications of all these methods to texture synthesis, inpainting and classification shows the efficiency of the proposed texture model.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Aharon, M., Elad, M.: Sparse and redundant modeling of image content using an image-signature-dictionary. J. Imag. Sci. 1(3), 228–247 (2008)
Aharon, M., Elad, M., Bruckstein, A.M.: The k-svd: An algorithm for designing of overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54(11), 4311–4322 (2006)
Ashikhmin, M.: Synthesizing natural textures. In: SI3D’01: Proceedings of the 2001 Symposium on Interactive 3D Graphics, pp. 217–226. Assoc. Comput. Mach., New York (2001)
Ballester, C., Caselles, V., Verdera, J.: Disocclusion by joint interpolation of vector fields and gray levels. Multiscale Model. Simul. 2(1), 80–123 (2003)
Bar-Joseph, Z., El-Yaniv, R., Lischinski, D., Werman, M.: Texture mixing and texture movie synthesis using statistical learning. IEEE Trans. Vis. Comput. Graph. 7(2), 120–135 (2001)
Bell, A.J., Sejnowski, T.J.: The independent components of natural scenes are edge filters. Vis. Res. 37, 3327–3338 (1997)
Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: Proc. of Siggraph 2000, pp. 417–424 (2000)
Bovik, A.C.: Analysis of multichannel narrow-band filters for image texture segmentation. IEEE Trans. Signal Process. 39(9), 2025 (1991)
Brox, T., Cremers, D.: Iterated nonlocal means for texture restoration. In: Proc. International Conference on Scale Space and Variational Methods in Computer Vision, Ischia, Italy. LNCS. Springer, Berlin (2007)
Buades, A., Coll, B., Morel, J.M.: A review of image denoising algorithms, with a new one. Multiscale Model. Simul. 4(2), 490–530 (2005)
Chen, S.S., Donoho, D.L., Saunders, M.A.: Atomic decomposition by basis pursuit. SIAM J. Sci. Comput. 20(1), 33–61 (1998)
Clausi, D.A., Jernigan, M.E.: Designing Gabor filters for optimal texture separability. Pattern Recogn. 33(11), 1835–1849 (2000)
Cohen, L.D.: A new approach of vector quantization for image data compression and texture detection. In: International Conference on Pattern Recognition, pp. 1250–1254 (1988)
Cook, R.L., DeRose, T.: Wavelet noise. ACM Trans. Graph. 24(3), 803–811 (2005)
Criminisi, A., Pérez, P., Toyama, K.: Region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Process. 13(9), 1200–1212 (2004)
Daubechies, I., Defrise, M., De Mol, C.: An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Commun. Pure Appl. Math. 57, 1413–1541 (2004)
De Bonet, J.S.: Multiresolution sampling procedure for analysis and synthesis of texture images. In: Proc. of SIGGRAPH’97, pp. 361–368. Assoc. Comput. Mach./Addison Wesley, New York/Reading (1997)
Donoho, D.: Wedgelets: Nearly-minimax estimation of edges. Ann. Stat. 27, 353–382 (1999)
Efros, A.A., Leung, T.K.: Texture synthesis by non-parametric sampling. In: ICCV’99: Proceedings of the International Conference on Computer Vision, vol. 2, p. 1033. IEEE Computer Society, Los Alamitos (1999)
Efros, A.A., Freeman, W.T.: Image quilting for texture synthesis and transfer. In: Proc. of SIGGRAPH 2001, pp. 341–346 (2001)
Engan, K., Aase, S.O., Hakon Husoy, J.: Method of optimal directions for frame design. In: Proc. ICASSP’99, Washington, DC, pp. 2443–2446. IEEE Computer Society, Los Alamitos (1999)
Fadili, M.J., Starck, J.-L., Murtagh, F.: Inpainting and zooming using sparse representations. Comput. J. (2006, revised)
Heeger, D.J., Bergen, J.R.: Pyramid-Based texture analysis/synthesis. In: Cook, R. (ed.) SIGGRAPH 95 Conference Proceedings. Annual Conference Series, pp. 229–238. Assoc. Comput. Mach./Addison Wesley, New York/Reading (1995)
Jain, A.K., Farrokhnia, F.: Unsupervised texture segmentation using Gabor filters. Pattern Recogn. 24(12), 1167–1186 (1991)
Julesz, B.: Visual pattern discrimination. IRE Trans. Inf. Theory 8(2), 84–92 (1962)
Kreutz-Delgado, K., Murray, J.F., Rao, B.D., Engan, K., Lee, T.-W., Sejnowski, T.J.: Dictionary learning algorithms for sparse representation. Neural Comput. 15(2), 349–396 (2003)
Kwatra, V., Essa, I., Bobick, A., Kwatra, N.: Texture optimization for example-based synthesis. ACM Trans. Graph. 24(3), 795–802 (2005). Proc. of SIGGRAPH 2005
Kwatra, V., Schödl, A., Essa, I., Turk, G., Bobick, A.: Graphcut textures: Image and video synthesis using graph cuts. ACM Trans. Graph. 22(3), 277–286 (2003). SIGGRAPH 2003
Lee, D.D., Seung, H.S.: Algorithms for non-negative matrix factorization. In: Advances in Neural Information Processing Systems 13. MIT Press, Cambridge (2001)
Lefebvre, S., Hoppe, H.: Parallel controllable texture synthesis. ACM Trans. Graph. 24(3), 777–786 (2005)
Lefebvre, S., Hoppe, H.: Appearance-space texture synthesis. ACM Trans. Graph. 25(3), 541–548 (2006)
Lewicki, M.S., Sejnowski, T.J.: Learning overcomplete representations. Neural Comput. 12(2), 337–365 (2000)
Lu, S.Y., Hernandez, J.E., Clark, G.A.: Texture segmentation by clustering of Gabor feature vectors. In: Proc. IJCNN’91, International Joint Conference on Neural Networks, vol. I, pp. 683–688. IEEE Press, New York (1991)
Mairal, J., Elad, M., Sapiro, G.: Sparse representation for color image restoration. IEEE Trans. Image Process. 17(1), 53–69 (2008)
Mairal, J., Sapiro, G., Elad, M.: Learning multiscale sparse representations for image and video restoration. Preprint (2007)
Mallat, S.: A Wavelet Tour of Signal Processing. Academic Press, San Diego (1998)
Manduchi, R., Portilla, J.: Independent component analysis of textures. In: ICCV, pp. 1054–1060 (1999)
Matusik, W., Zwicker, M., Durand, F.: Texture design using a simplicial complex of morphable textures. ACM Trans. Graph. 24(3), 787–794 (2005)
Olshausen, B.A., Field, D.J.: Emergence of simple-cell receptive-field properties by learning a sparse code for natural images. Nature 381(6583), 607–609 (1996)
Paragios, N., Deriche, R.: Geodesic active regions and level set methods for supervised texture segmentation. Int. J. Comput. Vis. 46(3), 223–247 (2002)
Perlin, K.: An image synthesizer. In: Proc. of SIGGRAPH’85, pp. 287–296. Assoc. Comput. Mach., New York (1985)
Peyré, G.: Non-negative sparse modeling of textures. In: Proc. of SSVM’07, pp. 628–639 (2007)
Peyré, G.: Manifold models for signals and images. Comput. Vis. Image Underst. (2008, to appear)
Portilla, J., Simoncelli, E.P.: A parametric texture model based on joint statistics of complex wavelet coefficients. Int. J. Comput. Vis. 40(1), 49–70 (2000)
Sallee, P., Olshausen, B.A.: Learning sparse multiscale image representations. In: Becker, S., Thrun, S., Obermayer, K. (eds.) NIPS, pp. 1327–1334. MIT Press, Cambridge (2002)
Simoncelli, E.P., Olshausen, B.A.: Natural image statistics and neural representation. Ann. Rev. Neurosci. 24, 1193–1215 (2001)
Skretting, K., Husoy, J.H.: Texture classification using sparse frame based representations. EURASIP J. Appl. Signal Process. 2006(1), 102–102 (2006)
Starck, J.-L., Elad, M., Donoho, D.L.: Redundant multiscale transforms and their application for morphological component analysis. Adv. Imaging Electron Phys. 132, 287–348 (2004)
Tropp, J.A.: Greed is good: algorithmic results for sparse approximation. IEEE Trans. Inf. Theory 50(10), 2231–2242 (2004)
Tropp, J.A.: Just relax: convex programming methods for identifying sparse signals in noise. IEEE Trans. Inf. Theory 52(3), 1030–1051 (2006)
Tschumperlé, D., Deriche, R.: Vector-valued image regularization with PDEs: A common framework for different applications. IEEE Trans. Pattern Anal. Mach. Intell. 27(4), 506–517 (2005)
Tseng, P.: Convergence of a block coordinate descent method for nondifferentiable minimization. J. Optim. Theory Appl. 109(3), 475–494 (2001)
Tuceryan, M.: Moment-based texture segmentation. Pattern Recogn. Lett. 15(7), 659–668 (1994)
Wei, L.-Y., Levoy, M.: Fast texture synthesis using tree-structured vector quantization. In: SIGGRAPH’00: Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, pp. 479–488. Assoc. Comput. Mach./Addison-Wesley, New York/Reading (2000)
Wei, L.-Y., Han, J., Zhou, K., Bao, H., Guo, B., Shum, H.-Y.: Inverse texture synthesis. ACM Trans. Graph. 27(3), 1–9 (2008). http://doi.acm.org/10.1145/1360612.1360651
Zeng, X.-Y., Chen, Y.-W., van Alphen, D., Nakao, Z.: Selection of ICA features for texture classification. In: ISNN (2), pp. 262–267 (2005)
Zhu, S.C., Liu, X.W., Wu, Y.N.: Exploring texture ensembles by efficient Markov chain Monte Carlo-toward a ‘trichromacy’ theory of texture. IEEE Trans. Pattern Anal. Mach. Intell. 22(6), 554–569 (2000)
Zhu, S.C., Wu, Y., Mumford, D.: Filters, random fields and maximum entropy (FRAME): Towards a unified theory for texture modeling. Int. J. Comput. Vis. 27(2), 107–126 (1998)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Peyré, G. Sparse Modeling of Textures. J Math Imaging Vis 34, 17–31 (2009). https://doi.org/10.1007/s10851-008-0120-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10851-008-0120-3