Abstract
This paper describes a generalized axiomatic scale-space theory that makes it possible to derive the notions of linear scale-space, affine Gaussian scale-space and linear spatio-temporal scale-space using a similar set of assumptions (scale-space axioms).
The notion of non-enhancement of local extrema is generalized from previous application over discrete and rotationally symmetric kernels to continuous and more general non-isotropic kernels over both spatial and spatio-temporal image domains. It is shown how a complete classification can be given of the linear (Gaussian) scale-space concepts that satisfy these conditions on isotropic spatial, non-isotropic spatial and spatio-temporal domains, which results in a general taxonomy of Gaussian scale-spaces for continuous image data. The resulting theory allows filter shapes to be tuned from specific context information and provides a theoretical foundation for the recently exploited mechanisms of shape adaptation and velocity adaptation, with highly useful applications in computer vision.
It is also shown how time-causal spatio-temporal scale-spaces can be derived from similar assumptions. The mathematical structure of these scale-spaces is analyzed in detail concerning transformation properties over space and time, the temporal cascade structure they satisfy over time as well as properties of the resulting multi-scale spatio-temporal derivative operators. It is also shown how temporal derivatives with respect to transformed time can be defined, leading to the formulation of a novel analogue of scale normalized derivatives for time-causal scale-spaces.
The kernels generated from these two types of theories have interesting relations to biological vision. We show how filter kernels generated from the Gaussian spatio-temporal scale-space as well as the time-causal spatio-temporal scale-space relate to spatio-temporal receptive field profiles registered from mammalian vision. Specifically, we show that there are close analogies to space-time separable cells in the LGN as well as to both space-time separable and non-separable cells in the striate cortex. We do also present a set of plausible models for complex cells using extended quasi-quadrature measures expressed in terms of scale normalized spatio-temporal derivatives.
The theories presented as well as their relations to biological vision show that it is possible to describe a general set of Gaussian and/or time-causal scale-spaces using a unified framework, which generalizes and complements previously presented scale-space formulations in this area.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Al-Sharif, S., Khalil, R.: On the generator of two parameter semigroups. Appl. Math. Comput. 156(2), 403–414 (2004)
Almansa, A., Lindeberg, T.: Fingerprint enhancement by shape adaptation of scale-space operators with automatic scale-selection. IEEE Trans. Image Process. 9(12), 2027–2042 (2000)
Alvarez, L., Guichard, F., Lions, P.-L., Morel, J.-M.: Axioms and fundamental equations of image processing. Arch. Ration. Mech. Anal. 123(3), 199–257 (1993)
Babaud, J., Witkin, A.P., Baudin, M., Duda, R.O.: Uniqueness of the Gaussian kernel for scale-space filtering. IEEE Trans. Pattern Anal. Mach. Intell. 8(1), 26–33 (1986)
Ballester, C., Gonzalez, M.: Affine invariant texture segmentation and shape from texture by variational methods. J. Math. Imaging Vis. 9, 141–171 (1998)
Baumberg, A.: Reliable feature matching across widely separated views. In: Proc. IEEE Comp. Soc. Conf. on Computer Vision and Pattern Recognition, pp. I:1774–1781. Hilton Head, SC (2000)
Broadbridge, P.: Entropy diagnostics for fourth order partial differential equations in conservation form. Entropy 10, 365–379 (2008). doi:10.3390/e10030365
Burt, P.J.: Fast filter transforms for image processing. Comput. Vis. Graph. Image Process. 16, 20–51 (1981)
Carslaw, H.S., Jaeger, J.C.: Conduction of Heat in Solids. Clarendon Press, Oxford (1959)
Crowley, J.L.: A representation for visual information. PhD thesis, Carnegie-Mellon University, Robotics Institute, Pittsburgh, Pennsylvania (1981)
DeAngelis, G.C., Ohzawa, I., Freeman, R.D.: Receptive field dynamics in the central visual pathways. Trends Neurosci. 18(10), 451–457 (1995)
Duits, R., Florack, L., de Graaf, J., ter Haar Romeny, B.: On the axioms of scale space theory. J. Math. Imaging Vis. 22, 267–298 (2004)
Duits, R., Felsberg, M., Florack, L., Platel, B.: α-scale-spaces on a bounded domain. In: Griffin, L., Lillholm, M. (eds.) Proc. Scale-Space Methods in Computer Vision: Scale-Space’03, Isle of Skye, Scotland, Jun. 2003. Lecture Notes in Computer Science, vol. 2695, pp. 494–510. Springer, Berlin (2003)
Dunninger, D.R.: Maximum principles for solutions of some fourth-order elliptic equation. J. Math. Anal. Appl. 37, 665–668 (1972)
Evans, L.C.: Partial Differential Equations. Graduate Studies in Mathematics, vol. 19. American Mathematical Society, Providence (1998)
Fagerström, D.: Temporal scale-spaces. Int. J. Comput. Vis. 2–3, 97–106 (2005)
Fagerström, D.: Spatio-temporal scale-spaces. In: Gallari, F., Murli, A., Paragios, N. (eds.) Proc. 1st International Conference on Scale-Space Theories and Variational Methods in Computer Vision. Lecture Notes in Computer Science, vol. 4485, pp. 326–337. Springer, Berlin (2007)
Felsberg, M., Sommer, G.: The monogenic scale-space: A unifying approach to phase-based image processing in scale-space. J. Math. Imaging Vis. 21, 5–26 (2004)
Fleet, D.J., Langley, K.: Recursive filters for optical flow. IEEE Trans. Pattern Anal. Mach. Intell. 17(1), 61–67 (1995)
Florack, L.M.J.: Image Structure. Series in Mathematical Imaging and Vision. Springer, Dordrecht (1997)
Florack, L.M.J., ter Haar Romeny, B.M., Koenderink, J.J., Viergever, M.A.: Scale and the differential structure of images. Image Vis. Comput. 10(6), 376–388 (1992)
Florack, L.M.J., Niessen, W., Nielsen, M.: The intrinsic structure of optic flow incorporating measurement duality. Int. J. Comput. Vis. 27(3), 263–286 (1998)
Folland, G.B.: Introduction to Partial Differential Equations, 2nd edn. Princeton University Press, Princeton (1995)
Freeman, W.T., Adelson, E.H.: The design and use of steerable filters. IEEE Trans. Pattern Anal. Mach. Intell. 13(9), 891–906 (1991)
Gårding, J., Lindeberg, T.: Direct computation of shape cues using scale-adapted spatial derivative operators. Int. J. Comput. Vis. 17(2), 163–191 (1996)
Goldstein, J.A.: Semigroups of Linear Operators and Applications. Oxford Mathematical Monographs. Oxford Science, Oxford (1985)
Griffin, L.: Critical point events in affine scale space. In: Sporring, J., Nielsen, M., Florack, L., Johansen, P. (eds.) Gaussian Scale-Space Theory: Proc. PhD School on Scale-Space Theory, Copenhagen, Denmark, May 1996, pp. 165–180. Springer, Berlin (1996)
Guichard, F.: A morphological, affine, and Galilean invariant scale-space for movies. IEEE Trans. Image Process. 7(3), 444–456 (1998)
Hille, E., Phillips, R.S.: Functional Analysis and Semi-Groups. American Mathematical Society Colloquium Publications, vol. 31 (1957)
Hummel, R.A., Moniot, R.: Reconstructions from zero-crossings in scale-space. IEEE Trans. Acoust. Speech Signal Process. 37(12), 2111–2130 (1989)
Iijima, T.: Observation theory of two-dimensional visual patterns. Technical report, Papers of Technical Group on Automata and Automatic Control, IECE, Japan (1962)
Koenderink, J.J.: The structure of images. Biol. Cybern. 50, 363–370 (1984)
Koenderink, J.J.: Scale-time. Biol. Cybern. 58, 159–162 (1988)
Koenderink, J.J., van Doorn, A.J.: Receptive field families. Biol. Cybern. 63, 291–298 (1990)
Koenderink, J.J., van Doorn, A.J.: Generic neighborhood operators. IEEE Trans. Pattern Anal. Mach. Intell. 14(6), 597–605 (1992)
Laptev, I., Lindeberg, T.: Velocity-adapted spatio-temporal receptive fields for direct recognition of activities. Image Vis. Comput. 22(2), 105–116 (2004)
Laptev, I., Caputo, B., Schuldt, C., Lindeberg, T.: Local velocity-adapted motion events for spatio-temporal recognition. Comput. Vis. Image Underst. 108, 207–229 (2007)
Lindeberg, T.: Scale-space for discrete signals. IEEE Trans. Pattern Anal. Mach. Intell. 12(3), 234–254 (1990)
Lindeberg, T.: Scale-Space Theory in Computer Vision. The Kluwer International Series in Engineering and Computer Science. Springer, Dordrecht (1994)
Lindeberg, T.: Scale-space theory: A basic tool for analysing structures at different scales. J. Appl. Stat. 21(2), 225–270 (1994). Also available from http://www.csc.kth.se/~tony/abstracts/Lin94-SI-abstract.html
Lindeberg, T.: On the axiomatic foundations of linear scale-space. In: Sporring, J., Nielsen, M., Florack, L., Johansen, P. (eds.) Gaussian Scale-Space Theory: Proc. PhD School on Scale-Space Theory, Copenhagen, Denmark, May, 1996. Springer, Berlin (1996)
Lindeberg, T.: On automatic selection of temporal scales in time-casual scale-space. In: Sommer, G., Koenderink, J.J. (eds.) Proc. AFPAC’97: Algebraic Frames for the Perception-Action Cycle, Kiel, Germany, Sep., 1997. Lecture Notes in Computer Science, vol. 1315, pp. 94–113. Springer, Berlin (1997)
Lindeberg, T.: Linear spatio-temporal scale-space. In: ter Haar Romeny, B.M., Florack, L.M.J., Koenderink, J.J., Viergever, M.A. (eds.) Scale-Space Theory in Computer Vision: Proc. First Int. Conf. Scale-Space’97, Utrecht, The Netherlands, July, 1997. Lecture Notes in Computer Science, vol. 1252, pp. 113–127. Springer, Berlin (1997). Extended version available as technical report ISRN KTH NA/P–01/22–SE from KTH
Lindeberg, T.: Feature detection with automatic scale selection. Int. J. Comput. Vis. 30(2), 77–116 (1998)
Lindeberg, T.: Linear spatio-temporal scale-space. Report, ISRN KTH/NA/P–01/22–SE, Dept. of Numerical Analysis and Computing Science, KTH, Nov. (2001)
Lindeberg, T.: Time-recursive velocity-adapted spatio-temporal scale-space filters. In: Johansen, P. (ed.) Proc. 7th European Conference on Computer Vision, Copenhagen, Denmark, May 2002. Lecture Notes in Computer Science, vol. 2350, pp. 52–67. Springer, Berlin (2002)
Lindeberg, T.: Scale-space. In: Wah, B. (ed.) Encyclopedia of Computer Science and Engineering, pp. 2495–2504. Wiley, Hoboken (2008). doi:10.1002/9780470050118.ecse609. Also available from http://www.nada.kth.se/~tony/abstracts/Lin08-EncCompSci.html
Lindeberg, T.: Generalized Gaussian scale-space axiomatics comprising linear scale-space, affine scale-space and spatio-temporal scale-space. Technical report TRITA-CSC-CV 2010:3, ISSN 1653-7092, School of Computer Science and Communication, KTH (Royal Institute of Technology), Stockholm, Sweden, 2010. Also available from http://www.csc.kth.se/~tony/abstracts/Lin10-GenGaussScSp.html
Lindeberg, T., Fagerström, D.: Scale-space with causal time direction. In: Proc. 4th European Conf. on Computer Vision, Cambridge, UK, Apr, 1996, vol. 1064, pp. 229–240. Springer, Berlin (1996)
Lindeberg, T., Gårding, J.: Shape-adapted smoothing in estimation of 3-D depth cues from affine distortions of local 2-D structure. In: Eklundh, J.-O. (ed.) Proc. 3rd European Conf. on Computer Vision, Stockholm, Sweden, May 1994. Lecture Notes in Computer Science, vol. 800, pp. 389–400. Springer, Berlin (1994)
Lindeberg, T., Gårding, J.: Shape-adapted smoothing in estimation of 3-D depth cues from affine distortions of local 2-D structure. Image Vis. Comput. 15, 415–434 (1997)
Lindeberg, T., Akbarzadeh, A., Laptev, I.: Galilean-corrected spatio-temporal interest operators. In: International Conference on Pattern Recognition, Cambridge, pp. I:57–62. (2004). Extended version available as technical report ISRN KTH NA/P–04/05–S from KTH
Lindeberg, T., Akbarzadeh, A., Laptev, I.: Galilean-corrected spatio-temporal interest operators. Technical Report ISRN KTH/NA/P–04/05–SE, Dept. of Numerical Analysis and Computing Science, KTH, Mar. 2004
Lukas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Image Understanding Workshop (1981)
Mikolajczyk, K., Schmid, C.: Scale and affine invariant interest point detectors. Int. J. Comput. Vis. 60(1), 63–86 (2004)
Nagel, H., Gehrke, A.: Spatiotemporal adaptive filtering for estimation and segmentation of optical flow fields. In: Proc. 5th European Conf. on Computer Vision, Freiburg, Germany, Jun. 1998, pp. 86–102. Springer, Berlin (1998)
Pauwels, E.J., Fiddelaers, P., Moons, T., van Gool, L.J.: An extended class of scale-invariant and recursive scale-space filters. IEEE Trans. Pattern Anal. Mach. Intell. 17(7), 691–701 (1995)
Pazy, A.: Semi-groups of Linear Operators and Applications to Partical Differential Equations. Applied Mathematical Sciences. Springer, Berlin (1983)
Perona, P.: Steerable-scalable kernels for edge detection and junction analysis. Image Vis. Comput. 10, 663–672 (1992)
Sato, K.-I.: Lévy Processes and Infinitely Divisible Distributions. Cambridge Studies in Advanced Mathematics. Cambridge University Press, Cambridge (1999)
Schaffalitzky, F., Zisserman, A.: Viewpoint invariant texture matching and wide baseline stereo. In: Proc. 8th Int. Conf. on Computer Vision, Vancouver, Canada, July 2001, pp. II:636–643. (2001)
Schoenberg, I.J.: On Pòlya frequency functions. II. Variation-diminishing integral operators of the convolution type. Acta Sci. Math. (Szeged) 12, 97–106 (1950)
Schoenberg, I.J.: On smoothing operations and their generating functions. Bull. Am. Math. Soc. 59, 199–230 (1953)
Simoncelli, E.P., Freeman, W.T., Adelson, E.H., Heeger, D.J.: Shiftable multi-scale transforms. IEEE Trans. Information Theory 38(2), 587–607 (1992)
Sporring, J., Nielsen, M., Florack, L., Johansen, P. (eds.): Gaussian Scale-Space Theory: Proc. PhD School on Scale-Space Theory. Series in Mathematical Imaging and Vision. Springer, Copenhagen (1996)
ter Haar Romeny, B. (ed.): Geometry-Driven Diffusion in Computer Vision. Series in Mathematical Imaging and Vision. Springer, Dordrecht (1994)
ter Haar Romeny, B.: Front-End Vision and Multi-Scale Image Analysis. Springer, Dordrecht (2003)
ter Haar Romeny, B., Florack, L., Nielsen, M.: Scale-time kernels and models. In: Scale-Space and Morphology: Proc. Scale-Space’01, Vancouver, Canada, July 2001. Lecture Notes in Computer Science. Springer, Berlin (2001)
Tuytelaars, T., van Gool, L.: Matching widely separated views based on affine invariant regions. Int. J. Comput. Vis. 59(1), 61–85 (2004)
Valois, R.L.D., Cottaris, N.P., Mahon, L.E., Elfer, S.D., Wilson, J.A.: Spatial and temporal receptive fields of geniculate and cortical cells and directional selectivity. Vis. Res. 40(2), 3685–3702 (2000)
Weickert, J.: Anisotropic Diffusion in Image Processing. Teubner-Verlag, Stuttgart (1998)
Weickert, J., Ishikawa, S., Imiya, A.: Linear scale-space has first been proposed in Japan. J. Math. Imaging Vis. 10(3), 237–252 (1999)
Witkin, A.P.: Scale-space filtering. In: Proc. 8th Int. Joint Conf. Art. Intell., Karlsruhe, Germany, Aug. 1983, pp. 1019–1022 (1983)
Young, R.A.: The Gaussian derivative theory of spatial vision: Analysis of cortical cell receptive field line-weighting profiles. Technical Report GMR-4920, Computer Science Department, General Motors Research Lab., Warren, Michigan (1985)
Young, R.A.: The Gaussian derivative model for spatial vision: I. Retinal mechanisms. Spat. Vis. 2, 273–293 (1987)
Young, R.A., Lesperance, R.M.: The Gaussian derivative model for spatio-temporal vision: II. Cortical data. Spat. Vis. 14(3, 4), 321–389 (2001)
Young, R.A., Lesperance, R.M., Meyer, W.W.: The Gaussian derivative model for spatio-temporal vision: I. Cortical model. Spat. Vis. 14(3, 4), 261–319 (2001)
Yuille, A.L., Poggio, T.A.: Scaling theorems for zero-crossings. IEEE Trans. Pattern Anal. Mach. Intell. 8, 15–25 (1986)
Zhang, H., Zhang, W.: Maximum principles and bounds in a class of fourth-order uniformly elliptic equations. J. Phys. A: Math. Gen. 35, 9245–9250 (2002)
Author information
Authors and Affiliations
Corresponding author
Additional information
The support from the Swedish Research Council, Vetenskapsrådet (contract 2004-4680), the Royal Swedish Academy of Sciences as well as the Knut and Alice Wallenberg Foundation is gratefully acknowledged.
Rights and permissions
About this article
Cite this article
Lindeberg, T. Generalized Gaussian Scale-Space Axiomatics Comprising Linear Scale-Space, Affine Scale-Space and Spatio-Temporal Scale-Space. J Math Imaging Vis 40, 36–81 (2011). https://doi.org/10.1007/s10851-010-0242-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10851-010-0242-2