Abstract
Using a large set of human segmented natural images, we study the statistics of region boundaries. We observe several power law distributions which likely arise from both multi-scale structure within individual objects and from arbitrary viewing distance. Accordingly, we develop a scale-invariant representation of images from the bottom up, using a piecewise linear approximation of contours and constrained Delaunay triangulation to complete gaps. We model curvilinear grouping on top of this graphical/geometric structure using a conditional random field to capture the statistics of continuity and different junction types. Quantitative evaluations on several large datasets show that our contour grouping algorithm consistently dominates and significantly improves on local edge detection.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Alvarez, L., Gousseau, Y., & Morel, J. (1999). Scales in natural images and a consequence on their bounded variation norm. In Scale-space theories in computer vision.
Attneave, F. (1954). Some informational aspects of visual perception. Psychological Review, 61, 183–193.
August, J., Siddiqi, K., & Zucker, S. W. (1999). Contour fragment grouping and shared, simple occluders. Computer Vision and Image Understanding, 76(2), 146–162.
Belongie, S., Malik, J., & Punicha, J. (2002). Shape matching and object recognition using shape contexts. IEEE Transactions on PAMI, 24(4), 509–522.
Besag, J. E. (1974). Spatial interaction and the statistical analysis of lattice systems. Journal of Royal Statistical Society: Series B, 36(2), 192–236.
Borenstein, E., & Ullman, S. (2002). Class-specific, top-down segmentation. In Proceedings of 7th European conference on computer vision (Vol. 2, pp. 109–124).
Borgefors, G. (1988). Hierarchical chamfer matching: A parametric edge matching algorithm. IEEE Transactions on PAMI, 10(6), 849–865.
Canny, J. (1986). A computational approach to edge detection. IEEE Transactions on PAMI, 8, 679–698.
Donoho, D., & Huo, X. (2002). Beamlets and multiscale image analysis. In Lecture notes in computational science and engineering : Vol. 20. Multiscale and multiresolution methods (pp. 149–196). New York: Springer.
Elder, J., & Goldberg, R. (2002). Ecological statistics of gestalt laws for the perceptual organization of contours. Journal of Vision, 2(4), 324–353.
Elder, J. H., & Zucker, S. W. (1996). Computing contour closures. In Proceedings of 4th European conference on computer vision (Vol. I, pp. 399–412).
Felzenszwalb, P. (2001). Learning models for object recognition. In Proceedings of IEEE conference on computer vision and pattern recognition.
Freeman, W. T., Pasztor, E. C., & Carmichael, O. T. (2000). Learning low-level vision. International Journal of Computer Vision, 40(1), 25–47.
Geisler, W. S., Perry, J. S., Super, B. J., & Gallogly, D. P. (2001). Edge co-occurrence in natural images predicts contour grouping performance. Vision Research, 41, 711–724.
Geman, S., & Geman, D. (1984). Stochastic relaxation, gibbs distribution, and the bayesian retoration of images. IEEE Transactions on PAMI, 6, 721–741.
He, X., Zemel, R., & Carreira-Perpinan, M. (2004). Multiscale conditional random fields for image labelling. In Proceedings of IEEE conference on computer vision and pattern recognition (Vol. 2, pp. 695–702).
Heitger, F., & von der Heydt, R. (1993). A computational model of neural contour processing. In Proceedings of 4th international conference on computer vision (pp. 32–40). Berlin, Germany, May 1993.
Huang, J., & Mumford, D. (1999). Statistics of natural images and models. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 541–547).
Huttenlocher, D., & Wayner, P. (1992). Finding convex edge groupings in an image. International Journal of Computer Vision, 8(1), 7–27.
Huttenlocher, D. P., Klanderman, G., & Rucklidge, W. (1993). Comparing images using the Hausdorff distance. IEEE Transactions on PAMI, 15(9), 850–863.
Jermyn, I., & Ishikawa, H. (2001). Globally optimal regions and boundaries as minimum ratio weight cycles. IEEE Transactions on PAMI, 23(10), 1075–1088.
Kanizsa, G. (1979). Organization in vision: essays on gestalt perception. New York: Praeger.
Kellman, P. J., & Shipley, T. F. (1991). A theory of visual interpolation in object perception. Cognitive Psychology, 23, 141–221.
Konishi, S., Yuille, A. L., Coughlan, J. M., & Zhu, S. C. (1999). Fundamental bounds on edge detection: An information theoretic evaluation of different edge cues. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 573–579).
Kschischang, F. R., Frey, B. J., & Loeliger, H.-A. (2001). Factor graphs and the sum-product algorithm. IEEE Transactions on Information Theory, 47, 498–519.
Kumar, S., & Hebert, M. (2006). Discriminative random fields. International Journal of Computer Vision, 68(2), 179–202.
Lafferty, J., McCallum, A., & Pereira, F. (2001). Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of 18th international conference on machine learning.
Lee, A., Mumford, D., & Huang, J. (2001). Occlusion models for natural images: A statistical study of a scale-invariant dead leaves model. International Journal of Computer Vision, 41(1/2), 35–59.
Leuttgen, M., Karl, W. C., Willsky, A. S., & Tenney, R. R. (1993). Multiscale representation of markov random fields. Special Issue of the IEEE Transactions on Signal Processing on Wavelet Transforms, 41(12), 3377–96.
Li, S. Z. (1995). Markov random field modeling in computer vision. New York: Springer.
Martin, D., Fowlkes, C., & Malik, J. (2002). Berkeley segmentation dataset. http://www.cs.berkeley.edu/projects/vision/bsds.
Martin, D., Fowlkes, C., Walker, L., & Malik, J. (2003). Local boundary detection in natural images: Matching human and machine performance. In European conference on visual perception (Perception, 32 supp., p. 55).
Martin, D., Fowlkes, C., & Malik, J. (2004). Learning to detect natural image boundaries using local brightness, color and texture cues. IEEE Transactions on PAMI, 26(5), 530–549.
Mori, G., Ren, X., Efros, A., & Malik, J. (2004). Recovering human body configurations: Combining segmentation and recognition. In Proceedings of IEEE conference on computer vision and pattern recognition (Vol. 2, pp. 326–333).
Mumford, D. (1994). Elastica and computer vision. In C. Bajaj (Ed.), Algebraic geometry and its applications (pp. 491–506). New York: Springer.
Murphy, K., Weiss, Y., & Jordan, M. (1999). Loopy belief propagation for approximate inference: An empirical study. In Proceedings of of UAI (pp. 467–475).
Palmer, S. (1999). Vision science: Photons to phenomenology. Cambridge: MIT Press.
Parent, P., & Zucker, S. W. (1989). Trace inference, curvature consistency, and curve detection. IEEE Transactions on PAMI, 11(8), 823–39.
Pearl, J. (1988). Probabilistic reasoning in intelligent systems: Networks of plausible inference. Palo Alto: Morgan Kaufmann.
Ren, X. (2007). Learning and matching line aspects for articulated objects. In Proceedings of IEEE conference on computer vision and pattern recognition.
Ren, X., & Malik, J. (2002). A probabilistic multi-scale model for contour completion based on image statistics. In Proceedings of 7th European conference on computer vision (Vol. 1, pp. 312–327).
Ren, X., Berg, A., & Malik, J. (2005). Recovering human body configurations using pairwise constraints between parts. In Proceedings of 10th international conference on computer vision (Vol. 1, pp. 824–831).
Ruderman, D. L. (1997). Origins of scaling in natural images. Vision Research, 37(23), 3385–98.
Ruderman, D. L., & Bialek, W. (1994). Statistics of natural images: Scaling in the woods. Physics Review Letters, 73(6), 814–817.
Sharon, E., Brandt, A., & Basri, R. (2000). Completion energies and scale. IEEE Transactions on PAMI, 22(10), 1117–1131.
Shashua, A., & Ullman, S. (1988). Structural saliency: the detection of globally salient structures using a locally connected network. In Proceedings of 2nd international conference on computer vision (pp. 321–327).
Shental, N., Zomet, A., Hertz, T., & Weiss, Y. (2003). Learning and inferring image segmentations with the gbp typical cut algorithm. In Proceedings of 9th international conference on computer vision (pp. 1243–1250).
Shewchuk, J. (1996). Triangle: Engineering a 2d quality mesh generator and delaunay triangulator. In First workshop on applied computational geometry (pp. 124–133).
Sun, J., Shum, H.-Y., & Zheng, N.-N. (2002). Stereo matching using belief propagation. In Proceedings of 7th European conference on computer vision (pp. 510–524).
Thornber, K. K., & Williams, L. R. (2001). Orientation, scale, and discontinuity as emergent properties of illusory contour shape. Neural Computation, 13(8), 1683–1711.
Tu, Z. W., Chen, X. R., L Yuille, A., & Zhu, S. C. (2005). Image parsing: Unifying segmentation, detection, and recognition. International Journal of Computer Vision, 63(2), 113–140.
von der Heydt, R., Peterhans, E., & Baumgartner, G. (1984). Illusory contours and cortical neuron responses. Science, 224, 1260–1262.
Weiss, Y. (1997). Interpreting images by propagating bayesian beliefs. Advances in Neural Information Processing Systems, 10, 908–915.
Weiss, Y. (2000). Correctness of local probability propagation in graphical models with loops. Neural Computation, 1–41.
Wertheimer, M. (1938). Laws of organization in perceptual forms (partial translation). In W. B. Ellis (Ed.), A sourcebook of gestalt psychology (pp. 71–88). New York: Harcourt Brace.
Williams, L. R., & Jacobs, D. W. (1997). Stochastic completion fields: A neural model of illusory contour shape and salience. Neural Computation, 9(4), 837–858.
Williams, L. R., & Thornber, K. K. (1999). A comparison of measures for detecting natural shapes in cluttered backgrounds. International Journal of Computer Vision, 34(2/3), 81–96.
Wu, Q., & Yu, Y. (2003). Two-level image segmentation based on region and edge integration. In Proceedings of DICTA (pp. 957–966).
Yu, S., Gross, R., & Shi, J. (2002). Concurrent object segmentation and recognition with graph partitioning. Advances in Neural Information Processing Systems, 15.
Zhu, S. C. (1999). Embedding gestalt laws in markov random fields. IEEE Transactions on PAMI, 21(11), 1170–1187.
Zhu, S. C., Wu, Y. N., & Mumford, D. B. (1998). Frame: Filters, random field and maximum entropy: Towards a unified theory for texture modeling. International Journal of Computer Vision, 27(2), 1–20.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Ren, X., Fowlkes, C.C. & Malik, J. Learning Probabilistic Models for Contour Completion in Natural Images. Int J Comput Vis 77, 47–63 (2008). https://doi.org/10.1007/s11263-007-0092-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-007-0092-6