Abstract
As a fundamental and challenging task in many subjects such as image processing and computer vision, image segmentation is of great importance but is constantly challenging to deliver, particularly, when the given images or data are corrupted by different types of degradations like noise, information loss, and/or blur. In this article, we introduce a segmentation methodology – smoothing and thresholding (SaT) – which can provide a flexible way of producing superior segmentation results with fast and reliable numerical implementations. A bunch of methods based on this methodology are to be presented, including many applications with different types of degraded images in image processing.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ambrosio, L., Tortorelli, V.: Approximation of functions depending on jumps by elliptic functionals via t-convergence. Commun. Pure Appl. Math. 43, 999–1036 (1990)
Aubert, G., Aujol, J.: A variational approach to removing multiplicative noise. SIAM J. Appl. Math. 68, 925–946 (2008)
Audet, P.: Toward mapping the effective elastic thickness of planetary lithospheres from a spherical wavelet analysis of gravity and topography. Phys. Earth Planet Inter. 226, 48–82 (2014)
Baldi, P., Kerkyacharian, G., Marinucci, D., Picard, D.: Asymptotics for spherical needlets. Ann. Stat. 37(3), 1150–1171 (2009)
Bar, L., Chan, T., Chung, G., Jung, M., Kiryati, N., Mohieddine, R., Sochen, N., Vese, L.: Mumford and shah model and its applications to image segmentation and image restoration. In: Handbook of Mathematical Imaging, pp. 1095–1157. Springer, New York (2011)
Bauer, B., Cai, X., Peth, S., Schladitz, K., Steidl, G.: Variational-based segmentation of biopores in tomographic images. Comput. Geosci. 98, 1–8 (2017)
Bellettini, G., Paolini, M., Verdi, C.: Convex approximations of functionals with curvature. Math. Appl. 2(4), 297–306 (1991)
Benninghoff, H., Garcke, H.: Efficient image segmentation and restoration using parametric curve evolution with junctions and topology changes. SIAM J. Imag. Sci. 7(3), 1451–1483 (2014)
Blake, A., Zisserman, A.: Visual Reconstruction. MIT Press, Cambridge, MA (1987)
Bresson, X., Esedoglu, S., Vandergheynst, P., Thiran, J., Osher, S.: Fast global minimization of the active contour/snake model. J. Math. Imag. Vis. 28(2), 151–167 (2007)
Brown, E., Chan, T., Bresson, X.: Completely convex formulation of the chan-vese image segmentation model. Int. J. Comput. Vis. 98, 103–121 (2012)
Brox, T., Rousson, M., Deriche, R., Weickert, J.: Colour, texture, and motion in level set based segmentation and tracking. Image Vis. Comput. 28, 376–390 (2010)
Cai, X.: Variational image segmentation model coupled with image restoration achievements. Pattern Recogn. 48(6), 2029–2042 (2015)
Cai, X., Steidl, G.: Multiclass segmentation by iterated ROF thresholding. In: Energy Minimization Methods in Computer Vision and Pattern Recognition, pp. 237–250. Springer, Berlin/Heidelberg (2013)
Cai, J., Chan, R., Shen, Z.: A framelet-based image inpainting algorithm. Appl. Comput. Harmon. Anal. 24, 131–149 (2008)
Cai, X., Chan, R., Morigi, S., Sgallari, F.: Framelet-based algorithm for segmentation of tubular structures. In: SSVM. LNCS6667. Springer (2011)
Cai, X., Chan, R., Morigi, S., Sgallari, F.: Vessel segmentation in medical imaging using a tight-frame based algorithm. SIAM J. Imag. Sci. 6(1), 464–486 (2013a)
Cai, X., Chan, R., Zeng, T.: A two-stage image segmentation method using a convex variant of the Mumford–Shah model and thresholding. SIAM J. Imag. Sci. 6(1), 368–390 (2013b)
Cai, X., Fitschen, J., Nikolova, M., Steidl, G., Storath, M.: Disparity and optical flow partitioning using extended potts priors. Inf. Inference J. IMA 4, 43–62 (2015)
Cai, X., Chan, R., Nikolova, M., Zeng, T.: A three-stage approach for segmenting degraded color images: smoothing, lifting and thresholding (SLaT). J. Sci. Comput. 72(3), 1313–1332 (2017). https://doi.org/10.1007/s10915-017-0402-2
Cai, X., Chan, R.H., Schönlieb, C.B., Steidl, G., Zeng, T.: Linkage between piecewise constant Mumford–Shah model and Rudin–Osher–Fatemi model and its virtue in image segmentation. SIAM J. Sci. Comput. 41(6), B1310–B1340 (2019)
Cai, X., Wallis, C.G.R., Chan, J.Y.H., McEwen, J.D.: Wavelet-based segmentation on the sphere. Pattern Recogn. 100 (2020). https://doi.org/10.1016/j.patcog.2019.107,081
Candés, E., Donoho, D.: Continuous curvelet transform: II. Discretization and frames. Appl. Comput. Harmon. Anal. 19(2), 198–222 (2005)
Cardelino, J., Caselles, V., Bertalmio, M., Randall, G.: A contrario selection of optimal partitions for image segmentation. SIAM J. Imag. Sci. 6(3), 1274–1317 (2013)
Chambolle, A.: Total variation minimization and a class of binary MRF models. In: Rangarajan, A., Vemuri, B.C., Yuille, A.L. (eds.) Energy Minimization Methods in Computer Vision and Pattern Recognition – EMMCVPR 2005. Lecture Notes in Computer Science, vol. 3757, pp. 136–152. Springer, Berlin (2005)
Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imag. Vis. 40(1), 120–145 (2011)
Chambolle, A., Caselles, V., Novaga, M., Cremers, D., Pock, T.: An introduction to total variation for image analysis. Theor. Found. Numer. Methods Sparse Recover. Radon Ser. Comput. Appl. Math. 9, 263–340 (2010)
Chan, T.F., Vese, L.A.: Active contours without edges. IEEE Trans. Image Process. 10(2), 266–277 (2001)
Chan, T.F., Sandberg, B.Y., Vese, L.A.: Active contours without edges for vector-valued images. J. Vis. Commun. Image Represent. 11(2), 130–141 (2000)
Chan, T.F., Esedoglu, S., Nikolova, M.: Algorithms for finding global minimizers of image segmentation and denoising models. SIAM J. Appl. Math. 66(5), 1632–1648 (2006a)
Chan, T.F., Esedoglu, S., Park, F., Yip, A.: Total variation image restoration: overview and recent developments. In: Handbook of Mathematical Models in Computer Vision, pp. 17–31. Springer, New York (2006b)
Chan, R., Yang, H., Zeng, T.: A two-stage image segmentation method for blurry images with Poisson or multiplicative Gamma noise. SIAM J. Imag. Sci. 7(1), 98–127 (2014)
Chan, J., Leistedt, B., Kitching, T., McEwen, J.D.: Second-generation curvelets on the sphere. IEEE Trans. Sig. Proc. 65(1), 5–14 (2017)
Chan, R., Yang, H., Zeng, T.: Total Variation and Tight Frame Image Segmentation with Intensity Inhomogeneity (2019). arXiv e-prints arXiv:1904.01760
Chan, R., Kan, K.K., Nikolova, M., Plemmons, R.J.: A two-stage method for spectral-spatial classification of hyperspectral images. J. Math. Imag. Vis. 62, 790–807 (2020)
Chapman, B., Parker, D., Stapelton, J., Parker, D.: Intracranial vessel segmentation from time-of-flight mra using pre-processing of the mip z-buffer: accuracy of the ZBS algorithm. Med. Image Anal. 8(2), 113–126 (2004)
Chen, J., Amini, A.: Quantifying 3d vascular structures in mra images using hybrid pde and geometric deformable models. IEEE Trans. Med. Imag. 23(10), 1251–1262 (2004)
Cremers, D., Rousson, M., Deriche, R.: A review of statistical approaches to level set segmentation: integrating color, texture, motion and shape. Int. J. Comput. Vis. 72(2), 195–215 (2007)
Datt, B., McVicar, T., Van Niel, T., Jupp, D., Pearlman, J.: Preprocessing eo-1 hyperion hyperspectral data to support the application of agricultural indexes. IEEE Trans. Geosci. Remote Sens. 41(6), 1246–1259 (2003)
Dong, B., Chien, A., Shen, Z.: Frame based segmentation for medical images. Commun. Math. Sci. 32, 1724–1739 (2010)
Dong, B., Chien, A., Shen, Z.: Frame based segmentation for medical images. Commun. Math. Sci. 9(2), 551–559 (2011)
Durand, S., Fadili, J., Nikolova, M.: Multiplicative noise removal using l1 fidelity on frame coefficients. J. Math. Imag. Vis. 38, 201–226 (2010)
Eismann, M., Stocker, A., Nasrabadi, N.: Automated hyperspectral cueing for civilian search and rescue. Proc. IEEE 97(6), 1031–1055 (2009)
Fauvel, M., Tarabalka, Y., Benediktsson, J., Chanussot, J., Tilton, J.: Advances in spectral-spatial classification of hyperspectral images. Proc. IEEE 101(3), 652–675 (2013)
Franchini, E., Morigi, S., Sgallari, F.: Composed segmentation of tubular structures by an anisotropic pde model. In: Tai, X.-C., et al. (eds.) SSVM 2009. LNCS5567, pp. 75–86 (2009)
Franchini, E., Morigi, S., Sgallari, F.: Segmentation of 3D tubular structures by a PDE-based anisotropic diffusion model. In: Dæhlen, M., et al. (eds.) MMCS 2008. LNCS5862, pp. 224–241 (2010)
Ge, Q., Liang, X., Wang, L., Zhang, Z., Wei, Z.: A hybrid active contour model with structured feature for image segmentation. Sig. Process 108, 147–158 (2015)
Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. 6, 721–741 (1984)
Goldstein, T., Osher, S.: The split Bregman method for l1-regularized problems. SIAM J. Imag. Sci. 2(2), 323–343 (2009)
Gooya, A., Liao, H., et al.: A variational method for geometric regularization of vascular segmentation in medical images. IEEE Trans. Image Process. 17(8), 1295–1312 (2008)
Gowen, A., O’Donnell, C., Cullen, P., Downey, G., Frias, J.: Hyperspectral imaging-an emerging process analytical tool for food quality and safety control. Trends Food Sci. Technol. 18(12), 590–598 (2007)
Han, D., Sun, D., Zhang, L.: Linear rate convergence of the alternating direction method of multipliers for convex composite programming. Math. Oper. Res. 43(2), 622–637 (2018)
He, Y., Hussaini, M.Y., Ma, J., Shafei, B., Steidl, G.: A new fuzzy c-means method with total variation regularization for image segmentation of images with noisy and incomplete data. Pattern Recogn. 45, 3463–3471 (2012)
Hörig, B., Kühn, F., Oschütz, F., Lehmann, F.: Hymap hyperspectral remote sensing to detect hydrocarbons. Int. J. Remote Sens. 22(8), 1413–1422 (2001)
Jung, Y.M., Kang, S.H., Shen, J.: Multiphase image segmentation via Modica-Mortola phase transition. SIAM J. Appl. Math. 67(5), 1213–1232 (2007)
Kay, D., Tomasi, A., et al.: Color image segmentation by the vector-valued Allen–Cahn phase-field model: a multigrid solution. IEEE Trans. Image Process. 18(10), 2330–2339 (2009)
Kim, W., Kim, C.: Active contours driven by the salient edge energy model. IEEE Trans. Image Process. 22, 1667–1673 (2013)
Kirbas, C., Quek, F.: A review of vessel extraction techniques and algorithms. CV Comput. Surv. 36, 81–121 (2004)
Krissian, K., Malandain, G., Ayache, N., Vaillant, R., Trousset, Y.: Model-based detection of tubular structures in 3d images. CVIU 80, 130–171 (2000)
Le, T., Chartrand, R., Asaki, T.J.: A variational approach to reconstructing images corrupted by Poisson noise. J. Math. Imag. Vis. 27, 257–263 (2007)
Leistedt, B., McEwen, J., Vandergheynst, P., Wiaux, Y.: S2let: a code to perform fast wavelet analysis on the sphere. Astron. Astrophys. 558(A128), 1–9 (2013)
Lellmann, J., Schnörr, C.: Continuous multiclass labeling approaches and algorithms. SIAM J. Imag. Sci. 44(4), 1049–1096 (2011)
Li, S., Hai, Y.: A full-view spherical image format. In: ICPR, pp. 2337–2340 (2010)
Li, C., Kao, C., Gore, J., Ding, Z.: Minimization of region-scalable fitting energy for image segmentation. IEEE Trans. Image Process. 17, 1940–1949 (2008)
Li, F., Ng, M., Zeng, T., Shen, C.: A multiphase image segmentation method based on fuzzy region competition. SIAM J. Imag. Sci. 3(2), 277–299 (2010)
Li, X., Yang, X., Zeng, T.: A three-stage variational image segmentation framework incorporating intensity inhomogeneity information. SIAM J. Imag. Sci. 13(3), 1692–1715 (2020)
Lorigo, L., Faugeras, O., Grimson, E., et al.: Curves: curve evolution for vessel segmentation. Med. Image Anal. 5, 195–206 (2001)
Manolakis, D., Shaw, G.: Detection algorithms for hyperspectral imaging applications. IEEE Sig. Process. Mag. 19(1), 29–43 (2002)
Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: ICCV, vol. 2, pp. 416–423 (2001)
McEwen, J., Hobson, M., Mortlock, D., Lasenby, A.: Fast directional continuous spherical wavelet transform algorithms. IEEE Trans. Sig. Process. 55(2), 520–529 (2007a)
McEwen, J., Vielva, P., Wiaux, Y., et al.: Cosmological applications of a wavelet analysis on the sphere. J. Fourier Anal. Appl. 13(4), 495–510 (2007b)
McEwen, J., Durastanti, C., Wiaux, Y.: Localisation of directional scale-discretised wavelets on the sphere. Appl. Comput. Harm Anal. 44(1), 59–88 (2018)
Michailovich, O., Rathi, Y.: On approximation of orientation distributions by means of spherical ridgelets. IEEE Trans. Sig. Proc. 19(2), 461–477 (2010)
Mumford, D., Shah, J.: Optimal approximation by piecewise smooth functions and associated variational problems. Commun. Pure Appl. Math. XLII, 577–685 (1989)
Paschos, G.: Perceptually uniform color spaces for color texture analysis: an empirical evaluation. IEEE Trans. Image Process. 10(6), 932–937 (2001)
Patel, N., Patnaik, C., Dutta, S., Shekh, A., Dave, A.: Study of crop growth parameters using airborne imaging spectrometer data. Int. J. Remote Sens. 22(12), 2401–2411 (2001)
Pock, T., Chambolle, A., Cremers, D., Bischof, H.: A convex relaxation approach for computing minimal partitions. In: IEEE Conference on Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE, pp. 810–817 (2009a)
Pock, T., Cremers, D., Bischof, H., Chambolle, A.: An algorithm for minimizing the piecewise smooth mumford-shah functional. In: ICCV (2009b)
Rathi, Y., Michailovich, O., Setsompop, K., et al.: Sparse multi-shell diffusion imaging. MICCAI, Int. Conf. Med. Image Comput. Comput.-Assist. Intervent. 14(2), 58–65 (2011)
Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234–241. Springer (2015)
Rotaru, C., Graf, T., Zhang, J.: Color image segmentation in HSI space for automotive applications. J. Real-Time Image Process. 3(4), 311–322 (2008)
Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992)
Sandberg, B., Kang, S., Chan, T.: Unsupervised multiphase segmentation: a phase balancing model. IEEE Trans. Image Process. 19, 119–130 (2010)
Schmitt, J., Starck, J., Casandjian, J., Fadili, J., Grenier, I.: Multichannel Poisson denoising and deconvolution on the sphere: application to the Fermi Gamma-ray Space Telescope. Astron. Astrophys. 546(A114) (2012). https://www.aanda.org/articles/aa/full_html/2012/10/aa18234-11/aa18234-11.html
Shi, J., Osher, S.: A nonlinear inverse scale space method for a convex multiplicative noise model. SIAM J. Imag. Sci. 1, 294–321 (2008)
Simons, F., Loris, I., Nolet, G., et al.: Solving or resolving global tomographic models with spherical wavelets, and the scale and sparsity of seismic heterogeneity. Geophys. J. Int. 187, 969–988 (2011)
Starck, J., Moudden, Y., Abrial, P., Nguyen, M.: Wavelets, ridgelets and curvelets on the sphere. Astron. Astrophys. 446(3), 1191–1204 (2006)
Steidl, G., Teuber, T.: Removing multiplicative noise by Douglas-Rachford splitting methods. J. Math. Imag. Vis. 36(2), 168–184 (2010)
Stein, D., Beaven, S., Hoff, L., Winter, E., Schaum, A., Stocker, A.: Anomaly detection from hyperspectral imagery. IEEE Sig. Process. Mag. 19(1), 58–69 (2002)
Storath, M., Weinmann, A.: Fast partitioning of vector-valued images. SIAM J. Imag. Sci. 7(3), 1826–1852 (2014)
Sum, K., Cheung, P.: Vessel extraction under non-uniform illumination: a level set approach. IEEE Trans. Biomed. Eng. 55(1), 358–360 (2008)
Vese, L., Chan, T.: A multiphase level set framework for image segmentation using the mumford and shah model. Int. J. Comput. Vis. 50(3), 271–293 (2002)
Wallis, C., Wiaux, Y., McEwen, J.: Sparse image reconstruction on the sphere: analysis and synthesis. IEEE Trans. Image Process. 26(11), 5176–5187 (2017)
Wang, L., Li, C., Sun, Q., Xia, D., Kao, C.Y.: Active contours driven by local and global intensity fitting energy with application to brain MR image segmentation. Comput. Med. Imag. Graph. 33(7), 520–531 (2009)
Wang, X., Huang, D., Xu, H.: An efficient local Chan-Vese model for image segmentation. Pattern Recogn. 43, 603–618 (2010)
Wang, X., Tang, Y., Masnou, S., Chen, L.: A global/local affinity graph for image segmentation. IEEE Trans. Image Process. 24(4), 1399–1411 (2015)
Yan, P., Kassim, A.: MRA image segmentation with capillary geodesic active contours. Med. Image Anal. 10, 317–329 (2006)
Yuan, J., Bae, E., Tai, X.C.: A study on continuous max-flow and min-cut approaches. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 2217–2224. IEEE (2010a)
Yuan, J., Bae, E., Tai, X.C., Boykov, Y.: A continuous max-flow approach to potts model. In: European Conference on Computer Vision, pp. 379–392 (2010b)
Zach, C., Gallup, D., Frahm, J.-M., Niethammer, M.: Fast global labeling for real-time stereo using multiple plane sweeps. In: Vision, Modeling, and Visualization Workshop (2008)
Zhang, Y., Matuszewski, B., Shark, L., Moore, C.: Medical image segmentation using new hybrid level-set method. In: 2008 Fifth International Conference BioMedical Visualization: Information Visualization in Medical and Biomedical Informatics, pp. 71–76 (2008)
Zhi, X., Shen, H.: Saliency driven region-edge-based top down level set evolution reveals the asynchronous focus in image segmentation. Pattern Recogn. 80, 241–255 (2018)
Zonoobi, D., Kassim, A., Shen, W.: Vasculature segmentation in mra images using gradient compensated geodesic active contours. J. Sig. Process. Syst. 54, 171–181 (2009)
Acknowledgements
Supported in part by HKRGC Grants No. CUHK14306316, CUHK14301718, CityU11301120, CityU Grant 9380101, CRF Grant C1007-15G, AoE/M-05/12.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 Springer Nature Switzerland AG
About this entry
Cite this entry
Cai, X., Chan, R., Zeng, T. (2023). An Overview of SaT Segmentation Methodology and Its Applications in Image Processing. In: Chen, K., Schönlieb, CB., Tai, XC., Younes, L. (eds) Handbook of Mathematical Models and Algorithms in Computer Vision and Imaging. Springer, Cham. https://doi.org/10.1007/978-3-030-98661-2_75
Download citation
DOI: https://doi.org/10.1007/978-3-030-98661-2_75
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-98660-5
Online ISBN: 978-3-030-98661-2
eBook Packages: Mathematics and StatisticsReference Module Computer Science and Engineering