1 Introduction

The segmentation of ultrasound (US) images is an essential step in computer-aided diagnosis (CAD) systems [1, 2]. The underlying objective of US image segmentation is to separate the lesion from its background, which is necessary for CAD. Besides, US imaging technologies are cost-effective, non-invasive, and practically harmless [3]. Therefore, there are more and more demands for US image segmentation in CAD technologies. However, owing to the noise, ambiguous boundary, and heterogeneity, US image segmentation is yet one of the challenging problems [4]. Various segmentation methods for US images [5, 6] have been proposed. The energy functional–based active contour model (ACM) [7,8,9] is among the most successful technique, which has been extensively used for this application showing useful properties for lesion detection [10,11,12,13].

ACM evolves the closed curve as a level set into a higher one-dimensional function, and the purpose of evolution is to obtain the minimum value of the energy function [9, 14]. In other words, an initial evolving curve is known, and image segmentation based on the energy function can be modeled by using the closed curve that divides the image into regions within and outside the interface. ACM can be divided into two categories according to the image characteristics: edge-based energy function and region-based energy function. The edge-based energy function [8, 15] uses boundary information, which is usually provided by gradient operators to guide the evolution of the curve. This energy function depends on the premise that images have clear and distinct boundaries. The region-based energy function [9, 10, 13] is such a statistical formulation that can incorporate model properties (e.g., curvature and smoothness) and image information (e.g., intensity, color, and texture) for the optimal segmentation flexibly.

However, due to the poor quality such as noise, ambiguous boundary, and heterogeneity of US images, the traditional energy function may suffer from some inherent drawbacks and do not achieve good segmentation performance. Different strategies have been proposed in an attempt to improve the traditional energy function by introducing new information such as region scalable fitting [16] and Gaussian distribution with a bias field [17]. In addition, Fang et al. [10] is our previous works, which is used to segment US images using global and local energy functional (GL-EF). The above methods can produce impressive results, but utilizing some restricted information may weaken the anti-noise ability. Statistically, the energy functional incorporating edge information [18, 19] and the similarity measure [12, 20] can also tackle the model’s sensitivity to noise.

In US image segmentation, the approximation and uncertainty are common and refractory, which can be tackled by machine learning methods, such as deep learning using time-series information [21] and attention-enriched deep learning model [22]. Besides, there have been many methods to control the energy function with machine learning methods in the literature. Here, the machine learning methods contain random forest [23], monogenic signal [24], self-organizing mapping network [25], etc. Among them, the most representative methods include the edge-driven energy function (ED-EF) [26, 27] and the region-driven energy function (RE-EF) [28,29,30,31] based on SVM and fuzzy clustering. Besides, they include the global and local energy function–based US image segmentation method (GL-EF) [10], and the active contours driven by the local Rayleigh distribution fitting energy method (LRDF-ACM) [31].

Concretely, Agus et al. [26] propose an algorithm (i.e., ED-EF) to construct a set of edge stop functions based on an active contour model to segment US images. The algorithm can find the final edge of the segmentation target well, but the algorithm relies on initialization. Meanwhile, Li et al. [30] propose a new level set for selective image segmentation using fuzzy region competition (i.e., RE-EF). It can detect and track any combination of selected objects or image components, but the algorithm has inherent disadvantages when it encounters both weak boundary and low contrast. To solve this problem, Fang et al. [10] propose a new active contour method combining global and local information (GL-EF). Global information can segment US images with noisy and fuzzy boundaries. Moreover, local information can determine the intensity uniformity [32, 33]. For the inhomogeneity of US images, Hui et al. [31] introduce local Rayleigh distribution fitting (LRDF) energy terms in the traditional level set method.

On this basis, this paper proposes a new energy function by integrating the probability projection. The proposed method allows the combination of the probability projection and the energy function in a couple of ways: (1) A learned probabilistic model is obtained by the generalized linear model (GLM), which yield a new probability projection. (2) Both boundary indicator–based and probability statistical–based energy functions are utilized to direct the level set evolution. (3) Integrating the probability projection into the energy functional framework attracts the evolving curve toward the desired lesion boundary. By probability projection based on Gabor wavelet and GLM, the noise problems in US images are solved well. Besides, the boundary indicator–based and probability statistical–based energy functions deal with ambiguous boundary and heterogeneity of US images. Thus, the proposed method achieves rapid convergence to its boundaries and segments the lesion region accurately.

The rest of this paper is organized as follows. Section II briefly introduces the generation of probability projection, including the Gabor filter and the GLM. The boundary indicator–based and probability statistical–based energy functions are elaborated in Sect. III. Section IV reports the experiments and performance evaluations. The last section is reserved for concluding remarks.

2 Generation of image probability projection

To model the lesion region, a feature vector and the corresponding probability projection are generated at each pixel location in the US images using the Gabor filter [34, 35] and the GLM [36,37,38] in this paper. Suppose z is a feature vector extracted by Gabor filter, and ℘ is the probability projection generated by the GLM.

2.1 Feature vector

To further analyze the complex characteristics of the US image, one with ambiguous boundary and heterogeneity is selected as the experimental analysis object in this paper. Fig. 1b shows the gray value distribution map of the gallbladder image in Fig. 1a. As can be seen from Fig. 1b, the lesion and background partially coincide (e.g., the x-axis interval 30 has lesion, boundary, and background), and the classification of each pixel is fuzzy. So it is difficult to distinguish the lesion and background. In this paper, the US image is modeled by the Gabor filter, and then the machine learning method is used to confirm the probability that the pixel belongs to the lesion or background before the segmentation.

Fig. 1
figure 1

The gray value distribution and the corresponding probability projection of the gallbladder image. a The gallbladder image. b The gray value distribution. c The probability projection. Here, red points in a are the positions corresponding to probability values in c, respectively

Full size image

Gabor filter hG(x, y, λ, σ, θ) can analyze the spatial frequencies and the orientations of images, which is defined as

$$ {h}_G\left(x,y,\lambda, \sigma, \theta \right)=1/2\pi \exp \left(-1/2\left({x}^2/{\sigma}_x+{y}^2/{\sigma}_y\right)\right)\cdot \exp \left(2\pi \lambda j\left(x\cos \theta +y\sin \theta \right)\right) $$
(1)

where x and y are the horizontal and vertical coordinates of pixel points in the image, respectively; λ is the modulating frequency of the Gaussian kernel; σ is the standard deviation; and θ is the orientation.

By a 2D convolution operation with hG(x, y, λ, σ, θ), the feature vector z(x, y) of a Gabor filter to an image I(x, y) is obtained:

$$ \mathbf{z}\left(x,y\right)=I\left(x,y\right)\ast {h}_G\left(x,y,\lambda, \sigma, \theta \right) $$
(2)

The frequency, standard deviation, and orientation constitute a set of parameters. The reason why the parameters are chosen is that they can cover the spatial frequency space and enhance the image information. In this paper, four spatial frequencies, four spatial variances, and seven orientations are used.

2.2 Probability projection

Generating the probability projection ℘ requires selecting the appropriate statistical model. The posterior probability ℘ is defined as

$$ \wp =p\left(\mathbf{z}\left|\rho \left(\eta, \kappa \right)\right.\right) $$
(3)

where κ is the convolution kernel [39]. The linear predictor η is an averaging operation [40], and the link function ρ is the identity function [41]. The GLM is one of the most widely used machine learning methods for probability statistics, due to its efficiency for data representation with a relatively small number of parameters. For the GLM, half of the image is used for training and the other 1/2 is used for testing. The basic process of the GLM is as follows:

  • Training: A half of an image is fed into the GLM and the model can directly segment an approximate region around the annotated object.

  • Testing: The trained model predicts the probability value from the other half of the image. Then, the probability is thresholded and replaces the pixel value of each point. Thus, the probability projection ℘ is obtained.

The obtained probability projection fully reflects the position of the image, and the probability information belongs to the lesion region and background, which indicates that pixels are most likely to belong to the lesion region and background. Fig. 1c presents the probability projection obtained from the gallbladder image in Fig. 1a. One can see that there is a significant difference between the lesion and the normal region, so the proposed probability projection can be used to distinguish the lesion and background. However, the probability of the boundary is still fuzzy. Therefore, more necessary information needs to be contained in the proposed model to steer the evolving curve to an accurate segmentation result.

3 Proposed method

4 Using the above obtained image probability projection, boundary indicator–based and probability statistical-based energy functions are introduced in the following essay.

In this paper, the segmentation problem is reduced to the problem of partitioning the domain of definition Ω ⊂ ℜ of the US image I(x, y) (with (x, y) ∈ Ω) into two subsets, namely, lesion region Ω+ and its background Ω−. The two subsets can be defined utilizing a level-set function ϕ(x, y) : Ω → ℜ in the following manner:Ω+ = {(x, y) ∈ Ω : ϕ(x, y) > 0} and Ω− = {(x, y) ∈ Ω : ϕ(x, y) < 0}, respectively. By combining probability mapping, the lesion region and its background are denoted as

$$ \left\{\begin{array}{l}{\boldsymbol{\upzeta}}_{\mathbf{1}}=\int \wp \cdot H\left(\phi \right)d\Omega /\int H\left(\phi \right)d\Omega \\ {}{\boldsymbol{\upzeta}}_{\mathbf{2}}=\int \wp \cdot \left(1-H\left(\phi \right)\right)d\Omega /\int \left(1-H\left(\phi \right)\right)d\Omega \end{array}\right. $$
(4)

where H(Φ) is a smoothed Heaviside function [42, 43]. In this paper, the proposed optimal energy function is defined as

$$ {\phi}^{\ast }=\arg \underset{\phi }{\operatorname{inf}}\left\{E\left(\phi \right)\right\} $$
(5)

with Eb(ϕ) and Ep(ϕ) are boundary indicator–based and probability statistical–based energy functions, which can be described below.

$$ E\left(\phi \right)={E}_b\left(\phi \right)+{E}_p\left(\phi \right) $$
(6)

4.1 Boundary indicator function

Based on the above analysis, the image probability of the lesion, the boundary, and the background is more than 0.5, approximate 0.5, and less than 0.5, respectively. Therefore, the proposed boundary indicator function should have the following properties:

  1. (1)

    It should be approximately equal to 0 on the boundary of the lesion region and positive elsewhere.

  2. (2)

    The value of the boundary indicator function is increased in the location that has a high probability of belonging to the lesion, vice versa; the value is reduced when the probability is low. In this paper, a function that meets these properties is proposed:

$$ {E}_b={\int}_{\Omega}\left[\cos \left(\pi \cdot \boldsymbol{\upzeta} +1/2\pi \right)+1\right]d\Omega $$
(7)

where ζ = (ζ1, ζ2) is the probability connection matrix of lesion region and background. Fig. 2 shows the trend of the boundary indicator function of the interval LR. One can see clearly that the boundary indicator function is approximately equal to 0 at the boundary L and R.

Fig. 2
figure 2

The trend of the boundary indicator function. a The original image. b Boundary indicator function of the interval LR

Full size image

4.2 Probability statistical function

In this paper, the probability statistical-based energy function is constructed:

$$ {E}_p={\int}_{\Omega}\sqrt{{\boldsymbol{\upzeta}}_1\cdot {\boldsymbol{\upzeta}}_{\mathbf{2}}}d\Omega $$
(8)

Subsequently, given a level-set function ϕ, the following evolving curve is computed (see Appendix 1):

$$ {\displaystyle \begin{array}{l}{\phi}_p=-\partial {E}_p\left(\phi \right)/\partial \phi \\ {}=-\delta \cdot \left(1/2{E}_p\left(\phi \right)\cdot \left({A}_{+}-{A}_{-}\right)+1/2\cdot \wp \left(1/A+\cdot \sqrt{{\boldsymbol{\upzeta}}_2/{\boldsymbol{\upzeta}}_1}+1/A-\cdot \sqrt{{\boldsymbol{\upzeta}}_1/{\boldsymbol{\upzeta}}_2}\right)\right)\end{array}} $$
(9)

where δ is the Dirac delta [44, 45].

The proposed probability statistical function can give it the latitude to stop the evolving curve more effectively owing to the probability value of the current pixel. Besides, it can handle complex shapes and topological changes owing to the level set.

At last, by taking the first variation of the proposed energy functional concerning ϕ, the final evolution equation is obtained (see Appendix 1):

$$ {\displaystyle \begin{array}{l}\partial E\left(\phi \right)/\partial \phi =\partial {E}_b\left(\phi \right)/\partial \phi +\partial {E}_p\left(\phi \right)/\partial \phi \\ {}=-\pi \cdot \sin \left(\pi \cdot \boldsymbol{\upzeta} +1/2\pi \right)\cdot \partial \boldsymbol{\upzeta} /\partial \phi +\delta \left(\phi \right)\cdot V\end{array}} $$
(10)

The integrated probability statistical information and the energy function can be complementary for US image segmentation. Furthermore, statistical information can control the direction and velocity of the evolving procedure. Meanwhile, the integrated energy function can facilitate the detection of lesion regions more accurately. The feasibility analysis of the proposed method refers to Appendices 2 and 3. Finally, the iterative steps of the proposed method are summarized in Algorithm 1, and a sequence of steps are depicted intuitively as shown in Fig. 3.

Algorithm 1 Iterative Steps for the Proposed Method

Input: Initialize the parameters σ, θ, and λ of the Gabor filter, the level set function ϕ.

Output: Final contour of the lesion region.

Step 1: noise removal and extract feature vector z by Gabor filter using (3);

Step 2: generate the probability projection ℘;

Step 3: compute the boundary indicator function and the probability statistical function using (6) and (8), respectively;

Step 4: solve the proposed energy functional concerning ϕ using (9);

Step 5: check the convergence of the level set function ϕ, if the solution is not converged, go to step 3.

Fig. 3
figure 3

All steps involved in the proposed method. a Original image. b The feature vector is extracted by the Gabor filter. c The probability projection is generated by the GLM. d–e Intermediate evolution results. f Segmentation result

Full size image

5 Experiments

In this section, to evaluate the effectiveness of the method, 40 pairs of US images are selected from the datasets (http://onlinemedicalimages.com/index.php/en/site-map, http://www.ultrasound-images.com, http://www.radiologyinfo.org) for the experiment. Here, eight representative images are selected to present the experimental results in Fig. 5; images A to H are intramural nodule (237*183), gall bladder (244*206), cholecystitis (284*203), liquefied hematoma (251*205), breast cyst (296*350), thymic cyst (247*206), malignant solid mass(768*520), and liver hydatid cysts (166*90), respectively. The ground truths of these images are created by imaging experts for comparison and testing. All experiments are implemented with Matlabs R2019a in a Windows 10 system and run on an Intel (R) Core (TM) i5-4210H CPU@2.90GHz.

5.1 Experimental evolution

To prove the effectiveness of the proposed method, the visual examples of the evolving processes are depicted in Fig. 4. Both the simulated images (the first two rows) and real US images (the last two rows) are carried out. Here, the simulated images with noise, ambiguous boundary (the first row), and heterogeneity (the second row) are chosen. Besides, the first column shows the original images and the initial curves. The second and third columns show the concrete evolving processes. The segmentation results and the corresponding energy function are shown in the fourth and fifth columns. According to the ground truths shown in the sixth column, the proposed method successfully obtain segmentation results.

Fig. 4
figure 4

The proposed segmentation method on the simulated images (the first two rows) and real US images (the last two rows). First column: original images with initialization. Second-third columns: intermediate evolution results. Fourth column: segmentation results. Fifth column: ground truths. Sixth column: the final energy functional

Full size image

5.2 Effectiveness and accuracy of the proposed method

The proposed segmentation method (GLM-EF) combines two components: the probability-based method (GLM) and the energy function–based method (EF). To further verify the effectiveness of the proposed method, the segmentation performance of each component is tested separately and then combined, as shown in Fig. 5.

Fig. 5
figure 5

Comparison of the proposed method with the separate component. First column: the probability-based method (GLM). Second column: the energy function-based method (EF). Third column: the proposed segmentation method (GLM-EF). Fourth column: ground truths

Full size image

The probability-based GLM shows a marked improvement, unlike the EF. Indeed, the learning process helps to differentiate between the lesion region and background in US images.

However, the probability projection provides a fuzzy localization which limits the segmentation accuracy. Besides, despite the robustness of the energy-based method, segmentation results are unsatisfactory. Most lesions of US images contain ambiguous boundaries, which can influence the contour evolution. Compared with each component, it can be seen that the proposed method slightly improves the segmentation. Therefore, we can get that the proposed energy function incorporated with the probability projection can enhance the precision of the segmentation process. The proposed method is also compared with different state-of-the-art methods: two machine learning-based energy functions (the ED-EF method [26, 27] and the RE-EF method [30] ), the GL-EF method [10], and the LRDF-ACM method [31], as shown in Fig. 6.

Fig. 6
figure 6

Comparison of the proposed method with other methods. First column: the edge-driven energy function (ED-EF). Second column: the region-driven energy function (RE-EF). Third column: the global and local energy function-based US image segmentation method (GL-EF). Fourth column: the active contours driven by local Rayleigh distribution fitting energy (LRDF-ACM). Fifth column: the proposed segmentation method (GLM-EF). Sixth column: ground truths

Full size image

Their results, labeled as ED-EF, RE-EF, GL-EF, and LRDF-ACM, are shown in the first four columns of Fig. 6, respectively. The parameters of these methods are chosen according to their best performance. However, for the sake of fairness, the same initialization is used.

Fig. 6 shows classical examples of US image segmentation, in which different regions including the lesion and the background intertwine with each other. Because of severe heterogeneity and ambiguous boundary, the ED-EF and the RE-EF methods are not sufficient to segment the lesions, as shown in the first two columns. Although the GL-EF takes both local and global information into account, they eventually perform inaccurate segmentation because of ambiguous boundary and complex cases, as shown in the third and fourth columns. In contrast, the proposed method enhances the probability projection and makes the proposed energy functional effective to segment lesion well, as shown in the fifth column.

In this paper, a quantitative evaluation is also carried out to measure the performances of the proposed method. Here, four metrics are used: the DICE coefficient [44], the Jaccard (JAC) distance [45], root-mean-square error (RMSE) [46], and mean absolute error (MAE) [47]. For DICE and JAC, higher scores mean better performances. For RMSE and MAE, the smaller the value, the better the segmentation performance. To verify the performance of the proposed algorithm further, 40 pairs of US images are selected in three datasets mentioned above. Besides, the mean DICE, JAC, RMSE, and MAE of the experimental results are shown in Fig. 7. The results show that the proposed algorithm is favorable compared with the state-of-the-art performance.

Fig. 7
figure 7

Quantitative analysis of the proposed method for US image segmentation

Full size image

5.3 Iterations and time

Iterations and time (in seconds) of the methods in Figs. 5 and 6 are studied as shown in Table 1. Compared with the other methods, the proposed method needs fewer iterations and time due to the following reasons: (1) with a suitable evolution strategy, the noise and heterogeneity can be handled well, and (2) the probability projection based on the GLM is integrated into the energy function, which can accelerate the evolution.

Table 1 Iterations and average time of each method
Full size table

5.4 Analysis of initialization

It is noteworthy that due to image noise, ambiguous boundary, and heterogeneity, initialization will dominate the level set evolution. Considering the complexity of medical image segmentation, most computerized systems run in a semi-automatic or interactive manner. In this paper, different segmentation results can be obtained by using different initial contours on the same image. Therefore, the proposed method requires a robust initialization, which contains the approximate position of objects in the image. In this paper, two types of initialization are used:

(1) Manual method: In this setting, initialization is done manually by visual inspection. The input required from the user is a single click to indicate the approximate initialization of the lesion.

(2) Automatic method: the common practice is to employ automatic initialization. Inspired by Li et al. [30], we construct automatic initialization as follows:

$$ {\phi}_0=2\left(\overline{\wp_n}>\theta \right)-1 $$
(11)

where \( \overline{\wp_n} \) represents the average gray value of n neighborhood pixels and θ is customizable between 0 and 1. A large number of experiments verify that a conservative 0.5 works well in this paper.

The segmentation results on the simulated image (the first two rows) and the real US image (the last two rows) are shown in Fig. 9. The first three columns show the results of the manual initialization. The last column shows the segmentation results of the automatic initialization. The corresponding quantitative evaluations, e.g., DICE and JAC, are shown in Fig. 8. From Figs. 8 and 9, one can get that as long as the manual initialization contains partial lesion information (the first two columns), better segmentation results can be achieved. However, it can still be a tedious task for large-scale datasets. Besides, the automatic procedure results in unstable results, mainly because this type of initialization can lead to the local optimum of the likelihood function.

Fig. 8
figure 8

Quantitative analysis of different initialization. a Simulated image. b Real US image

Full size image
Fig. 9
figure 9

Segmentation results with different initialization on the simulated images (the first two rows) and real US images (the last two rows). The first three columns: manual initialization. The last column: automatic initialization

Full size image

5.5 Analysis of the effect of different noise

For most medical US images, there exists speckle noise [48]. Considering the complexity of US medical images and the effect of image noise on the evolution of level sets, three images are added with different noise (i.e., speckle and Gaussian) with different variances, as shown in Fig. 10. It can be seen from the experimental results that the proposed method is more robust to different noise with different variances. This is because that our method contains the Gabor wavelet and the GLM, which is useful for removing different noise.

Fig. 10
figure 10

Segmentation results of different noise

Full size image

5.6 Analysis of energy function

In this paper, a novel energy functional method based on boundary indicators and probability statistics is proposed. To verify the effectiveness of the proposed energy function and each component, three experiments are carried out, as shown in Fig. 11. From Fig. 11, one can get that the proposed method is superior to that of any of the two individual energy functions.

Fig. 11
figure 11

Segmentation results of different energy function

Full size image

6 Conclusion

In this paper, the complementary strengths of the machine learning method and the energy function are combined into a robust workflow. To segment the lesion region in the US image efficiently, the US image is denoised by the Gabor wavelet and the GLM. Besides, each value of the probability projection is considered as a weighting coefficient of the boundary indicator–based and probability statistical–based energy function. The energy function utilizes a dynamic variational method for US image segmentation compared to the GLM using pixel classification. This makes it possible to emphasize the lesion region and allows rapid convergence to its boundaries. To verify the proposed algorithm, different initializations and the influence of noise are analyzed. Its performance has been validated on different US image datasets. Qualitative and quantitative performance evaluations of the proposed method demonstrate improvements in accuracy.