Automatic gallbladder and gallstone regions segmentation in ultrasound image

Abstract

Purpose

As gallbladder diseases including gallstone and cholecystitis are mainly diagnosed by using ultra-sonographic examinations, we propose a novel method to segment the gallbladder and gallstones in ultrasound images.

Methods

The method is divided into five steps. Firstly, a modified Otsu algorithm is combined with the anisotropic diffusion to reduce speckle noise and enhance image contrast. The Otsu algorithm separates distinctly the weak edge regions from the central region of the gallbladder. Secondly, a global morphology filtering algorithm is adopted for acquiring the fine gallbladder region. Thirdly, a parameter-adaptive pulse-coupled neural network (PA-PCNN) is employed to obtain the high-intensity regions including gallstones. Fourthly, a modified region-growing algorithm is used to eliminate physicians’ labeled regions and avoid over-segmentation of gallstones. It also has good self-adaptability within the growth cycle in light of the specified growing and terminating conditions. Fifthly, the smoothing contours of the detected gallbladder and gallstones are obtained by the locally weighted regression smoothing (LOESS).

Results

We test the proposed method on the clinical data from Gansu Provincial Hospital of China and obtain encouraging results. For the gallbladder and gallstones, average similarity percent of contours (EVA) containing metrics dice’s similarity , overlap fraction and overlap value is 86.01 and 79.81%, respectively; position error is 1.7675 and 0.5414 mm, respectively; runtime is 4.2211 and 0.6603 s, respectively. Our method then achieves competitive performance compared with the state-of-the-art methods.

Conclusions

The proposed method is potential to assist physicians for diagnosing the gallbladder disease rapidly and effectively.

Introduction

Medical image segmentation, such as ultrasound image [1,2,3], computed tomography image [4,5,6], magnetic resonance image [7,8,9], has been playing an increasingly significant role in image processing. Since ultrasound images have some significant advantages, such as economy, convenience, good experimental repeatability, non-electromagnetic radiation, it has become the most commonly used technique for clinical diagnosis of gallstones. However, if quantification of gallbladder and gallstone size is desired, currently these structures must be manually delineated, which are labor-intensive and time-consuming. We therefore develop an automatic gallbladder and gallstone regions segmentation method, which is endowed with a great importance.

Fig. 1

The general flowchart of the new method

Although the automatic gallbladder and gallstone regions segmentation in ultrasound images develops slowly, there are still several approaches published in literatures. Ciecholewski et al. presented a modified snake model to search boundaries of the gallbladder and gallstones [10, 11]. Xie et al. proposed a segmentation and extraction method of gallstones by the level set [12]. Based on the automatic scribbling proposed by Abhishek et al. [13] and the closed form matting proposed by Levin et al. [14], Agnihotri et al. presented an approach to distinguish gallstones from other tissues [15]. However, aforementioned methods are semi-automatic, and their parameters setting depends on subjective experience and human knowledge.

In this paper, we present an automatic segmentation method to provide an effective segmentation result to assist physicians’ diagnosis. Presented automatic segmentation method has high diagnosis accuracy, less time-consuming, lower diagnosis cost and is the premise for detecting automatically stone types by feature extraction.

We modify the Otsu algorithm [16] and the anisotropic diffusion algorithm [17] to reduce much speckle noise and to enhance image contrast. The modified Otsu algorithm is automatically able to search the gallbladder segmentation thresholds between the weak edge regions and the central region. Segmentation results are processed by a morphological filter to deicide the gallbladder region. In gallstones segmentation, we adopt a parameter-adaptive pulse-coupled neural network (PA-PCNN) and present a modified method to coarsely and finely segment the gallstone region. In the post-processing, the locally weighted regression smoothing (LOESS) is employed to obtain accurate boundaries of the gallbladder and gallstones. We conduct lots of experiments to demonstrate that the new method has shorter runtime and higher accuracy rates than the state-of-the-art methods. Our segmentation results of the gallbladder and gallstones in ultrasound images are successful in about 75% of the cases.

The rest of this paper is organized as follows:“Materials” section introduces materials used for the experiments. “Segmentation of the gallbladder and gallstones” Section describes coarse-fine segmentation of the gallbladder and gallstones. “Experiments and Analysis” Section performs a large number of experiments to validate robustness of the new method, discusses the strengths and the weaknesses of that, and suggests next work. “Conclusion” Section makes conclusion.

Materials

The ultrasound images from Gansu Provincial Hospital in Gansu, China, were acquired with a Siemens Acuson X300 ultrasound machine. The dataset from 60 patients includes 60 gray-level ultrasound images of the gallbladder with stones, their clinical presentations and manual drawing results from two provincial experts in radiology. For the gallbladder, 60 images are divided into two groups: 45 images from the regular shapes and 15 images from the irregular shapes. For gallstones, 60 images are divided into five groups: 42 images containing a large stone, nine images containing a small stone, three images where the stone covers a large part of the bottom of the gallbladder, three images containing a large stone with the acoustic shadow, and three images containing stones filling a majority of the gallbladder region. Each image has 256 gray levels and resolution of 512512 pixels.

Fig. 2

The five candidate values of $R_{\mathrm{max}}$ and the value of $MAL_{\mathrm{min}}$

Fig. 3

The examples of the ultrasound images from Gansu Provincial Hospital in Gansu, China

Segmentation of the gallbladder and gallstones

The framework of the new method

The whole method comprises six key steps as follows:

• 1. Search image segmentation thresholds in the pre-processing step according to a modified Otsu method.

• 2. Reduce noise and enhance contrast for the pre-processing based on modified anisotropic diffusion.

• 3. Obtain fine segmentation result of the gallbladder in light of a morphology filtering algorithm.

• 4. Record coarse segmentation result of gallstones by the results of the gallbladder and PA-PCNN.

• 5. Deduce fine segmentation of gallstones in terms of a proposed region-growing algorithm.

• 6. Obtain final segmentation results by a Loess algorithm.

Figure 1 shows the general flowchart of our method, and the details are elaborated in the following sections.

Morphological structuring element

Morphological operations are essential in following segmentation steps, so disk-shaped structuring elements are employed in our works and their radius values are determined by $R_{\mathrm{max}}$, $R_{\mathrm{mid}}$ and $R_{\mathrm{min}}$. These radii denote different distance values among pixels as described in (2)–(4). Moreover, $D_{\mathrm{l}},\, D_{\mathrm{r}},\, D_{\mathrm{u}},\, D_{\mathrm{d}}$ denote the four minimum distance values between the boundaries of the gallbladder and four sides of the whole ultrasound image (see Fig. 2a). $MAL_{\mathrm{max}}$ and $MAL_{\mathrm{min}}$ denote minor axis lengths of the gallbladder and gallstone regions, specifying those of the ellipse regions that have the same normalized second central moments as shown in Fig. 2a, b. Finally, $R_{\mathrm{max}}$, $R_{\mathrm{mid}}$ and $R_{\mathrm{min}}$ for disk-shaped structuring elements are expressed as follows:

$$\begin{aligned}&D=\{D_l ,\, D_r ,\, D_u ,\, D_d ,\, MAL_{\mathrm{max}} \} \end{aligned}$$

(1)

$$\begin{aligned}&R_{\mathrm{max}} =D_{\mathrm{min}} \end{aligned}$$

(2)

$$\begin{aligned}&R_{\mathrm{mid}} =\sqrt{MAL_{\mathrm{max}}} \end{aligned}$$

(3)

$$\begin{aligned}&R_{\mathrm{min}} =\sqrt{MAL_{\mathrm{min}}}. \end{aligned}$$

(4)

Fig. 4

The flowchart of the pre-processing step

The pre-processing

Due to Rayleigh and scattering diffraction of acoustical waves, ultrasound images are in poor contrast and with much speckle noise (Fig. 3) [18]. Therefore, before automatic segmentation, a pre-processing method combining a modified Otsu algorithm and a modified anisotropic diffusion algorithm is used to reduce speckle noise and the flowchart is shown in Fig. 4.

Calculating segmentation thresholds

According to intensity distribution ranges in ultrasound images, the gallbladder region is divided into the central region, weak edge regions and strong edge regions (Fig. 5). Since the central region is regard to the region of interesting (ROI) for the full-size image in physicians’ clinical diagnosis, it is necessary to calculate the normalized threshold to distinguish the central region from above two regions. For the purpose, in contrast to [19, 20], we propose a modified Otsu algorithm to determine the normalized threshold $S_{de}$

$$\begin{aligned} S_{\mathrm{de}} =S_{\mathrm{bg}} \times (1-A)+S^{\prime }\times A \end{aligned}$$

(5)

According to (5), the normalized Otsu threshold $S'$ separates the whole image into the object and the background, and its intensity range is $1\ge S' \ge 0$. $S_{\mathrm{bg}}$ denotes normalized Otsu threshold of the background and its intensity range is $1\ge S_{\mathrm{bg}}\ge 0$. A and $1-A$ are the weight values of $S'$ and $S_{\mathrm{bg}}$, respectively. A is expressed as

$$\begin{aligned} A=S_{\mathrm{ob}}^{\mathrm{n}} \end{aligned}$$

(6)

where $S_{\mathrm{ob}}$ denotes the normalized Otsu threshold of the object and its intensity range is $1 \ge S_{\mathrm{ob}} \ge 0$. n denotes loop number. Obviously, the larger the value of n, the smaller the value of A becomes. What is more, the element values in $S_{\mathrm{de}}$ with uncertain intensity ranges of the gallbladder region include calculating results of n loops and their intensity ranges are $S' \ge S_{\mathrm{de}} \ge S_{\mathrm{bg}}$ (see Fig. 6).

Fig. 5

The different regions of the gallbladder

Subsequently, by a large number of experiments, the termination condition is obtained and described as

$$\begin{aligned} A<1-e^{\left( {-S_{\mathrm{bg}} } \right) } \end{aligned}$$

(7)

According to (7), the variations of the weight value A in Fig. 3a–d are shown in Fig. 7a, b, respectively. In the example of Fig. 3a, when given loop number $n=3$, the termination condition is satisfied and we acquire three thresholds in $S_{\mathrm{de}}$ (Fig. 7a–c), and these thresholds are saved in the weight matrix. As an analogy, n loops before satisfying the termination condition can acquire n thresholds.

Fig. 6

The relations of different thresholds in ultrasound images

Fig. 7

The variations of the weight value Awith loops: (a) and (b) the graphs showing the variations of the weight value A in Fig. 3a–d, respectively; (c) the graph showing the gray histogram in Fig. 3a (the line type specifier ’–’ denotes the curves of the weight value A in (a) and (b), respectively; The line type specifier ’-’ denotes the values of $1-e^{\left( {-S_{{\mathrm{bg}}}}\right) }$ in (a) and (b), respectively; the marker specifier ” denotes effective loops before dissatisfying the termination condition (7) in a, b, respectively; the marker specifier ’${^{\circ }}$’ denotes ineffective loops after satisfying the termination condition (7) in a , b; the red line, the green line and the blue line in Fig. 7 c denote the thresholds of $S_{\mathrm{de}}$ in Fig. 7 a in loop number n = 1, 2, 3, respectively)

Smoothing ultrasound images

In the pre-processing step, each threshold of $S_{\mathrm{de}}$ in the weight matrix can segment the original image and generate one pre-processing ultrasound image. Based on these thresholds, we reset the pixel intensities of the original image. If the intensity of the corresponding pixel is less than the threshold in $S_{\mathrm{de}}$, the output intensity is set to the minimum intensity of the whole image. Otherwise, it is set by a modified algorithm based on the anisotropic diffusion [17] as follows:

$$\begin{aligned} S_{\mathrm{new}} (i,j)=S(i,j)+\lambda (K_{\mathrm{E}} S_{\mathrm{E}} +K_{\mathrm{W}} S_{\mathrm{W}} +K_{\mathrm{S}} S_{\mathrm{S}} +K_{\mathrm{N}} S_{\mathrm{N}} ) \end{aligned}$$

(8)

where S(i, j) and $S_{\mathrm{new}}(i,j)$ in the position (i, j) denote the pixel intensities of the original image and the pre-processing image, respectively. $S_{\mathrm{E}}$, $S_{\mathrm{W}}$, $S_{\mathrm{S}}$ and $S_{\mathrm{N}}$ denote partial derivatives of S(i, j) in four directions as listed in (9). $K_{\mathrm{E}}$, $K_{\mathrm{W}}$, $K_{\mathrm{S}}$ and $K_{\mathrm{N}}$ denote the weighing coefficients of $S_{\mathrm{E}}$, $S_{\mathrm{W}}$, $S_{\mathrm{S}}$ and $S_{\mathrm{N}}$, respectively, as described in (10)

$$\begin{aligned} S_{\mathrm{E}}= & {} \left| {(S(i,j+1)-S(i,j))} \right| \nonumber \\ S_{\mathrm{W}}= & {} \left| {(S(i,j-1)-S(i,j))} \right| \nonumber \\ S_{\mathrm{S}}= & {} \left| {(S(i+1,j)-S(i,j))} \right| \nonumber \\ S_{\mathrm{N}}= & {} \left| {(S(i-1,j)-S(i,j))} \right| \end{aligned}$$

(9)

$$\begin{aligned} K_{\mathrm{E}}= & {} e^{\left( {-\frac{{S^{2}}_{\mathrm{E}} }{f^{2}}} \right) } \quad \quad \quad K_{\mathrm{W}} =e^{\left( {-\frac{{S^{2}}_{\mathrm{W}} }{f^{2}}} \right) }\nonumber \\ K_{\mathrm{S}}= & {} e^{\left( {-\frac{{S^{2}}_{\mathrm{S}} }{f^{2}}} \right) } \quad \quad \quad K_{\mathrm{N}} =e^{\left( {-\frac{{S^{2}}_{\mathrm{N}} }{f^{2}}} \right) } \end{aligned}$$

(10)

According to (10), $K_{\mathrm{E}}$, $K_{\mathrm{W}}$, $K_{\mathrm{S}}$ and $K_{\mathrm{N}}$ are obtained automatically by the parameter f. This parameter represents the average gradient value of the original image. In addition, the control coefficient $\lambda$ in (8) is set to 1 / f.

Finally, n thresholds in $S_{\mathrm{de}}$ acquire n pre-processing images as described in Algorithm 1 [21]. These images have high contrast and little speckle noise (see Fig. 8). This indicates that our method has a good pre-processing effect.

Fig. 8

n pre-processing ultrasound images: (a) Fig. 3a; (b)–(d) Pre-processing images from Fig. 3a in loop number $n=1,2,3$, respectively; (e) Fig. 3d; (f)–(i) Pre-processing images from Fig. 3d in loop number $n=1,2,3,4$, respectively

Fig. 9

The segmentation results of the gallbladder: the first column is the coarse segmentation results of the gallbladder from Fig. 8b, f, respectively; the second column denotes the fine segmentation results of the gallbladder from the first column; the third column denotes located binary images from the second column; the fourth column denotes located ultrasound images from the third column

Coarse-fine segmentation of the gallbladder

After obtaining n pre-processing ultrasound images, we can perform coarse-fine segmentation of the gallbladder (see Fig. 9), and the whole algorithm comprises four steps:

• 1. Process n pre-processing ultrasound images to acquire n binary images by n aforementioned normalized thresholds in $S_{de}$. If the intensity is less than the threshold in $S_{de}$, the output intensity of the corresponding pixel is set to 1; otherwise, it is set to 0.

• 2. Remove boundary connecting regions of all binary images and select a maximum region as coarse segmentation result of the gallbladder (Fig. 9a, e).

• 3. Adopt the disk-shaped structuring element with a radius of $R_{\mathrm{max}}$ to perform morphological closing operation (Fig. 9b, f).

• 4. Locate the gallbladder region by the parameter $R_{\mathrm{max}}$, which is also regarded to the distance value between the gallbladder contour and four sides of the located image (Fig. 9c, g).

Coarse-fine segmentation of gallstones

For coarse segmentation of gallstones, we firstly make use of a parameter-adaptive pulse-coupled neural network (PA-PCNN) to determine high-intensity regions including gallstones in the located ultrasound image. And then, we compute the intersection between these high-intensity regions and the fine segmentation region of the gallbladder and choose a maximum region as coarse segmentation result of gallstones. In addition, we are suggested to a modified region-growing algorithm for eliminating manual labeled regions and preventing from over-segmentation. Finally, we obtained fine segmentation result of gallstones.

Fig. 10

The coarse segmentation results of gallstones: the first column shows segmentation results using SPCNN in the third iteration from Fig. 9d, h, respectively; the second column shows segmentation results using PA-PCNN in the third iteration from Fig. 9d, h, respectively; c the intersection region between (b) and Fig. 9c; g the intersection region between (f) and Fig. 9(g); the fourth column is gray images from (c) and (g), respectively

Coarse segmentation of gallstones

Pulse-coupled neural network (PCNN) derived from Echorn’s cortical model [22] is a single-layer neural network and does not need any training, which is appropriate for image segmentation [23]. In contrast to PCNN, Chen et al..’s SPCNN model [24] derived from Zhan et al..’s SCM model [25] has low computational complexity and high segmentation precision. Therefore, the SPCNN model is employed in this paper and realized as follows:

$$\begin{aligned} F_{\mathrm{ij}} [n]= & {} S_{\mathrm{ij}} \end{aligned}$$

(11)

$$\begin{aligned} L_{\mathrm{ij}} [n]= & {} V_{\mathrm{L}} \sum \limits _{\mathrm{kl}} {W_{\mathrm{ijkl}} Y_{\mathrm{kl}} } [n-1] \end{aligned}$$

(12)

$$\begin{aligned} U_{\mathrm{ij}} [n]= & {} e^{{-\alpha _f}}U_{\mathrm{ij}} [n\!-\!1]\!+\!S_{\mathrm{ij}} (1\!+\!\beta V_{\mathrm{L}} \sum \limits _{\mathrm{kl}} {W_{\mathrm{ijkl}} Y_{\mathrm{kl}} } [n\!-\!1]) \nonumber \\ \end{aligned}$$

(13)

$$\begin{aligned} Y_{\mathrm{ij}} [n]= & {} \left\{ {{\begin{array}{l} 1,\quad \;if\;U_{\mathrm{ij}} [n]>E_{\mathrm{ij}} [n-1] \\ 0,\quad \;else \\ \end{array} }} \right. \end{aligned}$$

(14)

$$\begin{aligned} E_{\mathrm{ij}} [n]= & {} e^{-\alpha _e }E_{\mathrm{ij}} [n-1]+V_{\mathrm{E}} Y_{\mathrm{ij}} [n] \end{aligned}$$

(15)

where

$$\begin{aligned} W_{\mathrm{ijkl}} =\left[ {{\begin{array}{ccc} {0.5}&{}\quad 1&{}\quad {0.5} \\ 1&{}\quad 0\quad &{}\quad 1 \\ {0.5}&{}\quad 1&{}\quad {0.5} \\ \end{array} }} \right] \end{aligned}$$

(16)

Neuron $N_{\mathrm{ij}}$ in position (i, j) receives two inputs: simplified feeding input $F_{\mathrm{ij}}[n]$, which denotes an input stimulus $S_{\mathrm{ij}}$, and linking input $L_{\mathrm{ij}}[n]$, which denotes the products of eight neighboring outputs, a synaptic weight $W_{\mathrm{ijkl}}$ and the amplitude $V_{\mathrm{L}}$. These two inputs are coupled by linking strength $\beta$ to generate an internal activity $U_{\mathrm{ij}}[n]$ recording previous internal activity by a decay factor $e^{{-\alpha _{\mathrm{f}}}}$. $\alpha _{\mathrm{f}}$ and $\alpha _{\mathrm{e}}$ denote the decay coefficients of the internal activity $U_{\mathrm{ij}}[n]$ and the dynamic threshold $E_{\mathrm{ij}}[n]$, respectively. According to (14), when $U_{\mathrm{ij}}[n]\ge E_{\mathrm{ij}}[n-1]$, neuron $N_{\mathrm{ij}}$ fires ($Y_{\mathrm{ij}}[n]=1)$ and the dynamic threshold $E_{\mathrm{ij}}[n-1]$ can suddenly increase by the amplitude $V_{\mathrm{E}}$. When $U_{\mathrm{ij}}[n]<E_{\mathrm{ij}}[n-1]$, neuron $N_{\mathrm{ij}}$ does not fire ($Y_{\mathrm{ij}}[n]=0$) and the dynamic threshold $E_{\mathrm{ij}}[n-1]$ would decrease by a factor $e^{-\alpha f}$. Additionally, $Y_{\mathrm{kl}}[n-1]$ denotes previous neighboring outputs of neuron $N_{\mathrm{ij}}$.

For the SPCNN model, there are five critical parameters $\alpha _{\mathrm{f}}$, $\alpha _{\mathrm{e}}$, $\beta$, $V_{\mathrm{E}}$, $V_{\mathrm{L}}$, and these parameters are obtained automatically by [24] as follows.

$$\begin{aligned}&\alpha _{f} =\log \left( \frac{1}{{\upsigma }}\right) \end{aligned}$$

(17)

$$\begin{aligned}&\beta =\frac{(S_{\mathrm{max}} /S^{\prime })-1}{6V_L } \end{aligned}$$

(18)

$$\begin{aligned}&V_E =e^{{-\alpha _{\mathrm{f}}}}+1+6\beta V_L \end{aligned}$$

(19)

$$\begin{aligned}&V_L =1 \end{aligned}$$

(20)

$$\begin{aligned}&\alpha _{\mathrm{e}} =\ln \left( \frac{V_E }{S^{\prime }M[3]}\right) \end{aligned}$$

(21)

$$\begin{aligned}&M[3]=\frac{1-e^{{-3\alpha _{f}}}}{1-e^{{-\alpha _{f}}}}+6\beta V_L e^{{-\alpha _{\mathrm{f}}}} \end{aligned}$$

(22)

Fig. 11

The intensity ranges using SPCNN and PA-PCNN within the first pulsing cycle for Fig. 9d: a the intensity ranges of SPCNN; b the intensity ranges of PA-PCNN

Fig. 12

The flowchart for fine segmentation of gallstones

where $\sigma$ represents the standard deviation of the whole image. $S'$ and $S_{\mathrm{max}}$ denote the Otsu threshold and the maximum intensity of the image, respectively. For the sake of further improving segmentation precision, the PA-PCNN model based on SPCNN can be used. In the PA-PCNN model, we change threshold $S'$ to $S'_{\mathrm{ob}}$, which denotes the Otsu threshold of the object in located ultrasound image, to generate a larger dynamic threshold. Obviously, all neurons fire only once and need two iterations for SPCNN or four iterations for PA-PCNN at most after the second iteration (see Fig. 11). Iteration times of SPCNN are less than those of PA-PCNN, which guarantees the segmentation precision of gallstones. Meanwhile, within the first pulse cycle after the second iteration, we found that PA-PCNN would generate a good segmentation result in the third iteration. So, we can determine segmented regions involving the gallstone region in the third iteration. Subsequently, coarse segmentation steps are described as follows:

• 1. Use the PA-PCNN model to obtain segmented regions (Fig. 10b, f).

• 2. Calculate the intersection region between the fine result of the gallbladder and the above segmented regions and determine a maximum intersection region as coarse segmentation result of gallstones (see Fig. 10c, g).

Fine segmentation of gallstones

For fine segmentation of gallstones, we propose a modified region-growing method to remove the regions labeled by physicians (‘+’ from white or black regions on the left or the right of the gallstone region) and prohibit over-segmentation. The flowchart of the method is described in Fig. 12.

A Determinants of modified region-growing algorithm

The region-growing method has broad applications in image segmentation field [26, 27]. In this paper, a modified region-growing method is presented and its determinants depend on the seed region, the growing conditions and the termination conditions. Here, the selection steps of the seed region are given as follows:

• 1. Search the centroid of the rectangle seed region by coarse segmentation result of gallstones.

• 2. Set $R_{\mathrm{min}}$ as the radius value of the rectangle seed region.

According to above steps, the rectangle region is determined as the initial seed region. However, sometimes the initial seed region is not a perfect rectangle region because the seed region exceeds probably the coarse segmentation region of gallstones. Therefore, we can remove seed points outside the coarse region and acquire ultimately the initial seed region. Subsequently, growing conditions are described as follows:

$$\begin{aligned}&|S(i,j)-S_{\mathrm{mean}} |<k\times \min ((F_{\mathrm{x}} )_{\mathrm{mean}},\nonumber \\&\quad (F_{\mathrm{y}} )_{\mathrm{mean}} ),(i,j)\in N_8 (seed) \end{aligned}$$

(23)

$$\begin{aligned}&S(i,j)>S^{\prime }_{\mathrm{ob}} \end{aligned}$$

(24)

In located ultrasound image,seed denotes the intensity of the seed point in the seed region. $N_{8}(\bullet )$ and S(i, j) denote eight neighboring outputs and its corresponding pixel intensity of the seed point. The distribution ranges of parameters i and j are $-1\le i \le 1$ and $-1\le j \le 1$, respectively. $S_{\mathrm{mean}}$ denotes the average intensity of the seed point and its eight neighboring outputs. k is loop number. ($F_{\mathrm{x}})_{\mathrm{mean}}$ and ($F_{\mathrm{y}})_{\mathrm{mean}}$ denote the average gradient value of partial derivatives in the x (horizontal) direction and the y (vertical) direction. $S'_{\mathrm{ob}}$ is the Otsu threshold of the object. In addition, termination conditions are expressed as follows:

$$\begin{aligned}&Area_{\mathrm{gs}} (k)-Area_{\mathrm{gs}} (k-1)=0 \end{aligned}$$

(25)

$$\begin{aligned}&E(k)=PA(k)=\frac{Area_{\mathrm{gs}} (k)}{Area_{\mathrm{coar}} } \end{aligned}$$

(26)

$$\begin{aligned}&F(k)=e^{\left( {\frac{1}{PA(k)}} \right) }=e^{\left( {\frac{{Area_{\mathrm{coar}}}}{{Area_{\mathrm{gs}}}(k)}} \right) } \end{aligned}$$

(27)

$$\begin{aligned}&E(k)>F(k) \end{aligned}$$

(28)

Fig. 13

The graph showing the variations of E, F with PA

According to (26), $Area_{\mathrm{gs}}(k)$ denotes the area of the growing region in the kth loop. $Area_{\mathrm{coar}}$ denotes the area of the coarse segmentation region of gallstones and is a fixed value. PA(k) is the area ratio between $Area_{\mathrm{gs}}(k)$ and $Area_{\mathrm{coar}}$ in the kth loop. E(k) is the proportional function of PA(k), and F(k) is the exponential function of PA(k).

In order to better observe the variations of the parameters E(k) and F(k), sampling intervals are changed to 0.01 rather than the integer k (see Fig. 13). What is more, area ratio PA exceeding 10 is set to 10 because it is impossible that the area of the growing region is obviously more than that of the coarse segmentation region. In Fig. 13, the increase of E or the decrease of F is gradually with the increase of PA. If the value of E is more than F, PA can attain about 1.7. This indicates that the area of the growing region is obviously larger than that of the coarse region. According to (28), the region-growing step is terminated.

B The Steps of modified region-growing algorithm

In this paper, the proposed method based on the region-growing algorithm can be achieved in Algorithm 2. Hereinto, in the step 3, if eight neighboring outputs of a seed point satisfy growing conditions in (23) and (24), the seed region then combine these outputs into a new region. Further, if this new region cannot satisfy termination conditions, it can be regarded as new seed region and repeat step 3 until this region satisfy termination conditions. Finally, we obtain fine segmentation result of gallstones (Fig. 14) and the graphs showing the variations of E(k) and F(k) with loops (Fig. 15).

Fig. 14

Initial seed regions and region-growing results: a, c Initial seed region from Fig. 10c, g, respectively; b, d Final growing results from (a) and (c), respectively

Fig. 15

The graphs showing the variations of E(k) and F(k) with loops: a the graph from Fig. 14b; b the graph from Fig. 14d

Fig. 16

The final segmentation results from Figs. 3a, d (the red lines and the blue ones denote the contours of the final results for the gallbladder and gallstones, respectively)

In Fig. 15a, the area of the growing region has the same value between the twelfth loop and the thirteenth loop. This means that the region-growing result reaches the termination condition (25) and we acquires ultimately the fine segmentation result of gallstones in the twelfth loop.

In Fig. 15b, when loop number is more than 4, E can exceed F. This phenomenon illustrates that the area of the growing region in the fourth loop is significantly larger than that of the coarse region. Accordingly, According to (28), the region-growing step is terminated and acquires the fine segmentation result of gallstones from the third loop.

The post-processing

For the gallbladder, since the contour of the fine result is not enough smooth, a Loess algorithm is employed to improve the smoothness of the contour [28]. In the Loess algorithm,Equivdiameter specifies the diameter of a circle with the same area as the gallbladder region and Gall specifies the length of the gallbladder contour. Equivdiameter divided by Gall is regarded as the span value. To acquire final segmentation result of the gallbladder, we firstly combine the gallbladder region and the gallstone region into a region. Besides, the Loess algorithm using the span value is adopted to smooth the gallbladder contour. Finally, the disk-shaped structuring element with a radius of $R_{\mathrm{mid}}$ is used to perform the morphological operation.

For gallstones, the post-processing approach also adopt Loess algorithm and use morphological opening operation with the disk-shaped structuring element of the radius of $R_{min}$. In the end, the final segmentation results of the gallbladder and gallstones are shown in Fig. 16.

Experiments and analysis

In order to compare the performance of our method with the state-of-the-art methods, Snake-gvf(SG) [29], Snake-distance(SD) [30] and Snake-balloon(SB) [31] are used in the tests. Moreover, these methods contrast separately two references, which are manually drawn by two experts in radiology, to evaluate the final experimental results and validate the effectiveness and the robustness of our method. All experiments are run on MATLAB 7.11.0 with Intel(R) Core(TM) i3 M 350 at 2.27 GHz from Satellite L600 of Toshiba.

Table 1 The parameter values of Snake-gvf (gallbladder and gallstone)

Table 2 The parameter values of Snake-distance (gallbladder and gallstone)

Table 3 The parameter values of Snake-balloon (gallbladder and gallstone)

Segmentation evaluation criteria

For the evaluation criteria, four metrics in two measurement methodologies are employed in our work, containing the metrics overlap fraction (OF), overlap value (OV), dice’s similarity (DSI) from the first methodology [32] and the metric position error (PE) from the second methodology [33]. Besides, the metric runtime (T) is also used to analyze the experimental results for meeting clinical needs. Hereinto, the metrics OF, OV and DSI can identify the similarity of two segmented regions and are defined as follows:

$$\begin{aligned}&OF=\frac{\left| {S\cap M} \right| }{M} \end{aligned}$$

(29)

$$\begin{aligned}&OV=\frac{\left| {S\cap M} \right| }{\left| {S\cup M} \right| } \end{aligned}$$

(30)

$$\begin{aligned}&DSI=2\times \frac{|S\cap M|}{|S+M|} \end{aligned}$$

(31)

where S denotes the set of pixels in the region delineated by automatic and semi-automatic experimental methods, while M denotes the set of pixels in the region drawn by the experts; $S\cap M$ and $S\cup M$ denote the intersection and the union, respectively, between S and M. $S+M$ denotes the sum of pixels for S and M. Meanwhile, the metric PE denotes the position error of two contours and is expressed as follows:

$$\begin{aligned} PE=\frac{1}{2k}\sum \limits _{i=1}^k {D(m_{\mathrm{i}} ,n)} +\frac{1}{2l}\sum \limits _{j=1}^l {D(m,n_{\mathrm{j}} )} \end{aligned}$$

(32)

where $m=\{m_{1},m_{2},{\ldots }{\ldots },m_{k}\}$ and $n=\{n_{1},n_{2},{\ldots }{\ldots },n_{l}\}$ denote the pixels of two comparative contours. k and l denote the pixel number of the contour m and the contour n, respectively. $D(m_{\mathrm{i}},n)$ denotes the minimum distance value between the pixel $m_{\mathrm{i}}$ of the contour m and each pixel of the contour n. $D(m,n_{\mathrm{j}})$ denotes the minimum distance value between each pixel of the contour m and the pixel $n_{\mathrm{j}}$ of the contour n. The metric T is the runtime of the program and does not include the consumed time of the manually setting initial contour. If the metrics OV, OF, DSI are close to 1, and metrics PE, T are close to 0, this indicates that the segmentation result is close to the manual drawing result of physicians and has a better performance.

Table 4 The gallbladder regions (the physician 1 and the physician 2)

Table 5 The gallstone regions (the physician 1 and the physician 2)

Parameter settings of comparative algorithms

Before performing comparative experiments, SG, SD and SB algorithms need to set manually parameter values of the initial contours as given in Tables 1, 2 and 3. In the three snake algorithms, iter denotes iteration times of the whole algorithm; $\alpha$ and $\beta$ denote the weighing factors of the elasticity force and the rigidity force, respectively. $\kappa$ represents the weighting factor from the external force (image force). $\gamma$ is a viscosity parameter. $D_{\mathrm{min}}$ and $D_{\mathrm{max}}$ denote the minimum distance and the maximum distance between two successive points of a contour. In the SG model, $\mu$ is the regularization coefficient; ITER is iteration times of gradient vector flow. In the SD model, f is the threshold of the distances between the contour points and the image pixels; In the SB model, $\kappa 1$ is the pressure parameter adjusting the amplitude of the balloon force.

Experimental results and evaluation

To show the performances of our method, we list the experimental results using our method, SG, SD and SB comparing with two experts’ manual results in Tables 4, 5. The experiments are divided into four groups. The previous two groups and next two groups are the gallbladder and gallstones, respectively. Hereinto, Table 4 shows the gallbladder results of the first group (physician 1) and the second group (physician 2), respectively. Table 5 shows the gallstone results of the third group (physician 1) and the fourth group (physician 2), respectively. At the same time, these tables include not only five the analyzed metrics OF, OV, DSI, PE and T, but also four statistical coefficients the mean value (Mean) and the standard deviation (Sd), the maximum value (Max), the minimum value (Min), to evaluate experimental results. Additionally, Fig. 17 shows the experimental results of our method and the manual results from physician 1 for directly observing the similarity of two methods. These results include the irregular shape and the regular shape of the gallbladder (Fig. 17).

Fig. 17

Two cases from comparative results between the new method and the manual method: the first row is a case of the regular shape from the gallbladder; the second row is a case of the irregular shape from the gallbladder (the white lines and the black ones denote the manual results of the gallbladder and gallstone contours, respectively; the red lines and the blue ones are the segmentation results of our method from the gallbladder and the gallstone contours, respectively)

At the above tables, for the gallbladder segmentation, the evaluation values of the metrics OF, OV andDSI in our method are higher and PE are lower in contrast to other algorithms. The metric T also has a good performance than other methods, except SB. For gallstones segmentation, the results of our method in the above five metrics are prior to other three competitive methods.

In order to achieve the purpose of the ultimate evaluation for our method, we can merge three similar metrics OF, OV and DSI into an overall metric EVA and calculate average values of all metrics. The final experimental results can be given in Table 6, and their three-dimensional results are shown in Fig. 18.

Table 6 The final evaluation results (gallbladder and gallstone)

Fig. 18

Three-dimensional evaluation results: a–c the evaluation results from the overall metrics EVA, PE and T, respectively ((g) and (s) denote the gallbladder and gallstones, respectively. G, D, B and TP denote SG, SD, SB and our method, respectively)

Fig. 19

The special cases: a–d the special gallbladder cases; e–f the special gallstone cases

In contrast to other methods, our method has obvious advantages for the overall metrics EVA and PE in Table 6. The overall metric T combined with the gallbladder and gallstones is 4.8814 s, which is evidently less than SG and SD. Although T is longer than 1.9829 s of SB, if the balloon algorithm merges the consumed time of manual setting initial contours into the whole runtime, it is distinctly longer than our method. Therefore, these experimental results indicate that our method has a good performance than other prevalent methods.

Special cases

In this part, Fig. 19a–d introduces main unsuitable cases for the gallbladder. Figure 19a shows an example in which it is very difficult to calculate automatically the segmentation threshold between the right edge region and the central region of the gallbladder. Figure 19b shows that the gallbladder region filling stones is divided into two parts. In Fig. 19c, the right region in whole image is wrongly regarded as the gallbladder region due to the similar intensity range. Figure 19d shows a situation in which the acoustic shadow region is too large to search weak edge regions of the gallbladder. Figure 19e, f introduces main inappropriate gallstone cases. Figure 19e presents that there is a complex situation and it is hard to determine automatically the number of stones. Figure 19(f) shows that a stone covers almost at the bottom of the gallbladder region and is unable to be included into the gallbladder region.

Discussion

This paper proposes an automatic segmentation method for the gallbladder and gallstones. Hereinto, a modified Otsu algorithm, which adjusts weight value A to determine automatically segmentation thresholds between the weak regions and the central region of the gallbladder in ultrasound images, is presented. Based on a parameter-adaptive pulse-coupled neural network (PA-PCNN), the high-intensity regions including the gallstones are acquired by the resetting parameter $S'$ in (18). The growing conditions and the termination conditions are designed by a modified region-growing algorithm to control reasonably growth speed of the seed region, eliminate effectively manual labeled regions of physicians and avoid over-segmentation of gallstones.

The experimental results demonstrate high accuracy rates and short runtime of our method, presenting EVA of 86.01 and 79.81%, PE of 1.7675 and 0.5414 mm, T of 4.2211 and 0.6603 s from the gallbladder and gallstones, respectively.

In summary, the new method has three significant strengths: (1) It automatically achieves parameters setting for reducing the workloads from physicians. (2) It assists accurately physicians to judge the relative position between the gallbladder region and the gallstone region. (3) It provides the contour information of the gallbladder and gallstones to perform feature extraction and determine stone types. Whereas, this method still has several weaknesses: The ultrasound images in the dataset have a small quantity, especially special cases in “Special cases” section, and the method is still unable to process several special cases such as the stones with acoustic shadow. Due to these weaknesses, in the next step of the development, we firstly continue to collect practical cases in our dataset. Secondly, we will try to solve more special cases in clinical cases. For examples, a modified shadow removal algorithm will be employed to try to eliminate acoustic shadow of stones [34]. At the same time, an ellipse fitting algorithm will be used in the case where stones cover at a larger part of the bottom of the gallbladder [35]. Thirdly, based on the new method, we will further deduce image features extraction, including shape features (Perimeter, Area, Solidity, Eccentricity, Circularity, Extent, etc.) and texture features (Mean, Variance, Contrast, Entropy, Skewness, etc.), to determine automatically stone types. Fourthly, according to the above mentioned plans, we will build a complete segmentation framework to assist physicians to establish a treatment plan for the gallbladder disease.

Conclusion

This paper shows that automatic gallbladder and gallstone regions segmentation has better testing results than semi-automatic competitive algorithms because all the parameters in each step is obtained by self-adaptive ways. The method has a great potential to satisfy future demands of the cases detection and characteristic diagnosis for the gallbladder disease. Although the solutions of several special cases are being discussed, the new framework is still a promising method in clinic.