An automatic segmentation method of a parameter-adaptive PCNN for medical images

Abstract

Purpose

Since pre-processing and initial segmentation steps in medical images directly affect the final segmentation results of the regions of interesting, an automatic segmentation method of a parameter-adaptive pulse-coupled neural network is proposed to integrate the above-mentioned two segmentation steps into one. This method has a low computational complexity for different kinds of medical images and has a high segmentation precision.

Methods

The method comprises four steps. Firstly, an optimal histogram threshold is used to determine the parameter $\alpha$ for different kinds of images. Secondly, we acquire the parameter $\beta$ according to a simplified pulse-coupled neural network (SPCNN). Thirdly, we redefine the parameter V of the SPCNN model by sub-intensity distribution range of firing pixels. Fourthly, we add an offset $A\times S_{\mathrm{off}}$ to improve initial segmentation precision.

Results

Compared with the state-of-the-art algorithms, the new method achieves a comparable performance by the experimental results from ultrasound images of the gallbladder and gallstones, magnetic resonance images of the left ventricle, and mammogram images of the left and the right breast, presenting the overall metric UM of 0.9845, CM of 0.8142, TM of 0.0726.

Conclusion

The algorithm has a great potential to achieve the pre-processing and initial segmentation steps in various medical images. This is a premise for assisting physicians to detect and diagnose clinical cases.

Introduction

Medical image segmentation, such as ultrasound image [1,2,3,4], CT image [5,6,7,8], magnetic resonance image [9,10,11], has been playing an increasingly important role in image processing field. Most fashionable segmentation algorithms in medical images invariably capture the local details of object regions to determine the segmentation results and are applied to assist physician’s diagnosis. Further, segmentation steps of these algorithms are always divided into pre-processing, initial segmentation, coarse segmentation, fine segmentation, and post-processing. Hereinto, pre-processing and initial segmentation are indispensable regardless of the properties of the images.

Medical image methods with a global threshold are usually used to segment the images into different objects [12, 13]. There are some of image threshold methods, such as static threshold methods including multi-scale 3D Otsu thresholding [14], and dynamic threshold methods including PCNN [15]. More applications of medical image threshold are obtained from the literatures of Musrrat [16] and Guo [17].

PCNN has broad applications in many aspects for image processing [18]. For examples, image segmentation [19, 20], image enhancement [21, 22] and object recognition [23, 24]. Hereinto, the PCNN has a great potential in image segmentation. Temporal similarity and spatial proximity of the output pulses of PCNN always provides an image segmentation property. One firing neuron corresponding to the pixel directly impacts its adjacent neurons and the segmentation result of the whole image. However, a majority of the prevalent PCNN algorithms require to manually or semi-automatically set parameters, and there are only several automatic segmentation algorithms containing Berg et al. [25], Ma et al. [26], and Chen et al. [27].

Although the above-mentioned methods have better segmentation effects than the basic PCNN model, it is still possible to further simplify parameters setting. What is more, physicians still need to acquire more precise segmentation results to analyze and diagnose relative clinical cases in shorter time. Therefore, we develop a novel image segmentation method based on PA-PCNN. In this method, we attempt to merge pre-processing and initial segmentation steps into one to improve segmentation precision and decrease computational complexity, and try to search a method to achieve partial segmentation for different kinds of medical images with different organ sites.

Our method contributes two new ideas. Firstly, it changes five parameters of the SPCNN model to three parameters of the PA-PCNN model. Hereinto, parameters $\alpha _{\mathrm{f}}$ and $\alpha _{\mathrm{e}}$ in SPCNN are combined into a parameter $\alpha$, which represents the decay factor of the PA-PCNN. The parameter $\beta$ is retained because of its necessity and significance. The parameter $V_{\mathrm{E}}$ is set to V, which represents the weighing factor of the PA-PCNN. Secondly, we add a controlling parameter, which judges whether the image is over-segmentation or under-segmentation, and an offset $S_{\mathrm{off}}$, which adjusts the value of weighing factor V, to improve segmentation accuracy rates. This image segmentation method simplifies the expression of all parameters and calculates them by optimal histogram threshold $S'$ [28], which is appropriate for medical image segmentation.

Materials

Experimental images, containing 80 ultrasound images of the gallbladder and gallstones from Gansu Provincial Hospital in China, 680 magnetic resonance images of the left ventricle from Medical Image Computing and Computer Assisted Intervention Society (MICCAI), and 322 mammogram images of left and right breast from Mammographic Image Analysis Society (MIAS) database [29], were adopted on the research. Hereinto, magnetic resonance images have three cases containing 240 images from the HF-I database, 260 images from the HF-NI database and 180 images from the HYP database. The ultrasound images, the magnetic resonance images and the mammogram images have resolution of $512\times 512$ pixels, $512\times 512$ pixels and $1024\times 1024$ pixels, respectively.

Fig. 1

SPCNN segmentation steps: Images in the first column represent original ultrasound images of the gallbladder with stones; images in the second and third columns represent the segmentation results of the SPCNN in the third and fourth iteration, respectively

The SPCNN model

Basic PCNN model

It is known that the PCNN does not require any training and only has one single layer. In contrast to other PCNN models, Chen et al.’s SPCNN model [27], derived from Zhan et al. ’s SCM model [30], has higher segmentation accuracy and lower computational complexity. Therefore, the SPCNN could be employed in this paper and is written as follows:

$$\begin{aligned}&F_{\mathrm{ij}} [n]=S_{{\mathrm{ij}}} \end{aligned}$$

(1)

$$\begin{aligned}&L_{\mathrm{ij}} [n]=V_{\mathrm{L}} \sum _{{\mathrm{kl}}} {W_{{\mathrm{ijkl}}} Y_{\mathrm{kl}}} [n-1] \end{aligned}$$

(2)

$$\begin{aligned}&U_{{\mathrm{ij}}} [n]=e^{-\alpha _f }U_{{\mathrm{ij}}} [n-1] \nonumber \\&\qquad \qquad +\,S_{{\mathrm{ij}}} \left( 1+\beta V_{\mathrm{L}} \sum _{{\mathrm{kl}}} {W_{{\mathrm{ijkl}}} Y_{{\mathrm{kl}}} } [n-1]\right) \end{aligned}$$

(3)

$$\begin{aligned}&Y_{{\mathrm{ij}}} [n]=\left\{ {{\begin{array}{ll} 1,&{}\quad \hbox {if } U_{{\mathrm{ij}}} [n]>E_{{\mathrm{ij}}} [n-1] \\ 0,&{} \quad \hbox {else} \\ \end{array} }} \right. \end{aligned}$$

(4)

$$\begin{aligned}&E_{{\mathrm{ij}}} [n]=e^{-\alpha _e }E_{{\mathrm{ij}}} [n-1]+V_{\mathrm{E}} Y_{{\mathrm{ij}}} [n] \end{aligned}$$

(5)

where

$$\begin{aligned} W_{{\mathrm{ijkl}}} =\left[ {{\begin{array}{lll} {0.5}&{}\quad 1&{}\quad {0.5} \\ 1&{}\quad 0&{}\quad 1 \\ {0.5}&{}\quad 1&{}\quad {0.5} \\ \end{array} }} \right] \end{aligned}$$

(6)

In the SPCNN model, Neuron $N_{\mathrm{ij}}$ in position (i, j) has simplified feeding input $F_{\mathrm{ij}}[n]$ denoted by an input stimulus $S_{\mathrm{ij}}$, and linking input $L_{\mathrm{ij}}[n]$ denoted by the product of a synaptic weight $W_{\mathrm{ijkl}}$, eight neighboring outputs, and a weighing factor $V_{\mathrm{L}}$. These inputs in internal activity $U_{\mathrm{ij}}[n]$ are modulated by the linking strength $\beta$. Internal activity $U_{\mathrm{ij}}[n]$ also records its previous state by the decay factor $e^{-\alpha _{\mathrm{f}}}$. $\alpha _{\mathrm{f}}$ and $\alpha _{\mathrm{e}}$ represent decay factors of internal activity $U_{\mathrm{ij}}[n]$ and dynamic threshold $E_{\mathrm{ij}}[n]$, respectively. Moreover, there are five adjustable parameters $\alpha _{\mathrm{f}}$, $\alpha _{\mathrm{e}}$, $\beta$, $V_{\mathrm{E}}$ and $V_{\mathrm{L}}$, and these parameters could be set automatically

$$\begin{aligned}&\alpha _{\mathrm{f}} =\log \left( \frac{1}{\sigma (S)}\right) \end{aligned}$$

(7)

$$\begin{aligned}&\beta =\frac{\left( S_{\max } /S^{\prime }\right) -1}{6V_L } \end{aligned}$$

(8)

$$\begin{aligned}&V_E =e^{-\alpha _{\mathrm{f}} }+1+6\beta V_L \end{aligned}$$

(9)

$$\begin{aligned}&V_L =1 \end{aligned}$$

(10)

$$\begin{aligned}&\alpha _{\mathrm{e}} =\ln \left( \frac{V_E }{S^{\prime }M[3]}\right) \end{aligned}$$

(11)

$$\begin{aligned}&M[3]=\frac{1-e^{-3\alpha _f }}{1-e^{-\alpha _f }}+6\beta V_L e^{-\alpha _f } \end{aligned}$$

(12)

where $S'$ and $S_{\mathrm{max}}$ denote Otsu thresholding and the maximum intensity of the image. $\sigma (S)$ represents standard deviation of the image. M [3] is the most significantly term of internal activity $U_{\mathrm{ij}}[n]$ in the third iteration. In the above formulae, parameters $\beta$, $\alpha _{\mathrm{e}}$ and $V_{\mathrm{E}}$ always generate the large impact for the SPCNN. Obviously, the larger the value of $\beta$, the more strongly a neuron is influenced by its eight adjacent outputs. The larger the values of $\alpha _{\mathrm{e}}$ or $V_{\mathrm{E}}$, the lower segmentation accuracy rates become.

Sub-intensity ranges in the SPCNN model

In this paper, segmentation steps and sub-intensity ranges of the SPCNN are shown by two examples of the gallbladder and gallstones in ultrasound images (Figs. 1, 2). Figure 1 shows SPCNN segmentation steps which generates ineffective segmentation results at previous two iterations and effective segmentation results at subsequent iterations. Figure 2a, b shows the sub-intensity ranges of all firing neurons for Fig. 1a, d, respectively.

Fig. 2

Sub-intensity ranges of the SPCNN from the first iteration to the fifth iteration for Fig. 1: a Sub-intensity ranges of SPCNN for Fig. 1a; b Sub-intensity ranges of SPCNN for Fig. 1d

Parameter setting method of PA-PCNN

After using the SPCNN, one gray image is divided into the object and the background at the third iteration and generates further segmentation results at subsequent iterations, whereas we still need to set five parameters. Therefore, we proposed an automatic parameter setting method based on the PA-PCNN model to improve image segmentation precision and decrease computational complexity. Finally, the new method is determined at subsequent sections for detail and the general flowchart of our method is shown in Fig. 3.

Fig. 3

The flowchart of the PA-PCNN for medical images

Parameter $\alpha$

The parameters $\alpha _{\mathrm{f}}$ and $\alpha _{\mathrm{e}}$ in SPCNN represent decay factors of the dynamic threshold E and the internal activity U, respectively. $\sigma$(S) is the standard deviation of the image, whereas the parameters $\alpha _{\mathrm{f}}$ and $\alpha _{\mathrm{e}}$ have a large decay rates. This directly determines segmentation accuracy rates in the iteration. Therefore, by a large number of experiments, normalized optimal histogram threshold $S'$ could be employed in our work as follows:

$$\begin{aligned} \alpha =\alpha _{\mathrm{e}} =\alpha _{\mathrm{f}} =\log \left( {\frac{1}{S^{\prime }}} \right) \end{aligned}$$

(13)

It is noted that the increase of the parameter $\alpha$ is with the decrease of the threshold $S'$. This threshold can enhance the relationship between parameters $\alpha$ and $\beta$.

Parameter $\beta$

According to (8) and (10), since $S_{\max }\approx 1$ in most medical images, we can reset the parameter $\beta$ as

$$\begin{aligned} \beta =\frac{1-S^{\prime }}{6S^{\prime }} \end{aligned}$$

(14)

Parameter V

In the SPCNN model, $\beta V_{\mathrm{L}}$ is always a factor as a whole. What is more, according to (10), the value of the parameter $V_{\mathrm{L}}$ is equal to 1. Therefore, we can remove the parameter $V_{\mathrm{L}}$ and the expression of new linking input is given as

$$\begin{aligned} L_{{\mathrm{ij}}} [n]=\beta \sum _{{\mathrm{kl}}} {W_{{\mathrm{ijkl}}} Y_{{\mathrm{kl}}} } [n-1] \end{aligned}$$

(15)

Subsequently, we will redefine parameter $V_{\mathrm{E}}$ as V and the formula in [18] can be adopted as

$$\begin{aligned} \left\{ {{\begin{array}{l} {Us_{{\mathrm{high}}} [1+l]\le E[1+l-1]} \\ {Us_{{\mathrm{low}}} [1+l+1]>E[1+l]} \\ \end{array} }} \right. \end{aligned}$$

(16)

According to (16), dynamic threshold $E[1+l-1]$ in the decay step $1+l-1$ would be larger than that of the internal activity $U[1+l]$ in the decay step $1+l$, while the minimum value of dynamic threshold $E[1+l]$ in the decay step $1+l$ would be less than that of internal activity $U[1+l+1]$ in the decay step $1+l+1$. This indicates that there is a sub-intensity range of firing pixels in the decay step $1+l+1$. Besides, if one medical image generates the first segment in the second iteration, we could determine parameter l as 0 and the firing neurons can satisfy the firing conditions

$$\begin{aligned} \left\{ {{\begin{array}{l} {U_{\mathrm{high}}[1]\le E[0]} \\ {U_{\mathrm{low}}[2]>E[1]} \\ \end{array} }} \right. \end{aligned}$$

(17)

U[0] and E[0] are always set to 0 and U[1] is more than 0 under normal circumstances. This obeys the first firing condition in (17). Thus, to acquire the first image segment in the second iteration, we can redefine the firing condition

$$\begin{aligned} U_{\mathrm{low}}[2]>E[1] \end{aligned}$$

(18)

According to (3), the minimum intensity of firing pixels in U[2] is expressed as

$$\begin{aligned} U_{\mathrm{low}}[2]=S_{{\mathrm{low}}} (1+e^{-\alpha }+6\beta )=S_{{\mathrm{low}}} M\hbox {[2]} \end{aligned}$$

(19)

and according to (5), $E[1]=V_{E}=V$. The formula (18) is rewritten as

$$\begin{aligned} S_{\mathrm{low}}M[2]>V \end{aligned}$$

(20)

In addition, Otsu thresholding $S'$, which always generates a larger value than $S_{\mathrm{low}}$, can be substituted into (20) and the above formula is redefined as

$$\begin{aligned} S^{\prime }M[2]>V \end{aligned}$$

(21)

Subsequently, we obtain the unique value of V in light of the highest limiting in (21)

$$\begin{aligned} V=S^{\prime }M[2] \end{aligned}$$

(22)

To further improve image segmentation precision, we could add an offset $S_{\mathrm{off}}$ and the controlling parameter A, and the new formula is given as

$$\begin{aligned} V=(S^{\prime }+AS_{{\mathrm{off}}} )M[2] \end{aligned}$$

(23)

where the parameter $S_{\mathrm{off}}$ denotes a small offset. A denotes a parameter to judge whether the testing result is over-segmentation or under-segmentation as shown in (24)

$$\begin{aligned} A=\left\{ {{\begin{array}{ll} 1&{}|S^{\prime }-S_{{\mathrm{mean}}} |\le \sqrt{S^{\prime }\times S_{\hbox {mean}} } \\ -1&{}|S^{\prime }-S_{{\mathrm{mean}}} |>\sqrt{S^{\prime }\times S_{{\mathrm{mean}}} } \\ \end{array} }} \right. \end{aligned}$$

(24)

According to (24), the formula with $A=1$ denotes over-segmentation of images without the offset $S_{\mathrm{off}}$. At this time, the value of V would be added by the offset $S_{\mathrm{off}}$ to reduce the number of firing pixels and obtain reasonable segmentation results. What is more, the formula with $A=-1$ shows an opposite situation. Finally, the steps of the whole algorithm are shown in Algorithm 1.

Fig. 4

The PA-PCNN model

Table 1 Final segmentation results of our experiments

In the Algorithm 1, $F_{\mathrm{x}}$ and $F_{\mathrm{y}}$ denote the average gradient value of the whole image in the x direction and y direction, respectively. $F_{\mathrm{mean}}$ is expressed as

$$\begin{aligned} F_{{\mathrm{mean}}} =\sqrt{F_{\mathrm{x}}^{2} +F_{\mathrm{y}}^{2} } \end{aligned}$$

(25)

and the parameter B is shown as

$$\begin{aligned} B=CF_{{\mathrm{mean}}} \end{aligned}$$

(26)

In (26), the parameter C is a weighing factor (here, $C=100$). In summary, the parameters of the PA-PCNN are not independent but interact with each other by optimum histogram threshold $S'$. The PA-PCNN model is shown in Fig. 4, and its formulae are described as follows:

$$\begin{aligned} U_{{\mathrm{ij}}} [n]= & {} e^{-\alpha }U_{{\mathrm{ij}}} [n-1] \nonumber \\&+\,S_{\hbox {ij }} \left( 1+\beta \sum _{{\mathrm{kl}}} {W_{{\mathrm{ijkl}}} Y_{{\mathrm{kl}}} } [n-1]\right) \end{aligned}$$

(27)

$$\begin{aligned} Y_{{\mathrm{ij}}} [n]= & {} \left\{ {{\begin{array}{ll} 1,&{}\quad \hbox {if } \quad U_{{\mathrm{ij}}} [n]>E_{{\mathrm{ij}}} [n-1] \\ 0,&{}\quad \hbox {else} \\ \end{array} }} \right. \end{aligned}$$

(28)

$$\begin{aligned} E_{{\mathrm{ij}}} [n]= & {} e^{-\alpha }E_{{\mathrm{ij}}} [n-1]+VY_{{\mathrm{ij}}} [n] \end{aligned}$$

(29)

where

$$\begin{aligned} W_{{\mathrm{ijkl}}} =\left[ {{\begin{array}{l@{\quad }l@{\quad }l} {0.5}&{} 1&{} {0.5} \\ 1&{} 0&{} 1 \\ {0.5}&{} 1&{} {0.5} \\ \end{array} }} \right] \end{aligned}$$

Experiments and analysis

To compare our algorithm with the state-of-the-art algorithms, five competitive methods including SPCNN [27], NSCM [31], Otsu [28], Kittler [32], ISPCNN [15], and three metrics including uniformity measurement (UM) [33], contrast measurement (CM) [34], runtime measurement (TM) are employed on our research. Moreover, 80 ultrasound images of the gallbladder and gallstones, 680 magnetic resonance images of the left ventricle, and 322 mammogram images of left and right breast are used as experimental images. All tests are run on MATLAB 7.11.0 with Intel(R) Core(TM) i3 M 350 at 2.27GHz from Satellite L600 of Toshiba.

Segmentation evaluation criteria

For the evaluation of ultrasound images, magnetic resonance images and mammogram images, three metrics analyzing the segmentation precision and computational complexity of the object and the background can be used in this paper. Hereinto, the metric UM denotes the uniformity of the segmented region and is written as

$$\begin{aligned} UM=1-\frac{1}{C}\sum _i {\left\{ {\sum _{(x,y)\in R_j } {[f(x,y)-\frac{1}{A_{\mathrm{i}} }\sum _{(x,y)\in R_j }^n {f(x,y)]^{2}} } } \right\} } \end{aligned}$$

(30)

where C denotes the maximum normalized intensities of the whole image. For a segmented image, $R_{\mathrm{j}}$ and A denote the $j\hbox {th}$ region and its area, respectively. n denotes the number of regions. i denotes the number of gray-level (i.e., i is set to 2 in the binary image). Further, the larger the value of UM, the better segmentation effect becomes. Subsequently, the metric CM denotes the contrast of pixel intensities

$$\begin{aligned} CM=\frac{\left| {f_o -f_b } \right| }{f_0 +f_b } \end{aligned}$$

(31)

where $f_\mathrm{o}$ and $f_\mathrm{b}$ denote the mean value of the object and the background, respectively. The larger the value of CM, the higher segmentation accuracy rates of the image becomes.

Fig. 5

The three-dimensional graph of the final segmentation results: Data in the first, second and third row denote the evaluation results of the metrics UM, CM and TM, respectively; data in the first, second and third columns denote the evaluation results of the our method, the SPCNN method, the NSCM method, the Otsu method, the Kittler method and the ISPCNN method, respectively

Fig. 6

The segmentation results of the ultrasound and the mammogram images: Images in the column (1) denote original images; images in the columns (2), (3), (4), (5), (6) and (7) denote segmentation results of our method, the SPCNN method, the NSCM method, the Otsu method, the Kittler method and the ISPCNN method, respectively; images in the rows a and b denote segmentation results of mammogram images; images in the rows c and d denote segmentation results of ultrasound images

Experiment analysis and discussion

In our experiments, all images are divided into five groups including 80 ultrasound images in the first group, 240 magnetic resonance images of the HF-I database in the second group, 260 magnetic resonance images of the HF-NI database in the third group, 180 magnetic resonance images of the HYP database in the third group and 322 mammogram images in the fourth group. Moreover, we compare our method with other methods to obtain ultimate evaluation results in Table 1 and their three-dimensional graph is shown in Fig. 5. We also select one image in each group as the example to show experimental results (Figs. 6, 7).

Fig. 7

The segmentation results of the MR images: Images in the column (1) denote original images; images in the columns (2), (3) and (4) denote segmentation results of the our method, the SPCNN method, the SCM method, the Otsu method, the Kittler method and the ISPCNN method, respectively; images in the rows a denote segmentation results for the HF-I dataset; images in the rows b denote segmentation results for the HF-NI dataset; images in the rows c denote segmentation results for the HYP dataset

According to the above Table 1 and Fig. 5, compared with other state-of-the-art algorithms, our algorithm has a good performance for the evaluations of the metrics.

Discussion

In this paper, we present an image segmentation method based on PA-PCNN to further simplify computations and improve initial segmentation accuracy for various medical images. In contrast to other methods of the PCNN, our method only set three parameters $\alpha$, $\beta$, V, which are associated with each other by the Otsu thresholding $S'$, and retain main properties of the basic PCNN model. Our method, which determines rapidly and effectively the initial segmentation result in the pre-processing step and initial segmentation step for various medical images, obviously reduces the runtime and improve segmentation accuracy rates of the whole method including the coarse-fine segmentation and the post-processing. So, the whole segmentation method including our method is more easily to generate a good final segmentation result, which brings a great help to diagnose clinical cases for physicians. For example, for gallbladder and gallstone regions segmentation, the computing program of the whole segmentation method including PA-PCNN is copied to the chip controlled by ultrasonic diagnostic apparatus. When physicians use detectors to seek out lesion regions, the monitor will show rapidly segmentation results in most cases.

Our method for medicine has three main significances: (1) It obtains segmentation results more rapidly and accurately for physicians. (2) It avoids the otherness from subjective experience and human knowledge. (3) It improves the reliability and accuracy of clinical diagnosis for physicians.

By a large number of experiments employing three metrics, five comparative methods and 1082 medical images containing 80 ultrasound images of the gallbladder and gallstones, 680 magnetic resonance images of the left ventricle and 322 mammogram images of left and right breast, we demonstrate that the image segmentation method has low computational complexity and high initial segmentation accuracy rates, presenting the overall metric UM of 0.9845, CM of 0.8142, TM of 0.0726.

In the future, we firstly continue to add other types of medical images with other organ sites into our dataset. Secondly, for several types of medical images, we will combine coarse segmentation steps into modified PA-PCNN for further adding the applicable range of the method. Thirdly, we will use the whole segmentation method including PA-PCNN to validate the effectiveness and robustness of our method for gallbladder and gallstones in clinical diagnosis.

Conclusion

This paper indicates that the proposed method has better testing performance than other comparative methods because all calculated parameters are automatically acquired by self-adaptive ways. The method has a great potential to achieve the pre-processing and initial segmentation for various medical images. Although the segmentation ways of other clinical cases are being discussed, the new method is still a promising method for medical image segmentation.