Keywords

1 Introduction

Liver is the only body’s largest vascular glandular solid organ found in all vertebrates. Some of its functions are regulation of glycogen storage, decomposition of red blood cells, plasma protein synthesis, hormone production and detoxification. In human beings, it is located in the right upper quadrant of the abdomen, below the diaphragm. Normally, liver is divided into eight lobes, but from outside it is divided into a larger right lobe and smaller left lobe [1]. Liver is the only organ capable of regenerating the lost tissues. Resection of liver lobes will not regrow the lobes; instead, growth will be with respect to the function restoration not original form.

Tracing of liver is important in medical imaging for obtaining qualitative measurements such as the location finding the region of interest and quantitative measurements like area, volume or the behavioral analysis of structure anatomy over time. This provides a good computer-aided diagnosis for feature extraction and characterization of lesions. Accuracy in liver segmentation plays an important role in diagnosing the liver diseases, liver transplantation and liver resection which help the patients to survive. Hence, fully automated approaches to segment liver are being designed which help radiologists to diagnose different types of lesions accurately with less amount of time. Liver segmentation is still a challenge in diseased liver due to distortion in liver shape, complexities in liver pathologies, blurred edges and the presence of hypodense or hyperdense lesions [2].

Diagnosis of liver diseases can be made using various noninvasive imaging modalities like computed tomography (CT), magnetic resonance imaging (MRI), ultrasound (US), positron emission tomography (PET) and positron emission tomography-computed tomography (PET-CT). CT is a noninvasive imaging modality which combines X-rays and computer technology to produce horizontal images of the body parts [3]. CT scanners result with thousands of image slices which will be tedious and time-consuming for radiologists to perform accurate diagnosis [4]. Computed tomography is much preferred by the diagnosticians because of its accurate anatomical information about the structures which are visualized. Supporting arguments of CT are highly sensitive to distinguish tissue density differences and get accurate anatomical data. Opposing arguments of CT are less sensitive to pathological information. MRI modality uses strong magnetic field, radio waves and field gradient to form anatomical pictures and physiological processes of the body in both health wise and disease wise. Reason to support MRI is that it is highly effective in showing the difference between healthy and diseased soft tissues in the body, and reasons to oppose MRI include high cost, long procedure time and patient need to hold breathe for a longer time.

Various methodologies observed in the study include SLIC super-pixel with AdaBoost algorithm, graph cut method, fully convolution network with deeper bottleneck architecture, hybrid densely connected U-Net, statistical shape model, Bayesian probability atlas model with adaptive thresholding, level set method, active contour, Laplacian mesh optimization method, 3D active surface model used in computed tomography and magnetic resonance imaging modalities.

2 Literature Review

Many of the computer-aided diagnosis systems developed are semiautomatic and fully automatic systems using CT and MRI modalities which are observed through the study discussed in the following section.

2.1 Liver Segmentation Using Computed Tomography (CT) Images

Barstugan et al. [5] have put forward an automatic technique for splitting up of liver on CT images using SLIC super-pixel and AdaBoost algorithm, which uses two AdaBoost algorithms to train spinal cord and liver with image clustering performed using SLIC super-pixel algorithm. A fully automatic method (except initial slice selection) is proposed by Liao et al. [6] using graph cut and border marching to split up the liver. A fully automatic approach to segment liver from 3D CT scans based on fully convolution network with deeper bottleneck architecture (DBA) which decreases the number of parameters in the network and increases network depth is proposed by Jin et al. [7]. Zhang et al. [8] recommended a cascaded structure to section the liver by utilizing a fully convolution neural network with postprocessing which refines the liver. Liver segmentation in CT images is developed by Li et al. in [9] using hybrid densely connected U-Net which produces a coarse segmentation of liver quickly by training simple ResNet architecture [10] which reduces computation time. Zheng et al. in [11] projected a methodology to fragment liver in 2D CT images using statistical shape model (SSM) with enforced local statistical features. Farzaneh et al. [12] put forward a Bayesian probability atlas model with adaptive thresholding and super-pixel algorithm to segment liver by creating two Bayesian probability atlases for intensity and location of liver using adaptive thresholding and incorporate anatomical information with super-pixel algorithm to find final ROI. A noninvasive approach to segment liver is projected by Saito et al. by [13] using level set method for multiphase CT images.

2.2 Liver Segmentation Using Magnetic Resonance Imaging (MRI) Images

Christ et al. [14] submitted algorithm to segment liver with hepatic lesions in MR images using the cascaded fully convolutional neural network, where preprocessing is done using N4 Bias Correction algorithm [15], and several data augmentation steps are included to increase the training set like elastic deformation, translation, rotation, addition of Gaussian noise with standard deviation. Fully automated approach for breaking up the liver is preferred by Mohamed et al. [16] which uses active contours by considering image enhancement phase, liver localization phase, liver segmentation phase and segmented result enhancement phase. Liver fragmentation using Laplacian mesh optimization which is a semiautomatic model is suggested by Chartrand et al. [17]. Bereciartua et al. [18] proposed a novel approach to segment liver using compact descriptor integrated into 3D active surfaces in multiple sequence MRI images which use multi-sequential spatial descriptor, namely axial gap, axial arterial, axial venous, late axial (VIBE sequences) and blade which comprises of spatial variation.

3 Methodologies

Observations made in the study for segmentation of liver based on techniques are discussed in the following section using computed tomography and magnetic resonance imaging modalities.

3.1 Liver Segmentation Methods Using Computed Tomography Images

Ecabert et al. [4] use AdaBoost classifier constructed using decision trees and trained on patches of 3 × 3, 5 × 5, 7 × 7, 9 × 9. It is used to train liver and spinal cord which consist of base learners which were taken as 100, and classification result was based on weight voting given in Eq. (1)

$$ H\left( x \right) = sign\sum\limits_{t = 1}^{T} {(a_{t} h_{t} (x))} $$
(1)

where at, ht are referred as weight and base learner, respectively. SLIC super-pixel algorithm used in [19] is faster and efficient compared to existing super-pixel algorithm which adapts k-means method for super-pixel algorithm which is used to cluster the image.

Liao et al. of [6] come up with an unsupervised method for selecting a initial slice and segmented using density peak clustering technique [20] which divides it into three clusters. Author uses minimization energy function in Eq. (2) using graph cut method [21] which integrates intensity model, PCA-based appearance model and location constrained used to iteratively recognize liver in the remaining slices.

$$ E\left( f \right) = \sum\limits_{p \in P} {\left( {\alpha \cdot F_{intensity} \left( { f_{p} } \right) + \beta \cdot F_{PCA} \left( { f_{p} } \right)} \right)\cdot f_{location} \left( { f_{p} } \right)} + \sum\limits_{p \in P,q \in N p} {B( f_{p} , f_{q} )} $$
(2)

where P is the set of pixels of image \( f,{N_p} \) is the set of neighborhood pixels, α and β are the weights of the intensity and appearance penalties, and α + β = 1. Lastly, under-segmented vessels are compensated using marching of liver border.

Jin et al. [7] build a new FCN-based liver segmentation U-Net referring to the classical deep convolutional U-Net model proposed by Ronneberger et al. [22]. FCN-based liver segmentation U-Net is build along with three DBAs (deeper bottleneck architectures) to decrease the number of network parameters and increase the hidden layer, which varies with height, width and number of channels with 53 convolution layers called as U-Net-53. The data augmentation (move, rotation, mirror, noise, cut) is also done in this method to increase the performance compared to Ben-Cohen et al. [23] and Christ et al. [24] who use fully convolution neural network.

Zhang et al. [8] build 21 layers of fully convolutional neural network, i.e., 15 convolution layers, 3 pooling layers and 3 deconvolutional layers followed by ReLU [25], inspired by the U-Net of [22] to segment liver and generate probability map. Batch is normalized after convolution using mean and standard deviation [26]. To balance class, weighting factor ω is introduced in the cross-entropy loss function Fℓ of the FCN, which is given in Eq. (3)

$$ F_{\ell } = - \frac{1}{n}\sum\limits_{i = 1}^{N} {\omega i\left[ {\hat{p}_{i} \log p_{i} + \left( {1 - \hat{p}_{i} } \right)\log \left( {1 - p_{i} } \right)} \right]} $$
(3)

where \( {\text{p}}_{\text{i }} \) is the probability of foreground pixel and \( {\hat{\text{p}}}_{\text{i }} \) is ground truth. Three postprocessing models level set based, graph cut based and conditional random field are used for comparison with the probability map to increase the accuracy.

Li et al. [9] proposed a hybrid densely connected U-Net which includes both 2D Dense U-Net f2d extracts intra-slice features and 3D Dense U-Net f3d extracts volumetric inter-slice features, and optimized using hybrid feature fusion (HFF) layer for segmentation of both liver and its lesions; 2D Dense U-Net follows the structure of Dense-Net-161 [27], which is extended to 167 layers called 2D Dense U-Net-167. The feature maps and score maps of 2D Dense U-Net are given in Eq. (4) as follows:

$$ \begin{aligned} X_{2d} & = f_{2d} (I_{2d}; \theta d);X_{2d} \in R ^{12nX224X224X64} \\ {\hat{\text{Y}}}_{2d} & = f_{2dcls} (X_{2d} ; \theta_{2dcls} );{\hat{\text{Y}}}_{2d} \in R^{12nX224X224X3} \\ \end{aligned} $$
(4)

where X2d is feature map, Ŷ2d is predicted pixel-wise probabilities corresponding to the input three adjacent slices and I2d input samples of 2D Dense U-Net. The feature maps and score maps of 2D Dense U-Net transformed to volumetric are given in Eq. (5) as follows:

$$ \begin{aligned} X_{2d}^{\prime } & = f^{ - 1} (X_{2d} ) \, ;X_{2d}^{\prime } \in R^{nx224X224X12X64} \\ \hat{Y}_{2d}^{\prime } & = f^{ - 1} \left( {\hat{Y}_{2d} } \right);\hat{Y}_{2d}^{\prime } \in R^{nX224X224X12X3} \\ \end{aligned} $$
(5)

The learning process of 3D Dense U-Net is described in Eq. (6)

$$ \begin{aligned} X_{3d} & = f_{3d} (I,\hat{Y}_{2d}^{\prime } \, ;\theta_{3d} ), \\ Z & = X_{3d} + X_{2d}^{\prime } \\ \end{aligned} $$
(6)

where X3d denotes the feature volume from layer in 3D Dense U-Net-65 and Z denotes the hybrid feature. Features are learned and optimized using HFF layer as shown in Eq. (7)

$$ \begin{aligned} H & = f_{HFF} (Z;\theta_{HFF} ) \\ \hat{y}h & = f_{HFFcls} (H;\theta_{HFFcls} ) \\ \end{aligned} $$
(7)

where H denotes the optimized hybrid features and ŷh denotes pixel-wise probabilities generated from HFF layer.

Zheng et al. in [11] project a segmentation model (F) based on statistical shape prior model by applying PCA on signed distance function (SDF), global Gaussian fitting energy (FG) in Eq. (9) and local statistical consistency energy (FL) in Eq. (10) to move the contour toward the liver boundary which is given in Eq. (8)

$$ F(\alpha ;X_{T} ; \, f_{1} ; \, f_{2} ) = w_{1} F_{G} + w_{2} F_{L} $$
(8)
$$ F_{G} = \sum\limits_{i = 1}^{2} {\frac{{\lambda_{i} A_{i} }}{2}\log (\int\limits_{\varOmega 1} {\frac{{(A_{i} I\left( x \right) - \mathop \smallint \nolimits_{\varOmega 1} I\left( x \right)dx)^{2} }}{{A_{i}^{3} }}dx} )} $$
(9)
$$ F_{L} = v_{1} \int\limits_{\varOmega 1} {\left| {F\left( x \right) - f1} \right|^{2} dx} + v_{2} \int\limits_{\varOmega 2} {\left| {F\left( x \right) - f2} \right|^{2} dx} $$
(10)

where Ai represents number of pixels in background and foreground, f1 and f2 denote representative features, w1, w2, \( \lambda_{1} ,\lambda_{2} \), v1, v2 are parameters to balance inside and outside local and global energy, \( \varOmega 1 = \left\{ {x:\hat{\varPhi }T\left( x \right) > 0} \right\},\varOmega 2 = \left\{ {x:\hat{\varPhi }T\left( x \right) < 0} \right\} \) and XT is parameter vector of geometric transformation.

Farzaneh et al. [12] use the Bayesian-based method [28] for liver segmentation creating two atlases, where one atlas based on location and the other atlas based on intensity. The overall probability of the liver pixel is defined in Eq. (11)

$$ P(L|(i,j),I) = \frac{{P\left( {I,\left( {i,j} \right) |L} \right)P\left( L \right)}}{{P\left( {I,\left( {i,j} \right) |L} \right)P\left( L \right) + P\left( {I,\left( {i,j} \right) |L^{\prime } } \right)P\left( {L^{\prime } } \right)}} $$
(11)

where P(L|(i, j)) is location probability and P(L|I) is the intensity probability. New intensity probability atlas P(I|L)new is created, where probability of the pixel is calculated as Eq. (12)

$$ Pnew(L|I,(i,j)) \propto P(L|I)newP(L|i,j) $$
(12)

Authors propose adaptive thresholding for each slice with cut-off value t which is calculated using Eq. (13) and optimum threshold value th with step wise \( \Delta \).

$$ \begin{aligned} f\left( t \right) & = \frac{{\left| {\left| {Pnew \ge t - \Delta } \right|} \right|0 - \left| {\left| {Pnew \ge t} \right|} \right|0}}{{\left| {\left| {Pnew \ge t} \right|} \right|0}} \\ th & = argmin\left( {f\left( t \right)} \right) \\ \end{aligned} $$
(13)

where \( Pnew \ge t \) is a binary image with the pixel value of 1 and ||.|| 0 denotes the norm zero.

Saito et al. [13] proposed a level set based method to segment liver in multiple phases (non-contrast, early arterial phase, portal phase, equilibrium phase of CT images). In non-contrast phase bone is detected and used as a predefined template in contrast phases for bone detection. For segmenting the liver, Chan–Vese-based [29] level set method is applied with the following Eq. (14)

$$ \begin{aligned} {\text{F}}(c^{ + } ,c^{ - } ,{\text{C}}) & =\upmu{\text{length}}\left( {\text{C}} \right) + {\text{vArea}}\left( {{\text{Inside}}\left( {\text{c}} \right)} \right) + \lambda^{ + } \int\limits_{inside\left( c \right)} {\left| {u_{0} \left( {x,y} \right) - c^{ + } } \right|^{2} dx\,dy} \\ & \quad + \,\lambda^{ - } \int\limits_{outside\left( c \right)} {\left| { u_{0} \left( {x,y} \right) - c^{ - } } \right|^{2} dx\,dy} \\ \end{aligned} $$
(14)

where \( \mu_{0} \) is a pixel value of image, C is the boundary of a closed set and \( c^{ + } ,c^{ - } \) are the values of u respectively inside and outside of C.

3.2 Liver Segmentation Methods Using Magnetic Resonance Imaging Images

Christ et al. [14] proposed a cascaded fully convolutional neural network to segment liver and its lesion for both CT and MRI volumes which is the extension of [30]. U-Net architecture [22] is used to find soft label probability maps by combining spatial and contextual information which consists of 19 convolution layers. With reference to [31], class balancing is important in training the network to segment small structures like lesions, so additional weighting factor \( \omega^{class} \) is introduced in cross-entropy loss function L Eq. (15) of FCN

$$ L = - \frac{1}{n}\sum\limits_{i = 1}^{N} {\omega_{i}^{class} [\hat{P}_{i} \log P_{i} + (1 - \hat{P}_{i} )\log (1 - P_{i} )} $$
(15)

where Pi is the probability of voxel i belonging to the foreground, \( \hat{P}_{i} \) is the ground truth. It also uses pretrained U-Net models provided by Ronneberger et al. [22], who were trained on cell image segmentation data; 3D dense conditional random field (CRFs) is used as a postprocessing technique proposed by [32] to get final segmented volume.

Mohamed et al. [16] use the active contour to automatically segment liver in MRI images. The first active contour model was developed by Kass et al. [33]. In active contour, initialization of curve is done by user and snake moves and deforms toward boundary. Snake is derived from the three energy functions given in Eq. (16)

$$ E_{snake} = E_{int} + E_{ext} + E_{cons} $$
(16)
$$ E_{int} = E_{elastic } + E_{bending} = \int\limits_{s} {\frac{1}{2}(\alpha \left| { v_{s} } \right|^{2} + \beta \left| {v_{ss} } \right|^{2} )} $$
(17)
$$ E_{ext} = \int\limits_{s} {E_{image } ({\text{V(s)}}){\text{ds}}} $$
(18)

where \( {\text{E}}_{\text{int}} \), \( E_{ext} \), \( E_{cons} \) are internal energy mentioned in Eq. (17), external energy calculated using Eq. (18) and constrained energy, respectively, and \( \alpha ,\beta \) are the weights.

Chartrand et al. [17] developed Laplacian mesh optimization framework, a semiautomatic method to segment liver in both MRI and CT images. Initial shape was generated by a few contours carried out by users. Preserving the smoothness of the shape by using discrete Laplacian operator discussed by Nealen et al. [34] set to a target value of 0. The Laplacian energy function which is to be minimized is given in Eq. (19)

$$ E_{{\mathcal{L}}} \left( {V^{\prime } } \right) = \, \propto \sum\limits_{i = 1}^{n} {w_{i}^{2} (t_{i } - v_{i}^{\prime } )^{2} } + \left\| {{\mathcal{L}}V^{\prime } } \right\|^{2} $$
(19)

where V′, vi, ti, wi, ℒ are new vertex, target, weight and Laplacian matrix which represents delta coordinates obtained by applying discrete Laplace operator on the previously introduced mesh, respectively.

Bereciartua et al. [18] put forward 3D active surface model which takes the work of Bresson and Chan [35] to segment liver which combines active contours without edges (ACWE) introduced by Chan et al. [36] and the geodesic active contours (GAC) developed by Caselles et al. [37]. Chan and Vese [38] stated the energy functional model of ACWE to minimize is expressed in Eq. (20) as follows:

$$ \begin{aligned} E_{ACWE} (\varOmega_{c} ,c_{1 } ,c_{2} ,\lambda ) & = Per\left( {\varOmega_{c} } \right) + \lambda \int\limits_{{\varOmega_{c} }} {\left( {c_{1 } {-}I\left( {x,y} \right)} \right)^{2} dx\,dy} \\ & \quad + \,\lambda \int\limits_{{\varOmega \backslash \varOmega_{c} }} {\left( {c_{2 } {-}I\left( {x,y} \right)} \right)^{2} dx\,dy} \\ \end{aligned} $$
(20)

where I(x, y) is the image, \( \varOmega_{c} \) is a subset of the image domain bounded by closed contour, Per \( \varOmega_{c} \) is the perimeter of the set \( \varOmega_{c} \), λ is a positive parameter, c1 and c2 ∈ R. The energy functional model of GAC is expressed in Eq. (21) as follows:

$$ E_{GAC} \left( c \right) = \int\limits_{s} {g\,ds} $$
(21)

where g is an edge detecting function, and s is the arc length parameter along the contour C.

4 Discussion

Table 1 furnishes the digest of different liver slicing that confers about the literature survey along with techniques, year, datasets, outcomes, merits and demerits for each work carried on computed tomography volumes. Each of the study discussed uses different metrics used to measure the performance and varying datasets. The different approaches used by authors to segment liver CT images pose certain difficulties to analyze the performance. These difficulties are due to the use of different datasets, assumptions, different metrics to measure performance, image dimensions with different phases and ground truth marking done manually by radiologists.

Table 1 Overview of liver slicing in CT images
Full size table

The literature review in Table 2 presents with a digest consisting of diverse technologies, results with advantages and drawbacks of the whole lot on MRI Images for division of liver. Similar to CT images, performance analysis is also difficult mainly due to non-availability of public dataset, non-identical methodologies, assumptions made, varying datasets, ground truth marking done by various radiologists and different performance metrics used.

Table 2 Outline of liver division in MRI images
Full size table

In this paper, observations have been made on different methods to segment liver and planning to develop a cascaded structure using fully convolution neural network one for liver and another for lesion segmentation using both computed tomography and magnetic resonance imaging volumes with multiple phases (both with contrast and without contrast). The factors decided to be considered while designing the structure includes reducing the computational cost, increasing the depth of the network, improving the accuracy of segmentation, decreasing number of parameters used in the network, reducing the training time by using pretrained models, considering leaky problem when the liver contour is not clear, decreasing the number of assumptions and work on huge datasets (both public and clinical datasets).

5 Conclusion

Liver segmentation used for lesion classification is difficult because of its shape and appearance which varies from person to person and similar intensities reside with liver and other organs surrounded it. From the discussion, we can observe that few algorithms are semiautomatic which have user interactions, very few methods worked for both CT and MRI images, leakage problem when liver boundary is not clear, less work on multiple sequences in MRI images, worked using limited datasets, abnormalities in tissues, training time and computationally time are high. Due to all these factors, there is still scope for developing fully automatic methods on larger datasets (both public and clinical) to segment liver and lesion accurately in both MRI and CT images.