Automatic Hybrid Deep Learning Network for Image Lesion Prognosis and Diagnosis

Abstract

Human lives are very precious. Human body parts are affected due to various food habits, environment, social habits, hereditary health issues and many more. One of the most well-known disorders is liver cancer. Liver cancer causes major loss to human lives. Early detection of this can be curable at the initial stages. Due to the advancement of computer vision algorithms and progression of IoT technologies working along with image processing techniques, early detection and curing are inevitable. To improve the classification time with faster duration which can help the medical practitioner, lesion detection algorithms are developed. The work’s key contribution is to apply efficient image segmentation algorithms along with powerful deep learning algorithms to classify the cancerous lesion efficiently. This ensemble or hybrid technique is developed by combining the morphological operations and deep learning procedures for an automated liver cancer prognosis and diagnosis. This fully connected network with U-net architecture improves the classification accuracy. Also increased true positive rate is achieved in the proposed methodology with high sensitivity. This novel work achieves high classification accuracy with minimum time. In addition, it also improves the area under curve region. Work was experimented with open-source datasets as well as with mobile few clinical images.

1 Introduction

Liver is the biggest part in the abdominal region. Liver cancer is one of the subtypes of hepatocellular carcinoma (HCC). HCC is the major cause for primary liver cancer and leads to major health issues. Early stage of HCC prognosis and diagnosis is more benefited to patients to attempt earlier treatment and to live peacefully. HCC can be caused by non-alcoholic fatty liver disease, steatohepatitis, alcoholic liver disease or hepatitis B and C. During the course of disease, tumours from the breast, pancreas and colon can readily spread into the liver.

Manual segmentation of liver cancer in the radiological images is a time-consuming process. Deep learning combined with image processing techniques can be used to detect tumours automatically. The tumour usually originates in the other parts of the image and spreads to the liver. Tumour usually is the abnormal growth in the liver. Tumours can be cancerous or non-cancerous part.

The main goal of liver segmentation is to classify the pixels into two groups: the pixels which belong to the liver and the remaining pixels which belong to non-liver parts. Based on the conventional approach, other ways are employed to segment the liver, such as level sets or deformable models, clustering, region growing, graph cuts, probability atlases, statistical shape models and deep learning methods. Figure 1 depicts the details of abdominal CT image and various internal parts are marked. Also, this figure represents the CT image of liver in normal and lesion cases. Table 1 explains the different stages of liver cancer. Various stages are stage 1, 2, 3 and 4 based on the penetration of tumours into the inner human body parts. From the given dataset or data acquisition, the liver part is segmented using morphological operations or with efficient image segmentation algorithm. Then from the segmented liver lesion/tumour features are extracted. Then the various performance measures listed in Table 2 are computed and evaluated.

Fig. 1

Liver, heart, stomach and tumour in CT slice and presentation of normal and lesion image

Table 1 Various stages of liver cancer

Table 2 Evaluation parameters

Generally, the liver image segmentation applies various algorithms and is explained as follows. In spatial and geometric shape model, prior knowledge provides the statistical shape model. This method creates a probabilistic model with prior knowledge. Local image features can be extracted using B-spline transformation. Grey-level method is utilized for intensity distribution. Morphological filters and intensity distribution are utilized for extracting pattern features. Also, graph cut methods help to separate background and liver. K-means clustering and region growing segmentation methodologies were applied over the image followed by contouring algorithm in semi-automated method and the parameters (liver, heart, stomach and tumour) of CT slice and presentation of normal and lesion image.

After segmenting the liver, the following step is to segment any tumours that may be present in the image. The current research work contains a summary of several liver tumour segmentation methods, which are described as follows. Thresholding method is an effective tool used to separate tumour and background. Histogram analysis method is used for thresholding. Spatial regularization method depends on the morphological operations. Level set method is applied with numerical computations to segment various tumour shapes. Graph cut method, watershed segmentation, Bayesian classifier, SVM, fuzzy C-means clustering and hidden Markov models are used in the semi-automated method.

The paper is coordinated as follows. Section 2 portrays the connected works connected with liver disease anticipation and analysis. Section 3 presents the proposed technique. Section 4 makes sense of the outcomes and conversation of the proposed work with correlation of the new examination works. Section 5 presents the results and future improvements of the proposed work.

2 Objectives

The major contribution of the proposed work is given as follows:

• Hybrid liver image segmentation for lesion detection is proposed with morphological segmentation operation and two-fold cascaded CNN networks for liver and lesion segmentation.

• Morphological operations provide the effectiveness in liver segmentation from the abdominal CT and MRI images.

• In the cascaded FCN U-net architecture is experimented to reach the faster computation.

3 Methods

Men et al. [1] have worked on the 3D computer tomography images and avoided the time-consuming manual interpretation. But this method is less effective for multiple lesions. A fully automated fully convolutional network was experimented for 20 patients and received 0.86 and 0.6 true positive rate and false positive rate, respectively, with three-fold cross-validation [2]. Vivanti et al. [3] developed a convolutional neural network with a robust classifier for the detection of liver tumours. With global CNN, this approach generates high-scoring examples while discarding low-scoring cases. Multi-scale candidate generation with residual network [4] was applied with liver tumour segmentation. Initially U-network segments the liver and fractal residual network segregates the tumour. Finally, contour method refines the tumour refinement. A semi-automatic segmentation [5] of liver was performed by Voronstov et al. This work combines both deformable model and machine learning algorithm for segmentation. Support vector machine classifier precisely classified and validated the metastatic liver tumour for 27 tumours. A two-fold multi-voxel-based liver tumour segmentation was implemented using the method proposed by Conze et al. [6]. Random forest technique is applied for multi-voxel-based feature discrimination and hierarchical multi-scale fashion to deal with heterogeneity. Jiang et al. [7] have discussed a cascaded network for liver localization, segmentation and tumour identification on 3D datasets. This cascaded network outperforms the existing U-net and Res-net for various datasets. Hierarchical convolutional and deconvolutional network-based live tumour prognosis and diagnosis method was implemented by Yuan [8]. This model was experimented over LiTs 130 training datasets and the best Jaccard function was calculated. Two different fully convolutional layers are applied separately for liver and lesion classification [9]. The performance of these two FCN layers outperforms the single FCN network. Bellver et al. [10] implemented cascaded stages of network for liver segmentation and lesion detector. Here the segmentation network operates in pixel-wise manner. The lesion detector applies constraint-less detection. Pancreas cancer detection is implemented on PET/CT images. Linear iterative clustering method is implemented for pancreas segmentation. Then principal component analysis is applied for feature extraction. This method was finally evaluated over public datasets which include 82 three-dimensional CT images. Deep neural network with combined networks [11] of U-net for long-range concatenation and short-range residual network was proposed in this methodology. This DCNN has the limitation of long computation time for lesion detection. Larsson et al. [12] introduced a macro architecture based on self-similarity without residuals. This architecture includes sub-paths of various lengths. The majorly contributed fractal networks show better performance on CIFAR datasets. This deeper network brings quick answer with more accuracy. Trivizakis [13] has extended the 2D CNN architecture to 3D and achieved improved performance for MRI liver datasets. This architecture includes 2048 neurons with ReLu activation function with softmax binary classifier. Hu et al. [14] has applied three-dimensional CNN deep learning methods for detecting the liver. Here deep learning is trained to extract probabilistic map. This method is validated on 42 CT images. Lu et al. [15] developed a fully automatic method for liver segmentation. This combines graph cut and deep learning procedure. Deep learning method extracts liver surface. Refinement method [16] uses graph cut technique to map the liver probability map. Christ presents an automatic fully connected CNN for segmenting liver and its lesion. HU windowing/N4 bias correction method is applied in the pre-processing step. Then from the abdominal images liver part is segmented with FCN.

4 Proposed Work

The proposed work depicted in Fig. 2 uses the public dataset 3DIRCADSET and CIFAR set. The liver image is converted into luminance (Y) channel and then the artifacts are removed using median filter. Then the liver image is segmented using morphological operation including erosion and dilation operation.

Fig. 2

Block diagram of the proposed work

Next from the segmented liver image the tumour is detected using fully connected convolutional layer together with deeper U-net architecture which enhances the classification accuracy. The experimentation is tested over the computer tomography image slices and MRI images as shown in Figs. 3 and 4.

Fig. 3

Liver segmentation using morphological operations (a–e)

Fig. 4

Blue colour is the lesion ground truth, red colour indicates the tumour ground truth, yellow represents the predicted liver and green represents the lesion detection

5 Results

The proposed method applies morphological operation for the liver segmentation from the abdominal images and the performance measures of various parameters for MRI Images are shown in Table 3 and Fig. 5. Then the two-fold lesion segmentation for lesion classification is further enhanced using fully convolutional network with U-net architecture. The proposed hybrid method enhances the sensitivity and true positive rate with highest value compared to the existing methods. The evaluation can be extended for 3D slices in the future.

Table 3 Performance measures comparison of the proposed method with existing methods

Fig. 5

Performance measures of the various parameters for MRI images