A deep learning based review on abdominal images

Abstract

Computer-aided diagnosis have stumbled rapidly in the last few years. One of foremost step in computer-aided diagnosis is organ classification and segmentation. Among various organ segmentation techniques, the segmentation of abdominal organs like liver, stomach, kidney, pancreas and bladder from different modality of images has gotten keen interest in past few years. Mostly the interpretations of abdominal images are being done by medical experts or radiologists. Image interpretation by human experts is quite limited due to its subjectivity, complexity of the image, extensive variations exist across different interpreters, and fatigue. After the success of deep learning in real world applications, it is also providing exciting solutions with good accuracy for medical imaging and is seen as a key method for future applications in medical field. Emergence of deep Convolutional Neural Networks (CNN) tends to provide better classification in abdominal imaging analysis as compared to traditional models. This paper presents the state of the art of abdominal images for classifying abdominal organs based on deep learning and is a useful for computer-aided diagnosis applications. First this paper describe background of abdominal organs as well as modalities of imaging system. Then, we reviewed the techniques of deep learning for image segmentation, object detection, classification and other related tasks for multiorgan and single organ abdominal images. For single organ, different organs of abdomen such as liver, kidney, pancreas, and stomach are discussed seprately. In the last section, we have discussed current market challenges and the future recommendations.

1 Introduction

The primary cognitive task of a diagnostic radiologist is medical imaging analysis and interpretation. With the rapid development in medical imaging, an enormous amount of data is produced due to the different image modalities such as magnetic resonance imaging (MRI), computed tomography (CT) and positron emission tomography (PET). Computer-aided diagnosis systems improved the healthcare by analyzing the image modalities (especially CT images). This computerized analysis has several advantages over human interpretation, such as accuracy, speed, non-sensitivity to exhaustion, and bulky knowledge base for diagnostic information. One of foremost step in computer-aided diagnosis is organ classification and segmentation. Among the various organ segmentation techniques, the segmentation of abdominal organs like liver, stomach, kidney, pancreas and bladder from CT images has gotten keen interest since last few years.

In radiology, the procedures of endoscopic pancreatobiliary is deployed for imaging and intervention of organs. For this aid, an endoscope is orally injected and navigated over the gastrointestinal region to particular locations on the stomach or duodenal wall to permit the pancreatobiliary organ imaging. This navigation procedure is often challenging, particularly for novice endoscopists due to the small field view of endoscopic, and lack of visual location alignment. To overcome this challenge, segmentation of multiple organs is introduced to support the navigation and targeting procedure. Segmentation of multiple organ includes: gastrointestinal organs (stomach, duodenum, and esophagus), adjacent organs used as navigational landmarks (kidney, liver, gallbladder and spleen) and the pancreas.

Organ segmentation from abdominal images can aid scientific work flow in multiple clinical domains like computer-assisted diagnostic intervention, treatment planning, and delivery, surgical planning and delivery, intra-operatablity, planning radiation therapies and so on [1, 2]. These potential benefits recently encouraged the interest in the development of more comprehensive computational anatomical models. Deep learning (DL) techniques has got the revolutional breakthrough in computer vision and pattern recognition tasks. The deep learning encompasses several machine learning algorithms for modelling high level abstractions in data by deploying deep architectures comprised of multiple non-linear transformations [3]. Deep learning techniques are used to learn the discriminative visual features from raw images directly at multiple level of abstractions. Recently, the deep learning techniques have been employed in different domains especially in medical imaging classification and automated disease diagnostic systems. This recent development of deep learning serves as a motivation to exploit convolutional neural network (CNN) for abdominal organ segmentation and classification.

This paper presents a detailed review on the deep leaning based techniques for multi-organ as well as single organ abdominal images classification/segmentation. The background of basic terminologies of multi-organ abdominal images as well as medical imaging modalities are explained in Section 2. Section 3 explain the state of art of multi-organ and single organ abdominal images according to the traditional standard steps: image acquisition, preprocessing, feature extraction, segmentation, and classification. Section 4 presents the existing and suggested recent performance evaluation matrices. The main findings of the survey is elaborated in Section 5. Section 6 presents the challenges offering in the domain along with the future perspectives. Finally the conclusion is draw in Section 7.

2 Background of basic terminologies

The background of the abdominal diagnostic is quite an old concept. The information that a tense abdomen is a dangerous and life-threatening condition is extremely old. In 1948, the pediatric specialist Gross perceived the clinical significance of a tense abdomen as a complication of the fix of huge omphaloceles [4]. Kron et al. [5] suggested the term “abdominal compartment syndrome” (ACS) in 1984. World Society of the Abdominal Compartment Syndrome (WSACS) refined the definitions of ACS [6] and provide guidelines management in the recent publications of 2006 and 2007 [7] and updated in 2013 [8].

The conventional methods of diagnosis and intervention of abdominal images relay on the human expertise. Thus these methods were time consuming and tiresome. This difficulty is overcome by the automated computerized systems that work on the abdominal images and provide faster and accurate diagnosis and intervention on abdominal images. The automated systems either classify or segment the multi-organ of abdomen or either work on single organ of abdomen. Here multi-organ means that more than one organ of abdomen whereas single-organ means only one organ of abdomen is used for classification or segmentation task respectively. On the basis of these classification, we provide the literature review of multi-organ and single-organ in Section 3 and illustrate the taxonomy in Figure 3.

Before presenting the state of art, we provides comprehensive detail according to the biological point of view as well as different types of images from different types of imaging systems that will present the ease for the IT researcher to work on it. Basically abdominal images are the image of the abdomen. The abdomen is the part of body that space between the thorax (chest) and pelvis. The abdomen is the combination of three main categories on the basis of organs: gastrointestinal organs (stomach, duodenum, and esophagus), adjacent organs used as navigational landmarks (kidney, liver, small and large intestines, gallbladder and spleen) and the pancreas. These organs are connected together loosely with the aid of connecting tissues known as mesentery that permit them to expand and to slide against each other. Abdomen have many important blood vessels including the venacava and their small branches, aorta etc. Abdomen is protected in the front by the fascia that is thin and tough tissue layer. In the rear of the abdomen are the back muscles and spine. The overall organs of abdomen are graphically illustrated in the Figure. 1.

Fig. 1

Internal and External Structure of Abdomen with organs (Available at: https://www.webmd.com/digestive-disorders/picture-of-the-abdomen)

The major organ of abdomen is stomach that is located on the left upper side. It is the muscular organ of the abdomen that receives the food from the esophagus through the valve known as esophageal sphincter. To the left of stomach, the organ at the upper far left part of the abdomen is known as spleen. The spleen is almost 4 inches long, purple in color, fist-shaped organ. It is one of supporting organ abdomen that play a significant role in blood filtering as part of the immune system. Another important organ of abdomen is kidney that is a bean shaped organ located either the side of the spine, behind the abdomen and below the ribs. The kidney’s pelvis is responsible to collect the waste into the urine and drains down towards a tube called the ureter to the bladder. The next foremost organ of abdomen is liver, large reddish brown meaty organ of abdomen that lies on the right side protected by the rib cage with a weight of 3 pounds. The bladder could be a tiny pouch that sits slightly below the liver. The bladder stores digestive fluid (bile) created by the liver. The pancreas lies across the back of abdomen behind the stomach. The head of the pancreas is attached to the first section of small intestine known as duodenum through the small tube known as pancreatic duct.The tail of the pancreas is shrink and extends to the left side of the body. The intestines is the amalgamation of large intestine, small intestine and rectum. The small intestine is also known as small bowel that is one inch in diameter and 20 feet long in height.

Automated imaging analysis tool based on machine learning and deep learning algorithms are the key enablers to improve the quality of image diagnosis and interpretation by facilitating through efficient identification of finding. Medical imaging is the process of creating visual representations of the interior of a body for medical analysis and clinical intervention, as well as visual representation of organs or tissues. Medical imaging is also called radiology. There are different types of imaging, such as X-rays, ultrasound, CT (computed tomography) scans, and MRI (magnetic resonance imaging). Each imaging type uses a different technology to create an image. One of the oldest and most commonly used imaging technique is X-rays discovered in 1895 and used in 1896 to capture image of human tissue first time. X-rays use ionizing radiation to generate images of a person’s internal structure by sending beams through the body. These are absorbed at different levels depending on the density of the tissue. X-ray radiation produce three kinds of medical images; conventional X-ray imaging, angiography, and fluoroscopy. X-rays are used to evaluate the digestive system.

Ultrasound is introduced in 1952, used for therapy and muscle stimulation or as a diagnostic tool in medical imaging using an ultrasonographer. It uses high frequency sound waves to produce images of interior of body. It is broadly used in abdominal images such as scanning organs in the pelvis and abdomen and diagnosing symptoms of pain, swelling and infection. Another important medical imaging is CT (computed tomography) scans used from 1972. CT combines multiple X-ray images taken from different angles, thus generate images with greater information. CT scans are widely used in abdominal images such as scanning organs in the pelvis, chest and abdomen as well as in abdominal aortic aneurysms (CT angiography). Magnetic Resonance Imaging (MRI) was developed in 1977, uses radio waves and a magnetic field to provide detailed images of organs and tissues. MRI is used to evaluate the major organs of abdomens. The advantages and disadvantages of these medical imaging are illustrated from Table 1.

Table 1 Different Types of Imaging Systems

3 State of the art

The work on abdominal images started back to 1983 when Liu et al. [9] investigated digital processing in order to improve ultrasonic abdominal images. With the advent of machine learning and deep learning approaches, researchers focus on the development of automated system for the diagnosis, prediction, classification, detection, and segmentation of abdominal images. A lot of survey papers had been published on liver images for segmentation techniques [10,11,12] , liver diseases prediction [13,14,15], and monitoring liver disorder [16, 17]. However, there is no comprehensive review on multiorgan abdominal images, stomach images, pancreas images, and stomach images using machine learning or deep learning models. In contrast to several reviews on this subject, our review paper provides additional information which is lacking in other review articles, such as provide comprehensive state of art on multiorgan and single organ like liver, kidney, pancreas, and stomach.

Early work of abdominal organ segmentation relied on atlas-based methods [18,19,20], statistical models [21, 22], patch-based methods [23], multi-atlas methods [18], multi-atlas label fusion methods [22, 24,25,26,27], probabilistic atlas [19, 28] and registration-free methods [29,30,31]. These techniques attain remarkable results for abdominal organs but a lot of challenges were offered in these techniques like large anatomical variability, reliability on handcrafted features, organ-diagnostic image features [31, 32] etc. Recently deep learning techniques had been employed for multi-organ and single-organ abdominal images. This motivate us to provide a comprehensive review on abdominal images including multiorgan and single organ images using deep learning techniques.

Before presenting the related work, we first elaborate the standard workflow of the system. The standard workflow of automated abdominal images classification/segmentation system consists of four major phases: Image Acquisition, Preprocessing, Image Segmentation, Feature Extraction and Classification. These phases are graphically illustrated in Fig. 2 and elaborated comprehensively in the coming subsections along with the literature review. We present the literature of the abdominal images with respect to standard workflow according to the taxonomy of multi-organ and single organ as illustrated from Fig. 3.

Fig. 2

Standard Workflow of Abdominal Images Classification

Fig. 3

Taxonomy of Abdominal Images Classification

3.1 Image acquisition

The process of capturing or acquiring the images is called image acquisition. It is one of the foremost and fundamental task in any computer vision or image processing domain. Due to the medical imaging nature, the abdominal images belongs to different modalities like Computed Tomography (CT), Ultrasound-imaging, Magnetic Resonance (MR) imaging, Positron Emission Tomography (PET), Digital Subtraction Angiography, Single-Photon Emission Computed Tomography (SPECT) etc. These modalities play an important role in disease detection and diagnosis. Another important aspect is that optimal acquisition device will be select on the basis of the investigation objective, in order to highlight the particular areas of the human body. Dataset is the cornerstone of any research work. The availability of dataset is one of the essential prerequisite for development and evaluation in any research domain and same is the case with abdominal images based systems. Some of the publicly available datasets are detailed followed and tabulated in Table 2.

Table 2 Publicly Available Datasets for Abdominal Imaging System

3.1.1 Multi-organ abdominal ct reference standard segmentations

This dataset consists of 90 abdominal CT images including liver, left kidney, stomach, pancreas, spleen, gallbladder, esophagus, and duodenum for reference segmentation. The dataset is released with conjunction with the paper [33] published in IEEE Transactions on Medical Imaging paper. The abdominal images and reference segmentation was taken from two publicly available datasets: The Cancer Image Archive (TCIA) Pancreas-CT data set [34, 35] and the Beyond the Cranial Vault (BTCV) Abdomen data set [36].

3.1.2 Cancer imaging archive pancreas-ct dataset

The Pancreas-CT data set encompasses 82 3D abdominal CT scans acquired at the National Institutes of Health Clinical Center. CT scans were acquired from 53 male and 27 female subjects. 17 subjects are pre-nephrectomy healthy kidney donors while 65 subjects have neither major abdominal pathologies nor pancreatic cancer. The images were acquainted from Philips and Siemens MDCT scanners (120 kVp tube voltage) with resolutions of 512×512 pixels and slice thickness between 1.5 to 2.5 mm.

3.1.3 Beyond the Cranial Vault (BTCV) abdomen dataset

The BTCV data set contains abdominal CT scans acquired at the Vanderbilt University Medical Center from metastatic liver cancer patients or post-operative ventral hernia patients. The voxel sizes of images are from 0.6 to 0.9 mm in the left-right axis and anterior-posterior, 0.5 to 5.0 mm in the inferior-superior axis. The fields of view is from 172–318 mm for anterior-posterior, 246–367 mm for left-right axis and 138–283 mm for inferior-superior axis that achieved from the manual cropping of rib-cage.

3.1.4 LiTS - Liver Tumor Segmentation dataset

The LiTS dataset comprises of 200 3D liver CT scans from different clinics. The dataset contains images of varying spatial resolution and fields-of-view. The axial slices have identical size of 512×512, with slice spacing from 0.45-5.0 mm, in-plane resolution of 0.60-0.98 mm. The dataset splits into 130 CT scans in training and 30 CT scans in testing respectively.

3.1.5 3D-IRCADb-01 database

The 3DIRCADb dataset contains 20 venous phase enhanced CT scans of 10 men and 10 women, where 15 volumes have hepatic tumors in the liver (75% of cases). The analyzed CT volumes differ substantially in the level of size and number of tumor lesions and contrast enhancement.

3.1.6 NIH pancreas segmentation dataset

NIH pancreas segmentation dataset [34] contains 82 contrast-enhanced abdominal CT scans. The size of CT volumes is 512× 512 ×D, where D belongs to 181 and 466. The spatial resolutions height and weight ranges from 0.5-1.0 mm with the depth of 1.0 mm.

3.1.7 BOT gastric slice dataset

The gastric dataset encompases 560 gastric cancer images and 140 normal images. The images were acquired by hematoxylin-eosin (H&E) staining with 20 times magnification factor. The resolution of images is 2048×2048. The tumor area in images are annotated by data provider.

3.2 Image pre-processing

Pre-processing is elementary step to enhance the input data or to sort raw data samples in a standard form that is appropriate for feature extraction phase. In medical imaging, the acquired images are mostly affected by low contrast, noise, blurriness, poor sharpness that leads to the false diagnosis. Therefore several image processing and computer vision algorithms are deployed to enhance the images like contrast enhancement, histogram equalization, filtering, de-noising, gray level transformation etc.

The primary task of medical imaging analysis is correct image annotation. Image annotation is the task of radiologists to annotate an image with label. The next step is to clean the images and to enhance the contrast. The medical images were obtained from different modalities that cause artifacts and false intensity levels. Thus, different machine leaning and image processing algorithms were deployed to enhance the contrast of images. Noise removal algorithms are used to remove the unnecessary information and artifacts from images. Thresholding is used to segment the images into regions by creating binary image. Furthermore, data augmentation is a new emerging image processing technique used to increase the training samples artificially for deep learning models. Different types of data augmentation are scaling, translation, rotation, flipping, sharpening, zooming, brightness, and high frequency images generation. We present the preprocessing techniques deployed in the domain of abdominal images using multiorgan and single organ in the following subsections.

3.2.1 MultiOrgan preprocessing

Different types of preprocessing techniques are employed to enhance the multiorgan abdominal images. The foremost preprocessing technique used for multiorgan abdominal images is image annotation. Image annotation is perform by radiologist that assign label to abdominal organs. Along with image annotation, different image processing and machine learning techniques are used as preprocessing technique in the domain of multiorgan abdominal image processing. We here present some of the notable techniques of preprocessing.

Zhou et al. [37] performed multi-organ segmentation using 210 CT images dataset. The images were annotated by four radiologists and preprocessed using contrast enhanced technique. They represented the CT images into coronal, axial, and sagittal planes. In [38], 240 3D CT scans from Computational Anatomy dataset were collected. They performed 3D to 2D image sampling and 2D to 3D label voting in preprocessing.

González et al.[39] gathered the CT scan images from retrospective COPD observational study [40]. They split the dataset on the basis of annotation structure. In complete dataset, 2000 cases were annotated into six abdominal structures by an expert while in partial dataset, 3000 cases were annotated in such a way one structure per case.

Cheng and Malhi [41] conducted experiment on 185 studies of consecutive clinical abdominal ultrasound with the total gray scale images of 9298. They performed text annotation for categorization of ultrasound images into 11 categories. In preprocessing, images were resample into 256 × 256 resolution.

Roth et al. [42] used 150 cases, 281 contrast enhanced CT images for training set and 50 images for validation set. Gibson et al. [33] experimented on two publicly available dataset: 43 subjects of Cancer Imaging Archive Pancreas-CT dataset and 47 subjects of Beyond the Cranial Vault (BTCV) dataset for the segmentation challenge. They performed manual cropping in preprocessing. Larsson et al. [43] performed thresholding and removal of positive small samples in post processing.

3.2.2 Single organ preprocessing

After discussing the multiorgan preprocessing techniques, we present the single organ preprocessing techniques. We start with the preprocessing techniques of liver proceeding with kidney, pancreas, and stomach in the following paragraphs.

Gruber et al. [44] conducted experiment on different type of liver lesions using the subset of LiTS (Liver Tumor Segmentation) challenge dataset. They used 756 axial slices in training, 50 slices in validation and 50 for testing in their experiment respectively. Adar et al. [45] conducted experiment on 182 CT images of liver lesions (53 cysts, 64 metastases and 65 hemangiomas). In order to increase the training data and reduce over-fitting, they performed data augmentation techniques like rotation, scaling, flipping and translation in preprocessing. Li et al. [46] performed liver lesion segmentation on 26 portal of enhanced CT images acquired from Zhujiang Hospital. They generated 1 million patches of size 17 × 17 in training. Cohen et al. [47] worked on 333 CT images of 40 patients annotated by radiologist. They split the dataset in such a way that 255 images for training and 108 for testing. They applied two data augmentation techniques i.e. scaling and translation in preprocessing. Schmauch et al. [48] investigated the focal liver lesion with the aid of deep learning techniques for the detection and classification of liver lesion into two classes (malignant and benign). They experimented on 367 ultrasound images. ResNet50 was deployed to extract the feature vector of 2048. Doğantekin et al. [49] used perceptual hash function in preprocessing. They conducted experiment on 200 augmented CT images. Ben-Cohen [50] experimented on 333 annotated CT images.

Kline et al. [51] diagnosed polycystic kidney disease (PKD) using MRI images of 2000 cases in training and 400 cases in testing. Yin et al. [52] collected 185 ultrasound images of kidney from Children’s Hospital of Philadelphia (CHOP). Kuo et al. [53] conducted experiment on 4,505 ultrasound images. They labeled the images using eGFRs with the aid of serum creatinine concentrations. Al Imran et al. [54] diagnosed the Chronic Kidney Disease by retrieving from the UCI machine learning repository. They performed imputation of missing value and feature scaling in preprocessing and then partition and balance the dataset. Salehinejad [55] annotated images by sampling with cylindrical transform in 3D space. They conducted experiment on 20 contrast enhanced CT images of abdomen. Marsh et al. [56] used deep learning model for the classification of non-sclerosed and sclerosed glomeruli from the 48 whole slide images of donor kidney biopsies. Pedraza et al. [57] conducted experiment on 10,600 region of interests (ROIs) images from 40 whole slide images.

Roth et al. [58] performed experiment on the contrast enhanced CT images of 82 patients for pancreas segmentation. Sekaran et al. [59] classified the pancreatic cancer from 1900 images of the dataset obtained from the Cancer Imaging Archive (TCIA). Oktay et al. [60] conducted experiment on 150 abdominal 3D CT scans of TCIA CT Pancreas dataset.

Shichijo et al. [61] conducted experiment on 32,208 esophagogastroduodenoscopy (EGD) images of 1750 patients. Garcia et al. [62] detected the lymphocyte gastric cancer using Immunohistochemistry (IHC) images. They performed data augmentation techniques like rotation and reflection on 3,257 images and generated 10868 image patches. Horie et al. [63] gathered 8428 training images and 1118 test images from Cancer Institute Hospital of Japan for the detection of esophageal cancer. Itoh et al. [64] conducted experiment on 179 upper gastrointestinal endoscopy images from 139 patients. The dataset was increased with the rotation data augmentation technique. Li et al. [65] experimented on BOT gastric slice dataset that is available publicly. They performed rotation as a data augmentation technique in preprocessing. Zhu et al. [66] experimented on 790 images and 203 test images. Rotation was performed as a data augmentation technique to increase the data samples. The qualitative analysis of preprocessing techniques of abdominal images from 2017 to 2019 are presented in Table 3.

Table 3 Qualitative Analysis of Preprocessing Techniques of Abdominal Images based Systems from 2017 to 2019

3.3 Feature extraction

Feature extraction is the method of extracting the representative characteristics of abdominal organs that are used for the discrimination of several organs. The optimal selection of a dominant set of feature is the key for an effective system. Broadly speaking, feature extraction techniques can be categorized into three broad categories: statistical features, structural features, model based features or automatic features [67, 68]. Statistical features are the mathematical or statistical measurements for classification of relevant information for reducing the gap among difference classes. Another broad categorization of statistical features are between Global and Local features. The global features are extracted from the entire image. On contrary, in local features extraction, image is divided into number of units or sections and the features are extracted from particular section of image. The structural features are the local structure of the abdominal images like Local Binary Patterns (LBP), pixel density within abdominal organ grids, Scale-Invariant Feature Transform (SIFT) and Speed Up Robust Features (SURF) etc. There has been an increased attention in recent years on methods that do not rely on hand-crafted features. This interest leads the researchers towards the model based or automatic features. The model based or automatic features are learned from raw data (pixels, in the case of images) using specific models like Convolutional Neural Network (CNN) , Extend Learning Model (ELM) , Recurrent Neural Network (RNN) etc. The qualitative analysis of feature extraction techniques used in the abdominal imaging systems from 2017 to 2019 are reported in Table 4.

Table 4 Qualitative Analysis of Feature Extraction Techniques of Abdominal Images based Systems from 2017 to 2019

3.4 Image segmentation and classification

One of the important and crucial task in medical imaging is image segmentation in which image is divided into the regions that are significant for a particular task such as the detection or segmentation of organs or the metrics computation. Image segmentation can be categorized into three categories on the basis of the extracted features: region-based, edge-based, or classification techniques [95]. The region-based and edge-based techniques depends on the inter-region differences and intra-region similarities between features. On contrary, classification techniques assigns the unique class labels to the individual voxels or pixels depending on the values of the features. Some of the notable segmentation techniques are thresholding (global or local, point-based or region-based) [96], atlas based methods [20] and deep learning based segmentation [97].

Classification is the process of predicting the class labels or categories of images or data points. A number of classification techniques have been proposed in the recent years. These techniques rely on supervised or unsupervised learning. Supervised learning is the classification technique in which targets are provided with the input data. The classification techniques based on supervised learning are Artificial Neural Networks (ANNs), Swarm Intelligence (SI), Support Vector Machines (SVMs), Linear Discriminant Analysis (LDA) etc. On the other hand, unsupervised learning does not require target classes with the training data. Examples of unsupervised learning are K-means clustering, hierarchical clustering, mixture models, OPTICS algorithm etc. Regarding different classification strategies, the success of deep neural networks are revealed tremendously due to the large spread of Convolutional Neural Networks (CNNs). Deep Convolutional Neural Networks works on the model based or automatic features are learned from raw data (pixels, in the case of images) using specific models. Deep Convolutional Neural Networks had been deployed in many medical applications for image classification and segmentation with more than 100 publications. Deep Convolutional Neural Network architectures can be used in three scenarios: training from scratch, fine-tuning and fixed feature extractor or freeze. In first scenario, the overall architecture of a specific CNN is designed from scratch. This requires an enormous amount of training samples and large dataset with several classes. In second scenario, pretrained CNN architecture is deployed on the target dataset using transfer learning. As the pretrained CNN is trained on the large base dataset thus it can be employed on the small target dataset for attaining high accuracy. Different pretrained CNN architecture are AlexNet [98], VGG [99], ResNet [100, 101] , GoogLeNet [102], DenseNet [103], Inception [104], Xception [105], and MobileNet [106]. In the third scenario, classification layer from the CNN architecture is removed and freeze the specific layer of the CNN. The freeze layers act as the features that can be classified using linear classifier [107, 108].

Pre-invasive segmentation gives more difficult classification than the binary classification task of invasive segmentation. Thus semantic segmentation is used to assign each pixel of image to appropriate target label using region of interest image as a ground truth [109]. Different CNN models are used for semantic segmentation like FCN [110], UNet [111], Fully Connected DenseNet [112], DeepLab [113], and Gated-SCNN [114]

3.4.1 Multiorgan segmentation and classification

Zhou et al. [37] performed multi-organ segmentation using 210 CT images dataset. They segmented 16 abdominal organs by deploying their deep learning model named as Deep Multi-Planar CoTraining (DMPCT). They achieved the mean Dice-Sørensen Coefficient (DSC) of 77.94%. In [38], FCN was deployed with the aid of voting technique on 240 3D CT scans. They attained 89% accuracy using the proposed voting scheme. González et al. [39] employed UNet and its modification on the axial slice to perform semantic segmentation. An average mean dice score of 0.909 using UNet, 0.908 using 6xUNets, 0.910 using PUNet and 0.916 using CUNet was achieved respectively. Cheng and Malhi [41] deployed VGGNet and CaffeNet for feature extraction and classification. They achieved highest accuracy of 90.4% in Top-2. Roth et al. [42] employed 3D FCN (UNet) for segmentation of seven abdominal organs. Their experiment composed of two stages. In first stage, 3D FCN was trained on candidate region while in second stage, organs were segmented in more detail. An overall mean dice score from 68.5 to 82.2% was reported. Gibson et al. [33] segmented eight abdominal organs by deploying DenseVNet architecture. The experiment was conducted on two publicly available dataset: 43 subjects of Cancer Imaging Archive Pancreas-CT dataset and 47 subjects of Beyond the Cranial Vault (BTCV) dataset for the segmentation challenge. They achieved the dice score of 0.95 on spleen, 0.93 on left kidney, 0.95 on liver, 0.87 on stomach, 0.75 on pancreas, 0.73 on gall bladder and 0.63 on duodenum respectively. Larsson et al. [43] employed two CNN architectures: CNN-sw and FCN on MICCAI 2015 dataset by simplifying the computation of training. They used multi-atlas technique for the localization of ROI and CNN for the voxal classification. An average dice score of 0.767 using FCN and 0.757 using CNN-sw was attained from their proposed system.

3.4.2 Single organ segmentation and classification

In this section we present the work of abdominal images using single organ of abdomen. We elaborate the deep learning based segmentation and classification of four major organs of abdomen: Liver, Kidney, Pancreas, and Stomach.

3.4.3 i. Liver

In this section we present the related work of liver. We explain the feature extraction, segmentation, and classification techniques along with the performance evaluation. Gruber et al. [44] conducted experiment on different type of liver lesions using the subset of LiTS (Liver Tumor Segmentation) challenge dataset. They used 756 axial slices in training, 50 slices in validation and 50 for testing in their experiment respectively. They deployed UNet CNN architecture to segment the tumor region and IoU of 0.93848. Adar et al. [45] conducted experiment on 182 CT images of liver lesions (53 cysts, 64 metastases and 65 hemangiomas). They employed the CNN architecture to detect the lesions and named it GAN model. They attained 92.4% specificity and 85.7% sensitivity using augmented dataset.

Li et al. [46] performed liver lesion segmentation on 26 portal of enhanced CT images acquired from Zhujiang Hospital. They generated 1 million patches of size 17 × 17 in training and deployed CNN. Furthermore they compared the proposed system with the existing machine learning techniques. The overall mean similarity coefficient of 80.06 ± 1.63%, precision of 82.67 ± 1.43%, and recall of 84.34 ± 1.61% respectively. Cohen et al. [47] performed semantic segmentation using UNet to detect the lesion and a dice of 83 was reported.

Schmauch et al. [48] investigate the focal liver lesion with the aid of deep learning techniques for the detection and classification of liver lesion into two classes (malignant and benign). They experimented on 367 ultrasound images. ResNet50 was deployed to extract the feature vector of 2048. The liver lesion was detected using the local prediction technique in which annotation was provided to the model. However, characterization of liver lesion was accomplished with 7 neurons of densely connected layer. The proposed system revealed mean ROC-AUC scores of 0.935 for liver detection and 0.916 for liver characterization.

Doğantekin et al. [49] extracted features 5 layer convolutional neural network and the extracted features were fed to the SVM, KNN and ELM. The ELM outperformed and reported 97.3% accuracy. They concluded that proposed system reduced the execution time of CNN, hard disk space and improved the classification performance. Vorontsov et al. [77] performed segmentation and detection of liver using fully convolutional network (FCN). They experimented on Liver Tumor Segmentation (LiTS) challenge dataset that contains CT images of the patients suffering from colorectal liver metastases. They reported the highest Dice similarity coefficient of 0.68.

Christ et al. [78] segmented the liver and lesion with the aid of cascaded fully convolutional neural networks (CFCNs) and dense 3D conditional random fields (CRFs) based on the architecture of UNet. The ROI of liver segment was detected from first FCN and input to the second FCN for the lesion segmentation. The results were refined using CRFs. They worked on 3DIRCAD abdomen CT dataset and reported 94% dice score. In 2017, they deployed same experiment on MRI images for the liver and tumor segmentation [79]. They experimented on 38 MRI images and achieved over than 94% dice score. In the meanwhile, Sun et al. [80] segmented the liver tumors by employing multi-channel FCN on CT images, where the feature vector was created by the fusion of features from different channels.

Han [81] proposed deep convolutional neural network (DCNN) for the segmentation of liver lesion. The proposed system revealed 0.67 dice score on 70 test CT scan images of LiTS (Liver Tumor Segmentation Challenge) dataset. Li et al. [82] worked on the datasets of MICCAI 2017 i.e. 3DIRCADb and LiTS Dataset. They designed H-DenseUNet architecture for the segmentation of liver and tumor. A dice global of 96.5% was achieved on LiTS Dataset and 0.937 on 3DIRCADb dataset for the tumor segmentation and 0.982 for liver segmentation.

Ben-Cohen [50] deployed FCN for the detection of liver lesions and segmentation. The experiment was accompanied on 333 annotated CT images. They proposed a deep learning model based on UNet framework and attained dice of 83%.

3.4.4 ii. Kidney

We present the feature extraction, segmentation, and classification techniques of abdominal images regarding to kidney in this section. Kline et al. [51] diagnosed polycystic kidney disease (PKD) using MRI images of 2000 cases in training and 400 cases in testing. They performed semantic segmentation by deploying UNet architecture of CNN. They reported 0.96 dice score using single network and 0.97 dice score using multi-observer respectively.

Zheng et al. [83] detected the congenital abnormalities in children due to the kidney and urinary tract. Graph cuts method were used for the segmentation of kidney. They extracted conventional features like geometrical features and histogram of oriented gradient (HOG). Furthermore, they also extracted CNN based features. Linear SVM was used for the classification. The accuracy of 81%, 84%, and 87% on right, left, and bilateral kidney. In [84], imagenet-caffe-alex model was deployed to explore the transfer learning techniques. They extracted handcrafted and automated features. They claimed that integrating handcrafted and automated features lead to improve the performance of classification and reported the classification accuracy of 0.87 ± 2.1 on combined features.

Yin et al. [52] extracted features from pretrained VGG-16 and fine-tune through DeepLab. They extracted the boundary of kidney using distance regression network and performed semantic segmentation. Their proposed system revealed 98% accuracy. Kuo et al. [53] performed transfer learning techniques using ResNet model. They achieved overall accuracy of 85.6%.

Kannan et al. [85] employed CNN model for the classification of normal, nonglomerular, and globally sclerosed images. They collected 275 trichrome-stained images from 171 chronic kidney disease patients of Boston Medical Center. They attained 92.67% accuracy on test set.

Al Imran et al. [54] diagnosed the chronic kidney disease using deep learning, logistic regression, and feed-forward neural networks. They reported f1-score of 0.97 on deep learning model, 0.95 on logistic regression, and 0.99 on feed-forward neural networks respectively. Salehinejad [55] performed 3D semantics segmentation using deep convolutional neural networks (DCNNs). They compared the FCN with DCNN and reported the highest dice similarity coefficient of 98.40% on cylindrical transform (CLT) using GoogLeNet.

Marsh et al. [56] used deep learning model for the classification of non-sclerosed and sclerosed glomeruli from the 48 whole slide images of donor kidney biopsies. The pre-trained VGG16 CNN was used as patch based and fully convolutional model. They attained the IOU of 0.9766 on Tubulointerstitial, 0.5949 on Non-sclerosed, and 0.3560 on Sclerosed using fully convolutional CNN. Furthermore, IOU of 0.9160 on Tubulointerstitial, 0.2017 on Non-sclerosed, and 0.0713 on Sclerosed was reported respectively.

Bevilacqua et al. [86] performed sementic segmentation using deep learning model. They experimented on 155 MRI images of four patients. They deployed encoder-decoder CNN on full image and region of interest for the classification. Their architecture revealed 86% accuracy on full image whereas, 84% accuracy on region of interest. Pedraza et al. [57] employed pretrained AlexNet for the classification of glomerulus and non-glomerulus. They conducted experiment on 10,600 region of interests (ROIs) images from 40 whole slide images. The highest F-score of 0.999 was reported.

Sharma et al. [87] experimented on 244 CT scans of Autosomal Dominant Polycystic Kidney Disease (ADPKD) patients. They trained fully convolutional network for segmentation on slicewise axial-CT sections. Their system revealed overall Dice Similarity Coefficient of 0.86 ± 0.07.

3.4.5 iii. Pancreas

Pancreas is one of major organ of abdomen. Computer scientists and machine learning researchers play a significant work using the images of pancreas. We here present some of notable work of pancreas.

Roth et al. [58] performed experiment on the contrast enhanced CT images of 82 patients for pancreas segmentation. ConvNet was deployed and dice of 68% was retrieved on test set. In [34], they deployed multi-level deep convolution networks (ConvNets) on patches and regions and reported a Dice Similarity Coefficient of 71.8 ± 10.7% on test set.

Sekaran et al. [59] classified the pancreatic cancer from 1900 images of the dataset obtained from the Cancer Imaging Archive (TCIA). They combined the Gaussian Mixture model (GMM) with Expectation Maximization (EM) algorithm for the feature extraction. The region of interest was detected by CNN and achieved the recognition rate of 99.9% from the proposed combined system.

Oktay et al. [60] deployed UNet model for pancreas segmentation. They conducted experiment on 150 abdominal 3D CT scans of TCIA CT Pancreas dataset. They attained 81.48 ± 6.23 DSC for pancreas labels.

Li et al. [88] diagnosed pancreatic cysts using densely-connected convolutional networks (Dense-Net) without pre-segmenting the lesions. They worked on the images of 206 patients and retrieved overall accuracy of 72.8%.

Zhu et al.[89] used ResNet for 3D coarse to fine segmentation of pancreas. They conducted experiment on NIH and JHMI Pathological Pancreas dataset and reported DSC 84.59 ± 4.86% on NIH dataset. In [90] identified the pancreatic cancer i.e. pancreatic ductal adenocarcinoma (PDAC) using 439 CT scans. They developed a multi-scale CNN for tumor segmentation and achieved 56.46% accuracy, 94.1% sensitivity and 98.5% specificity respectively.

Man et al. [91] proposed Deep Q Network(DQN) for the segmentation of pancreas using deformable UNet. They worked on NIH dataset and attained 86.93 ± 4.92 mean dice coefficient.

3.4.6 iv. Stomach

One of foremost organ of abdomen is stomach that play significant role in human body. Computer vision and machine learning researchers deployed several architectures of CNN for feature extraction, segmentation, and classification using images of stomach. We here present the notable contributions.

Shichijo et al. [61] deployed GoogLeNet CNN for the classification of positive or negative Helicobacter pylori. Then they also performed the classification of 8 different anatomical locations of the stomach using GoogLeNet. They reported accuracy, sensitivity and specificity of 83.1%, 81.9%, 83.4% on first CNN and 87.7%, 88.9%, 87.4% on second CNN respectively.

Garcia et al. [62] detected the lymphocyte gastric cancer using Immunohistochemistry (IHC) images. Deep CNN model was trained with ADAM algorithm and accuracy of 94% was attained.

Horie et al. [63] gathered 8428 training images and 1118 test images from Cancer Institute Hospital of Japan for the detection of esophageal cancer. They developed a time efficient CNN model and achieved 98% accuracy.

Itoh et al. [64] employed GoogLenet for the detection of HP infection and reported sensitivity, specificity, and AUC of 86.7%, 86.7%, and 0.956 respectively.

Zhang [92] constructed an efficient CNN model named as GDPNet for the classification of Gastric precancerous diseases. 1331 gastroscopy images were collected from Sir Run Run Shaw Hospital. The GDPNet classified images into three classes: polyp, ulcer, and erosion. They accomplished 88.90% accuracy on their proposed network.

Lee et al. [93] performed deep learning classification using transfer learning technique with the aid of VGGNet, ResNet, and inception models. They collected the images from Gil Hospital and developed dataset of 367 cancer, 200 normal, and 220 ulcer images. They reported highest accuracy of 0.9649 on ResNet-50 for the classification of normal and cancer classes.

Li et al. [65] experimented on BOT gastric slice dataset that is available publicly. They developed CNN based architecture called GastricNet for the identification of gastric cancer. An average accuracy of 97.93% was retrieved on patches and 100% on slices correspondingly.

Takiyama et al. [94] deployed GoogLeNet architecture for the classification of four anatomical locations and the sub-classification of stomach images into three regions. They conducted experiment on 27,335 EGD images in training and 17,081 images in validation. They achieved AUC of 0.99 for stomach and duodenum images and 1.00 for larynx and esophagus images respectively.

Zhu et al. [66] used the transfer learning techniques of ResNet50 and developed CNN-CAD system for the diagnosis of gastric cancer. They experimented on 790 images and 203 test images. Rotation was performed as a data augmentation technique to increase the data samples. They reported the overall accuracy of 89.16%. The qualitative analysis of deep learning based classification techniques used in the abdominal imaging systems from 2017 to 2019 are reported in Table 5.

Table 5 Qualitative Analysis of Deep Learning Classification Techniques of Abdominal Images based Systems from 2017 to 2019

4 Performance evaluation matrices

The effectiveness of any abdominal images based segmentation or classification system was evaluated by computing evaluation measures based on four major outcomes; true positives (tp), false positives (fp), true negatives (tn) and false negatives (fn). The performance of the proposed system is computed using the following measures:

Accuracy

is used to determine the classes of proposed system correctly. To evaluate the accuracy of a test set, we compute the proportion of true positive and true negative in all evaluated cases computed as:

$$ Accuracy=\frac{tp+fn}{tp+tn+fp+fn} $$

(1)

Sensitivity or Recall

measure the ability of system to correctly classify the classes and is calculated from the proportion of true positives. It is calculated as:

$$ Sensitivity=\frac{tp}{tp+fn} $$

(2)

Specificity

is the ability of the model to accurately classify the actual class and is computed as:

$$ Specificity=\frac{tn}{tn+fp} $$

(3)

Precision

is the true positive relevant measure and is calculated as:

$$ Precision=\frac{tp}{tp+fp} $$

(4)

F-score or F ₁ score or F-measure

is used to measure the accuracy of test set. It is the harmonic mean of precision and recall measured as:

$$ F-score=2.\frac{Precision.Recall}{Precision+Recall} $$

(5)

Dice coefficient

also called the overlap index or Sørensen–Dice coefficient, is a metric for validation of medical image segmentation. The pair-wise overlap of the repeated segmentation is calculated using the Dice, which is defined by:

$$ Dice=\frac{2tp}{2tp + fp + fn}\\ $$

(6)

IOU

also known as Jaccard index is computed by comparing the number of pixels in each category in which the predicted and annotated labels agree (intersection) divided by the total number of predicted and annotated pixels assigned a label for that category (union).

$$ IOU=\frac{Area of Overlap}{Area of Union}\\ $$

(7)

The existing models were usually based on the aforementioned performance evaluation measures as illustrated in Table 7. However recent researches released three recent measures to evaluate the similarity region of images i.e. pixel-wise accuracy of the segmentation, region similarity, and structure similarity [115]. We elaborate these recent evaluation measures for the IT researcher that will work on this domain.

Region Similarity

measures the correct similarity of two region maps. It is computed as:

$$ RS=\frac{(1+{\upbeta}^{2})Precision \times Recall}{{\upbeta}^{2})Precision + Recall}\\ $$

(8)

where β² is the trade off between precision and recall and its suggested value is 0.3 [116]

Pixel-wise Accuracy

compute normalized and mean absolute error between predicted map (M) and ground map (G).

$$ PA=\frac{1}{W \times H} \sum\limits_{x=1}^{W} \sum\limits_{y=1}^{H} \| M(x,y)-G(x,y)\| \\ $$

(9)

where W is width and H is height of images.

Structure similarity

also known as enhanced alignment measures sturctural similarity between regions and objects of predicted map and ground truth [117].

$$ Q_{FM}=\frac{1}{W \times H} \sum\limits_{x=1}^{W} \sum\limits_{y=1}^{H} \phi_{FM}(x,y) \phi_{FM}=(f(\xi_{FM}) $$

(10)

where is the function of absolute value ξ.

$$ \phi_{FM}=(f(\xi_{FM}) $$

(11)

The overall qualitative state of art is presented in Table 7 that shows the publication year, abdominal type, dataset, modality of images in the dataset, features, models and performance evaluation respectively. The list of abberivations used in the related work and Table 7 is presented in Table 6.

Table 6 List of Abbreviations

Table 7 Review of the Deep Learning Systems based on Abdominal Images

5 Main findings

This survey presents the emerging landscape of deep learning techniques in the domain of abdominal imaging system using multiorgan and single organ that is useful for computer-aided diagnosis applications. We discuss the various feature extraction, segmentation, and classification techniques using deep learning techniques regarding to multiorgan and four single organ of abdomen i.e. liver, kidney, pancreas, and stomach. This survey draws an integrated depiction of how distinct healthcare activities are accomplished in a pipeline to facilitate individual patients from multiple perspectives. The existing reviews did not provide the detailed explanation using multiorgan and four single organ using deep learning models.

We have performed two types of analysis to show the trend of the research direction: quantitative analysis and qualitative analysis. In quantitative analysis, we use Google Scholar search engine to evaluate the most cited paper between 2017 to 2019 regarding to multiorgan and single organ. There are 983 publications found at Google Scholar on the query ”MultiOrgan abdominal images segmentation/classification using deep learning models. The most cited paper was presented by Gibson et al. [33] with 108 citations. Similarly, we used the same query is used for single organ by replacing multiorgan with liver, kidney, pancreas, and stomach respectively. We found 8000 publications on liver, 5400 on kidney, 2000 on pancreas, and 4300 on stomach correspondingly. On deep insight, mostly publications are duplicated in the search query. The most cited paper of liver was presented by Li et al. [82] with 186 citation. The publication by Hu et al. [118] in 2017 got 86 citation and become the most cited paper of kidney. Oktay et al. [60] presented attention UNet model for pancreas segmentation and become the most popular publication of pancreas with 169 citations. Sharma et al. [119] presented CNN model for classification of gastric carcinoma and become most cited paper of stomach with 87 citations. Finally we analyze that there are more than 500 publications on abdominal images either multiorgan or single organ. This seems that a lot of work is presented in this domain thus we present some of most notable contributions in our survey specially the deep learning based techniques.

In qualitative analysis, we have presented the online available dataset, preprocessing techniques, feature extraction, segmentation and classification techniques of multiorgan and single organ in Section 3 comprehensively. The online publicly available datasets are presented in Table 2 along with the web URL. The qualitative techniques of preprocessing, feature extraction, and classification from 2017 to 2019 are reported in Table 3, 4, 5. Furthermore, we present the notable contributions in Table 7. The table 7 illustrate the publication year, dataset type (multiorgan or single organ), modalaity type of images, dataset, CNN model, and performance measure respectively.

6 Current Trends, Limitations and Challenges

This review provides the detail on the deep learning techniques deployed for the abdominal images either in multi-organ or single-organ. The review discussed the deep learning based literature of five abdominal organs (liver, kidney, stomach, pancreas and intestines). Deep learning models provides the tremendous results and improvement in the research domain however, there is still room for improvement. We highlight some of the notable challenges of the domain in this section that pave the path for future research in this domain graphically illustrated in Fig. 4.

Fig. 4

Challenges of Abdominal Images Classification

Starting from the image acquisition, the most facing challenge is whole-body organ scans. In the analysis of multiorgan, several organs and their structures are need to be captured simultaneously. The appearance, shape and size of abdominal organs vary among the patients that is a challenging task in scanning. It is also noted that small organs of abdomen are less often investigated than major organs. However, in clinical applications, small organs play a significant role in diagnosis like cancer screening is diagnosis form the small organs. There is a need of optimal acquisition device that overcome these challenges. Another important challenge regarding to the CT scans of abdominal image is the artifacts that can arise in images. Different types of artifacts that arise in the abdominal images are beam-hardening, partial-volume and streak artifacts. The beam-hardening artifacts are resulting as focal regions of low attenuation contiguous to bones. The partial-volume artifacts arises in blurred edges results from the spatial averaging of distinct abdominal tissues in close proximity. The streak artifacts arises due to the motion of patient results from the respiratory, cardiac and peristalsis motion etc. Another important aspect is that several abdominal organs have similar gray levels, homogeneity of image slices and similar visual appearance which consigns thresholding to limited utility.

One of the notable challenge is availability of the public datasets. The researchers worked on the private datasets collected from different hospitals. The patients in the hospitals feels insecure to provide their data, thus only a small amount of data samples were experimented. Similarly, annotated dataset is also limited that leads to the recurrent challenge in medical imaging. This challenge mostly arises when modeling the anatomical structures of multi-organ in which large datasets are required not only to locate each organ at particular location but also to find the relations of the complex inter-organ of abdominal images. Similarly, the deep learning models requires enormous amount of training data in order to provide the best performance. In case of abdominal images, only a few data repositories like VISCERAL[120], NIH Cancer Imaging Archive [35] and UK Bio Bank Imaging Study [121] exists. These repositories does not contains the data of all the abdominal organs, anyhow, combining the organs data was performed. Another limitation of these repositories data is that annotations are not provides with most of the images. This leads to the critical challenge in anatomy and computer aided systems. Therefore, there is need to develop the large datasets with the proper annotations for the multiorgan analysis.

Moving towards the multiorgan segmentation of the abdominal images, annotation is one of the hot challenge. There is a large variation exists in the size, shape, location and appearance of the organs, thus the challenge exist in locating the anatomical structures in the target image. Another important factor is to define the fuzzy boundaries of adjacent organs and soft tissues of abdomen. This become challenging due to the similar visual appearance and the gray level intensities of the organs. One more challenge is to ensure the consistency of global spatial in labeling or annotating of the patches.

Another common challenge facing by the researcher is high inter-subject variability that occurs in abdominal organs due to the differences in disease status, age, physique, gender and intricate relations of inter-organ. Some challenges are persuaded by respiratory cycle, body pose status of digestive system, edema etc. Another important challenge is the inherent inter-organ and intra-organ variability with the development of age and body. It is one of the hot challenge in the future computer aided and computational anatomy systems because it needs the development of more comprehensive multiorgan system that characterize the organs with the diversity of age [122] and fetal stage[123].

7 Conclusion and future recommendations

In this survey paper, we provided the deep learning techniques for the abdominal images classification. The effective implementation of organs segmentation from abdominal images can aid scientific workflows in multiple clinical domains like computer-assisted diagnostic intervention, treatment planning and delivery, surgical planning and delivery, intraoperatablity, planning radiation therapies and so on. These potential benefits recently encouraged the interest in the development of more comprehensive computational anatomical models. Therefore, despite of multi-organ systems, we also surveyed the five single organs of abdomen i.e. liver, kidney, stomach, pancreas and intestines. We highlight some of the notable challenges of the domain in the previous section that pave the path for future research in this domain. Here we present some of the directions that will be helpful for the future researchers of the domain.

There is a need of optimal acquisition device that will investigate the small organs for cancer screening as well as minimize the artifacts in the images. Next, we are suggesting to explore the improved image enhancement and preprocessing algorithms that consigns thresholding to vast utility because several abdominal organs have similar gray levels, homogeneity of image slices and similar visual appearance that become a hurdle of the domain. With the advancement in image processing, there are many thresholding algorithms that tackle this challenge. There is a critical need to develop the publicly available datasets for the researchers as it is one of foremost requirement to perform any machine learning or computer vision task. It can also be noted that the available datasets contain less data samples that is a obstacle in the deployment of deep learning models. Similarly, annotated dataset is also limited that leads to the recurrent challenge in medical imaging. Therefore, there is need to develop the large datasets with the proper annotations for the multiorgan analysis.

With the rise of technological innovation and personalised medicine, big data analytics has the potential to make a huge impact on our life i.e. how it helps to predict, prevent, manage, treat and cure disease. Big data is the name given to the larger and enormous data-sets that are usually complex so that traditional information processing techniques are not enough to deal with them. It helps, government agencies, policy maker and hospital to manage resources, improving medical research, planning preventative methods and managing epidemic [124]. we are suggesting to explore and investigate the implementation of big data in the domain of abdominal imaging based system.

Furthermore, there is need to explore recent CNN architectures like Local Estimation and Global Search (LEGS) [125], Multi-Context (MC) deep learning [126], Multiscale Deep Features (MDF) [127], Deep Contrast Learning (DCL) for salient object detection [128], Encoded LowLevel Distance map (ELD) [129], Deep Hierarchical Saliency (DHS) network for salient object detection [130], Recurrent Fully Convolutional Networks (RFCN) [131], Deep Image Saliency Computing (DISC) [132], Integrating Multi-level Cues (IMC) [133], Non-Local Deep Features (NLDF) for salient object detection [134], Aggregating Multi-level Convolutional Features for Salient Object Detection (AmuletNet) [135] , Deep Saliency (DS) [136], WSS [137], and MSR [138]. Another important technique in medical images is attention mechanism that need to be explore in abdominal imaging systems.

At the end of the article, we graphically represent the future directions in Fig. 5. Having a glance on Fig. 5 it can be concluded that large researches has been conducted but much more is necessary until the problem of the abdominal images classification and segmentation can be considered largely solved.

Fig. 5

Future Recommendations for Researchers