Keywords

1 Introduction

Cancer is a term used for diseases in which abnormal cells divide without control and can invade other tissues. It spreads beyond their usual boundaries, and which can then invade adjoining parts of the body. It can affect almost any part of the body. Lack of awareness, suboptimal medical infrastructure, fewer chances of screening, and low doctor-patient ratio are the prime reasons for the rise in various cancers [3].

According to Central Brain Tumor Registry of the United States (CBTRUS), 86,970 new cases of primary malignant and non-malignant brain tumors are expected to be diagnosed in the United States in 2019 [1]. An estimated 16,830 deaths will be attributed to primary malignant brain tumors in the US in 2018. In developing country India alone, total deaths due to cancer in 2018 are 7,84,821 [2]. More than 50% of the cases in India are diagnosed in stage 3 or 4, which decreases the patient’s chances of survival. Reports say India has highest mortality-to-incidence ratio in the whole world.

Late-stage presentation and inaccessible diagnosis and treatment are common in various types of cancer. In 2017, only 26% of low-income countries reported having pathology services generally available in the public sector. More than 90% of high-income countries reported treatment services are available compared to less than 30% of low-income countries. Only 1 in 5 low- and middle-income countries have the necessary data to drive cancer policy. Annotation of brain tumors in MRI images is time consuming task for the radiologists. It has been observed that there is high inter-rater variation when the same tumor is marked by different radiologists [16]. Thus, we aimed at developing a fully automatic algorithm which accurately segment the Glial brain tumor without any manual intervention. Also, on average, segmentation of the intra-tumor parts for a single patient is completed in 60 seconds. An automatic segmentation algorithm will be useful as a reference and will save the radiologist time to attend more patients. This is of most importance in developing countries with large population.

Fig. 1.
figure 1

Top row: sub-tumor parts like Edema, Necrotic and Enhancing tumor. Bottom row: appearance of Sub tumor parts in different MR modalities: FLAIR, T1, T2, T1ce [8]

Full size image

Gliomas are the most frequent primary brain tumors in adults and account for 70% of adult malignant primary brain tumors. Glioma arises from glial cells and infiltrate the surrounding tissues such as white matter fibre tracts with very rapid growth [16]. Patients with High Grade Gliomas, also known as Glioblastoma tumors have average survival time of one year. Patient undergoes MRI scan for imaging of the brain tumor and treatment planning. Various intra-tumor parts like Enhancing Tumor (ET), Tumor Core (TC)/Necrosis and Edema appears differently in different MR modalities as shown in Fig. 1.

Fig. 2.
figure 2

Proposed two step flow diagram for whole tumor and Intra-tumor segmentation. In first step WT is segmented and WT ROI is given input to second step.

Full size image

Brain tumor segmentation is a challenging task because of its non-rigid and complex shape, variation in size and position from patient to patient. These challenges make classical segmentation techniques such as thresholding, edge detection, region growing, classification and clustering ineffective at accurate delineation of complex boundaries between tumor and healthy tissues. Brain tumor segmentation methods are broadly classified into four categories as: Threshold based, Region based, Pixel classification based and Model based techniques with pros and cons over each other [4, 6]. Many approaches to brain tumor segmentation have been implemented over decades but there is no winning theory.

Recently, methods based on Deep Convolutional Neural Networks are outperforming over all traditional machine learning methods in various domains like medical image segmentation, image classification, object detection and tracking etc. [7, 18]. The computational power of GPUs has enabled researchers to design deep neural network models with convolutional layers which are computationally expensive in all the domains [5, 14, 15]. Ronneberger et al. [17] proposed U-Net architecture for biomedical image segmentation with limited images. This paper is major breakthrough in the field of medical image segmentation like liver, brain, lung etc. Inspired from the above literature we developed two step approach based on Deep Convolutional Neural Network 2D U-Net model. Various heterogeneous histologic sub-regions like peritumoral edema, enhancing tumor and necrosis were accurately segmented in spite of thin boundaries between intra-tumor parts.

2 Patients and Method

2.1 Database

In this study, we trained model on popular Brain Tumor Segmentation Challenge (BraTS) 2019 dataset [9,10,11,12,13]. BraTS is the popular challenge organised at Medical Image Computing and Computer Assisted Intervention (MICCAI) conference since 2012. Organisers collected dataset from various hospitals all over the Globe with various scanners and acquisition protocols which made the task challengingFootnote 1. The BraTS dataset comprised of 335 training patients out of which 259 are with High Grade Glioma (HGG) and 76 with Low Grade Glioma (LGG). MRI data of each patient was provided with four channels as FLAIR, T1, T2, T1ce with volume size 240 \(\times \) 240 \(\times \) 155. Also, for each case annotations marked by expert radiologists were provided for Whole Tumor (WT), Enhancing Tumor (ET), Tumor Core (TC) and Edema. Validation dataset of 125 patients was provided without ground truths. In addition to this 166 patients data was provided for testing purpose. Evaluation of the proposed method is done by submitting the segmentation results on online evaluation portalFootnote 2.

2.2 Preprocessing

BraTS dataset is already skull-stripped and registered to 1 mm \(\times \) 1 mm \(\times \) 1 mm isotropic resolution. Bias fields were corrected with N4ITK tool [19]. All four MR channel is normalised to zero mean and unit variance.

2.3 Patch Extraction

Researchers have proposed several ways of training U-Net for biomedical image segmentation like training with complete multi-modal images and with images patches. In our database there was high class imbalance among the different intra-tumor labels to be segmented along with the non-tumor region. In order to address this, we extracted patches from all the four channels of MRI data. We explicitly augmented patches of underrepresented class by applying various affine transformation like scaling, rotation, translation. Training patches of size 64 \(\times \) 64 were extracted from the FLAIR, T1, T2, T1ce for training.

2.4 Cascaded U-Net

The proposed two step approach is shown in Fig. 2. We have cascaded two U-Net architecture for WT segmentation and intra-tumor segmentation respectively. In step one first U-Net is trained to segment the Whole tumor. Four 64 \(\times \) 64 patches from T1, T2, T1ce, FLAIR are concatenated together and given input along with the corresponding WT mask to the first layer of U-Net as shown in Fig. 3. Second U-Net is trained with WT patch along with corresponding intra-tumor patch as input to give segmentation output for enhancing tumor and tumor core. It should be noted that the area without ET and TC in whole tumor is Edema.

Fig. 3.
figure 3

U-Net architecture for WT segmentation

Full size image

We modified 2D U-Net with large number of feature channels in down-sampling and up-sampling layers as shown in Fig. 3. The architecture consists of a contracting path to capture context and a symmetric expanding path that enables precise localization. At the first layer four 64 \(\times \) 64 multichannel MR volume data was given input for training along with the corresponding ground truth. The number of features maps increases in the subsequent layers to learn the deep tumor features. These are followed by Leaky ReLU activation function and the features were down-sampled in encoding layer. Similarly, in decoding layer after convolution layers and Leaky ReLU activation function, features maps were up-sampled by factor of 2. Features maps from encoding layers were concatenated to corresponding decoding layer in the architecture. At the output layer, the segmentation map predicted by the model was compared with the corresponding ground truth and the error was back propagated in the intermediate U-Net layers. The output layer is a 1 \(\times \) 1 convolution with one filter for the first stage i.e. WT segmentation, and three filters for the second stage i.e. ET, TC and Edema segmentation. The learning rate \((\alpha )\) was initialised to 0.001. After every epoch learning rate was decreased linearly by the factor of \(10^{-1}\) which avoid convergence of the model to local minima. The model was trained for 100 epochs since validation loss did not improved beyond that. Further, for better optimization a momentum strategy was included in the implementation. This used a temporally averaged gradient to damp the optimization velocity.

Fig. 4.
figure 4

Sample segmentation results with proposed approach. Green: Edema, Blue: Enhancing tumor (ET), Red: Tumor Core (TC) (Color figure online)

Full size image

3 Result and Discussion

The quantitative evaluation of the proposed model was done on BraTS 2019 challenge dataset. No ground truths are provided for validation dataset and predicted results are to be uploaded on online evaluation portal for fair evaluation. The sample result on BraTS challenge dataset is shown in Fig. 4 with Glial tumors. Edema, Enhancing Tumor and Tumor Core segmented by our approach are shown with Green, Blue and Red colours respectively. The performance in terms of dice similarity index is shown in Table 1.

The proposed architecture was implemented using Keras and Tensorflow library which supported the use of GPUs. GPU implementation greatly accelerated the implementation of deep learning algorithms. The approximate time to train the model was 48 hours on 16 GB NVIDIA P100 GPU using cuDNN v5.0 and CUDA 8.0 with 128 GB RAM. The prediction on validation data took less than 60 seconds for a single patient with four MR channels data, each of dimension 240 \(\times \) 240 \(\times \) 155.

Table 1. Performance evaluation of proposed algorithm on BraTS challenge
Full size table

There was high class imbalance in the dataset. Around more than 98% pixels belonged to background/healthy class, and rest were labelled as Edema, enhancing tumor and necrotic tumor. Training of the model was challenging because of this class imbalance as there were chances of overfitting to the healthy class. This overfitting of the model to healthy tissue would lead to misclassification of necrotic pixels to healthy pixels. This problem was overcome by augmenting the data with under represented regions. We increased the training data with augmentation techniques like rotation, scaling, shifting etc. This had improved the performance of the model with better class balance. Patches from the boundary region of the tumor were added explicitly for better training of the model. Thus, the segmentation accuracy at the tumor boundaries improved because of additional patches.

4 Conclusion

In conclusion, we presented two step cascaded brain tumor segmentation approach with 2D U-Net architecture based on Deep Convolutional Neural Networks. The encoder-decoder type ConvNet model for pixel-wise segmentation is proposed. We incorporated information from all four MR channels for segmentation which could delineate the tumor boundaries more accurately. We considered different training schems with variable patch sizes, data augmentation methods, activation functions, loss functions and optmizers. This automated approach will definitely provide second opinion to the radiologists for automatic segmentation of brain tumor within minutes. This will enable radiologists for quick reporting and deal with more patients where there is poor patient to doctor ratio.