Keywords

1 Introduction

Brain tumor is a fatal and complex disease. The overall average annual age-adjusted incidence rate for all primary brain and other CNS tumors was 23.03 per 100,000 population [1]. The estimated five- and ten- year relative survival rates are 35.0% and 29.3% for patients with malignant brain tumor, respectively, according to a report from 2011–2015 [1]. Even with substantial improvements in the treatment process, the median survival period of patients with glioblastoma (GBM), a progressive primary brain tumor, is still 12–16 months [2]. In general, patient’s survival period highly depends on the grade of the tumor. Therefore, accurate tumor classification is imperative for proper prognosis. Diagnosis and grading of brain tumor are conventionally done by pathologists, who examine tissue sections fixed on glass slides under a light microscope. However, this manual process is time-consuming, tedious, and susceptible to human errors. Therefore, a computer-aided automatic brain tumor classification method is highly desirable. Some traditional machine learning methods, such as Support Vector Machine (SVM) [3], Random Forest (RF) [4], etc. are widely used for brain tumor classification. However, these methods require hand-crafted features extraction.

The World Health Organization (WHO) revised the central nervous system (CNS) tumor classification criteria by including both pathology image and molecular information in 2016 [5]. However, non-invasive magnetic resonance image (MRI) is regarded as an alternative source to classify tumor. Hence, many works have proposed the use of MRI in designing automated tumor classification methods [3, 6,7,8,9]. In recent years, deep learning has shown great success in many areas, and specifically in brain tumor classification [10,11,12].

The whole slide histopathology image size may be usually up to 1–2 GB and computation burden is a challenge because of its massive size. A related challenge is to effectively select the region-of-interest (ROI). This work attempts to analyze both pathology image and MRI as the sources for the brain tumor classification task. The available training data includes multimodal MRI (mMRI) sequences such as T1-weighted MRI (T1), T1-weighted MRI with contrast enhancement (T1ce), T2-weighted MRI (T2), and T2-weighted MRI with fluid-attenuated inversion recovery (T2-FLAIR).

In this paper, we investigate deep learning-based tumor classification method using mMRI and histopathology data. Our results suggest that analysis of mMRI image alone may be sufficient for the desired tumor classification task. It is composed of two parts, brain tumor segmentation and tumor classification. For brain tumor segmentation, we take advantage of the Multimodal Brain Tumor Segmentation Challenge 2019 (BraTS 2019) training data set [13,14,15,16,17], and use a 3D UNet-like CNN [18]. Subsequently, we use a regular 3D CNN with the tumor segmentation output of the previous part to accomplish the final classification task.

2 Method

Brain tumor segmentation is utilized as a prerequisite for tumor classification. Accordingly, the accuracy of brain tumor segmentation impacts the final tumor classification performance. The framework of brain tumor segmentation is shown in Fig. 1. We use a UNet-like architecture for brain tumor segmentation, and then apply a regular 3D CNN architecture for the tumor classification task. The input brain tumors are categorized into one of three sub-types: glioblastoma (G), oligodendroglioma (O), and astrocytoma (A).

Fig. 1.
figure 1

Pipeline of the proposed method.

Full size image

We assume that the cancerous cells have lower intensity on WSI. We select ROI from WSI following the steps as shown Fig. 2. We perform over-segmentation [20] on the thumbnail WSI images. From the super-pixels, we select the region with mean intensity at 10-percentile of all regions. Finally, we map the location to original WSI, and take the corresponding region as the ROI. Figure 3 shows an example.

Fig. 2.
figure 2

Strategy of selecting ROI from WSI.

Full size image
Fig. 3.
figure 3

A case example of selecting ROI from WSI.

Full size image

3 Materials and Pre-processing

3.1 Data

The training data is obtained from two sources: Multimodal Brain Tumor Segmentation Challenge 2019 (BraTS 2019) [13,14,15,16,17] and the Computational Precision Medicine: Radiology-Pathology Challenge on Brain Tumor Classification 2019 (CPM-RadPath 2019). Data of BraTS 2019 is used for tumor segmentation model, and CPM-RadPath 2019 is for tumor classification. For BraTS 2019 data, it has total 335 cases consisting of 259 high-grade glioma (HGG) and 76 low-grade glioma (LGG). For CPM-RadPath data, it has 221 cases in training, 35 cases in validation, and 73 cases in testing data, respectively. The training dataset contains 54 cases with astrocytoma, 34 cases with oligodendroglioma, and 133 cases with glioblastoma. Each case contains both radiology and pathology data. Radiology data consists of four image modalities: T1-weighted MRI (T1), T1-weighted MRI with contrast enhancement (T1ce), T2-weighted MRI (T2), and T2-weighted MRI with fluid-attenuated inversion recovery (T2-FLAIR). The structural MRIs are co-registered, skull-stripped, denoised, and bias corrected. Each case has paired pathology as well, and the size is varied from 50 Mb to 1 Gb. Note that the ground truths of validation and testing data are privately owned by the challenge organizer.

3.2 Pre-processing of Radiology Images

Even though the mMRIs are co-registered, skull-stripped, denoised, and bias corrected, the intensity inhomogeneity still exists across all patients. The intensity inhomogeneity may result in tumor misclassification. To reduce the impact of intensity inhomogeneity, z-score normalization is employed. Figure 4 shows an example with z-score normalization.

Fig. 4.
figure 4

An example of applying z-score normalization. Top row: raw images. Bottom row: z-score normalized images. From left to right: T2-FLAIR, T1, T1ce, and T2 image.

Full size image

4 Experiments and Results

The dice score obtained using pathology image analysis is 0.716, which is less than that of using only radiology MRI. Hence, the remaining methods in this work use mMRI for analysis.

4.1 Hyper-parameter Setting

We empirically crop size of all mMRIs to 160 × 192 × 128 to reduce the computational burden on the graphics processing unit (GPU). However, we ensure that all appropriate brain information is retained in the cropped version. The corresponding cross-entropy loss function is computed as follows:

$$ L{L_{ce}} = - \mathop \sum \nolimits_{c = 1}^M {y_{o, c}}log\left( {{p_{o, c}}} \right), $$
(1)

where \( M \) is the number of classes, \( y \) is a binary indictor (0 or 1) if class label \( c \) is the correct classification for observation \( o \). \( p \) is predicted probability observation \( o \) belonging to class \( c \). We use Adam [19] optimizer with initial learning rate of \( lr_{0} = 0.001 \) in training phase, and the learning rate (\( lr_{i} \)) is gradually reduced as follows:

$$ lr_{i} \, = \, lr_{0} \,*\,\left( {1 - \frac{i}{N}} \right)^{0.9} , $$
(2)

where \( i \) is epoch counter, and \( N \) is total number of epochs in training.

4.2 Brain Tumor Segmentation

We randomly split the training dataset to have 80% for training, and the rest 20% used as validation based on high-grade glioma (HGG) and low-grade glioma (LGG). We then save the best model according to the validation performance. The saved model is then used for our final tumor segmentation on radiology data of CPM-RadPath 2019. The result offers dice score of 0.73659, 0.89612, and 0.81043 for enhancing tumor (ET), whole tumor (WT), and tumor core (TC), respectively. After post-processing steps (removing small objects and filling hole), the performance reaches dice of 0.76758, 0.90456, and 0.81657 for ET, WT, and TC, respectively.

4.3 Brain Tumor Classification

With the segmented tumor, we use a regular 3D CNN for classification. The tumor is categorized as glioblastoma, oligodendroglioma, or astrocytoma. We also randomly split data as training data and validation data with a ratio of 4:1 according to the tumor sub-type. To overcome the overfitting issue, dropout and leaky ReLu are used in the network. We achieve a training accuracy of 81%.

4.4 Online Evaluation

We applied the proposed method to CPM-RadPath 2019 validation data set, which includes 35 cases with radiology and pathology images. We achieve a best validation accuracy of 0.749. All the performance parameters are reported in Table 1.

Table 1. Online evaluation result by using our proposed method.
Full size table

4.5 Discussion

In this work, we also try the experiment by integrating radiology images with pathology image for tumor classification. The histopathology images are traditionally used for tumor classification in clinical settings. However, there are some challenges for processing the pathology image. First, the size of whole slide image (WSI) has very large variation within the dataset. Second, the region-of-interest (ROI) selection is very challenging. The quality of selected ROI impacts the final tumor classification. Finally, ROI selection in WSI is quite computationally expensive for this task.

The dice score shows large difference in the Validation and Testing phase. This may be due to small data sample size and data imbalance. The training data consists of only 221 cases, which may be insufficient for deep learning-based method. Of this, 133 cases are glioblastoma, and rest 88 cases are divided between oligodendroglioma and astrocytoma, creating a data imbalance.

5 Conclusion

This work proposes a deep learning-based method for brain tumor classification. We first segment the brain tumor using a context encoding-based 3D deep neural network. We then apply a regular 3D convolutional neural network on the segmented tumor outputs to obtain the tumor classification. The online evaluation results show a promising performance. In future, we plan to utilize the digital pathology image along with mMRI for categorizing tumor sub-type, especially for distinguishing oligodendroglioma and astrocytoma.