Keywords

1 Introduction

The most prevalent malignant primary brain tumor in adults is glioblastoma (GBM) which accounts for 48.3% of all malicious brain tumors [1]. The first line radiotherapeutic treatment for GBM patients is Temozolomide (TMZ). The methylation state of the O6-methylguanine-DNA-methyltransferase (MGMT) gene promoter has been a significant biomarker for tumor response to TMZ treatment [2]. MGMT methylation status is associated with prolonged survival time in GBM patients [3]. The standardized method for evaluation of MGMT status is surgical resection which is invasive and time-consuming. Recently, different studies [4, 5] have found that genetic alternations are linked to phenotypic changes and can be detected using magnetic resonance imaging (MRI) features. Predicting MGMT status using MRI features can be broadly categorized into two categories: application of machine learning (ML) or deep learning (DL) models. Several studies focused on quantitative and qualitative feature extraction from pre-operative MRI, and then predicting MGMT status utilizing the extracted features in different ML models. V. G. Kanas et al. [6] apply quantitative and qualitative features extracted from segmented tumors, and then apply several dimensionality reduction methods for multivariate analysis of MGMT status. Another study by T. Sasaki et al. [7], the authors apply supervised principal component analysis to predict MGMT status utilizing shape and texture features. On the other hand, Authors in [8] apply different architectures of residual CNN to predict MGMT status. P. Chang et al. [9] focus on 2D CNN filters to extract features, and then apply principal component analysis (PCA) for dimensionality reduction of features that is used in the classification of genetic mutations. The study by E. Calabrese et al. [10] has cascaded deep learned based tumor segmentation with MGMT classification. After 3D tumor segmentation on multiparametric MRI sequence, pyradiomics is utilized for extraction of image features followed by random forest method to analyze the likelihood of MGMT status in patients. Most of the studies require extensive image pre-processing, feature extraction, and tumor segmentation before classifying MGMT methylation status.

In this work, we propose a deep learning-based approach with CNN that does not require tumor segmentation and feature extraction. In addition, to find the optimal hyperparameters in CNN we utilize Bayesian Optimization method.

2 Method

2.1 Description of Dataset

The dataset [11,12,13,14,15] is divided into three cohorts as follows: training, validation and testing. The training dataset consists of 585 patients. Each patient has four modalities of MRI scans in DICOM (Fig. 1): Fluid Attenuated Inversion Recovery (FLAIR), T1-weighted pre-contrast (T1w), T1-weighted post-contrast (T1wCE), and T2-weighted (T2). The MGMT status distribution of the training cohort is as follows: 307 patients are methylated, and 278 patients are unmethylated. The validation cohort consists of 87 patients. Note that the ground truth of validation data and testing data are kept private by the challenge organizers.

Fig. 1.
figure 1

Representation of MRI modalities with MGMT Status. MGMT status (0) and (1) indicates unmethylated and methylated status.

Full size image

2.2 Radiogenomic Classification Model

A convolution neural network is a stacking of convolutional layers. The layers can be categorized into different stages where each stage has a same type of convolutional layer. Each convolution filter is made up of image pixel values that are modified throughout training. The model’s parameters are adjusted variables that can be approximated or learned from the data and incorporated within the learning process [16]. The model parameters are not selected manually such as weights in neural network. Hyperparameters are factors that impact the model’s training or behavior. Hyperparameters are non-model parameters that cannot be anticipated from the data set but can be customized by subject matter experts or via trial and error until an acceptable accuracy is attained [17].

To obtain high accuracy in classification tasks, users must properly handle the hyperparameter setting procedure, which varies depending on the algorithm and data collection. This procedure can be carried out by using the algorithm’s default values or manually configuring them. Another alternative is to use data-dependent hyperparameter-tuning approaches, which aim to reduce the algorithm’s estimated generalization error over a hyperparameter search space [17]. There are different methods such as random search, grid search, and Bayesian optimization to automatically configure the hyperparameters. Grid Search evaluates the learning algorithm using all possible hyper-parameter combinations [18]. Every parameter has the same chance of influencing the process. In Grid Search’s there are a lot of hyper-parameters to set, and the algorithm’s evaluation phase is highly expensive. Random searches utilize the same search space as Grid search using a random process. This method does not produce as accurate results as Grid Search, but it takes less time to compute. The first two approaches are computationally expensive because random combination of hyperparameters are considered. Bayesian Optimization use sequential method for global optimization of objective function [19].

The main idea behind Bayesian Optimization method is to limit the objective function’s evaluation phase by devoting more effort to select the next set of hyper-parameter values. This method is specifically designed to solve the problem of determining the maximum of an objective function \( f :X \rightarrow \mathbb {R} \), given as,

$$\begin{aligned} argmax_{x \in X}f(x) \end{aligned}$$
(1)

where, X corresponds to hyperparameter space which can be considered as three-dimensional hypermeter space [19].

The two basic component of Bayesian Optimization is statistical modeling and acquisition function to determine the next sampling of hyperparameters [20]. The statistical model generates a posterior probability distribution that specifies the objective function’s possible values at potential locations [21]. With the increment in data observations the algorithm performance improves and a potential region on hyperparameters space is determined using Gaussian Process. A Gaussian Process (GP) is a robust prior distribution on functions and a stochastic process [23]. A multi-variate gaussian process is completely defined by its mean and covariance matrix. The acquisition function determines the maximum of objective function at a particular point. The aquistion function consists of exploration and exploitation phase. The objective of exploration phase is sampling to optimize the search space and the target of exploitation phase to focus on reduced search space to select optimum samples [24].

The Bayesian optimization method employs a surrogate model that is fitted to the real model’s observations [25]. In this case, the real model is the Convolutional Neural Network (CNN) with hyperparameters selected for an iteration. In each iteration, it quantifies the uncertainty in the surrogate model using Gaussian process [26]. Then for next iteration, hyperparameter set is chosen using an acquisition function that balances the need to explore the whole search space versus focusing on high-performing sections of the search space.

Fig. 2.
figure 2

Architecture of Convolutional Neural Network (CNN) model in pipeline.

Full size image
Fig. 3.
figure 3

Overview of the proposed radiogenomic classification pipeline.

Full size image

The proposed pipeline in the Fig. 3 shows data augmentation steps in input data. After the input is provided in the CNN model (Fig. 2), hyperparameter optimization is performed utilizing the Bayesian Optimization method which provides the tunned model with optimized hyperparameters [27]. The tunned model is utilized for a classification model of the methylation status.

2.3 Training Stage

In the training phase, we randomly split the data into 80% training and 20% validation on the basis on methylation and unmethylation status. In training phase, we consider only the T1-weighted post-contrast (T1wCE) modality of MRI. The pre-processing steps consist of selecting fixed number of slices for each patient. At first, we sort out the middle slice number from the list of slices for each patient. Afterwards, the slices that are in the range of 25% above the middle slice and 75% below the middle slice is selected with a fixed interval. The interval is three but if a patient data contains less than 10 slices then the interval is changed to one. The selected slices are resized into 32 \(\times \) 32-pixel value and normalized between 0 and 255.

Fig. 4.
figure 4

The Area Under Curve (AUC) in the training stage.

Full size image

The pre-processed data is augmented prior to feeding into CNN model. The data augmentation includes random flip along the horizontal axis and random rotation with scale of 0.1. For hyperparameters optimization the number of trials is 20 and objective function is to minimize the validation loss. During parameter optimization, the bounded learning rate is between 0.0003 and 0. 003. The tunned model is trained for 200 epochs with Adam optimizer. Figure 4 shows the area under curve in the training phase with hyperparameter parameter optimized model.

3 Online Evaluation Results

We apply our proposed method to the online validation data which consists of 87 patients. The data is available in the RSNA-MICCAI Brain Tumor Radio-genomic classification competition under the Kaggle Platform [15]. We apply the pre-processing steps that are discussed in the previous section. The area under curve (AUC) value in the validation phase is 0.718. In addition, the proposed method is utilized on testing data which is kept private by the challenge organizers. The area under (AUC) value in the testing phase is 0.477. The performance of classification model is shown in the Table 1.

Table 1. Performance of classification model using online evaluation
Full size table

4 Discussion

In this study, we have applied CNN with Bayesian hyperparameters optimization for classification of MGMT status in Glioblastoma (GBM) patients from MRI. As it can be seen the model performance on testing data drops compared to validation data which might be caused by the lack of generalization of model in the testing data. Moreover, the predictive feature (MGMT) is not directly visible in imaging data which is rather a genomic biomarker identified by molecular analysis in tissue specimens. However, several studies [28,29,30] have shown that genetic alterations express themselves as phenotypic changes that can be identified by MRI features extracted from the segmented tumor. In our proposed framework, we do not consider tumor segmentation to extract features from the segmented tumor. We utilize the pre-processed DICOM directly to evaluate the methylation status in GBM patient. The goal of the proposed method is to simulate the real-world clinical scenario and asses how well the model generalizes the data obtained from different sources.

The selected slices contain the middle slice and slices selected at a regular interval for each patient. The CNN with Bayesian optimized hyperparameters assist the model to obtain better classification accuracy as depicted in Fig. 4 and in the validation phase of the challenge as shown in Table 1. The training phase contains a simple architecture of CNN and tunned with hyperparameters that is obtained by reducing an objective function in Bayesian process. Therefore, the tunning of the hyperparameters improve the performance when the data distribution is similar in training and validation phase. The dataset consists of multiple image samples from 18 institutions [31]. The source of data and distribution of test data is completely different [31] and hence the classification model is unable to generalize the new data distribution and the selected hyperparameters are not the optimized ones in testing data.

Bayesian Optimization is included in the pipeline to obtain hyperparameters that assist in better classification performance by optimizing the search space. The objective of the Bayesian approach is to reduce the validation loss in each trial and shrink the hyperparameter space after each trail. After completion of pre-defined trials on training data, best hyperparameters are included in the final training of the classification model. However, such hyperparameter optimization may not be able to generalize to data from different source.

5 Conclusion

In this paper, we propose a CNN model with Bayesian Optimized hyperparameters to classify MGMT methylation status in glioblastoma patients. Bayes Optimization is applied to obtain the hyperparameters for the CNN model. The goal of this work is to predict the MGMT status using non-invasive MRI features. The proposed approach does not require extensive tumor segmentation, image pre-processing, and feature extraction steps to predict MGMT status. The proposed method is evaluated on validation and testing dataset provided by RSNA-MICCAI. The validation and testing area under curve (AUC) were 0.718 and 0.477 respectively.