Keywords

1 Introduction

The brain is the hub of all nerve activity, making it one of the necessary and delicate organs in the body. The spinal cord and brain constitute the nervous system of a human body, and the body’s vital operations, such as thought, speech, and movement, are controlled by the central nervous system. Each year, about 350,000 new brain tumour cases are recorded around the world, with a 36% survival rate. A brain tumour is an unregulated cell multiplication that leads in abnormal growth. Abnormality in the brain and spine growth can result in negative effects on how a human walks, speaks, and performs basic cognitive activities. Regular headaches, speech difficulties, fatigue, difficulty in balance, inexplicable nausea, and vomiting are all common signs of a brain tumour. These symptoms are many times overlooked at early stages and they prove to be very dangerous later when detected. These tumours may often be divided into primary tumours and secondary tumours. Primary brain tumors are further divided into two categories: low grade and high grade. When compared to high-grade tumors, low-grade tumors grow more slowly. Secondary brain tumors are malignant tumors that originate in another part of the body before spreading to the brain. The severity of the condition determines the grade of the brain tumour, which ranges from Grade I to Grade IV.

  • Grade 1: The brain cells seem normal and will grow gradually and slowly. At this stage, the long-term survival is very likely.

  • Grade 2: The cells will start looking slightly abnormal along with slow growth of tumor. At this stage, the spread of tumor to the nearby tissues starts and it may grow to a life-threatening grade.

  • Grade 3: The growth of tumor in the nearby tissues starts actively and by the looks the cells will look abnormal.

  • Grade 4: At this stage, the growth of tumor is very fast and it will spread to nearby cells very rapidly.

Neurological exams such as computer tomography (CT), MRI, and other tests such as angiogram, spinal tap, and biopsy are used to detect brain tumours. MRI detection of brain tumours is one of the most effective methods, and current diagnosis process mostly involves trained human experts. Often, grading tumours is a difficult and error-prone process, due to the minute features that need to be identified for accurate grading. Automated detection and grading of brain tumor have thus received significant research attention, as early diagnosis and accurate grading can be very beneficial in managing patient treatment and improving disease prognosis. Detection at a benign stage (lower grade) can be treated very effectively and can help in successful prevention of the tumour turning malignant. Additionally, automated segmentation of MRI images of brain tumours can help enhance diagnostic and treatment choices, particularly in circumstances when access to qualified specialists and radiologists is restricted.

A comprehensive assessment of current state-of-the-art efforts for brain tumour classification and segmentation is offered in this paper. Deep learning-based works are primarily for classification of tumour in various ways, i.e., if the tumour is present or not and if present, classifying the tumor as per the severity grade of the tumour. Certain features of the brain tumour like the radius of the tumour, density of the tumour which can be recognized with radiomic analysis can help in the detection of tumour to a great extent, and these properties can be used in combination with one another to obtain good results. Segmentation techniques are used in order to separate the tumor from other parts of the brain in the MR image so that regions of interest can be identified. The remainder of this paper is structured as follows : Sect. 2 details the various approaches in deep learning for brain tumour detection. In Sect. 3, radiomic analysis-based techniques which are mainly used for feature extraction are discussed and segmentation-based approaches are discussed in Sect. 4. Also, various standard datasets available for the research community for brain tumour detection research are discussed in Sect. 5. Open challenges and research directions are discussed in Sect. 6, followed by concluding remarks and references.

2 Deep Learning Approaches for Brain Tumour Detection

There has been a lot of research in deep learning-based brain tumor detection, which can be perceived as a classification problem. Several studies have considered brain tumor detection as binary classification problem where two classes are tumor and no tumor. A few have addressed this task as a multi-class classification where different grades of tumor are considered as different classes. We present a review of existing works that address both binary classification and multi-class classification in this section.

Amin et al. proposed that for brain tumour detection, a seven-layer deep learning model of 3 convolution layers, 3 ReLU layers, and 1 softmax layer was used to segment and classify MR images. They segmented the input image into patches of 4 * 65 * 65 pixels, determined the centre pixel label for each patch, and gave it to the deep neural network. The overall results of segmentation and classification were enhanced using this way of partitioning the image into patches. They also assessed eight distinct benchmarks, all of which are widely used in the identification of brain tumours. The results on these bechmarks, namely BRATS2012, BRATS2012 (synthetic images), BRATS2013, BRATS2013 (synthetic images), BRATS2014, BRATS2015, ISLES2015, and ISLES2017, were in the range 95.5–100% which shows that model also generalizes very well and can be further used for real-time usage. Fusion of certain handcrafted techniques was addressed by Saba et al. for the prediction of glioma categories like high-grade and low-grade gliomas. High-grade gliomas have a two-year survival rate when treated early.

The authors used deep learning approaches, particularly convolutional neural networks, to recognize the patterns. The suggested technique offers a substantial contribution by transforming the input images into one channel. An accuracy of around 99% was obtained for the BRATS2015, BRATS2016, and BRATS2017 benchmarks from BRATS, but there was no testing done with other datasets, which limits the scalability of this work.

For predicting the progression-free and overall survival of a newly diagnosed glioblastoma patient, Chaddad et al. [1] proposed multi-scale characteristics based on the Laplacian of Gaussian (LoG) filter. T1-weighted imaging (T1WI) and fluid attenuation inversion recovery (FLAIR) images of 40 GBM patients were used to extract characteristics that were manually categorized as active tumour, necrosis, or edoema. To extract multi-scale texture features, the LoG filter was applied to each of these regions of interest, and then univariate approaches including the Spearman rank coefficient, logrank test, and Kaplan–Meier estimator were employed to validate the features. Multivariate analysis, as well as the relationship between them (i.e., random forest classifier). The first three and seventh features were shown to have statistically significant correlations with PFS and OS, respectively. The three characteristics derived only from the active tumor area were associated with OS (where p is less than 0.05), and the hazard ratios (HR) were found to be 2.9, 3, and 3.24, respectively. Using the random forests classifier to predict the combined characteristics of patients with PFS and OS in GBM patients showed that the AUC was 85.37% and 85.54%, respectively.

Dastmalchian et al. adopted feature selection phase during which features which were discovered to be strongly associated were picked using Spearman’s rank coefficient. One of the traits from the correlated pairs was chosen to avoid redundancy. Furthermore, the authors used the Wilcoxon rank sum test on each of the tumour groups for the specified attributes in the preceding steps. The text characteristics were evaluated using ROC and AUC. As a result, 20 out of 39 texture features were chosen for further investigation. When used to MRF, texture analysis is a well-established and powerful method for inferring local and regional heterogeneity in tumours, and it can reveal significant differences between tumour types. This work demonstrated the use of texture analysis on MRF-derived quantitative maps in malignant brain tumours. Hashemzehi et al. evaluated their proposed model on the T1-weighted CE-MR images and obtained a precision of 94.49%. In this model, a CNN and a neural autoregressive distribution are merged. They used a three-step methodology for their research that involved categorization, feature exploitation, and density estimation. The NADE model was initially trained to produce appropriate joint distribution, which made feature extraction easier later on. Separate CNNs were used to learn features from the produced output and the raw picture, limiting the impact of noise and rotation. Although the proposed model’s operation was extremely distinctive, it was not tested on numerous datasets or datasets with a larger number of photos.

3 Radiomic Analysis for Brain Tumour Detection

Radiomics has been defined as “a quantitative approach to medical imaging that uses cutting-edge mathematical analysis to improve the data already accessible to physicians.” It has been reported to have significant impact on clinical decision making as it utilizes the inherent image information like pixel intensity and spatial distribution for determining dominant features. Radiomics also aim to convert medical images into high-dimensional data. There are CAD systems that detect if tumor/cancer is present or not, radiomics are directed towards extracting large number of features that may be used for different purposes. With the help of radiomics, decision support tools that may help in accurate prediction of tumor can be developed. Radiomic features can be combined along with other characteristics of the patients in order to improve decisions. As discussed the process of feature extraction includes six steps: acquiring images, identifying the interest areas in image, segmentation of volumes, extracting and qualifying descriptive features, using features to for collecting database, mining this data for the development of classifier models.

Brunese et al. employed ensemble learning to combine ten distinct machine learning algorithms, including closest neighbor, Linear SVM, RBF-SVM, neural network, Gaussian process, naive Bayes, C4.5, random forest, QDA, and logistic regression. First order, shape, the co-occurence matrix for gray levels, the run-length matrix for gray levels, and the gray level size zone matrix were also employed as radiomic features. The first step was to build a classifier using the previously mentioned machine learning techniques. The second step in this process was to choose the best classifier based on the criteria that were considered, such as decision boundaries, performance obtained by single models, and calibration plots. The voting ensemble classifier was used to predict the final prediction from among the many predictions provided by the best classifiers. Following that, they built a model in two stages: training and testing. In addition, the dataset was divided into three parts, namely training, testing, and validation, in the ratio 2:1:1. The accuracies obtained for Grade 1, Grade 2, Grade 3, and Grade 4 were 95.1 %, 95.1 %, 95.2 %, and 95.1 %, respectively, because they predicted four different grades of brain tumour.

Supervised learning was used by Mercaldo et al. for the prediction of the type of lung cancer among the five categories from the magnetic resonance images. A total of 30 radiomic features were considered which belong to two basic categories, namely shape and gray level size zones. Carefully considering the radiomic features is also very crucial stage as it will affect the final prediction to a very large extent. The neural network considered was a basic single layer feed forward perceptron with the Heaviside step function as the activation function. The precision obtained was maximum of 93.1% but since the prediction was to identify the different stages of lung cancer the study is very useful. Gillies et al. showed that features can be extracted for varied imaging modalities like position emission tomography, magnetic resonance, and computed tomography. Radiomics generally have six steps: collection of image, identification of volume of interest, volume segmentation, extraction of different features, using these features for database preparation, and development of classifiers based on these databases. The various usecases of radiomics include enabling diognosis, tumor prognosis, treatment selection, deciding on location of biopsy or resect. However, radiomics is a very young discipline and there are certain challenges faced like the reproducability of data, data sharing, lot of available data(big data), and lacking of standards. In the future, radiomics can prove to be a field of great use in medical health care.

4 Segmentation Approaches for Brain Tumour Detection

Segmentation is the process of assigning a specified set of classes to each pixel in an image and is an important tool in general medical image analysis tasks. It is predominantly used for extracting targeted information like regions of interest (RoI) from medical diagnostic scans. Various segmentation methods like watershed, threshold, edge-based, region-based, and clustering-based segmentation can be used. Deep neural networks have recently been widely employed for RoI segmentation as well. We discuss some relevant works that adopt these methods for brain tumor detection below.

Watershed segmentation is based on topological interpretation where resulting boundaries form closed and connected regions. Jemimma and Vetharaj [2] proposed an approach using watershed dynamic angle projection and CNNs, for accurate measurement of tumor regions. The textured characteristics of the brain were extracted using the DAP pattern, and the tumor and non-tumor areas of the MRI brain picture are classified using the CNN. Threshold segmentation is used to break down images into their constituent segments or areas in order to recognize objects and edges is known as image segmentation. The pixels in the image are partitioned based on their attributes. The pixels in the image are either replaced with black or white in threshold segmentation. Natarajan et al. [3] proposed a method based on sharpening and median filtering for preprocess and to improve image used histogram equalization and used thresholding for segmentation of image. Finally, using the image subtraction approach, the tumor area is derived. Threshold segmentation is not applicable when the object area ratio changes.

Edge-based segmentation is based on the detection of discontinuities. Maiti et al. converted an RGB color image to an HSV color image, it has three sections: color, saturation, and intensity. After enhancing the contrast, the watershed algorithm is applied to each section of the image. The disadvantage of this method is that it is difficult to construct a robust edge linking. An picture is divided into homogenous areas using region-based segmentation techniques. In this strategy, unique object boundaries are generated by flow from an interior point to an outside region. Tak et al. [4] proposed object labeling for more specific information about the tumor site and for detection and extraction they used MATLAB software. SVM was utilized in an unsupervised way to develop and retain the pattern for future usage.

Clustering-based segmentation typically employs unsupervised approaches for segmenting RoIs. Maksoud et al. [5] proposed well-formed hybrid segmentation techniques, based on fuzzy C-Means algorithm. They found that this did not correctly segment images that are influenced by noise, outliers, or other imaging distortions. They included a 3D evaluation of brain tumor identification using 3D slices. To recognize edges in CT and MRI images, Sharma et al. [6] used the Gabor transform in association with soft and hard clustering. Their study included a total of 4500 MRI scans and 3000 CT pictures.

K-means clustering was used to divide related subgroups of characteristics. The author used the Fuzzy C-Means algorithm to represent the images as histogram properties. Islam and Ahmed [7] proposed a comparison of k-means, k-medoids, and hierarchical clustering algorithms. They reported that the k-means clustering method outperformed other methods and was simple to use. Bhoi and Gupta [8] for investigating two clustering segmentation techniques, k-means and fuzzy c-means, in terms of execution duration and computation of the tumour area. Because it takes less time to execute and extracts the tumour region more precisely, the k-means segmentation method outperformed fuzzy c-means. One of the most effective image classification strategies was , which was used to create a fused segmentation. The ensemble methodology, which is based on the potential field notion of physics, combines data from PFS and other methodologies. The authors reported an accuracy of 89.8% using the BRATS dataset for experimental validation of their approach.

Deep learning-based segmentation techniques have been adopted by several researchers in recent years. Ahuja et al. [9] applied transfer learning for brain tumor identification and super-pixel approach for segmentation. They created a model based on the BRATS 2019 brain tumor segmentation competition. For training purposes, they used VGG19 transfer learning and with the help of super-pixel method tumor was separated into LGG and HGG images. Jia and Chen [10] used support vector machine to construct a heterogeneous segmentation that is completely automated. To train and test the accuracy of tumor diagnosis in MRI images, they used a classification approach known as probabilistic neural network classification system. This model focused on automatic meningioma segmentation and employed a multispectral brain dataset. Mittal et al. introduced the notion of the stationary wavelet transform (SWT) and the novel growing convolution neural network (GCNN). The emphasis was on enhancing the convention system’s correctness. In addition, a comparison study was conducted using SVM and CNN, which revealed that the suggested model outperformed both in terms of accuracy. Convolutional neural networks are compared to a combination of texture-based feature extraction methodologies and conventional techniques by Affoso et al., using different learning models for the classification of biological images. This study shows that deep learning-based algorithms like CNN are the best at image classification.

5 Datasets

Several data sources are currently available, that provide images from brain tumor of varying tumor ranges. The most commonly used modalities are magnetic resonance images (MRIs). Some of the most common resources are BraTS, REMBRANDT, images from radiopaedia, etc. RAMBRANDT and images from radiopaedia repository contain images that are divided into 4 grades of tumors. The RAMBRANDT dataset contains 110,020 MR Images with Region of Interest segmentation and medical reports from 130 different patients. The 4 categories in which the dataset is divided are I, II, III, and IV depending on the grade of the tumor and they contain MR images from 37, 40, 25, 28 patients, respectively. The radiopedia repository dataset contains images from 121 MRIs which are labeled with four labels divided into I, II, III, IV categories and they contain 36, 32, 25, and 28 MRIs, respectively, for the given grades.

BraTS 2019 [11] makes use of data from multiple institutes, the data is preoperative MR images, and the main focus is on instrinsically heterogeneous brain tumors, specifically gliomas. Furthermore, BraTS’19 focuses on predicting patient overall survival using integrative analyses of radiomic features and machine learning algorithms in order to pinpoint the clinical relevance of this segmentation task. Finally, at BraTS’19, an experimental evaluation of tumour segmentation uncertainty will be carried out. The dataset is divided into four subfiles, with dataset volumes classified as T1, T2, Flair, or T1Ce sequences. There are 155 slices in each volume. MRI images are used to create the slices. One volume necessitates 155 slices. Glioma dataset is with 210 high-grade volumes. There are 285 volumes in total, with 75 volumes in another type of glioma. Because it is a standard dataset for detecting brain tumours, BraTS2019 can be used. The Kaggle dataset [12] is containing MRI images with 155 tumor and 98 no tumor images (253 total images) (Figs. 1 and 2).

Fig. 1
figure 1

Healthy brain MRI images

Full size image
Fig. 2
figure 2

MRI images containing tumor

Full size image
Fig. 3
figure 3

Sample images from BraTS dataset

Full size image
Fig. 4
figure 4

Images of 4 grades of brain tumor

Full size image

6 Open Challenges and Research Directions

Based on the detailed review of various approaches for brain tumor detection, several interesting insights were revealed. It was observed that deep learning techniques have been extensively used for accurate classification of brain tumor into multiple classes, for enabling automated severity grading tasks. This is critical in clinical workflows for supporting decision-making tasks that can help improve the diagnostic process and reduce time taken for diagnosis. Most studies have focused on binary classification of the brain tumor that reduces the task to a prediction with respect to if brain tumor is present or not. However, the main concern is the type of brain tumor as there is a very high chance of grade 1 and grade 2 tumors not being detected in time. Also, for many of the datasets the number of samples in different classes is very unbalanced. Although the classes can be balanced using augmentation techniques, the sample images obtained after augmentation are just copies of the other images by introducing some small variations. However, if there can be datasets with almost balanced classes, then it would make the results better and more practical to use in real-world applications (Figs. 3 and 4).

Segmentation is a technique in which an image is divided into number of partitions called as segments. The image segments simplify image processing and analysis by reducing the complexity of the image. Most existing studies have focused on standard parametric methods like threshold, watershed, region-based, and clustering-based segmentation methods. Threshold segmentation is an easy-to-use method which takes pixels’ mean value as threshold value, the disadvantage being that it is not suitable when the object area ratio changes. Watershed is based on topological interpretation which results in closed and connected regions and the problem is it gives excessive segmentation while edge-based employs discontinuity detection and performs better in photos with strong object-background contrast. In these standard techniques, k-means clustering is easy to implement, takes less processing time, and also provides accurate predictions. While applying these segmentation techniques on the MRI images, skull stripping has not been used. This is a significant gap as image quality has a significant impact on the segmentation method’s performance. The result of image segmentation is determined by a lot of aspects, including pixel color, intensity, image similarity, and issue domain. Hence, automated methods that can address these issues are the need of the day.

7 Concluding Remarks

There are different methods for identifying brain tumor by looking at magnetic resonance images. Deep learning methods and segmentation along with radiomic analysis play a huge role in the detection of brain tumor. Deep learning methods majorly focus on learning of a model by learning from different images given as a train data. Segmentation focuses on division of regions in an image so that it can identify the tumor by those regions. Radiomic analysis is the extraction of features like shape and size of tumor from a given image so that these features can be used for the further training and testing of data. These methods can be collectively used to obtain good results. Many of the publications combined deep learning with radiomic analysis, allowing characteristics derived from images to be input into the model for training and testing. These give better results as compared to the studies that have individually used one of these methods.