Abstract
The potential of improving disease detection and treatment planning comes with accurate and fully automatic algorithms for brain tumor segmentation. Glioma, a type of brain tumor, can appear at different locations with different shapes and sizes. Manual segmentation of brain tumor regions is not only time-consuming but also prone to human error, and its performance depends on pathologists’ experience. In this paper, we tackle this problem by applying a fully convolutional neural network SegNet to 3D data sets for four MRI modalities (Flair, T1, T1ce, and T2) for automated segmentation of brain tumor and subtumor parts, including necrosis, edema, and enhancing tumor. To further improve tumor segmentation, the four separately trained SegNet models are integrated by post-processing to produce four maximum feature maps by fusing the machine-learned feature maps from the fully convolutional layers of each trained model. The maximum feature maps and the pixel intensity values of the original MRI modalities are combined to encode interesting information into a feature representation. Taking the combined feature as input, a decision tree (DT) is used to classify the MRI voxels into different tumor parts and healthy brain tissue. Evaluating the proposed algorithm on the dataset provided by the Brain Tumor Segmentation 2017 (BraTS 2017) challenge, we achieved F-measure scores of 0.85, 0.81, and 0.79 for whole tumor, tumor core, and enhancing tumor, respectively.
Experimental results demonstrate that using SegNet models with 3D MRI datasets and integrating the four maximum feature maps with pixel intensity values of the original MRI modalities has potential to perform well on brain tumor segmentation.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Louis, D. N.; Perry, A.; Reifenberger, G.; von Deimling, A.; Figarella-Branger, D.; Cavenee, W. K.; Ohgaki, H.; Wiestler, O. D.; Kleihues, P.; Ellison, D. W. The 2016 world health organization classification of tumors of the central nervous system: A summary. Acta Neuropathologica Vol. 131, No. 6, 803–820, 2016.
Menze, B.; Reyes, M.; van Leemput, K. The multimodal brain tumorimage segmentation benchmark (BRATS). IEEE Transactions on Medical Imaging Vol. 34, No. 10, 1993–2024, 2015.
Juan-Albarracín, J.; Fuster-Garcia, E.; Manjón, J. V.; Robles, M.; Aparici, F.; Martí-Bonmatí, L.; García-Gómez, J. M. Automated glioblastoma segmentation based on a multiparametric structured unsupervised classification. PLoS ONE Vol. 10, No. 5, e0125143, 2015.
Bauer, S.; Wiest, R.; Nolte, L. P.; Reyes, M. A survey of MRI-based medical image analysis for brain tumor studies. Physics in Medicine and Biology Vol. 58, No. 13, R97–R129, 2013.
Havaei, M.; Davy, A.; Warde-Farley, D.; Biard, A.; Courville, A.; Bengio, Y.; Pal, C.; Jodoin, P. M.; Larochelle, H. Brain tumor segmentation with deep neural networks. Medical Image Analysis Vol. 35, 18–31, 2017.
Bakas, S.; Zeng, K.; Sotiras, A.; Rathore, S.; Akbari, H.; Gaonkar, B.; Rozycki, M.; Pati, S.; Davatzikos, C. GLISTRboost: Combining multimodal MRI segmentation, registration, and biophysical tumor growth modeling with gradient boosting machines for glioma segmentation. In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. Lecture Notes in Computer Science, Vol. 9556. Crimi, A.; Menze, B.; Maier, O.; Reyes, M.; Handels, H. Eds. Springer Cham, 144–155, 2016.
Pereira, S.; Pinto, A.; Alves, V.; Silva, C. A. Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Transactions on Medical Imaging Vol. 35, No. 5, 1240–1251, 2016.
Shelhamer, E.; Long, J.; Darrell, T. Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 39, No. 4, 640–651, 2017.
Ronneberger, O.; Fischer, P.; Brox, T. UNet: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015. Lecture Notes in Computer Science, Vol. 9351. Navab, N.; Hornegger, J.; Wells, W.; Frangi, A. Eds. Springer Cham, 234–241, 2015.
Shah, S. A.; Chauhan, N. Techniques for detection and analysis of tumours from brain MRI images: A review. Journal of Biomedical Engineering and Medical Imaging Vol. 3, No. 1, 9–20, 2016.
Gooya, A.; Pohl, K. M.; Bilello, M.; Biros, G.; Davatzikos, C. Joint segmentation and deformable registration of brain scans guided by a tumor growth model. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2011. Lecture Notes in Computer Science, Vol. 6892. Fichtinger, G.; Martel, A.; Peters, T. Eds. Springer Berlin Heidelberg, 532–540, 2011.
Zikic, D.; Glocker, B.; Konukoglu, E.; Criminisi, A.; Demiralp, C.; Shotton, J.; Thomas, O. M.; Das, T.; Jena, R.; Price, S. J. Decision forests for tissue-specific segmentation of high-grade gliomas in multi-channel MR. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2012. Lecture Notes in Computer Science, Vol. 7512. Ayache, N.; Delingette, H.; Golland, P.; Mori, K. Eds. Springer Berlin Heidelberg, 369–376, 2012.
Tustison, N. J.; Shrinidhi, K. L.; Wintermark, M.; Durst, C. R.; Kandel, B. M.; Gee, J. C.; Grossman, M. C.; Avants, B. B. Optimal symmetric multimodal templates and concatenated random forests for supervised brain tumor segmentation (simplified) with ANTsR. Neuroinformatics Vol. 13, No. 2, 209–225, 2015.
Kamnitsas, K.; Ledig, C.; Newcombe, V. F. J.; Simpson, J. P.; Kane, A. D.; Menon, D. K.; Rueckert, D.; Glocker, B. Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Medical Image Analysis Vol. 36, 61–78, 2017.
Kamnitsas, K.; Ferrante, E.; Parisot, S.; Ledig, C.; Nori, A. V.; Criminisi, A.; Rueckert, D.; Glocker, B. DeepMedic for brain tumor segmentation. In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. Lecture Notes in Computer Science, Vol. 10154. Crimi, A.; Menze, B.; Maier, O.; Reyes, M.; Winzeck, S.; Handels, H. Eds. Springer Cham,138–149, 2016.
Chang, P. D. Fully convolutional deep residual neural networks for brain tumor segmentation. In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. Lecture Notes in Computer Science, Vol. 10154. Crimi, A.; Menze, B.; Maier, O.; Reyes, M.; Winzeck, S.; Handels, H. Eds. Springer Cham, 108–118, 2016.
Drozdzal, M.; Vorontsov, E.; Chartrand, G.; Kadoury, S.; Pal, C. The importance of skip connections in biomedical image segmentation. In: Deep Learning and Data Labeling for Medical Applications. Lecture Notes in Computer Science, Vol. 10008. Carneiro, G. et al. Eds. Springer Cham, 179–187, 2016.
Çiçek, Ö.; Abdulkadir, A.; Lienkamp, S. S.; Brox, T.; Ronneberger, O. 3D U-Net: Learning dense volumetric segmentation from sparse annotation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2016. Lecture Notes in Computer Science, Vol. 9901. Ourselin, S.; Joskowicz, L.; Sabuncu, M.; Unal, G.; Wells, W. Eds. Springer Cham, 424–432, 2016.
Lai, M. Deep learning for medical image segmentation. arXiv preprint arXiv:1505.02000, 2015.
Pereira, S.; Pinto, A.; Alves, V.; Silva, C. A. Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Transactions on Medical Imaging Vol. 35, No. 5, 1240–1251, 2016.
Tustison, N. J.; Avants, B. B.; Cook, P. A.; Zheng, Y. J.; Egan, A.; Yushkevich, P. A.; Gee, J. C. N4ITK: Improved N3 bias correction. IEEE Transactions on Medical Imaging Vol. 29, No. 6, 1310–1320, 2010.
Badrinarayanan, V.; Kendall, A.; Cipolla, R. SegNet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 39, No. 12, 2481–2495, 2017.
Noh, H.; Hong, S.; Han, B. Learning deconvolution network for semantic segmentation. In: Proceedings of the IEEE International Conference on Computer Vision, 1520–1528, 2015.
Bakas, S.; Akbari, H.; Sotiras, A.; Bilello, M.; Rozycki, M.; Kirby, J. S.; Freymann, J. B.; Farahani, K.; Davatzikos, C. Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features. Scientific Data Vol. 4, 170117, 2017.
Kamnitsas, K.; Bai, W.; Ferrante, E.; McDonagh, S.; Sinclair, M.; Pawlowski, N.; Rajchl, M.; Lee, M.; Kainz, B.; Rueckert, D.; Glocker, B. Ensembles of multiple models and architectures for robust brain tumour segmentation. In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. Lecture Notes in Computer Science, Vol 10670. Crimi, A.; Bakas, S.; Kuijf, H.; Menze, B.; Reyes, M. Eds. Springer Cham, 450–462, 2018.
Casamitjana, A.; Cat`a, M.; S´anchez, I.; Combalia, M.; Vilaplana, V. Cascaded V-Net using ROI masks for brain tumor segmentation. In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. Lecture Notes in Computer Science, Vol. 10670. Crimi, A.; Bakas, S.; Kuijf, H.; Menze, B.; Reyes, M. Eds. Springer Cham, 381–391, 2018.
Acknowledgements
We would like to thank nVidia for their kind donation of a Titan XP GPU.
Author information
Authors and Affiliations
Corresponding author
Additional information
Salma Alqazzaz is an assistant lecturer in medical physics in the Department of Physics, College of Science for Women, Baghdad University, Iraq. She graduated as a bachelor of medical engineering in 2003 from Nahrain University. In 2008, she was awarded an M.Sc. degree in medical engineering at Nahrain University. Currently, she is a Ph.D. candidate in medical engineering, in the School of Engineering, Cardiff University, UK. Her research interests are in image processing and machine learning applied to images.
Xianfang Sun received his Ph.D. degree in control theory and its applications from the Institute of Automation, Chinese Academy of Sciences. He is a senior lecturer at Cardiff University. His research interests include computer vision and graphics, pattern recognition and artificial intelligence, and system identification and control.
Xin Yang since 2013 has been a lecturer in medical engineering and director of the Medical Ultrasound and Sensors Laboratory in the School of Engineering, Cardiff University, and adjunct professor at Lanzhou Jiaotong University, China. He studied biomedical engineering in Beijing Jiaotong University from 2001 to 2005. He was awarded an M.Sc. degree in medical electronics & physics at Queen Mary College, London, in 2006. He worked as CEO and CTO for two years in Beijing BJ Device Ltd. He was awarded a Ph.D. degree in 2011 for work on Doppler ultrasound in quantifying neovascularisation. He was a British Heart Foundation research fellow working on Doppler ultrasound phantoms and wall shear stress measurement at Queen’s Medical Research Institute, University of Edinburgh. He is principal author of 8 books on electronics and microcontrollers.
Len Nokes is a professor of clinical biomechanics, Cardiff University, UK. He holds doctorates in both engineering and medicine and has co-authored four text books on biomechanics. He also has a D.Sc. degree for his work on trauma science. He has published over 100 scientific papers, and is a Fellow of the Institution of Mechanical Engineers and a Chartered Engineer. He is also a Fellow of the Faculty of Sports and Exercise Medicine (UK). His main research areas involve trauma and sports biomechanics.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.
The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.
About this article
Cite this article
Alqazzaz, S., Sun, X., Yang, X. et al. Automated brain tumor segmentation on multi-modal MR image using SegNet. Comp. Visual Media 5, 209–219 (2019). https://doi.org/10.1007/s41095-019-0139-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41095-019-0139-y