Abstract
In this paper, we present an evaluation of seven automatic brain tissue classifiers based on level of agreements. A number of agreement measures are explained, and we show how they can be used to compare different segmentation techniques. We use the Simultaneous Truth and Performance Level Estimation (STAPLE) of Warfield et al. but also introduce a novel evaluation technique based on the Williams’ index. The methods are evaluated using these two techniques on a population of forty subjects, each having an SPGR scan and a co-registered T2 weighted scan. We provide an interpretation of the results and show how similar the output of the STAPLE analysis and Williams’ index are. When no ground truth is required, we recommend the use of Williams’ index as it is easy and fast to compute.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Zou, K.H., Wells, W.M., Kikinis, R., Warfield, S.K.: Three validation metrics for automated probabilistic image segmentation in brain tumors. Statistics in Medicine 23, 1280–1291 (2004)
Bello, F., Colchester, A.C.F.: Measuring global and local spatial correspondence using information theory. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 964–973 (1998)
Warfield, S.K., Zou, K.H., Wells, W.M.: Simultaneous truth and performance level estimation (staple): An algorithm for the validation of image segmentation. IEEE Trans. Medical Imaging 23, 903–921 (2004)
Williams, G.W.: Comparing the joint agreement of several raters with another rater. Biometrics 32, 619–627 (1976)
Jaccard, P.: Étude comparative de la distribuition florale dans une portion des alpes et de jura. Bulletin de la Societé Voudoise des Sciences Naturelles 37, 547–579 (1901)
Rogers, J.S., Tanimoto, T.T.: A computer program for classying plants. Science 132, 1115–1118 (1960)
Wells, W.M., Viola, P., Atsumi, H., Nakajima, S., Kikinis, R.: Multi-modal volume registration by maximization of mutual information. Medical Image Analysis 1, 35–52 (1996)
Warfield, S.: Fast knn classification for multichannel image data. Pattern Recognition Letters 17, 713–721 (1996)
Zijdenbos, A.P., Forghani, R., Evans, A.C.: Automatic ”pipeline” analysis of 3-d mri data for clinical trials: application to multiple sclerosis. IEEE Transactions on Medical Imaging 21, 1280–1291 (2002)
Sled, J.G., Zijdenbos, A.P., Evans, A.C.: A non-parametric method for automatic correction of intensity non-uniformity in mri data. IEEE Transactions On Medical Imaging 17, 87–97 (1998)
Zhang, Y., Brady, M., Smith, S.: Segmentation of brain MR images through a hidden markov random field model and the expectation maximization algorithm. IEEE Transactions on Medical Imaging 20, 45–57 (2001)
Ashburner, J., Friston, K.: Spatial normalization using basis functions. In: Frackowiak, R.S.J., Friston, K.J., Frith, C., Dolan, R., Friston, K.J., Price, C.J., Zeki, S., Ashburner, J., Penny, W. (eds.) Human Brain Function, 2nd edn. Academic Press, London (2003)
Wells, W.M., Kikinis, R., Grimson, W.E.L., Jolesz, F.: Adaptive segmentation of mri data. IEEE Transactions on Medical Imaging 15, 429–442 (1996)
Pohl, K.M., Bouix, S., Kikinis, R., Grimson, W.E.L.: Anatomical guided segmentation with non-stationary tissue class distributions in an expectation-maximization framework. In: IEEE International Symposium on Biomedical Imaging, Arlington, VA, pp. 81–84 (2004)
Grau, V., Mewes, A.U.J., Alcañiz, M., Kikinis, R., Warfield, S.K.: Improved watershed transform for medical image segmentation using prior information. IEEE Transactions on Medical Imaging 23, 447–458 (2004)
Krissian, K.: Flux-based anisotropic diffusion applied to enhancement of 3-d angiogram. IEEE Transactions on Medical Imaging 21, 1440–1442 (2002)
Smith, S.: Fast robust automated brain extraction. Human Brain Mapping 17, 143–155 (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Martin-Fernandez, M., Bouix, S., Ungar, L., McCarley, R.W., Shenton, M.E. (2005). Two Methods for Validating Brain Tissue Classifiers. In: Duncan, J.S., Gerig, G. (eds) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2005. MICCAI 2005. Lecture Notes in Computer Science, vol 3749. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11566465_64
Download citation
DOI: https://doi.org/10.1007/11566465_64
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29327-9
Online ISBN: 978-3-540-32094-4
eBook Packages: Computer ScienceComputer Science (R0)