Abstract
Because of the strong variability of the cortical sulci, their automatic recognition is still a challenging problem. The last algorithm developed in our laboratory for 125 sulci reaches an average recognition rate around 86%. It has been applied to thousands of brains for morphometric studies (www.brainvisa.info). A weak point of this approach is the modeling of the training dataset as a single template of sulcus-wise probability maps, losing information about the alternative patterns of each sulcus. To overcome this limit, we propose a different strategy inspired by Multi-Atlas Segmentation (MAS) and more particularly the patch-based approaches. As the standard way of extracting patches does not seem capable of exploiting the sulci geometry and the relations between them, which we believe to be the discriminative features for recognition, we propose a new patch generation strategy based on a high level representation of the sulci. We show that our new approach is slightly, but significantly, better than the reference one, while we still have an avenue of potential refinements that were beyond reach for a single template strategy.
You have full access to this open access chapter, Download conference paper PDF
Similar content being viewed by others
Keywords
1 Introduction
1.1 Overview of Cortical Sulci Recognition Approaches
The surface of the human brain cortex is divided into gyri, separated by fissures called sulci. The largest sulci are good indicators of the localization of functional areas and the morphometry of the sulci geometry is used to quantify brain development and neurodegenerative processes. Automatic recognition is mandatory to exploit the large databases yielded by recent neuroimaging projects. However, automatic sulci recognition remains a challenging problem because of the lack of understanding of the nature of the interindividual variability of the folding patterns. During the last twenty years, many different approaches have been proposed to handle this problem.
The approaches based on atlas registration yield reasonable identification rate for major sulci. For instance, in [12], a labeled brain is elastically deformed to fit a new brain, and in [6], a multiresolution strategy is designed to improve the results by progressively registering the largest folds to the smallest folds. This single template strategy however has strong limitations with regard to the numerous folding configurations incompatible with the atlas geometry, which occurs even for the largest sulci.
Modeling intersubject variability seems mandatory to increase robustness, which was tackled using a variety of frameworks ranging from PCA to Bayesian approaches [1, 5, 7, 9]. A large amount of strategies rely on graph-based representations, which provide a flexible way to model the spatial relationships between the sulci in addition to their shape and localization in a normalized space [3, 10, 11, 13,14,15]. Comparing these approaches is difficult in the absence of benchmark. Furthermore, all of them except the one in BrainVISA are restricted to a small set of sulci, a small training database and are not distributed. Therefore, in this paper, we will only compare our new model to the results of BrainVISA current package.
1.2 The Current BrainVISA Model: Principles, Advantages and Inconvenients
The current BrainVISA model relies on a coherent Bayesian framework based on a probabilistic atlas (a model made of a mixture of Statistical Probabilistic Anatomy Maps (SPAM)). This approach performs simultaneously sulcus recognition and local alignment with the template of SPAMs [9]. Unfortunately, this approach performs poorly with unusual folding patterns, which depart from the main modes of the probabilistic atlas. This is a classical limitation of single template strategies. For instance, the reconfiguration of the folding patterns induced by interruptions of the large sulci can lead to inconsistent sulcus recognition. Furthermore, we have observed disturbing mistakes that did not occur with the older graph-based approach also proposed in BrainVISA [10]. For instance a large sulcus can be locally duplicated because the current Bayesian framework does not model the relationships between folds, while the previous graph-based approach could learn alternative configurations thanks to a neural network (NN) based memory. Unfortunately, this NN strategy could probably not be trained with a large enough manually labelled database to reach the robustness of the Bayesian framework.
1.3 MAS Strategy: A Solution to the Limits of the Current Model?
In biomedical image analysis, segmentation is the process of tagging image voxels with biologically meaningful labels. The MAS strategy has become one of the most widely-used and successful solution. The idea of this technique is to use for segmentation the entire dataset of “atlases” (i.e. training images) rather than one model-based average kind of representation, so that it better captures the anatomical variability. A power of this approach is its pragmatism: it can be applied for a problem where a real understanding of the nature of the interindividual variability is lacking, which fits the current status of the neuroscience world with regard to the cortical folding.
Most of the MAS techniques include four main steps: (1) Atlas generation: the design of atlases from the training images; (2) Registration: alignment of each atlas onto the new image to be segmented; (3) Label propagation: from each aligned atlas to the new subject, (4) Label fusion: combining the propagated labels to achieve the final segmentation. One of the most recent MAS techniques developed by Coupé et al. [4] performs a patch-based label fusion, which leads to very efficient implementations.
The problem of sulci recognition differs from standard MAS applications by several aspects. First, there are many more anatomical structures to be labeled: each brain contains up to 125 cortical sulci in the nomenclature of BrainVISA. Second and more importantly, grey level intensities and textures surrounding the sulci do not provide discriminative information except for a few sulci like central sulcus, because of a specific local myelin content. Therefore, the key discriminative information is folding geometry, which requires larger voxel-based patches than in usual implementations. Hence, for the sake of efficiency, in the following we will operate at a larger intermediate scale of representation of the folding patterns, where the entities to be labelled are predefined sets of voxels corresponding to the most elementary folds (about 500 such entities in a standard brain) [8].
The new model proposed in this article is inspired by the patch-based methodology proposed by Coupé et al. [4] but the patches are not defined as local groups of voxels but as local groups of sulci. Note that this strategy could be extended to all the domains where an intermediate level of representation of the data could be useful for the alignment of the atlases to the unknown subject. Hence, we have to revisit the different stages of the mainstream strategy to take into account the heterogeneity of the representations of the subjects. In the following, we design a dedicated patch generation strategy, a geometry-based measure of similarity to perform the patch-based alignment, and a propagation strategy for 3D points clouds.
2 Database
In order to compare our results to those of [9] in the same conditions, we use the exact same database of 62 healthy subjects selected from different heterogeneous databases. Most of the subjects are right-handed male persons, between 25 and 35 years old. The elementary folds of each brain were manually labelled according to the sulcus nomenclature following a long process leading to achieve a consensus across a set of several experts of the morphology of the cortex. Each fold representation is a set of voxels corresponding to the medial surface of the cerebrospinal fluid filling up the fold. Hence, these fold representations are elementary pieces of a negative mould of the brain. As mentioned above, the sulcus recognition method described in this paper will be forced to give one unique label to the set of voxels representing a fold. The training base is composed of 62 brains labelled with a model containing 62 sulci for the right hemisphere and 63 for the left.
3 Method: A New Model Inspired by MAS Algorithms
As the new model is inspired by the MAS strategy, the same steps are used and will be described in this section (Fig. 1). In the following, the atlas brains and the new brain are all represented as a set of elementary folds. All the atlas folds have been manually labelled with the sulcus nomenclature. All the brains have been affinely aligned with the Talairach standard space.
3.1 Patch Generation
In this paper, we experiment with a reasonable but relatively naive strategy, which will be refined in the future using machine learning. The goal is to define from the atlas dataset a library of local patches embedding enough geometrical information to minimize ambiguities when searching for a high similarity hit in the unknown subject morphology. Note that the shape of small sulci is not informative enough to prevent spurious hits. Hence the idea is to aggregate a few sulci to create discriminative local shapes. In the following, we define a type of patches for each pair of sulci that are neighbors in the brain (Fig. 1). Practically, a pair of sulci is selected if the instances of the two sulci are neighbors at least in one brain of the atlas dataset, for the topology provided by the BrainVISA pipeline yielding the folds. This pipeline endows the list of folds with a graph structure corresponding to either direct connections or to the fact that two folds are separated by a piece of gyrus. Finally, for a given sulcus S, with neighbors \(S_{1}, S_{2}, ..., S_{N}\), N patches are generated from each atlas of the dataset, each patch corresponding to a different type of patch \(SS_{1}, SS_{2},..., SS_{N}\). Note that if the neighbor or S is missing in the atlas, the patch is not included in the library.
3.2 Registration
For the registration step, the set of folds of the unknown brain and the patches of the library are represented by point clouds. The well-known iterative closest points algorithm [2] is applied to find an optimal alignment of each patch into the point cloud made up of the unlabelled folds of the unknown brain. To build the measure used to rank the matches, the nearest points in the new brain of each patch point are saved as activated points. The measure corresponds to the sum of quadratic distances of the patch points and their corresponding activated points, divided by the number of different activated points.
3.3 Label Propagation
For each patch type, the ten instances leading to the smallest distances are selected to participate to the propagation of the two parent sulci. The number of instances selected has been set to ten arbitrary for a first implementation and should be optimized later on. Note that some sulcus instances are selected several times, because they win the competition for several patch types. But their multiple contributions will be associated with slightly different alignments. Hence, sulcus instances maximizing regional similarity to the unknown subject get more weight. For each selected patch after the ICP registration to the unknown subject, each point gives its label to its nearest neighbor in the target brain. To consider the patch structure, each connected set of points in the patch should correspond to a unique connected set in the target brain: the smallest non-connected sets are excluded. Hence, each point of the target brain can be labeled several times from several patches and only the most propagated label is saved.
3.4 Label Fusion
The label fusion process is performed on a fold by fold basis. For each fold of the unknown subject, the sulcus label the most represented in its points is given to the fold. Without activated points in the fold, the “unknown” label is kept.
4 Results and Comparison with the BrainVISA Model
The proposed approach and the Bayesian BrainVISA approach are compared using a Leave-One-Out strategy on the database described above. Many different error measures are possible to evaluate the model. The measures used to evaluate the current BrainVISA model are the \(E_{local}\) for each sulcus and \(E_{SI}\) for the final labeling. Here is their definitions for a given subject:
with L the ensemble of sulcus labels, FP(l), FN(l) and TP(l) respectively the size of the set of voxels false positive, false negative and true positive for the label l and \(w_{l} = \frac{s_{l}}{\sum \limits _{l \in L} s_{l}}\) with \(s_{l} = FN(l)+TP(l)\) the true size of the sulcus with the label l.
The advantages of \(E_{SI}\) is that it is sensible to local labeling and it takes into account the shared errors between labels [9]. With this measure, we deduct the main recognition rate for the two compared models: the current BrainVISA model obtains 85.53% (+/– 5.80%) for the left hemisphere and 86.27% (+/– 6.12%) for the right hemisphere while the new model obtains 86.76% (+/– 5.16%) and 88.04% (+/– 5.70%).
In order to compare the two models, we calculate the error rates for each subject in the database and compare the set of measures with a T-test. The \(E_{SI}\) comparison shows that the new model is significantly better than the Brainvisa model (\(p_{value}=4.14e-8\)). Moreover, by comparing the \(E_{local}\) per sulcus, more than 20 sulci are found significantly better (Fig. 2).
5 Conclusion
This paper describes a first attempt at casting the cortical sulci recognition problem in a MAS-based framework. While some choices are still ad hoc and will require further developments, the comparison with the existing Bayesian approach is full of promise. The main contribution of our work is the extension of the MAS framework to a high level representation of the data dedicated to our pattern recognition problem. This original setting is calling for a more sophisticated inference of the library of patches, which should probably be developed with an optimization strategy trying to maximize the recognition result while keeping the library as small as possible. This strategy shall pick the selected patch types from a larger combinatorial set aggregating more than two sulci. Another improvement opportunity lies in the patch selection strategy, for example by learning the optimal cut-off for each type of patches. The label fusion could be largely improved by the introduction of weights, in order to balance the contribution of each patch according to its similarity measure. Finally, as the new strategy involves a labeling per point, it will allow the method to question the initial split of the cortex morphology into elementary folds to detect under-segmentation issues. This is an essential element to overcome the weakness of the current Bayesian framework, which is stuck to the high level representation yielded by the preprocessing stage. This possibility can be considered as a top-down feature embedded in the global recognition system.
References
Behnke, K.J., et al.: Automatic classification of sulcal regions of the human brain cortex using pattern recognition. In: Medical Imaging 2003: Image Processing, vol. 5032, pp. 1499–1511. International Society for Optics and Photonics (2003)
Besl, P.J., McKay, N.D.: Method for registration of 3-D shapes. In: Sensor Fusion IV: Control Paradigms and Data Structures, vol. 1611, pp. 586–607. International Society for Optics and Photonics (1992)
Blida, A.: Ontology driven graph matching approach for automatic labeling brain cortical sulci. In: IT4OD, p. 162 (2014)
Coupé, P., Manjón, J.V., Fonov, V., Pruessner, J., Robles, M., Collins, D.L.: Patch-based segmentation using expert priors: application to hippocampus and ventricle segmentation. NeuroImage 54(2), 940–954 (2011)
Fischl, B., et al.: Automatically parcellating the human cerebral cortex. Cerebral cortex 14(1), 11–22 (2004)
Jaume, S., Macq, B., Warfield, S.K.: Labeling the brain surface using a deformable multiresolution mesh. In: Dohi, T., Kikinis, R. (eds.) MICCAI 2002. LNCS, vol. 2488, pp. 451–458. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-45786-0_56
Lohmann, G., von Cramon, D.Y.: Automatic labelling of the human cortical surface using sulcal basins. Med. Image Anal. 4(3), 179–188 (2000)
Mangin, J.F., Frouin, V., Bloch, I., Régis, J., López-Krahe, J.: From 3D magnetic resonance images to structural representations of the cortex topography using topology preserving deformations. J. Math. Imaging Vis. 5(4), 297–318 (1995)
Perrot, M., Rivière, D., Mangin, J.F.: Cortical sulci recognition and spatial normalization. Med. Image Anal. 15(4), 529–550 (2011)
Rivière, D., Mangin, J.F., Papadopoulos-Orfanos, D., Martinez, J.M., Frouin, V., Régis, J.: Automatic recognition of cortical sulci of the human brain using a congregation of neural networks. Med. Image Anal. 6(2), 77–92 (2002)
Royackkers, N., Desvignes, M., Revenu, M.: Une méthode générale de reconnaissance de courbres 3D: application à l’identification de sillons corticaux en imagerie par résonance magnétique. Traitement du Signal 15(5), 365–379 (1998)
Sandor, S., Leahy, R.: Surface-based labeling of cortical anatomy using a deformable atlas. IEEE Trans. Med. Imaging 16(1), 41–54 (1997)
Shi, Y., et al.: Joint sulci detection using graphical models and boosted priors. In: Karssemeijer, N., Lelieveldt, B. (eds.) IPMI 2007. LNCS, vol. 4584, pp. 98–109. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-73273-0_9
Vivodtzev, F., Linsen, L., Hamann, B., Joy, K.I., Olshausen, B.A.: Brain mapping using topology graphs obtained by surface segmentation. In: Scientific Visualization: The Visual Extraction of Knowledge from Data, pp. 35–48. Springer, Heidelberg (2006). https://doi.org/10.1007/3-540-30790-7
Yang, F., Kruggel, F.: A graph matching approach for labeling brain sulci using location, orientation, and shape. Neurocomputing 73(1–3), 179–190 (2009)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Borne, L., Mangin, JF., Rivière, D. (2018). A Patch-Based Segmentation Approach with High Level Representation of the Data for Cortical Sulci Recognition. In: Bai, W., Sanroma, G., Wu, G., Munsell, B., Zhan, Y., Coupé, P. (eds) Patch-Based Techniques in Medical Imaging. Patch-MI 2018. Lecture Notes in Computer Science(), vol 11075. Springer, Cham. https://doi.org/10.1007/978-3-030-00500-9_13
Download citation
DOI: https://doi.org/10.1007/978-3-030-00500-9_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-00499-6
Online ISBN: 978-3-030-00500-9
eBook Packages: Computer ScienceComputer Science (R0)