Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

Imprecision is often inherent to images, and its causes can be found at several levels: observed phenomenon (imprecise limits between structures or objects), acquisition process (limited resolution, numerical reconstruction methods), image processing steps (imprecision induced by a filtering for instance). Fuzzy sets have several advantages for representing such imprecision. First, they are able to represent several types of imprecision in images, as for instance imprecision in spatial location of objects, or imprecision in membership of an object to a class. For instance, partial volume effect, which occurs frequently in medical imaging, finds a consistent representation in fuzzy sets (membership degrees of a voxel to tissues or classes directly represent partial membership to the different tissues mixed up in this voxel, leading to a consistent modeling with respect to reality). Second, image information can be represented at different levels with fuzzy sets (local, regional, or global), as well as under different forms (numerical, or symbolic). For instance, classification based only on grey levels involves very local information (at the pixel level); introducing spatial coherence in the classification or relations between features involves regional information; and introducing relations between objects or regions for scene interpretation involves more global information and is related to the field of spatial reasoning. Third, the fuzzy set framework allows for the representation of very heterogeneous information, and is able to deal with information extracted directly from the images, as well as with information derived from some external knowledge, such as expert knowledge. This is exploited in particular in model-based pattern recognition, where fuzzy information extracted from the images is compared and matched to a model representing knowledge expressed in fuzzy terms.

Fuzzy set theory is of great interest to provide a consistent mathematical framework for all these aspects. It allows representing imprecision of objects, relations, knowledge and aims, at different levels of representation. It constitutes an unified framework for representing and processing both numerical and symbolic information, as well as structural information (constituted mainly by spatial relations in image processing). Therefore this theory can achieve tasks at several levels, from low level (e.g. grey-level based classification) to high level (e.g. model based structural recognition and scene interpretation). It provides a flexible framework for information fusion as well as powerful tools for reasoning and decision making.

In this chapter, we provide an overview of the potential of this theory in medical imaging, in particular for classification, segmentation and recognition of anatomical and pathological structures. The chapter is organized according to the level of information and processing. We assume that the basics of fuzzy sets theory are known (details can be found e.g. in [30]).

2 Low-level processing

The use of fuzzy sets in medical imaging at low level concerns mainly classification, often based on grey levels.

2.1 Representation

We denote by \(\mathcal{S}\) the spatial domain (\(\mathbb{R}^{n}\) in the continuous case or \(\mathbb{Z}^{n}\) in the discrete case). Fuzzy sets can be considered from two points of view. In the first one, a membership function is a function μ from the space \(\mathcal{S}\) on which the image is defined into [0, 1]. The value μ(x) is the membership degree of x (\(x \in \mathcal{S}\)) to a spatial fuzzy object. In the second one, a membership function is defined as a function μ′ from a space of attributes \(\mathcal{A}\) into [0, 1]. At numerical level, such attributes are typically the grey levels. The value μ′(g) represents the degree to which a grey level g supports the membership to an object or a class. There is an obvious relation between μ and μ′ in grey level based processing: μ(x) = μ′[g(x)], where g(x) denotes the grey level of x in the considered image.

Such models explicitly represent imprecision in the information provided by the images, as well as possible ambiguity between classes. For instance the problem of partial volume effect finds a consistent representation in this model. A pixel or voxel suffering from partial volume effect is characterized by its partial belonging to two (or more) different tissues or classes, i.e. by non zero membership values to several classes. Figure 1 shows an example of an MR image of the brain of a patient suffering from adrenoleukodystrophy, and where the slice thickness induces a high partial volume effect. The grey levels on the right figure represent the membership values to the pathology. The pathology is then considered as a fuzzy object, represented by a membership function defined on the spatial domain.

Fig. 1
figure 1

MR image of the brain (left) (courtesy Prof. C. Adamsbaum, Saint-Vincent de Paul Hospital, Paris), and estimation of the partial membership to the pathology (right) in the pathological area (white means that there is only pathological tissue in the considering voxel, black means no pathological tissue, and intermediate values represent the partial volume effect, i.e. voxels that have also a non zero membership value to the white matter class)

Full size image

More generally, a spatial fuzzy object may represent different types of imprecision, either on the boundary of the objects (due for instance to partial volume effect, or to the spatial resolution), or on the individual variability of these structures, etc.

There is no definite answer to the question of how defining the membership functions. As mentioned above, they can be directly derived from the grey levels, but other characteristics can be used as well. For instance the contours of an object can be defined as a fuzzy set with a membership function depending on the gradient intensity. Based on a detection operator of some specific objects, the membership functions can be derived from the magnitude of the answer provided by this operator. Imprecision can also be introduced from a first crisp estimation of the objects, typically at their boundary as a function of the distance to the crisp object, to account for imprecision in this estimation. Finally, several approaches rely on fuzzy classification methods to derive membership functions.

2.2 Fuzzy classification

Learning of membership functions is a difficult task that still does not have a definite answer. Several methods have been proposed in the literature, often based on the minimization of some criteria. Among these methods, the most used is the fuzzy C-means algorithm (FCM) [5]. The idea is to define a membership function of a point to each class (which is then a fuzzy set), instead of deriving crisp assignments.

The FCM algorithm iteratively modifies a fuzzy partition so as to minimize an objective function defined as: \(J_{m} =\sum _{ j=1}^{C}\sum _{i}^{N}\mu _{ij}^{m}\vert \vert x_{i} - m_{j}\vert \vert ^{2}\), under the constraint that \(\forall i,\sum _{j=1}^{C}\mu _{ij} = 1\), where C denotes the number of class, N the number of points to be classified, μ i j the membership function of point i to class j, and m is a parameter belonging to ]1, +∞[ called fuzzy factor, which controls the amount of “fuzziness” of the classification. The membership function is deduced from the cluster center position as: \(\mu _{ij} = \frac{1} {\sum _{j=1}^{C}[ \frac{\vert \vert x_{i}-m_{i}\vert \vert } {\vert \vert x_{i}-m_{j}\vert \vert )} ]^{ \frac{2} {m-1} }}\), and the cluster center position is obtained by: \(m_{j} = \frac{\sum _{i}\mu _{ij}^{m}x_{ i}} {\sum _{i}\mu _{ij}^{m}}\). From an initialization of cluster centers, the membership values and cluster centers are alternatively updated using these two equations, until convergence. Convergence towards a local minimum of the objective function has been proved. An example is provided in Fig. 2, in the case of a 1-dimensional 2-class problem. It also illustrates one of the main drawbacks of this approach: the membership functions are not decreasing with respect to the distance to the cluster center. This is due to the normalization constraint, and this phenomenon gets even worse with more classes.

Fig. 2
figure 2

Membership values in a fuzzy C-means classification as a function of x in the case of 2 classes, with centroids in positions x = 1 and x = 2. In this example m = 2

Full size image

An alternative solution to fuzzy C-means classification, which avoids the normalization drawbacks, is given by possibilistic C-means (PCM) [40]. The objective functional is defined as: \(J =\sum _{ j=1}^{C}\sum _{i=1}^{N}\mu _{ij}^{m}.\vert \vert x_{i} - m_{j}\vert \vert ^{2} +\sum _{ j=1}^{C}\eta _{j}\) \(\sum _{i=1}^{N}(1 -\mu _{ij})^{m}.\vert \vert x_{i} - m_{j}\vert \vert ^{2}\) and the obtained membership function is: \(\mu _{ij} = \frac{1} {1+\frac{\vert \vert x_{i}-m_{j}\vert \vert ^{2}} {\eta _{j}} ^{ \frac{1} {m-1} }}\). Now the membership functions are decreasing with respect to the distance to the class centers. However this algorithm is very sensitive to initialization and sometimes coincident clusters may occur.

To address the problems of FCM and PCM a new fuzzy possibilistic C-mean (FPCM) algorithm was proposed in [49] by combining these two algorithms. The objective function involves both membership and typicality. FPCM solves the noise sensitivity defect of FCM and overcomes the problem of coincident clusters of PCM. Although FPCM is less prone to the problems of FCM and PCM, in the case of a large data set this algorithm does not work properly since the typicality values are very small in such cases, again due to a normalization constraint. This constraint has been suppressed in possibilistic fuzzy c-mean (PFCM) [50]. Recently, approaches have been proposed by modifying the objective function to increase the robustness of FCM to noise [1, 33, 44, 45, 59, 64]. They also try to incorporate spatial information, by defining membership functions that depend on a local neighborhood around each point.

Another class of methods relies on probability-possibility transformations [29, 31, 39]. Other methods based on statistical information have been proposed, also by minimizing some criteria (e.g. [23, 25]). However, most criteria provide a function that depends on the shape of the histogram. Accounting for frequent situations where a pixel may belong completely and without any ambiguity to a class while having a grey-level with low occurrence frequency thus becomes difficult.

In [13] an original approach was proposed to obtain membership values from grey-level histogram. Two types of criteria are used simultaneously. The first type is based on a “resemblance” between the grey-level histogram and the membership function in the form of a distance between the two distributions. This type is very close to existing methods. The second type accounts for prior information on the expected shape of the membership function, in order to deal with problems mentioned above concerning low occurrence frequencies. This calls for a parametric representation of the functions. The combination of these two types of criteria leads to a simpler interpretation of the obtained functions that fits better the intuitive notion of membership. Membership functions are chosen as simple trapezoidal functions, whose parameters are estimated (simultaneously for all class membership functions) using simulated annealing in order to optimize the two criteria. The results obtained on a MR brain image are illustrated in Fig. 3. This method has been applied successfully to several problems like multi-image classification or segmentation of internal brain structures.

Fig. 3
figure 3

MR image of the brain, showing three main classes: brain, ventricles and pathology (the white area on the left image), and result of the estimation of the three classes

Full size image

Finally, other types of classification methods, such as k-nearest neighbors, have also been extended to the fuzzy case.

Despite their drawbacks, these methods are quite widely used, mostly as an initialization for further more sophisticated processing. For instance, an adaptive C-means algorithm was used in [69] in order to take the partial volume effect into account in a deformable model approach. An original fuzzy classification method taking spatial context into account was also used as the initialization of a deformable model in [38] for segmenting brain tumors of different types, shapes and locations in 3D MRI. Some results are shown in Sect. 6.

2.3 Local operations for filtering or edge detection

In this section, we summarize the main techniques for local filtering in a broad sense, aiming at enhancing the contrast of an image, at suppressing noise, at extracting contours, etc. Note that these aims are different and often contradicting each other. However, the principles of the techniques are similar, and they can be grouped into two classes: techniques based on functional optimization on the one hand, and rule based techniques on the other hand. These aspects have been largely developed in the literature (see e.g. [2, 6, 42, 67]), and we provide here just the main lines.

Functional approaches consist in minimizing or maximizing a functional, which can be interpreted as an analytical representation of some objective. For instance, enhancing the contrast of an image according to this technique amounts to reduce the fuzziness of the image. This can be performed by a simple modification of membership functions (for instance using intensification operators), by minimizing a fuzziness index such as entropy, or even by determining an optimal threshold value (for instance optimal in the sense of minimizing a fuzziness index) which provides an extreme enhancement (until binarization) [51, 52].

Other methods consist in modifying classical filters (median filter for instance) by incorporating fuzzy weighting functions [43].

Rule based techniques rely on ideal models (of filters, contours, etc.). These ideal cases being rare, variations and differences with respect to these models are permitted through fuzzy representations of the models, as fuzzy rules. For instance, a smoothing operator can be expressed by [62, 63]:

Table 1
Full size table

In this representation, the emphasized terms are defined by fuzzy sets or fuzzy operations. Typically, the grey level characteristics are defined by linguistic variables, the semantics of which are provided by fuzzy sets on the grey level interval. Actions are fuzzy functions applied on grey levels and on pixels. The implementation of these fuzzy rules follows the general principles of fuzzy logic [30].

More complex rules can be found, for instance in [41, 56], where a contour detector is expressed by a set of rules involving the gradient, the symmetry and the stiffness of the contour. Fuzzy rule based systems have also been proposed for contour linking, based on proximity and alignment criteria.

Note that rules are sometimes but a different representation of functional approaches. Their main advantage is that they are easy to design (in particular for adaptive operators) and to interpret, and they facilitate the communication with the user.

3 Intermediate level

Several operations have been defined in the literature on fuzzy objects, in particular spatial fuzzy objects, since the early works of Zadeh [71] on set operations, and of Rosenfeld on geometrical operations [60].

Typical examples of geometrical operations are area and perimeter of a fuzzy object. They can be defined as crisp numbers, where the computation involves each point up to its degree of membership. But since objects are not well defined, it can also be convenient to consider that measures performed on them are imprecise too. This point of view leads to definitions as fuzzy numbers [30].

Such geometrical measures can typically be used in shape recognition, where geometrical attributes of the objects are taken into account.

As an example, fuzzy measures have been used in [58] for detecting masses in digital breast tomosynthesis. The measures are performed on detected fuzzy regions, that are considered as candidate particles (Fig. 4). A decision concerning their recognition is performed by combining fuzzy attributes. Fuzzy decision trees can be used to this aim [20, 53].

Fig. 4
figure 4

Fuzzy particle (black lines represent the contours of the α-cuts of the fuzzy object) extracted from a digital mammography. Computing fuzzy attributes on this fuzzy object leads to a decision concerning this region. (From [57])

Full size image

It has been shown in [65, 66] that using fuzzy representations of digital objects allow deriving more robust measures than using crisp representations, and in particular dealing properly with the imprecision induced by the digitization process.

Such measures can also be used as descriptors for indexation and data mining applications.

Let us now consider topological features and the example of fuzzy connectivity. The degree of connectivity between two points x and y in a fuzzy object μ in a finite discrete space is defined as [60]: \(c_{\mu }(x,y) =\max _{L_{xy}}\min _{t_{i}\in L_{xy}}\mu (t_{i})\), where L x y is any path from x to y. This definition was exploited in fuzzy connectedness notions [68], now widely used in medical image segmentation and incorporated in freely available softwares such as ITKFootnote 1.

Morphological operations have also been defined on fuzzy objects (see e.g. [17]). We give here general definitions, for fuzzy erosion and dilation, from which several other morphological operations can be derived:

$$\displaystyle{ \forall x \in \mathcal{S},\;E_{\nu }(\mu )(x) =\inf _{y\in \mathcal{S}}T[c(\nu (y - x)),\mu (y)], }$$
(1)
$$\displaystyle{ \forall x \in \mathcal{S},\;D_{\nu }(\mu )(x) =\sup _{y\in \mathcal{S}}t[\nu (x - y),\mu (y)]. }$$
(2)

In these equations, μ denotes the fuzzy set to be dilated or eroded, ν the fuzzy structuring element, t a conorm (fuzzy intersection), T the t-conorm (fuzzy union) associated to t with respect to the complementation c.

Such fuzzy morphological operations have been used in medical imaging for instance for taking into account the spatial imprecision on the location of vessel walls for 3D reconstruction of blood vessels by fusing angiographic and ultrasonic acquisitions [19]. They also constitute a good formal framework for defining fuzzy spatial relations, as will be seen in Sect. 4. Another application of fuzzy morphology is for defining median fuzzy sets and series of interpolating fuzzy sets [12], which can typically be used for representing variability based on several instances of an anatomical structures or for atlas construction. An example is illustrated in Fig. 5.

Fig. 5
figure 5

Fuzzy median sets between the 18 instances of the IBSR database for four internal brain structures (thalamus and putamen in both hemispheres)

Full size image

Some approaches using fuzzy rules can also be found at intermediate level. Let us just mention two examples. The first one [28] deals with the segmentation of osseous surface in ultrasound images. It uses fuzzy representations of image intensity and gradient, as well as their fusion, in rules that mimic the reasoning process of a medical expert and that include knowledge about the physics of ultrasound imaging. This approach was successfully tested on a large image data set.

The second example is completely different and fuzzy rules are used in [24] to tune the parameters of a deformable model for segmenting internal structures of the brain. This approach elegantly solves the difficult problem of parameter tuning in such segmentation methods, and proved to provide very good results on normal cases.

4 Higher level

The main information contained in the images consists of properties of the objects and of relations between objects, both being used for pattern recognition and scene interpretation purposes. Relations between objects are particularly important since they carry structural information about the scene, by specifying the spatial arrangements between objects. These relations highly support structural recognition based on models. This models can be of iconic type, as an anatomical atlas, or of symbolic type, as linguistic descriptions or ontologies. Although the use of iconic representations for normal structure recognition is well acknowledged, they remain difficult to exploit in pathological cases. Anatomical knowledge is also available in textbooks or dedicated web sites, and is expressed mainly in linguistic form. These models involve concepts that correspond to anatomical objects, their characteristics, or the spatial relations between them. Human experts use intensively such concepts and knowledge to recognize visually anatomical structures in images. This motivates their use in computer aided image interpretation. Some attempts to formalize this knowledge has been recently performed, in particular in the form of ontologies (e.g. the Foundational Model of Anatomy [61]).

In our work, we concentrate mainly on spatial relations, which are strongly involved in linguistic descriptions. They constitute a very important information to guide the recognition of structures embedded in a complex environment, and are more stable and less prone to variability (even in pathological cases) than object characteristics such as shape or size. We proposed mathematical models of several spatial relations (adjacency, distances, directional relations, symmetry, between...) [8–10, 15, 18, 27], in the framework of fuzzy sets theory, which proved useful to recognize thoracic and brain structures [14, 16, 26]. These fuzzy representations can enrich anatomical ontologies and contribute to fill the semantic gap between symbolic concepts, as expressed in the ontology, and visual percepts, as extracted from the images. These ideas were used in particular in our segmentation and recognition methods [3, 36]: a concept of the ontology is used for guiding the recognition by expressing its semantics as a fuzzy set, for instance in the image domain or in an attribute domain, which can therefore be directly linked to image information.

The methods we develop in our group for segmentation and recognition of 3D structures in medical images can be seen as spatial reasoning processes. Two main components of this domain are spatial knowledge representation and reasoning. In particular spatial relations constitute an important part of the knowledge we have to handle, as explained before. Imprecision is often attached to spatial reasoning in images, and can occur at different levels, from knowledge to the type of question we want to answer. The reasoning component includes fusion of heterogeneous spatial knowledge, decision making, inference, recognition. Two types of questions are raised when dealing with spatial relations:

  1. 1.

    given two objects (possibly fuzzy), assess the degree to which a relation is satisfied;

  2. 2.

    given one reference object, define the area of space in which a relation to this reference is satisfied (to some degree).

In order to answer these questions and address both representation and reasoning issues, we rely on three different frameworks and their combination: (i) mathematical morphology, which is an algebraic theory that has extensions to fuzzy sets and to logical formulas, and can elegantly unify the representation of several types of relations; (ii) fuzzy set theory, which has powerful features to represent imprecision at different levels, to combine heterogeneous information and to make decisions; (iii) formal logics and the attached reasoning and inference power. The association of these three frameworks for spatial reasoning is an original contribution of our work [11].

An example of using the second type of question was used for segmenting the heart in low resolution CT images [46], relying on the anatomical knowledge “the heart is between the lungs”. The translation of this knowledge uses an original definition of the concept “between” [15], that defines a fuzzy region of interest in which the heart can then be segmented using a deformable model integrating the spatial relation contraints, as in [26]. An example is shown in Fig. 6.

Fig. 6
figure 6

Fuzzy region between the lungs, segmentation of the lungs and the heart on an axial slice and a coronal one

Full size image

Further examples in brain imaging will be illustrated in Sect. 6.

5 Fusion

As seen in the previous sections, a lot of approaches, whatever their level, involve fusion steps.

Information fusion becomes increasingly important in medical imaging due to the multiplication of imaging techniques. The information to be combined can be issued from several images (like multi-echo MR images for instance), or from one image only, using for instance combination of several relations between objects or several features of the objects, or from images and a model, like an anatomical atlas, or knowledge expressed in linguistic form or as ontologies.

The advantages of fuzzy sets and possibilities rely in the variety of combination operators, offering a lot of flexibility in their choice, and which may deal with heterogeneous information [32, 70]. We proposed a classification of these operators with respect to their behavior (in terms of conjunctive, disjunctive, compromise [32]), the possible control of this behavior, their properties and their decisiveness, which proved to be useful for several applications in image processing [7]. It is of particular interest to note that, unlike other data fusion theories (like Bayesian or Dempster-Shafer combination), fuzzy sets provide a great flexibility in the choice of the operator, that can be adapted to any situation at hand. Indeed, image fusion has often to deal with situations where an image is reliable only for some classes, or does not provide any information about some class, or is not able to discriminate between two classes while another does. In this context, some operators are particularly powerful, like operators that behave differently depending on whether the values to be combined are of the same order of magnitude or not, whether they are small or high, and operators that depend on some global knowledge about source reliability about classes, or conflict between images (global or related to one particular class). The combination process can be done at several levels of information representation, from pixel level to higher level. A noticeable advantage of this approach is that it is able to combine heterogeneous information, like it is usually the case in multi-image fusion.

At a numerical level, the typical application is multi-source classification. We show an example of image fusion problem in brain imaging, where we combine dual-echo brain MR images in order to provide a classification of the brain into three classes: brain, ventricles and CSF, and pathology. These images are shown in Fig. 7. The membership functions for these classes have been estimated in a completely unsupervised way on both images, as described before. We then use these membership functions in a fuzzy fusion scheme [13]. Since both images provide similar information about the ventricles, we use a mean operator to combine the membership functions obtained in both images for this class. Brain and pathology cannot be distinguished in the first echo and we obtain only one class for this image, denoted by \(\mu _{c}^{1}\). In the second image, we obtain two classes denoted by \(\mu _{c}^{2}\) and \(\mu _{path}^{2}\) respectively. We combine \(\mu _{c}^{1}\) and \(\mu _{c}^{2}\) using an arithmetical mean again. As for the pathology, we combine \(\mu _{c}^{1}\) and \(\mu _{path}^{2}\) using a symmetrical sum defined as: \(\frac{ab} {1-a-b+2ab}\). This guarantees that no pathology is detected in the areas where \(\mu _{path}^{2} = 0\), and this reinforces the membership to that class otherwise, in order to include the partial volume effect areas in the pathology (this corresponds to what radiologists do). After the combination, the decision is made according to the maximum of membership values. The result is shown in Fig. 7 (right).

Fig. 7
figure 7

Dual echo MR image of the brain, showing three main classes: brain, ventricles and pathology (the white area on the middle image). Right: final decision after fuzzy combination (note that the decision is taken at each pixel individually, without spatial regularization)

Full size image

At a structural level, the operations defined on fuzzy objects as well as the relations between fuzzy objects can serve as a basis for structural recognition. An example will be provided in the next section.

A noticeable advantage of fuzzy fusion is that it is able to combine heterogeneous information, like is the case when dealing with higher level approaches, where several types of knowledge and information with different semantics have to be combined, and to avoid to define a more or less arbitrary and questionable metric between pieces of information.

Let us give a few examples. If we have different constraints about an object (for instance concerning the relations it should have with respect to another object) which have all to be satisfied, these constraints can be combined using a t-norm (a conjunction). If one object has to satisfy one relation or another one then a disjunction represented by a t-conorm has to be used. This occurs for instance when two symmetrical structures with respect to the reference object can be found (this situation often occurs in medical imaging). Mean operators can be used to combine several estimations and try to find a compromise between them. Associative symmetrical sums can be used for reinforcing the dynamics between high and low membership degrees. Importance of a constraint or reliabilities can be easily introduced in adaptive operators, and so on.

6 An application to the recognition of brain structures based on anatomical knowledge representation

Let us now illustrate how fuzzy spatial relations can be used for recognizing structures in a scene based on a model. The chosen example concerns the recognition of internal brain structures (ventricular system and grey nuclei) in 3D MRI. Two types of approaches have been developed, that correspond to the two types of questions raised in Sect. 4.

6.1 Global approach

In the first approach, which relies on the first type of question, spatial relations evaluated between spatial entities (typically objects or regions) are considered as attributes in a graph. The model is a graph derived from an anatomical atlas. Each node represents an anatomical structure, and edges represent spatial relations between these structures. A data graph is constructed from the MRI image where recognition has to be performed. Each node represents a region obtained from a segmentation method. Since it is difficult to segment directly the objects, usually the graph is based on an over-segmentation of the image, for instance based on watersheds. Attributes are computed as for the model. The use of fuzzy relations is particularly useful in order to be less sensitive to the segmentation step.

One important problem to be solved then is graph matching. Because of the schematic aspect of the model and the difficulty to segment the image into meaningful entities, no isomorphism can be expected between both graphs. In particular, several regions of the image can be assigned to the same node of the model graph. Such problems call for inexact graph matching. In general, it consists in finding a morphism, which furthermore optimizes an objective function based on similarities between attributes. Here the fusion applies not directly on the relations but on the similarities between them (see Sect. 5). A weighted mean operator allows us to give more importance to the edges, which show less variability between subjects and therefore constitute stronger anchors for guiding recognition. The morphism aims at preserving the graph structure, while the objective function privileges the association between nodes, respectively between edges, with similar attribute values. This approach can benefit from the huge literature on fuzzy comparison tools (see e.g. [21]) and from recent developments on fuzzy morphisms [54]. The optimization is not an easy task since the problem is NP-hard. Genetic algorithms, estimation of distribution algorithms and tree search methods have been developed towards this aim [4, 22, 55]. An example of recognition of the caudate nucleus is shown in Fig. 8.

Fig. 8
figure 8

Two regions of a 3D MR image, selected from an over-segmentation of the image as the ones having the best matching degree to the caudate nucleus in the atlas (only one slice is shown)

Full size image

Another approach consists in representing all knowledge on spatial relations between structures in a graph and expressing the joint segmentation and recognition problem as a constraint satisfaction problem [47, 48]. Propagators are defined for each spatial relation, and applied sequentially in order to progressively reduce the domain of each anatomical structure.

6.2 Sequential approach

In the second type of approach, relying on the second type of question, we use spatial representations of spatial knowledge [16, 26]. It consists in first recognizing simple structures (typically brain and lateral ventricles), and then progressively more and more difficult structures, based on relations between these structures and previously recognized ones. The order in which structures can be recognized can be provided by the user, or estimated as suggested in [34, 35]. Each relation describing the structure to be recognized is translated into a spatial fuzzy set representing the area satisfying this relation, to some degree. The fuzzy sets representing all relations involved in the recognition process are combined using a numerical fusion operator. While we first used an atlas in [16], this constraint has been relaxed in our recent work [26, 35]. This presents two main advantages: the high computation time associated with the computation of a deformation field between the atlas and the image is left aside and the procedure is potentially more robust because it uses only knowledge expressed in symbolic form, which is generic instead of being built from a single individual as in the iconic atlas.

Finally, a refinement stage is introduced using a deformable model. This stage uses an initial classification (using a low level approach based on grey levels) as a starting point and has the potential to correct possible imperfections of the previous stage together with regularizing the contours of structures. This deformable model makes use of a fusion of heterogeneous knowledge: edge information derived from the image, regularization constraints and spatial relations contained in the linguistic description. All pieces of information are combined in the energy of a parametric deformable model. For instance the caudate nucleus can be recognized based on its grey level (roughly known depending on the type of acquisition), and, more importantly, on its relations to the lateral ventricles (exterior and close to them). Here, the primary role of spatial relations is to prevent the deformable model from progressing beyond the limit of structures with weak boundaries.

Figure 9 shows 3D views of some cerebral objects recognized in an MR image with our method. In particular, the importance of spatial relations is illustrated in the case of the caudate nucleus. The lower part of this structure has a very weakly defined boundary and the use of a spatial relation is essential to achieve a good segmentation.

Fig. 9
figure 9

Segmentation and recognition results obtained for the lateral ventricles, third ventricle, caudate nuclei and thalami by integrating spatial relations in 3D deformable models. Illustration of the importance of spatial relations in the deformable model: in the case of caudate nucleus, the force derived from spatial relations prevents the model to grow below the lower limit of the structure (left: result obtained without this force, right: with this force)

Full size image

One of the advantages of this approach is that it can be extended to pathological cases, since spatial relations remain quite stable in the presence of pathologies, unlike shapes and absolute locations. Moreover, it is possible to learn the parameters of the relations, and their stability according to the type of pathology [3, 37]. Two examples of segmentation and recognition results in pathological cases are shown in Fig. 10, based on a segmentation of the tumor (based on fuzzy classification) [38].

Fig. 10
figure 10

Examples of segmentation and recognition in pathological cases

Full size image

7 Conclusion

In this chapter, several examples illustrating the potential of fuzzy methods for medical imaging have been described. While low level methods are still the most widely used, recently several higher level approaches were developed, based on a rigorous and powerful mathematical basis. The association of a mathematical framework for modeling imprecision at different levels and of artificial intelligence methods for representing concepts and knowledge and for reasoning on them seems to be a very interesting current trend, where promising results are expected in a near future.