Order Based Morphology for Color Images via Matrix Fields

Abstract

Mathematical morphology is a successful branch of image processing with a history of more than four decades. Its fundamental operations are dilation and erosion, which are based on the notion of supremum and infimum with respect to an order. From dilation and erosion one can build readily other useful elementary morphological operators and filters, such as opening, closing, morphological top-hats, derivatives, and shock filters. Such operators are available for grey value images, and recently useful analogs of these processes for matrix-valued images have been introduced by taking advantage of the so-called Loewner order. There is a number of approaches to morphology for vector-valued images, that is, color images based on various orders, however, each with its merits and shortcomings. In this chapter we propose an approach to (elementary) morphology for color images that relies on the existing order based morphology for matrix fields of symmetric 2 × 2-matrices. An RGB-image is embedded into a field of those 2 × 2-matrices by exploiting the geometrical properties of the order cone associated with the Loewner order. To this end a modification of the HSL-color model and a relativistic addition of matrices is introduced. The experiments performed with various morphological elementary operators on synthetic and real images provide results promising enough to serve as a proof-of-concept.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

Beginning with the path-breaking work of Matheron and Serra [21, 22] in the late sixties mathematical morphology has provided us with an abundance of tools and techniques to process real valued-images for applications ranging from medical imaging to geological sciences [15, 23–25]. Erosion and dilation are the fundamental operations of grey scale morphology relying on the notion of minimum and maximum of real numbers. Since minimum and maximum in turn depend on the presence of an order, it is no surprise that morphology for vector valued i.e. color images does not always provide satisfactory results.

There have been numerous approaches how to extend the mathematical morphology framework to color or vector-valued images. The main ingredients for such a framework are ranking schemes and the proper notion of extremal operators such as supremum and infimum . Due to the lack of reasonable complete lattice for vectorial data numerous suggestions for ranking schemes (based on various notions of distances, projections, and real-valued transforms) have been made, for a well structured, comprehensive, in-depth, and still up-to-date survey the reader is referred to [2] and the extensive list of literature cited therein. In [9] and [13] a more historic account is presented, while for a study of the background in order theory see [3] and [10].

Depending on the choices made one obtains morphological transforms with specific properties. However, none of these attempts seems to have been accepted unanimously in the image processing community.

Somewhat surprising the situation for symmetric matrix valued images is not as hopeless as it might seem at first glance.

Here we consider a (symmetric) matrix valued images or matrix field F as a mapping

$$\displaystyle{F:\varOmega \subset \mathbb{R}^{d}\longrightarrow \mbox{ Sym}(n)}$$

from a image domain Ω in $\mathbb{R}^{d}$ into the set Sym(n) of real symmetric n × n-matrices.

There have been successful attempts to extend the operations of mathematical morphology to images with values in the set of positive definite real symmetric 2 × 2- or 3 × 3-matrices [5–7], since these types of data make a natural appearance in medical imaging as the output of diffusion tensor weighted magnetic resonance imaging (DT-MRI).

The advantage of the matrix valued setting over the vector valued one is the presence of a prominent order for symmetric matrices, the so-called Loewner order, and the richer algebraic structure of symmetric matrices.

Hence, the goal of this chapter is to present an approach to morphological operators for color images by embedding a color image suitably into a matrix field. Hence the morphology already developed for matrix fields will give rise to morphology for color images.

For the coding of a color image as a matrix field we will make use of a variant of the HSL-color space and the Loewner order cone for real symmetric 2 × 2-matrices. This novel concept can be applied to grey value images as well and indeed includes scalar (flat) morphology.

The structure of the article is as follows: In order to keep the chapter as self-contained as possible we devote the next section to a brief review of the grey scale morphological operations we aim to extend to the matrix-valued setting, starting from the basic erosion/dilation and reaching to the morphological equivalents of gradient, and Laplacian, and its use for shock filtering. In Sect. 3 we present the maximum and minimum operations for matrix-valued data and especially a three-dimensional representation of the Loewner order cone for 2 × 2- matrices. Section 4 contains the aforementioned embedding and an operation for symmetric matrices gleaned from the relativistic addition of velocities. We report the results of our experiments with various morphological operators applied to synthetic and real color images in Sect. 5. Section 6 offers concluding remarks and a short hint at future research.

2 A Glance at Scalar Morphology

In grey scale morphology a scalar function f represents an image: f(x, y) with $(x,y) \in \mathbb{R}^{2}\;$. In this paper we restrict ourselves to flat grey scale morphology where a binary type of the so-called structuring element is used. It is nothing but a set B in $\mathbb{R}^{2}$ determining the neighborhood relation of pixels. Then grey scale dilation ⊕, resp., erosion ⊖ replaces the grey value of the image f(x, y) by its supremum, resp., infimum within the mask B:

$$\displaystyle\begin{array}{rcl} (f \oplus B)\,(x,y)\;&:=& \;\sup \;\{f(x\! -\! x^{\prime},\,y\! -\! y^{\prime})\;\vert \;(x^{\prime},y^{\prime})\! \in \! B\}\;, {}\\ (f \ominus B)\,(x,y)\;&:=& \;\inf \;\{f(x\! +\! x^{\prime},\,y\! +\! y^{\prime})\;\vert \;(x^{\prime},y^{\prime})\! \in \! B\}\;. {}\\ \end{array}$$

By concatenation other operators are constructed such as opening and closing ,

$$\displaystyle{f \circ B:= (f \ominus B) \oplus B\;,\qquad f\bullet B:= (f \oplus B) \ominus B\;,}$$

the white top-hat and its dual, the black top-hat

$$\displaystyle{\mbox{ WTH}(f):= f - (f \circ B)\;,\qquad \mbox{ BTH}(f):= (f\bullet B) - f\;,}$$

finally, the self-dual top-hat,

$$\displaystyle{\mbox{ SDTH}(f):= (f\bullet B) - (f \circ B)\;.}$$

The boundaries or edges of objects in an image are the loci of high grey value variations and those can be detected by gradient operators. Erosion and dilation are also the elementary building blocks of the basic morphological gradients , namely: The so-called Beucher gradient

$$\displaystyle{\rho _{B}(f):= (f \oplus B) - (f \ominus B)\;.}$$

It is an analog to the norm of the gradient $\|\nabla f\|$ if an image is considered as a differentiable function. Other useful approximations to $\|\nabla f\|$ are the internal and external gradient,

$$\displaystyle{\rho _{B}^{-}(f):= f - (f \ominus B)\;,\qquad \rho _{ B}^{+}(f):= (f \oplus B) - f\;.}$$

A morphological Laplacian has been introduced in [26] as the morphological equivalent for the Laplace operator $\varDelta F = \partial _{xx}F + \partial _{yy}F$ in the matrix valued setting. Following [8] we consider a variant given by the difference between external and internal gradient

$$\displaystyle{\varDelta _{m}F:=\rho _{ B}^{+}(F) -\rho _{ B}^{-}(F) = (F \oplus B) - 2 \cdot F + (F \ominus B)\;,}$$

thus representing the second derivative $\partial _{\eta \eta }f$ where η denotes the direction of the steepest slope.

Since Δ _m f is matrix-valued, trace(Δ _m f) will provide us with useful information: Regions where trace(Δ _m F) ≤ 0 can be viewed as the influence zones of maxima while those areas with trace(Δ _m F) ≥ 0 are influence zones of minima. Hence it allows us to distinguish between influence zones of minima and maxima in the image F. This is decisive for the construction of so-called shock filters .

The basic idea underlying shock filtering is applying either a dilation or an erosion to an image, depending on whether the pixel is located within the influence zone of a minimum or a maximum [19]:

$$\displaystyle{ \delta _{B}(f):= \left \{\begin{array}{ccl} f \oplus B & \mbox{ if } &\mathrm{trace}(\varDelta _{m}f) \leq 0\;, \\ f \ominus B &\mbox{ else}\;.& \end{array} \right. }$$

(1)

The shock filter expands local minima and maxima at the cost of regions with intermediate grey values. When iterated, experimental results in grey scale morphology suggest that a non-trivial steady state exists characterized by a piecewise constant segmentation of the image.

In the scalar case the zero-crossings Δ f = 0 can be interpreted as edge locations [14, 17, 20]. We will also use the trace of the morphological Laplacian in this manner to derive an edge map.

However, if we consider a matrix field F, Δ _m F is matrix-valued, but the scalar value of trace(Δ _m F) provides us with the information necessary for the switch criterion in the matrix valued shock filter.

3 Loewner Order: Maximal and Minimal Matrices

Morphology rests on the fundamental notions of supremum and infimum with respect to an order. This is true for matrix fields as well.

The so-called Loewner order is a natural partial order on Sym(n), defined via the cone of positive semidefinite matrices Sym⁺(n) by

$$\displaystyle{A,B \in \mbox{ Sym}(n): \quad A \geq B:\Leftrightarrow A - B \in \mbox{ Sym}^{+}(n),}$$

i.e. if and only if A − B is positive semidefinite.

This partial order is not a lattice order, that is, there is no notion of a unique supremum and infimum with respect to this order [4]. Nevertheless, given any finite set of symmetric matrices $\mathcal{A} =\{ A_{1},\ldots,A_{n}\}$, we will be able to identify suitable maximal, resp., minimal matrices

$$\displaystyle{\overline{A}:=\sup \mathcal{A}\quad \mbox{ resp., }\quad \underline{A}:=\inf \mathcal{A}\;.}$$

Since we will consider images with three color components we may restrict ourselves from now on to the case of 2 × 2-matrices in Sym(2) which offer already three degrees of freedom. The procedure to find these extremal matrices for a set $\mathcal{A}$ is as follows: The cone Sym⁺(2) can be represented in 3D using the bijection

$$\displaystyle\begin{array}{rcl} A:= \left (\begin{array}{cc} \alpha &\beta \\ \beta &\gamma \end{array} \right )\longleftrightarrow \frac{1} {\sqrt{2}}\left (\begin{array}{c} 2\beta \\ \gamma -\alpha \\ \gamma +\alpha \end{array} \right ),\mbox{ resp., } \frac{1} {\sqrt{2}}\left (\begin{array}{cc} z - y& x\\ x &z + y \end{array} \right )\longleftrightarrow \left (\begin{array}{c} x\\ y \\ z \end{array} \right )\;.& &{}\end{array}$$

(2)

This mapping creates an isometrically isomorphic image of the cone Sym⁺(2) in the Euclidean space $\mathbb{R}^{3}$ given by $\{(x,y,z)^{\top }\in \mathbb{R}^{3}\vert \sqrt{x^{2 } + y^{2}} \leq z\}$ and is depicted in Fig. 1a. For A ∈ Sym(2) the set P(A) = { Z ∈ Sym(2) | A ≥ Z} denotes the penumbral cone or penumbra for short of the matrix A. It corresponds to a cone with vertex in A and a circular base in the x − y-plane:

$$\displaystyle{P(A) \cap \{ z = 0\} = \mbox{ circle with center }(\sqrt{2}\beta, \frac{\gamma -\alpha } {\sqrt{2}})\mbox{ and radius }\frac{\mbox{ trace}(A)} {\sqrt{2}} \;.}$$

Considering the associated penumbras of the matrices in $\mathcal{A}$ the search for the maximal matrix $\overline{A}$ amounts to determine the smallest penumbral cone covering all the penumbras of $\mathcal{A}$ tightly, see Fig. 1b. One realises that the height of a penumbra measured from the x − y-plane is equal to the radius of its base, namely $\frac{\mbox{ trace}(A)} {\sqrt{2}}$. Hence a penumbra is already uniquely determined by the circle constituting its base. This implies that the search for a maximal matrix comes down to finding the smallest circle enclosing the base-circles of the matrices in $\mathcal{A}$. This is a non-trivial problem in computer graphics. A numerical solution for finding the smallest circle enclosing the sampled basis circles has been implemented in C++ by Gärtner [11] and was used in [7] and [6]. However, in our case we employ the implementation of an efficient subgradient method detailed in [27] for the calculation of the smallest circle enclosing them. This gives us the smallest covering cone and hence the maximal matrix $\overline{A}$. For technical reasons in our later application we will not apply the above reasoning directly to the matrices A ₁, …, A _n but to their shifted versions $\mathcal{A} + I:= A_{1} + I,\ldots,A_{n} + I$ with the unit matrix I taking advantage of the relation

$$\displaystyle{\overline{A} =\sup (A_{1},\ldots,A_{n}) =\sup (A_{1} + I,\ldots,A_{n} + I) - I = \overline{A + I} - I\;.}$$

A suitable minimal matrix $\underline{A}$ is obtained by means of the formula

$$\displaystyle{\underline{A} = I -\left (\sup (I - A_{1},\ldots,I - A_{n})\right )}$$

inspired by the relation $\min (a_{1},\ldots,a_{n}) = 1 -\left (\max (1 - a_{1},\ldots,1 - a_{n})\right )$ valid for real numbers a ₁, …, a _n . For i = 1, …, n we have $\underline{A} \leq A_{i} \leq \overline{A}$ with respect to the Loewner order. We emphasise that $\overline{A}$ and $\underline{A}$ depend continuously on A ₁, …, A _n by their construction. Also the rotational invariance is preserved, since the Loewner order is already rotational invariant: $A \geq B\Longleftrightarrow UAU^{\top }\geq UBU^{\top }$ holds for any orthogonal matrix U. Nevertheless, the definitions of the matrices $\overline{A}$ and $\underline{A}$ are still meaningful for matrices that are not positive definite as long as they have a non-negative trace (since it corresponds to a radius in the construction above). It also becomes evident from their construction that in general neither $\overline{A}$ nor $\underline{A}$ coincide with any of the A _i .

Fig. 1

(a) Image of the Loewner cone Sym⁺(2). (b) Cone covering four penumbras of other matrices. The tip of each cone represents a symmetric 2 × 2 – matrix in $\mathbb{R}^{3}$. For each cone the radius and the height are equal

With these essential notions of suitable maximal and minimal matrices $\overline{A}$ and $\underline{A}$ at our disposal the definitions of the higher morphological operators carry over essentially verbatim, with one exception:

The morphological Laplacian Δ _m as defined in Sect. 2 is a matrix. In Eq. (1) we used the trace of the morphological Laplacian to steer the interwoven dilation-erosion process, and to create an edge map.

4 Color Images as Matrix Fields

One of the most common models is the RGB color space where each color appears in its primary components red, green, and blue. After standard normalizations the color space is represented by the unit cube in a Cartesian coordinate system, see [12]. Deeper inside in the use of color in sciences is provided in [18]. Closer to the human perception process is the HSL (or HSI) color model describing a color object by its hue , saturation and brightness resp. luminance . It is a popular cylindrical-coordinate representation of points in an RGB color model (see [1, Algorithm 8.6.3] for the conversion). If one replaces in this model the coordinate saturation by the so-called chroma one arrives at a modified version of the HSL-model which we call HC$\tilde{\mathrm{L}}$ color model. Its representation is given by a bicone , depicted in Fig. 2.

Fig. 2

HC$\tilde{\mathrm{L}}$-bicone: Bicone for the HC$\tilde{\mathrm{L}}$ color model

To be more specific:

$$\displaystyle{\tilde{L} = 2L - 1\;,}$$

and the chroma is obtained via

$$\displaystyle{C =\max \{ R,G,B\} -\min \{ R,G,B\}\;.}$$

Hence, any point (x, y, z) of the bicone can be obtained via

$$\displaystyle\begin{array}{rcl} x& =& C\cdot \cos (2\pi \cdot H)\;, {}\\ y& =& C\cdot \sin (2\pi \cdot H)\;, {}\\ z& =& \tilde{L}\;. {}\\ \end{array}$$

Note that the ranges of all components lie in the unit interval. This provides us with an one-to-one transformation of the HC$\tilde{\mathrm{L}}$ color space to the RGB color space. It is now apparent, that this HC$\tilde{\mathrm{L}}$-bicone corresponds via (2) directly to the order interval

$$\displaystyle{[-I,I]_{L}:=\{ A \in \mbox{ Sym}(2)\;\vert \;- I \leq A \leq I\}\;.}$$

Hence each matrix in [−I, I] _L corresponds uniquely to a point in the bicone and each point in the bicone represents uniquely a color. In total this establishes the desired continuous one-to-one correspondence of the matrices in [−I, I] _L with the colors in the HC$\tilde{\mathrm{L}}$ (and from there to the standard RGB space, if so desired),

$$\displaystyle{\varPsi: \mbox{ HC}\tilde{\mathrm{L}} \subset IR^{3}\longrightarrow [-I,I]_{ L} \subset \mbox{ Sym}(2)\;.}$$

Exploiting this correspondence one obtains, for example, the supremum of two colors c ₁, c ₂ ∈  HC$\tilde{\mathrm{L}}$ by transforming them into the matrices Ψ(c ₁), Ψ(c ₂) ∈ [−I, I] _L , then taking the supremum $\sup (\varPsi (c_{1}),\varPsi (c_{2}))$ of these two matrices which is then transformed back to the new ‘supremum color‘

$$\displaystyle{\sup (c_{1},c_{2}):=\varPsi ^{-1}\left (\sup (\varPsi (c_{ 1}),\varPsi (c_{2}))\right )\;.}$$

Ψ ⁻¹ combines the mapping (2), namely

$$\displaystyle{\left (\begin{array}{cc} \alpha &\beta \\ \beta &\gamma \end{array} \right ) \rightarrow \frac{1} {\sqrt{2}}\left (\begin{array}{c} 2\beta \\ \gamma -\alpha \\ \gamma +\alpha \end{array} \right ) =: \left (\begin{array}{c} x\\ y \\ z \end{array} \right )}$$

with a transform into polar coordinates via

$$\displaystyle\begin{array}{rcl} H& =& \frac{1} {2\pi }\mathrm{arg(y,x)}\;, {}\\ C& =& \sqrt{x^{2 } + y^{2}}\;, {}\\ \tilde{L}& =& z\;, {}\\ \end{array}$$

with a principal value of an appropriate argument function. The luminance L is obtained via $L = (\tilde{L} + 1)/2$ while the saturation is given by S = 0 if C = 0, otherwise $S = C/(1 -\vert 2L - 1\vert )$.

Thus, having obtained those HSL-values, we now convert them to the normalized RGB-values (see [1, Algorithm 8.6.4] for the conversion).

The infimum of two colors is treated analogously. Hence by applying the same rationale not only the fundamental processes of dilation and erosion can be transferred from matrix fields to color images but other basic morphological operations as well.

However, two major difficulties arise:

• 1. Due to the non-polygonal nature of the Loewner order cone it can happen, that the supremum (and likewise the infimum) of matrices is outside the interval [−I, I] _L , that is, the corresponding HCL- and RGB-value do not exist.

• 2. The design of the morphological top-hat or derivative operators requires taking the differences/sums of matrices in [−I, I] _L . Taking this differences/sums in the standard way of matrix algebra results in matrices outside [−I, I] _L , again entailing non-existing corresponding HCL- and RGB-values.

We overcome the first difficulty by a rescaling of the corresponding matrices: Instead of the matrix Ψ(c) associated to a color c ∈ HCL we consider the matrix $\frac{\sqrt{2}} {2} \varPsi (c)$ involving the scaling factor $d = \sqrt{2}/2$.

To resolve the second problem we extend Einstein‘s rule for the addition velocities in the theory of Special Relativity to symmetric matrices:

$$\displaystyle{ A +\!\! +B:= (A + B) \circ \left (1 + \frac{1} {2}A \circ B\right )^{-1} }$$

(3)

where the symmetric matrix product ‘∘’ defined by

$$\displaystyle{A \circ B:= \frac{1} {2}\left (\mathit{AB} + \mathit{BA}\right )}$$

is the Jordan product of A and B. This type of addition is inspired from the fact that the interval of real numbers $[-d,d] \subset \mathbb{R}$ with $d = \frac{\sqrt{2}} {2}$ equipped with the relativistic addition

$$\displaystyle{a +\!\! +b:= \frac{a + b} {1 + \frac{a\cdot b} {d^{2}} } }$$

of numbers a, b ∈ [−d, d] establishes a commutative group [16], hence allows naturally for subtraction as well. This relativistic rule of addition/subtraction of matrices will lead to matrices often located in the lower part of the bicone causing the resulting color to tend towards grey/black. This effect is responsible for the very dominant tone of grey in images processed with operators invoking a subtraction, such as morphological top-hats and derivatives. Subsequent experimental results confirm this reasoning.

5 Experimental Results

In this section, the morphological operators designed above will be applied to synthetic as well as natural color images of various sizes.

In all the experiments we use a cross-shaped structuring element consisting of five pixels centered at the middle pixel. Each of the images is extended by one layer of mirrored boundary values.

In the first experiments we confirm that our color-morphological operators applied to image in Fig. 3a in principle act as regular morphological operators on black-and-white images. As expected dilation and erosion result in an accurate shift of the inner object front, see Fig. 4a. However, the scaling causes a slight shift of black and white towards grey: The black part which is represented by the RGB-values [0, 0, 0] becomes dark grey [37, 37, 37]. The white part which is represented by the RGB-values [255, 255, 255] turns into light grey [218, 218, 218].

Fig. 3

Original 8 × 8-image test images. (a) Image 1. (b) Image 2. (c) Image 3

Fig. 4

Dilation and erosion applied to bipartite black-and-white image. (a) Original 8 × 8-image. (b) Dilation. (c) Erosion

An image of the same size 8 × 8, but with a blue-colored (RGB  = [0, 0, 255]) left and a green-colored (RGB  = [0, 255, 0]) right side undergoes both an dilation and an erosion. Both colors are located in the x − y-plane relatively far apart on the boundary of the HC$\tilde{\mathrm{L}}$ bicone, hence the maximal and minimal matrices are representing almost white and black respectively. This accounts for the light grey resp. dark grey center section in the dilated resp. eroded images in Figs. 5b, c. Note that due to the scaling the colors are slightly faded. We repeat the experiment with a bipartite image whose halves are colored with RGB-values [255, 0, 0] ($\mathrm{HC}\tilde{\mathrm{L}} = [0,1,0]$) and [128, 255, 0] ($\mathrm{HC}\tilde{\mathrm{L}} = [1/4,1,0]$). Their representing $\mathrm{HC}\tilde{\mathrm{L}}$-values are located at the boundary of the base of the bicone in the x − y-plane at a 90^∘ angle. This represents a worst-case-scenario in the sense that the maximum of the two color-representing cones does not lie completely within the bicone, unless one uses the above mentioned scaling prior to taking the maximum. The same is true for taking the minimum. In total this results in a grey tone of the appearing colors, see Figs. 6b, c.

Fig. 5

Dilation and erosion applied to bipartite blue-and-green image. (a) Original 8 × 8-image. (b) Dilation. (c) Erosion

Fig. 6

Bipartite color image for which the rescaling of the bicone is necessary. (a) Original 8 × 8-image. (b) Dilation. (c) Erosion

Now, we apply dilation and erosion as well as their concatenations opening and closing to a 50 × 50-image with random RGB-coloring of its pixels, see Figs. 7a and 8a respectively. As expected from the color distribution within the bicone erosion Fig. 8c entails a darkening of the image while dilation Fig. 8b accounts for its overall brightening. Inspecting Fig. 8d, e it becomes apparent that opening and closing lead to a coarsening of the image making the structuring element a clearly discernible shape.

Fig. 7

Original 50 × 50- and 36 × 24-test images. (a) Image 4. (b) Image 5

Fig. 8

Dilation, erosion, opening, and closing of an image with randomly colored pixels. (a) Original image of resolution 50 × 50. (b) Dilation. (c) Erosion. (d) Closing. (e) Opening

Let us consider an RGB-image of resolution 36 × 24 containing six circles with the RGB colors [128, 255, 0], [0, 128, 255], [0, 0, 0], [255, 255, 255], [235, 249, 18] and [249, 155, 18] above a red background ([255, 0, 0]). In Figs. 7b and 9a respectively we show the constructed image, whereas in Fig. 9b, c we see the results for dilation and erosion. Comparing the results of opening and closing displayed in Fig. 9d, e with the original image we note its almost perfect recovery. Only the colors lost a bit of their brilliance, again an effect of the aforementioned scaling.

Fig. 9

Dilation, erosion, opening, and closing. (a) Original. (b) Dilation. (c) Erosion. (d) Closing. (e) Opening

The outcome of the operations of dilation, erosion, opening, and closing when applied to a natural image Figs. 10b and 11a respectively is depicted in Fig. 11b–e. The closing operation behaves indeed as a filter eliminating the slight pepper-noise in the original image. Both closing and opening operations preserve to some extent the texture-like structure of the forest in the picture while simultaneously diminishing the depicted clouds.

Fig. 10

Original 171 × 228- and 228 × 171-test images. (a) Image 6. (b) Image 7

Fig. 11

Dilation, erosion, opening, and closing of a natural image. (a) Original 228 × 171-image. (b) Dilation. (c) Erosion. (d) Opening. (e) Closing

Another example of the effect of these for basic operations on natural images, see Fig. 10, is displayed in Fig. 12. Dilation causes the white stripes in the flag to widen and creates a slightly bright rim around the depicted building whereas erosion induces a widening of the red stripes and a greyish rim around the building.

Fig. 12

Natural image. Top: Dilation, erosion. Middle: Image polluted with 5 %-salt noise and 5 %-pepper noise. Bottom: Removal of these noise types by opening and closing. (a) Original 171 × 228-image. (b) Dilation. (c) Erosion. (d) 5 % Salt noise added. (e) 5 % Pepper noise added. (f) Opening. (g) Closing

When polluted with 5 % salt-noise, Fig. 12d, an opening removes it, Fig. 12f, as expected. The same is true with 5 % pepper-noise, Fig. 12e, eliminated by a closing operation, see Fig. 12g.

In the next experiment we investigate the effect of repeated dilation and erosion on both a synthetic and a natural image. In the synthetic image small structures assume more and more the shape of the structuring element, as expected. However, repeated dilations produce bright-colored rims at the cost of the objects itself, the black structure in Fig. 13a even vanishes and blends into the red background. It is the opposite with erosion: the eroded structures are first surrounded and then swallowed in a thickening dark rim and white structures disappear into the red background.

Fig. 13

Iterated dilations and iterated erosions. (a) Original. (b) Dilation 1×. (c) Dilation 2×. (d) Dilation 4×. (e) Erosion 1×. (f) Erosion 2×. (g) Erosion 4×

In principle the same is true for natural images, prominent objects are getting surrounded by a thickening rim of light grey resp. dark grey tone when the number of dilations resp. erosions performed increases, as can be seen in Fig. 14. In the next set of experiments we turn our attention to the morphological derivatives: internal, external, and Beucher gradient, and the morphological Laplacian. They invoke a difference operation which is realized by means of the relativistic addition (3). The predominant tone in the derived image can be expected to be grey, the shade of color in the center of the bicone. Indeed, the difference operation underlying all the derivatives leads to matrices close to the said center. Only where a considerable change by erosion and/or dilation occurs the subtraction entails a matrix somewhat outside the bicone center, that is a color close to the primary colors. The experiments with the matrix-based derivative operations corroborate this reasoning.

Fig. 14

Iterated dilations and iterated erosions based on the matrix field approach. (a) Original. (b) Dilation 2×. (c) Dilation 5×. (d) Dilation 10×. (e) Erosion 2×. (f) Erosion 5×. (g) Erosion 10×

Nevertheless, it becomes apparent that the morphological gradients may serve as edge detectors, the Beucher gradient being slightly more dissipative than the other two one-sided ones (Figs. 15 and 16).

Fig. 15

Internal, external, and Beucher gradient applied to a natural image. (a) Internal. (b) External. (c) Beucher gradient

Fig. 16

The morphological Laplacian which provides the switch criterion for the shock filter applied once and 10 times to a natural image. (a) Morph. Laplacian. (b) Shockfilter 1×. (c) Shockfilter 10×

An obvious approach to morphology to color images is the channel-wise application of the morphological operations. That means, every RGB-image is decomposed into three scalar images and the morphological operation is applied to each of the three images separately. The resulting images are then re-combined to a now transformed RGB-image.

This works reasonably well as far as dilation, erosion, opening and closing with a small structuring element are concerned. The rim that is expected to get thicker with each iteration assumes a green color using the component-wise approach, see Fig. 17 which is different compared to the new approach, see Fig. 14.

Fig. 17

Component-wise approach: Iteration of the erosion operation applied separately to each of the three channels of an RGB-image. The three resulting scalar images are then re-combined to the RGB-image. (a) Component-wise erosion 1×. (b) Component-wise erosion 5×. (c) Component-wise erosion 10×

Furthermore, these color disturbances become even more severe if a difference operation is involved, as the next examples exhibit, see Fig. 18. Trying a component-wise approach with the channels of the HC$\tilde{\mathrm{L}}$-model is as fruitless as with the channels of the RGB-model. The other color phenomenon manifests itself clearly in Fig. 19, where we depicted only the HC$\tilde{\mathrm{L}}$-channel-wise Beucher gradient and Laplacian.

Fig. 18

Component-wise approach: The morphological operations of internal, external, and Beucher gradient applied to each of the three channels of an RGB-image separately followed by a recombination to a single RGB-image. (a) Component-wise internal gradient. (b) Component-wise external gradient. (c) Component-wise Beucher gradient

Fig. 19

HC$\tilde{\mathrm{L}}$-component-wise approach: The morphological operations Beucher gradient and Laplacian applied to each of the three channels of an HC$\tilde{\mathrm{L}}$-image separately followed by a recombination to a single HC$\tilde{\mathrm{L}}$-image. (a) HC$\tilde{\mathrm{L}}$-component-wise Beucher gradient. (b) HC$\tilde{\mathrm{L}}$-component-wise Laplacian

So far we have refrained from presenting results of the application of the matrix-based morphological operations referred to as black top-hat, white top-hat and self-dual top-hat to color images. In fact, it is difficult to say what one should expect from a filter that is supposed to extract small black details from a (grey scale) image if it is applied to a color image.

In the experiments the predominant tone in the transformed image is again grey, the shade of color in the center of the bicone. The explanation for its appearance is practically the same as for the derivative operations since the top-hats are based on the difference operation as well: Only where a considerable change by opening and/or closing occurs the subtraction leads to matrices out of the bicone center, meaning, to colors close to primary colors. This is confirmed by the experiments with the matrix-based top-hat operations, see Fig. 20. However, the results of the component-wise RGB-based approach are not satisfactory either as can be seen in Fig. 21.

Fig. 20

Matrix-based black, white, and self-dual top-hat applied to an natural image. (a) Black top-hat. (b) White top-hat. (c) Self-dual top-hat

Fig. 21

Component-wise approach: The morphological operations of black, white and self-dual top-hat applied RGB-channel-wise to an natural image. (a) Component-wise black top-hat. (b) Component-wise white top-hat. (c) Component-wise self-dual top-hat

6 Summary and Future Work

The real symmetric 2 × 2-matrices of a matrix field offer three degrees of freedom, enough to house the three components of many popular color models, such as the RGB- or the HSI-model. The matrix-setting has indeed several advantages over the vector-setting of color images: First the richer algebraic structure, second, the existence of an almost canonical order, the Loewner order. And third, when morphological operations are applied, the interaction of the color channels is ensured, if these are coded in matrix form. This coding was inspired by the close, almost obvious geometric relation between the $\mathrm{HC}\tilde{\mathrm{L}}$ bicone, a variant of the HSI-bicone, and an order interval induced by the Loewner order cone. Once the color image has been rewritten in this manner, the morphological techniques developed for matrix fields in [7] and [8] were applicable and the results of various morphological operations ranging from the elementary dilation and erosion to second order derivatives could be studied.

The experimental results are promising and suffice as a proof-of-concept for this novel approach to color image morphology.

However, in the future we will investigate the usefulness of other color models as well as their embedding into the matrix setting, and we will attempt to develop meaningful notions of top-hats, and other more sophisticated morphological operators for color images.