Transforms and Operators for Directional Bioimage Analysis: A Survey

Abstract

We give a methodology-oriented perspective on directional image analysis and rotation-invariant processing. We review the state of the art in the field and make connections with recent mathematical developments in functional analysis and wavelet theory. We unify our perspective within a common framework using operators. The intent is to provide image-processing methods that can be deployed in algorithms that analyze biomedical images with improved rotation invariance and high directional sensitivity. We start our survey with classical methods such as directional-gradient and the structure tensor. Then, we discuss how these methods can be improved with respect to robustness, invariance to geometric transformations (with a particular interest in scaling), and computation cost. To address robustness against noise, we move forward to higher degrees of directional selectivity and discuss Hessian-based detection schemes. To present multiscale approaches, we explain the differences between Fourier filters, directional wavelets, curvelets, and shearlets. To reduce the computational cost, we address the problem of matching directional patterns by proposing steerable filters, where one might perform arbitrary rotations and optimizations without discretizing the orientation. We define the property of steerability and give an introduction to the design of steerable filters. We cover the spectrum from simple steerable filters through pyramid schemes up to steerable wavelets. We also present illustrations on the design of steerable wavelets and their application to pattern recognition.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

3.1 Introduction

Directionality and orientation information is very useful for the quantitative analysis of images. By those terms, we refer to local directional cues and features that one can identify in natural images. The area of applications based on the detection of orientation is continuously growing as the importance of directionality is becoming more and more relevant in image processing. The range of applications spans topics from astronomy (Bernasconi et al. 2005; Yuan et al. 2011; Schuh et al. 2014), aerial and satellite imagery (Tupin et al. 1998; Jiuxiang et al. 2007), material sciences (Dan et al. 2012) to biological and medical applications. Focusing on the last two categories, the palette is quite broad: detection of nodules in the lungs (Agam et al. 2005) and vessels in retinal fundus images (Lam et al. 2010; Patton et al. 2006), bioimaging (Honnorat et al. 2011), neuroimaging (Meijering et al. 2004; Gonzalez et al. 2009). Investigations of collagen in the arterial adventitia also rely on directional analysis (Rezakhaniha et al. 2012). Neuron tracking is of primal importance to understand the development of the brain and requires robust directional image-analysis tools to capture dendrites in 2D and 3D (Meijering 2010). In Jacob et al. (2006), the authors used steerable ridge detector [based on Canny (1986)] to study the aging of elastin in human cerebral arteries. In Aguet et al. (2009), 3D steerable filters were applied to the estimation of orientation and localization of fluorescent dipoles.

Researchers in image analysis are getting inspiration from the human visual system. In the early 1960s, it was demonstrated that directionality plays a key role in visual perception: The neurophysiological findings of Huber and Wiesel initiated a field of research for decades to come (Hubel and Wiesel 1962). Follow-up studies confirmed that the organization of the primary visual cortex makes our visual perception particularly sensitive to the directional clues carried by edges, ridges, and corners (Olshausen and Field 1996; Marr and Hildreth 1980). Our visual system is able to efficiently capture and summarize this information using a small number of neuronal cells.

Based on these structures, many image-analysis methods have been proposed, but they face several challenges. One of them is efficiency with respect to computational resources, because real-time applications and the processing of large multidimensional data (e.g., multichannel time-lapse sequences of images or volumes) demand fast algorithms. Another challenge is to design algorithmic detectors of orientation that acknowledge that patterns in natural images usually have an unknown size and location. Robustness to noise is another desirable trait.

This survey aims at providing the reader with a broad overview of techniques for the directional analysis of images. It is intended to be used as a guide to state-of-the-art methods and techniques in the field. In this paper, we focus on the applications in bioimaging, presenting and comparing the described methods on experimental data.

We focus on the continuous domain setup for explaining the relevant concepts because it allows for convenient, compact, and intuitive formulation. It primarily involves differential and convolution operators (smoothing filters and wavelets) that are acting on continuously defined images, $f(\boldsymbol{x}),\boldsymbol{x} = (x_{1},x_{2}) \in \mathbb{R}^{2}$. The final transcription of a continuous domain formula into an algorithm requires the discretization of the underlying filters which can be achieved using standard techniques. For instance, partial derivatives can be closely approximated using finite differences, while there are well-established techniques for computing wavelets using digital filters. For further implementation details, we are giving pointers to the specialized literature.

3.2 Derivative-Based Approaches

3.2.1 Gradient Information and Directional Derivatives

Some of the earliest and simplest techniques in image analysis to account for orientation rely on gradient information. Intuitively, the direction of the gradient corresponds to the direction of steepest ascent. The local direction of an image f at $\boldsymbol{x}_{0}$ can be estimated in terms of the direction orthogonal to its gradient. A direction is specified in $\mathbb{R}^{2}$ by a unit vector $\boldsymbol{u} = (u_{1},u_{2}) \in \mathbb{R}^{2}$ with $\|\boldsymbol{u}\| = 1$. The first-order directional derivative $\mathrm{D}_{\boldsymbol{u}}\,f$ along the direction $\boldsymbol{u}$ can be expressed in terms of the gradient

$$\displaystyle{ \mathrm{D}_{\boldsymbol{u}}\,f(\boldsymbol{x}_{0}) =\lim _{h\rightarrow 0}\frac{f(\boldsymbol{x}_{0}) - f(\boldsymbol{x}_{0} - h\boldsymbol{u})} {h} =\langle \boldsymbol{ u},\boldsymbol{\nabla }f(\boldsymbol{x}_{0})\rangle, }$$

(3.1)

where the right-hand side is the inner product between $\boldsymbol{u}$ and the gradient vector $\boldsymbol{\nabla }f(\boldsymbol{x}_{0})$ evaluated at x ₀. We note that (3.1) is maximum when $\boldsymbol{u}$ is collinear to $\boldsymbol{\nabla }f(\boldsymbol{x}_{0})$ (by the Cauchy–Schwartz inequality). Conversely, $\mathrm{D}_{\boldsymbol{u}_{0}}\,f(\boldsymbol{x}_{0})$ vanishes when $\boldsymbol{u}_{0} \perp \boldsymbol{\nabla }f$, so that $\boldsymbol{u}_{0}$ provides us with a local estimate of the directionality of the image. Figure 3.1 illustrates the application of the gradient operators.

Fig. 3.1

Illustration of the use of gradient operators, from left to right: (1) Input image: confocal micrograph showing nerve cells growing along fibers made from a specially modified silk that is similar to that made by spiders and silkworms. This input image is from the Cell Image Library (http://www.cellimagelibrary.org/images/38921). Right lower corner: bright arcs with different scales, artificially added. (2) the x ₁ (or horizontal) component of the gradient is the directional derivative along $\boldsymbol{u} = (1,0)$. (3) the x ₂ (or vertical) component of the gradient is the directional derivative along $\boldsymbol{u} = (0,1)$. (4) Magnitude of the gradient vector. Highlighted window A: Horizontal edges are attenuated in case of directional derivative along $\boldsymbol{u} = (1,0)$ and enhanced/kept in case of directional derivative along $\boldsymbol{u} = (0,1)$. Highlighted window B: Vertical edges are attenuated in case of directional derivative along $\boldsymbol{u} = (0,1)$ and enhanced/kept in case of directional derivative along $\boldsymbol{u} = (1,0)$. All the images were produced by the ImageJ/Fiji plugin OrientationJ

Gradient-based orientation estimators are frequently used as they can be discretized and implemented easily. However, the gradient-based estimation of the orientations is sensitive to noise. The robustness can be improved by smoothing the image by a Gaussian kernel before taking the derivative. A still very popular method based on gradients is Canny’s classical edge detector (Canny 1986).

3.2.2 Improving Robustness by the Structure Tensor

The estimation of the local orientation using derivatives can be made more robust by using the structure tensor (Jahne 1997). The structure tensor is a matrix derived from the gradient of the image and can be interpreted as a localized covariance matrix of the gradient. Since the pioneering work of Förstner (1986), Bigun (1987), and Harris and Stephens (1988), the structure tensor has become a tool for the analysis of low-level features, in particular for corner and edge detection as well as texture analysis. In 2D, the structure tensor at location $\boldsymbol{x}_{0}$ is defined by

$$\displaystyle{ \boldsymbol{J}(\boldsymbol{x}_{0}) =\int _{\mathbb{R}^{2}}w(\boldsymbol{x} -\boldsymbol{ x}_{0})\left (\boldsymbol{\nabla }f(\boldsymbol{x})\right )\boldsymbol{\nabla }^{T}f(\boldsymbol{x})\mathrm{d}x_{ 1}\mathrm{d}x_{2}, }$$

(3.2)

where w is a nonnegative isotropic observation window (e.g., a Gaussian) centered at $\boldsymbol{x}_{0}.$ More explicitly, the (2 × 2) matrix $\boldsymbol{J}(\boldsymbol{x}_{0})$ reads

$$\displaystyle{ \boldsymbol{J}(\boldsymbol{x}_{0}) =\int _{\mathbb{R}^{2}}w(\boldsymbol{x}-\boldsymbol{x}_{0})\left (\begin{array}{*{10}c} f_{x_{1}}^{2}(\boldsymbol{x}) &f_{ x_{1}}(\boldsymbol{x})f_{x_{2}}(\boldsymbol{x}) \\ \,f_{x_{2}}(\boldsymbol{x})f_{x_{1}}(\boldsymbol{x})& f_{x_{2}}^{2}(\boldsymbol{x}) \end{array} \right )\mathrm{d}x_{1}\mathrm{d}x_{2} }$$

(3.3)

$$\displaystyle{ = \left (\begin{array}{*{10}c} (w {\ast} f_{x_{1}}^{2})(\boldsymbol{x}_{ 0}) &(w {\ast} f_{x_{1}}f_{x_{2}})(\boldsymbol{x}_{0}) \\ (w {\ast} f_{x_{2}}\,f_{x_{1}})(\boldsymbol{x}_{0})& (w {\ast} f_{x_{2}}^{2})(\boldsymbol{x}_{0}) \end{array} \right ), }$$

(3.4)

where w ∗ f denotes the convolution of w and f. The partial derivative of f with respect to some variable x _i is denoted by $f_{x_{i}}$. This reveals that $\boldsymbol{J}$ is a smoothed version of

$$\displaystyle{ \left (\begin{array}{*{10}c} f_{x_{1}}^{2}(\boldsymbol{x}) &f_{ x_{1}}(\boldsymbol{x})f_{x_{2}}(\boldsymbol{x}) \\ \,f_{x_{2}}(\boldsymbol{x})f_{x_{1}}(\boldsymbol{x})& f_{x_{2}}^{2}(\boldsymbol{x}) \end{array} \right ). }$$

(3.5)

The eigenvalues of the structure tensor are noted $\lambda _{\max }$ and $\lambda _{\min }$, with $\lambda _{\min },\lambda _{\max } \in \mathbb{R}$. They carry information about the distribution of the gradient within the window w. Depending on the eigenvalues, one can discriminate between homogenous regions, rotational symmetric regions without predominant direction, regions where the eigenvector is well aligned with one of the gradient directions, or regions where the dominant direction lies in between the gradient directions. For such purpose, two measures are defined, the so-called energy E and the coherence C. The energy is defined based on the eigenvalues of the structure tensor as $E = \vert \lambda _{1}\vert + \vert \lambda _{2}\vert$. If E ≈ 0, which corresponds to $\lambda _{\max } =\lambda _{\min } \approx 0$, then the region is homogenous. If $E \gg 0$, then the characteristic of the structure is determined by the coherency information. The coherency information C is a measure of confidence, defined as

$$\displaystyle{ 0 \leq C = \frac{\lambda _{\max } -\lambda _{\min }} {\lambda _{\max } +\lambda _{\min }} = \frac{\sqrt{(J_{22 } - J_{11 } )^{2 } + 4J_{12 }^{2}}} {J_{22} + J_{11}} \leq 1, }$$

(3.6)

where J _ij denotes an element of the structure tensor. If C ≈ 0, which corresponds to $\lambda _{\max } \approx \lambda _{\min }$, then the region is rotational symmetric without predominant direction, the structure has no orientation. If C ≈ 1, which corresponds to $\lambda _{\max } > 0,\lambda _{\min } \approx 0$ or $\lambda _{\max } \gg \lambda _{\min }$, the eigenvector is well aligned with one of the gradient directions. For 0 < C < 1, the predominant orientation lies between the gradient directions. In general, a coherency close to 1 indicates that the structure in the image is locally 1D, a coherency close to 0 indicates that there is no preferred direction.

The energy of the derivative in the direction $\boldsymbol{u}$ can be expressed as

$$\displaystyle{ \|\mathrm{D}_{\boldsymbol{u}}\,f\|_{w}^{2} =\langle \boldsymbol{ u}^{T}\boldsymbol{\nabla }f,\boldsymbol{u}^{T}\boldsymbol{\nabla }f\rangle _{ w} =\boldsymbol{ u}^{T}\langle \boldsymbol{\nabla }f,\boldsymbol{\nabla }f\rangle _{ w}\boldsymbol{u} =\boldsymbol{ u}^{T}\mathbf{J}\boldsymbol{u}. }$$

(3.7)

This means that, in the window centered around $\boldsymbol{x}_{0}$, the dominant orientation of the neighborhood can be computed by

$$\displaystyle{ \boldsymbol{u}_{1} =\arg \max _{\|\boldsymbol{u}\|=1}\|\mathrm{D}_{\boldsymbol{u}}\,f\|_{w}^{2}. }$$

(3.8)

We interpret $\|\mathrm{D}_{\boldsymbol{u}}\,f\|_{w}^{2}$ as the average energy in the window defined by w and centered at $\boldsymbol{x}_{0}$. Moreover, $\mathrm{D}_{\boldsymbol{u}}\,f =\langle \boldsymbol{ \nabla }f,\boldsymbol{u}\rangle$ is the derivative in the direction of $\boldsymbol{u}$. The maximizing argument corresponds to the eigenvector with the largest eigenvalue of the structure tensor at $\boldsymbol{x}_{0}$. The dominant orientation of the pattern in the local window w is computed as u ₁ = (cosθ, sinθ), with $\theta = \mbox{ $\frac{1} {2}$}\arctan \left ( \frac{2J_{12}} {J_{22}-J_{11}} \right ).$

Figure 3.2 illustrates the improved robustness of the structure tensor in terms of the estimation of the orientation. Figure 3.3 provides another concrete example on the structure-tensor analysis produced by the freely available OrientationJ plugin for Fiji/ImageJ.¹ We have chosen a HSB (hue, saturation, and brightness) cylindrical-coordinate color representation to visualize the results. The HSB components correspond to the following values: angle of the orientation, coherency, and input image, respectively. The advantage of the proposed model is that it gives a direct link between the quantities to display and the color coding. In the cylindrical-coordinate color representation, the angle around the central vertical axis corresponds to hue. The distance along the axis corresponds to brightness, thus we preserve the visibility of the original structures. The distance from the axis corresponds to saturation: the higher the coherency is, the more saturated the corresponding colors are.

Fig. 3.2

Illustration of the robustness of the structure tensor in terms of estimation of the orientation, from left to right: (1) Input image: confocal micrograph, same as the original image in Fig. 3.1. (2) Local dominant orientation, color-coded, no filtering applied. (3) Orientation given by the structure tensor with a small window size (standard deviation of the Gaussian window = 1). (4) Orientation given by the structure tensor large window size (standard deviation of the Gaussian window = 1). All the images were produced by the ImageJ/Fiji plugin OrientationJ

Fig. 3.3

Illustration of the use of structure tensors. Large images, from left to right: (1) Input image (800 × 800 pixels): immunofluorescence cytoskeleton (actin fibers), courtesy of Caroline Aemisegger, University of Zürich. (2) Coherency map: coherency values close to 1.0 are represented in white, coherency values close to 0.0 are represented in black. (3) Construction of color representation in HSB, H: angle of the orientation, S: coherency, B: input image. Small images in the left bottom corners, from left to right: (1) Input image: wave pattern with constant wavelength. (2) Coherency map: coherency values are close to 1.0 as expected. (3) The color representation reflects the different orientations. All the images were produced by the ImageJ/Fiji plugin OrientationJ

In the 3D shape estimation of DNA molecules from stereo cryo-electron micrographs (Fonck et al. 2008), the authors took advantage of its structure-tensor method. Other applications can be found in Köthe (2003) and Bigun et al. (2004).

While simple and computationally efficient, the structure-tensor method has drawbacks: it only takes into account one specific scale, the localization accuracy for corners is low, and the integration of edge and corner detection is ad hoc (e.g., Harris’ corner detector).

3.2.3 Higher-Order Directional Structures and the Hessian

To capture higher-order directional structures, the gradient information is replaced by higher-order derivatives. In general, an nth-order directional derivative is associated with n directions. Taking all of these to be the same, the directional derivative of order n in $\mathbb{R}^{2}$ is defined as

$$\displaystyle{ \mathrm{D}_{\boldsymbol{u}}^{n}\,f(\boldsymbol{x}) =\sum _{ k=0}^{n}{n\choose k}u_{ 1}^{k}u_{ 2}^{n-k}\partial _{ x_{1}}^{k}\partial _{ x_{2}}^{n-k}f(\boldsymbol{x}), }$$

(3.9)

which is a linear combination of partial derivatives of order n. More specifically, if we fix n = 2 and the unit vector $\boldsymbol{u}_{\theta } = (\cos \theta,\sin \theta )$, we obtain

$$\displaystyle{ \mathrm{D}_{\boldsymbol{u}_{\theta }}^{2}\,f(\boldsymbol{x}) =\cos ^{2}(\theta )\,\partial _{ x_{1}}^{2}f(\boldsymbol{x}) + 2\cos (\theta )\sin (\theta )\,\partial _{ x_{1}}\partial _{x_{2}}\,f(\boldsymbol{x}) +\sin ^{2}(\theta )\,\partial _{ x_{2}}^{2}\,f(\boldsymbol{x}).\quad }$$

(3.10)

The Hessian filter is a square matrix of second-order partial derivatives of a function. For example, in 2D, the smoothed Hessian matrix, useful for ridge detection at location $\boldsymbol{x}_{0}$, can be written as

$$\displaystyle{ \mathbf{H}(x_{0}) = \left (\begin{array}{*{10}c} (w_{11} {\ast} f)(\boldsymbol{x}_{0})&(w_{12} {\ast} f)(\boldsymbol{x}_{0}) \\ (w_{21} {\ast} f)(\boldsymbol{x}_{0})&(w_{22} {\ast} f)(\boldsymbol{x}_{0}) \end{array} \right ), }$$

(3.11)

where w is a smoothing kernel and $w_{\mbox{ ij}} = \partial _{x_{ \mbox{ i}}}\partial _{x_{ \mbox{ j}}}w$ denotes its derivatives with respect to the coordinates x _i and x _j. In the window centered around $\boldsymbol{x}_{0}$, the dominant orientation of the ridge is

$$\displaystyle{ \boldsymbol{u}_{2} =\arg \max _{\|\boldsymbol{u}\|=1}\left (\boldsymbol{u}^{T}\mathbf{H}\boldsymbol{u}\right ). }$$

(3.12)

The maximizing argument corresponds to the eigenvector with the largest eigenvalue of the Hessian at $\boldsymbol{x}_{0}$. The eigenvectors of the Hessian are orthogonal to each other, so the eigenvector with the smallest eigenvalue corresponds to the direction orthogonal to the ridge.

A sample application of the Hessian filter is vessel enhancement (Frangi et al. 1998). There, the authors define a measure called vesselness which corresponds to the likeliness of an image region to contain vessels or other image ridges. The vesselness measure is derived based on the eigenvalues of the steerable Hessian filter. In 2D, a vessel is detected when one of the eigenvalues is close to zero ($\lambda _{1} \approx 0$) and the other one is much larger $\vert \lambda _{2}\vert \gg \vert \lambda _{1}\vert$. The direction of the ridge is given by the eigenvector of the Hessian filter output corresponding to $\lambda _{1}$. In (Frangi et al. 1998), the authors define the measure of vesselness as

$$\displaystyle{ V (\boldsymbol{x}) = \left \{\begin{array}{@{}l@{\quad }l@{}} 0, \quad &\mbox{ if}\,\lambda _{1} > 0 \\ \exp \left (-\mbox{ $\frac{(\lambda _{1}/\lambda _{2})^{2}} {2\beta _{1}} $}\right )\left (1 -\exp \left (-\mbox{ $\frac{\lambda _{1}^{2}+\lambda _{ 2}^{2}} {2\beta _{2}} $}\right )\right ),\quad &\mbox{ otherwise}, \end{array} \right. }$$

(3.13)

where β ₁ and β ₂ control the sensitivity of the filter.² A particular application of the vesselness index on filament enhancement is shown in Fig. 3.4. Alternative vesselness measures based on the Hessian have been proposed by Lorenz et al. (1997) and Sato et al. (1998).

Fig. 3.4

Rotation-invariant enhancement of filaments. From top to bottom, left to right: (1) Input image (512 × 256 pixels) with neuron, cell body, and dendrites (maximum-intensity projection of a z-stack, fluorescence microscopy, inverted scale). (2) Output of the Hessian filter. The largest eigenvalue of the Hessian matrix was obtained after a Gaussian smoothing (standard deviation = 5). The image was produced using the ImageJ/Fiji plugin FeatureJ available at: http://www.imagescience.org/meijering/software/featurej/. (3) Output of the vesselness index obtained by the Fiji plugin Frangi-Vesselness. (4) Output of the steerable filters (Gaussian-based, 4th order). The image was produced using the ImageJ/Fiji plugin SteerableJ

3.3 Directional Multiscale Approaches

In natural images, oriented patterns are typically living on different scales, for example, thin and thick blood vessels. To analyze them, methods that extract oriented structures separately at different scales are required. The classical tools for a multiscale analysis are wavelets. In a nutshell, a wavelet is a bandpass filter that responds almost exclusively to features of a certain scale. The separable wavelet transform that is commonly used is computationally very efficient but provides only limited directional information. Its operation consists of filtering with 1D wavelets with respect to the horizontal and vertical directions. As a result, two pure orientations (vertical and horizontal) and a mixed channel of diagonal directions are extracted. Using the dual-tree complex wavelet transform (Kingsbury 1998),³ one can increase the number of directions to six while retaining the computational efficiency of the separable wavelet transform. [We refer to Selesnick et al. (2005) for a detailed treatment of this transform.] Next, we describe how to achieve wavelets with an even higher orientational selectivity at the price of higher computational costs.

Figure 3.5 illustrates the gradient at different scales. We can observe that different features are kept at different scales.

Fig. 3.5

Illustration of the gradient at different scales, from left to right: (1) Input image: confocal micrograph, same as the original image in Fig. 3.1. (2) Magnitude of the gradient at scale 1. (3) Magnitude of the gradient at scale 2. (4) Magnitude of the gradient at scale 4. All the images were produced by the ImageJ/Fiji plugin OrientationJ

3.3.1 Construction of Directional Filters in the Fourier Domain

In order to construct orientation-selective filters, methods based on the Fourier transform are powerful. From now on, we denote the Cartesian and polar representations of the same 2D function f by $f(\boldsymbol{x})$ with $\boldsymbol{x} \in \mathbb{R}^{2}$ and $f_{\mbox{ pol}}(r,\theta )$ with $r \in \mathbb{R}^{+}$, $\theta \in [0,2\pi )$ [similarly in the Fourier domain: $\hat{f} (\boldsymbol{\omega })$ and $\hat{f} _{\mbox{ pol}}(\rho,\phi )$]. The key property for directional analysis is that rotations in the spatial domain propagate as rotations to the Fourier domain. Formally, we write that

$$\displaystyle{ f(\mathbf{R}_{\theta }\boldsymbol{x})\stackrel{\mathcal{F}}{\longleftrightarrow }\hat{f} (\mathbf{R}_{\theta }\boldsymbol{\omega }), }$$

(3.14)

where $\mathbf{R}_{\theta }$ denotes a rotation by the angle $\theta$. The construction is based on a filter ψ whose Fourier transform $\hat{\psi }$ is supported on a wedge around the ω ₁ axis; see Fig. 3.6. In order to avoid favoring special orientations, one typically requires that $\hat{\psi }$ be nonnegative and that it forms (at least approximately) a partition of unity of the Fourier plane under rotation, like

$$\displaystyle{ \sum _{\theta _{i}}\vert \hat{\psi }(\mathbf{R}_{\theta _{i}}\boldsymbol{\omega })\vert ^{2} = 1,\mbox{ for all }\boldsymbol{\omega } \in \mathbb{R}^{2}\setminus \{\boldsymbol{0}\}. }$$

(3.15)

Here, $\theta _{1},\ldots,\theta _{n}$ are arbitrary orientations which are typically selected to be equidistant, with $\theta _{i} = (i - 1)\pi /n.$ To get filters that are well localized in the spatial domain, one chooses $\hat{\psi }$ to be a smooth function; for example, the Meyer window function (Daubechies 1992; Ma and Plonka 2010). A directionally filtered image $f_{\theta _{i}}$ can be easily computed by rotating the window $\hat{\psi }$ by $\theta _{i}$ and multiplying it with the Fourier transform $\hat{f}$ of the image, and by transforming back to the spatial domain. This is written

$$\displaystyle{ f_{\theta _{i}}(\boldsymbol{x}) = \mathcal{F}^{-1}\{\hat{\psi }(\mathbf{R}_{\theta _{ i}}\boldsymbol{\cdot })\hat{f}\}(\boldsymbol{x}). }$$

(3.16)

[We refer to Chaudhury et al. (2010) for filterings based on convolutions in the spatial domain.] The resulting image $f_{\theta _{i}}$ contains structures that are oriented along the direction $\theta _{i}.$ The local orientation $\theta$ is given by the orientation of the maximum filter response

$$\displaystyle{ \theta (\boldsymbol{x}) =\arg \max _{\theta _{i}}\vert \,f_{\theta _{i}}(\boldsymbol{x})\vert. }$$

(3.17)

Such directional filters have been used in fingerprint enhancement (Sherlock et al. 1994) and in crossing-preserving smoothing of images (Franken et al. 2007; Franken and Duits 2009).

Fig. 3.6

Top row (from left to right): schematic tilings of the frequency plane by Fourier filters, directional wavelets, curvelets, and shearlets (the origin of the Fourier domain lies in the center of the images); Middle row: a representative Fourier multiplier. Bottom row: Corresponding filtering result for the image of Fig. 3.1. The Fourier filter extracts oriented patterns at all scales whereas the wavelet-type approaches are sensitive to oriented patterns of a specific scale. Curvelets and shearlets additionally increase the directional selectivity at the finer scales

3.3.2 Directional Wavelets with a Fixed Number of Directions

Now we augment the directional filters by scale-selectivity. Our starting point is the radial windowing function of (3.15). The simplest way to construct a directional wavelet transform is to partition the Fourier domain into dyadic frequency bands (“octaves”). To ensure a complete covering of the frequency plane, we postulate again nonnegativity and a partition-of-unity property of the form

$$\displaystyle{ \sum _{s\in \mathbb{Z}}\sum _{\theta _{i}}\vert \hat{\psi }(2^{-s}\mathbf{R}_{\theta _{ i}}\boldsymbol{\omega })\vert ^{2} = 1,\mbox{ for }\boldsymbol{\omega } \in \mathbb{R}^{2}\setminus \{\boldsymbol{0}\}. }$$

(3.18)

Classical examples of this type are the Gabor wavelets that cover the frequency plane using Gaussian windows which approximate (rescaled) partition-of-unity (Mallat 2008; Lee 1996). These serve as model for the filters in the mammalian visual system (Daugman 1985, 1988). Alternative constructions are Cauchy wavelets (Antoine et al. 1999) or constructions based on the Meyer window functions (Daubechies 1992; Ma and Plonka 2010). We refer to Vandergheynst and Gobbers (2002) and Jacques et al. (2011) for further information on the design of directional wavelets. In particular, sharply direction-selective Cauchy wavelets have been used for symmetry detection (Antoine et al. 1999).

3.3.3 Curvelets, Shearlets, Contourlets, and Related Transforms

Over the past decade, curvelets (Candès and Donoho 2004), shearlets (Labate et al. 2005; Yi et al. 2009; Kutyniok and Labate 2012), and contourlets (Do and Vetterli 2005) have attracted a lot of interest. They are constructed similarly to the directional wavelets. The relevant difference in this context is that they increase the directional selectivity on the finer scales according to a parabolic scaling law. This means that the number of orientations is increased by a factor of about $\sqrt{ 2}$ at every scale or by 2 at every other scale; see Fig. 3.6. Therefore, they are collectively called parabolic molecules (Grohs and Kutyniok 2014). Curvelets are created by using a set of basis functions from a series of rotated and dilated versions of an anisotropic mother wavelet to approximate rotation and dilation invariance. Contourlets use a tree-structured filterbank to reproduce the same frequency partitioning as curvelets. Their structure is more flexible, enabling different subsampling rates. To overcome the limitations of the Cartesian grid (i.e., exact rotation invariance is not achievable on it), shearlets are designed in the discrete Fourier domain with constraints on exact sheer invariance.

These transforms are well suited to the analysis and synthesis of images with highly directional features. Applications include texture classification of tissues in computed tomography (Semler and Dettori 2006), texture analysis (Dong et al. 2015), image denoising (Starck et al. 2002), contrast enhancement (Starck et al. 2003), and reconstruction in limited-angle tomography (Frikel 2013). Furthermore, they are closely related to a mathematically rigorous notion of the orientation of image features, the so-called wavefront set (Candès and Donoho 2005; Kutyniok and Labate 2009). Loosely speaking, the wavefront set is the collection of all edges along with their normal directions. This property is used for the geometric separation of points from curvilinear structures, for instance, to separate spines and dendrites (Kutyniok 2012) and for edge detection with resolution of overlaying edges (Yi et al. 2009; Guo et al. 2009; Storath 2011b). We show in Fig. 3.7 the result of the curvelet/shearlet-based edge-detection scheme of Storath (2011b) which is obtained as follows: For every location (pixel) $\boldsymbol{b}$ and every available orientation $\theta$, the rate of decay $d_{\boldsymbol{b},\theta }$ of the absolute values of the curvelet/shearlet coefficients over the scale is computed. The reason for computing the rate of decay of the coefficients is their connection to the local regularity: the faster the decay rate, the smoother the image at location $\boldsymbol{b}$ and orientation $\theta$ (see Candès and Donoho 2005; Kutyniok and Labate 2009; Guo et al. 2009). We denote the curvelet/shearlet coefficients at scale a, location $\boldsymbol{b}$, and orientation $\theta$ by $c_{a,\boldsymbol{b},\theta }.$ Then, $d_{\boldsymbol{b},\theta }$ corresponds to the least-squares fit to the set of constraints $\vert c_{a,\boldsymbol{b},\theta }\vert = C'_{\boldsymbol{b},\theta }a^{d_{\boldsymbol{b},\theta }},$ where a runs over all available scales (in the example of Fig. 3.7, a = 2^−s∕3 with s = 0, …, 15). Note that this reduces to solving a system of linear equations in terms of $\log C'_{\boldsymbol{b},\theta }$ and $d_{\boldsymbol{b},\theta }$, after having taken a logarithm on both sides. Having computed d, we perform for each orientation $\theta$ a non-maximum suppression on d; that is, we set to $(-\infty )$ all pixels that are not a local maximum of the image $d_{\cdot,\theta }$ with respect to the direction $\theta.$ Finally, a threshold is applied and the connected components of the (3D-array) d are determined (and colored). The image displayed in Fig. 3.7 is the maximum-intensity projection of the three-dimensional image d with respect to the $\theta$ component.

Fig. 3.7

Edge detection with resolution of crossing edges using the curvelet transform. The colors correspond to connected edge segments. Note that crossing edges are resolved, for instance near the shoulder bones. [Original image courtesy of Dr. Jeremy Jones, Radiopaedia.org]

Relevant software packages implementing these transforms are the Matlab toolboxes CurveLab,⁴ ShearLab,⁵ FFST,⁶ and the 2D Shearlet Toolbox.⁷

3.4 Steerable Filters

For the purpose of detecting or enhancing a given type of directional pattern (edge, line, ridge, corner), a natural inclination is to try to match directional patterns. The simplest way to do that is to construct a template and try to align it with the pattern of interest. Usually, such algorithms rely on the discretization of the orientation. To obtain accurate results, a fine discretization is required. In general, Fourier filters and wavelet transforms are computationally expensive in this role because a full 2D filter operation has to be computed for each discretized direction. However, an important exception is provided by steerable filters, where one may perform arbitrary (continuous) rotations and optimizations with a substantially reduced computational overhead. The basics of steerability were formulated by Freeman and Adelson in the early 1990s (Freeman and Adelson 1990; Freeman 1992; Freeman and Adelson 1991) and developed further by Perona (1992), Simoncelli and Farid (1996), Unser and Chenouard (2013), Unser and Van De Ville (2010), Ward et al. (2013), and Ward and Unser (2014). We now explain the property of steerability and show the development of steerable wavelets.

A function f on the plane is steerable in the finite basis { f ₁, …, f _N } if, for any rotation matrix $\mathbf{R}_{\theta _{0}}$, we can find coefficients $c_{1}(\theta _{0}),\ldots,c_{N}(\theta _{0})$ such that

$$\displaystyle{ f(\mathbf{R}_{\theta _{0}}\boldsymbol{x}) =\sum _{ n=1}^{N}c_{ n}(\theta _{0})f_{n}(\boldsymbol{x}). }$$

(3.19)

It means that a function f in $\mathbb{R}^{2}$ is steerable if all of its rotations can be expressed in the same finite basis as the function itself. Thus, any rotation of f can be computed with a systematic modification (i.e., matrix multiplication) of the coefficients. The importance of this property is that, when doing pattern matching, it is enough to compute the coefficients only once, for one particular angle. Based on that, one can then easily determine the coefficients for arbitrary angles. A simple example of steerable functions are the polar-separable functions, whose amplitude (radial part) is 1 and angular part is $\cos (\theta )$ or $\sin (\theta )$. All rotations of these functions can be expressed in the basis $\{\sin (\theta ),\cos (\theta )\}$. The rotations of $\cos (\theta )$ and $\sin (\theta )$ by $\theta _{0}$ are determined by

$$\displaystyle{ \left (\begin{array}{*{10}c} \cos (\theta +\theta _{0}) \\ \sin (\theta +\theta _{0}) \end{array} \right ) = \left (\begin{array}{*{10}c} \cos (\theta _{0})&-\sin (\theta _{0}) \\ \sin (\theta _{0})&\phantom{+}\cos (\theta _{0}) \end{array} \right )\left (\begin{array}{*{10}c} \cos (\theta ) \\ \sin (\theta ) \end{array} \right ). }$$

(3.20)

Instead of setting the amplitude to 1, one can choose any nonvanishing isotropic function for the radial part. Also, replacing $\sin$ and $\cos$ with exponentials will preserve the property (since $\mathrm{e}^{\mathrm{j}(\theta +\theta _{0})} =\mathrm{ e}^{\mathrm{j}\theta _{0}}\mathrm{e}^{\mathrm{j}\theta }$).

The simplest examples of steerable filters are the ones that are based on the gradient or the Hessian. Starting from an isotropic lowpass function $\varphi (x_{1},x_{2})$, one can create a subspace of steerable derivative-based templates which can serve as basic edge or ridge detectors. In 2D, let $\varphi _{k,l} = \partial _{x_{1}}^{k}\partial _{x_{2}}^{l}\varphi$ be anisotropic derivatives of the isotropic function $\varphi$. By the chain rule of differentiation, for any rotation matrix $\mathbf{R}_{\theta _{ 0}}$, the function $\partial _{x_{1}}^{k}\partial _{x_{2}}^{l}\varphi (\mathbf{R}_{\theta _{0}}\cdot )$ can be written as a linear combination of $\varphi _{i,j}$ with i + j = k + l. Therefore, any anisotropic filter of the form

$$\displaystyle{ h(x_{1},x_{2}) =\sum _{ m=1}^{M}\sum _{ k+l=m}\alpha _{k,l}\varphi _{k,l}(x_{1},x_{2}) }$$

(3.21)

is steerable. Consequently, for any rotation matrix $\mathbf{R}_{\theta _{ 0}}$, an application of the rotated filter to an image f yields

$$\displaystyle{ (\,f {\ast} h(\mathbf{R}_{\theta }\cdot ))(\boldsymbol{x}) =\sum _{ m=1}^{M}\sum _{ k+l=m}\alpha _{k,l}(\theta )f_{k,l}(\boldsymbol{x}), }$$

(3.22)

where $f_{k,l} = f {\ast} \partial _{x_{1}}^{k}\partial _{x_{2}}^{l}\varphi$ and $\alpha _{k,l}(\theta )$ is a trigonometric polynomial in $\cos (\theta )$ and $\sin (\theta )$. Once every f _k, l is precomputed, the linear combination (3.22) allows us to quickly evaluate the filtering of the image by the anisotropic filter rotated by any angle. We can then,,steer” h by manipulating $\theta$, typically to determine the direction along which the response is maximized and across which is minimized.

Figure 3.4 contains the outcome of three derivative-based methods used to enhance filaments in a rotation-invariant way. The computation complexity is the same for the different methods, with approximatively the same number of filters with the same computation time. The directionality is best captured in the steerable case.

Jacob and Unser (2004) improved the basic steerable filters by imposing Canny-like criteria of optimality (Canny 1986) on the properties of the detector: reasonable approximation of the ideal detector, maximum signal-to-noise ratio, good spatial localization, and reduced oscillations. Their formalism boils down to a constrained optimization of the expansion coefficients α _k, i using Lagrange multipliers.

3.5 Steerable Multiscale Approaches

In Simoncelli and Freeman (1995), the authors proposed a new take on steerable filters: the steerable pyramid. The goal of their design was to combine steerability with a multiscale detection scheme. His pioneering work had many successful applications: contour detection (Perona 1992), image filtering and denoising (Bharath and Ng 2005), orientation analysis (Simoncelli and Farid 1996), and texture analysis and synthesis (Portilla and Simoncelli 2000). In Karssemeijer and te Brake (1996) multiscale steerable filters were involved in the detection of stellate distortions in mammograms. Classical multiresolution steerable methods use a purely discrete framework with no functional analytic counterpart. They are not amenable to extensions to dimensions higher than two. Fortunately, it is possible to address these limitations. In this section we overview a continuous-domain formulation that extends the technique proposed by Simoncelli and Freeman (1995). Multiresolution directional techniques were motivated by their invariance with respect to primary geometric transformations: translation, dilation, and rotation. Translation and dilation invariance is satisfied by the application of the wavelet transform. Rotation invariance is achieved by the Riesz transform, which also gives a connection to gradient-like signal analysis (Held et al. 2010).

3.5.1 The Riesz Transform

The complex Riesz transform was introduced to the literature by Larkin et al. (2001) and Larkin (2001) as a multidimensional extension of the Hilbert transform. The Hilbert transform is a 1D shift-invariant operator that maps all cosine functions into sine functions without affecting their amplitude (allpass filter). Expressed in the Fourier domain, the Hilbert transform of a function f is

$$\displaystyle{ \mathcal{F}\left \{\mathcal{H}\{\,f\}\right \}(\omega ) = -\frac{\mathrm{j}\omega } {\vert \omega \vert }\hat{f} (\omega ) = -\mathrm{j}\ \mbox{ sgn}(\omega )\hat{f} (\omega ). }$$

(3.23)

Similarly to the Hilbert transform, the Riesz transform is defined in the Fourier domain as

$$\displaystyle{ \mathcal{F}\left \{\mathcal{R}\{\,f\}\right \}(\boldsymbol{\omega }) = \frac{(\omega _{x} +\mathrm{ j}\omega _{y})} {\|\boldsymbol{\omega }\|} \hat{f} (\boldsymbol{\omega }) =\mathrm{ e}^{\mathrm{j}\phi }\hat{f} _{ \mbox{ pol}}(\rho,\phi ), }$$

(3.24)

where the subscript “pol” denotes the polar representation of the function. The transform is a convolution-type operator that also acts as an allpass filter, with a phase response that is completely encoded in the orientation.

The Riesz transform is translation- and scale-invariant since

$$\displaystyle{ \forall \boldsymbol{x}_{0} \in \mathbb{R}^{d},\quad \mathcal{R}\left \{\,f(\cdot -\boldsymbol{ x}_{ 0})\right \}(\boldsymbol{x}) = \mathcal{R}f(\boldsymbol{x} -\boldsymbol{ x}_{0}) }$$

(3.25)

$$\displaystyle{ \forall a \in \mathbb{R}^{+}\setminus \{0\},\quad \mathcal{R}\left \{\,f\left ( \frac{\cdot } {a}\right )\right \}(\boldsymbol{x}) = \mathcal{R}f\left (\frac{\boldsymbol{x}} {a}\right ). }$$

(3.26)

The Riesz transform is also rotation-invariant.

The nth-order complex 2D Riesz transform $\mathcal{R}^{n}$ represents the n-fold iterate of $\mathcal{R}$, defined in the Fourier domain as

$$\displaystyle{ \mathcal{F}\left \{\mathcal{R}^{n}\{\,f\}\right \}(\rho \cos \phi,\rho \sin \phi ) =\mathrm{ e}^{\mathrm{j}n\phi }\hat{f} _{ \mbox{ pol}}(\rho,\phi ). }$$

(3.27)

The nth order Riesz transform decomposes a 2D signal into n + 1 distinct components. It inherits the invariance properties of the Riesz transform since they are preserved through iteration. This means that we can use the Riesz transform to map a set of primary wavelets into an augmented one while preserving the scale- and shift-invariant structure.

3.5.2 Connection to the Gradient and Directional Derivatives

In this section, we describe the connection between the Riesz transform, the directional Hilbert transform, the gradient, and the directional derivatives. Assuming a zero-mean function f, the Riesz transform is related to the complex gradient operator as

$$\displaystyle{ \mathcal{R}f(x_{1},x_{2}) = -\mathrm{j}\left ( \frac{\partial } {\partial x_{1}} +\mathrm{ j} \frac{\partial } {\partial x_{2}}\right )(-\Delta )^{-1/2}f(x_{ 1},x_{2}). }$$

(3.28)

Here, $(-\Delta )^{\alpha },\alpha \in \mathbb{R}^{+}$ is the isotropic fractional differential operator of order 2α. Conversely, the corresponding fractional integrator of order 2α is $(-\Delta )^{-\alpha },\alpha \in \mathbb{R}^{+}$. The value α = 1∕2 is of special interest, providing the link between the Riesz transform and the complex gradient operator. The integral operator acts on all derivative components and has an isotropic smoothing effect, thus, the Riesz transform acts as the smoothed version of the image gradient.

Assuming a zero-mean function f, the high-order Riesz transform is related to the partial derivatives of f by

$$\displaystyle{ \mathcal{R}^{n}f(x_{ 1},x_{2}) = (-\Delta )^{-\frac{n} {2} }\sum _{n_{ 1}=0}^{n}{n\choose n_{ 1}}(-\mathrm{j})^{n_{1} }\partial _{x_{1}}^{n_{1} }\partial _{x_{2}}^{n-n_{1} }f(x_{1},x_{2}). }$$

(3.29)

The fractional integrator acts as an isotropic lowpass filter whose smoothing strength increases with n. The Riesz transform captures the same directional information as derivatives. However, it has the advantage of being better conditioned since, unlike them, it does not amplify the high frequencies.

The directional Hilbert transform is the Hilbert transform along a direction $\boldsymbol{u}$. It is related to the Riesz transform by

$$\displaystyle{ \mathcal{H}_{\boldsymbol{u}_{\theta }}f(\boldsymbol{x}) =\cos \theta f_{1}(\boldsymbol{x}) +\sin \theta f_{2}(\boldsymbol{x}), }$$

(3.30)

where $f_{1} =\mathop{ \mathrm{Re}}\nolimits (\mathcal{R}f)$ and $f_{2} =\mathop{ \mathrm{Im}}\nolimits (\mathcal{R}f)$ are the real and imaginary parts of $\mathcal{R}f$. Assuming again a zero-mean function f, the directional Hilbert transform is related to the derivative in the direction $\boldsymbol{u}$ by

$$\displaystyle{ \mathcal{H}_{\boldsymbol{u}}\,f(\boldsymbol{x}) = -(-\Delta )^{-\frac{1} {2} }\mathrm{D}_{\boldsymbol{u}}\,f(\boldsymbol{x}). }$$

(3.31)

Here, the operator $\mathrm{D}_{\boldsymbol{u}}$ is the one defined in (3.1). This result corresponds to the interpretation that the Hilbert transform acts as a lowpass-filtered version of the derivative operator. The n-fold version of the directional Hilbert transform acting on a zero-mean function f along the direction specified by $\boldsymbol{u}$ can be expressed in terms of the partial derivatives of f as

$$\displaystyle{ \mathcal{H}_{\boldsymbol{u}}^{n}f(\boldsymbol{x}) = (-1)^{n}(-\Delta )^{-\frac{n} {2} }\mathrm{D}_{\boldsymbol{u}}^{n}\,f(\boldsymbol{x}). }$$

(3.32)

3.5.3 Steerable Wavelets

In this section, we present the construction of steerable wavelet frames that are shaped to capture the local orientation of features in images within a multiresolution hierarchy. Their construction has two main parts: first, generation of circular harmonic wavelet frames by applying the multiorder complex Riesz transform on a bandlimited isotropic mother wavelet; second, shaping of the wavelet frames to a particular desired profile with an orthogonal transform.

We start our construction of steerable wavelet frames from a bandlimited isotropic mother wavelet in $L_{2}(\mathbb{R}^{2})$, denoted by ψ, whose shifts and dilations form a wavelet frame (e.g., Simoncelli’s wavelet). This isotropic wavelet at scale s and grid point (location) $\boldsymbol{x}_{0} = 2^{s}\boldsymbol{k}$, $\boldsymbol{k} \in \mathbb{Z}^{2}$ (in 2D) takes the form of

$$\displaystyle{ \psi _{s}(\boldsymbol{x} -\boldsymbol{ x}_{0}) = \frac{1} {2^{s}}\psi \left (\mbox{ $\frac{\boldsymbol{x}-\boldsymbol{x}_{0}} {2^{s}} $}\right ) = \frac{1} {2^{s}}\psi \left (\mbox{ $ \frac{\boldsymbol{x}} {2^{s}}$} -\boldsymbol{ k}\right ). }$$

(3.33)

We then apply the multiorder complex Riesz transform on $\psi _{s}(\cdot -\boldsymbol{ x}_{0})$. The transform preserves the frame properties. Thus, by choosing N distinct values for the integer n (distinct set of harmonics), one can form a frame of steerable wavelets, referred to as circular harmonic wavelets (Jacovitti and Neri 2000). An nth-order harmonic wavelet has a rotational symmetry of order n around its center, corresponding to the nth-order rotational symmetry of $\mathrm{e}^{\mathrm{j}n\phi }$. The illustration of circular harmonics for order three is presented in Fig. 3.8 (i). An application of circular harmonic wavelets on local symmetry detection is presented in Püspöki and Unser (2015).

Fig. 3.8

From left to right: (1) Circular harmonic wavelets of order three, real part, imaginary part, and magnitude; (2) steerable wavelets (different shapes, magnitude); (3) detections in synthetic data exhibiting multiple scales; (4) original image and detections in a micrograph of embryonic stem cells

3.5.4 Detection of Junctions

An important step in the analysis of microscopic images is the detection of key points, or junctions of coinciding branches. The automatic detection of these junctions can facilitate further image-processing tasks such as cell segmentation, counting of cells, or image statistics. M-fold symmetric structures (including the case of ridges, assimilated to twofold symmetries) are present in filaments, fibers, membranes, or endothelial cells (e.g., in the eyes). The difficulty in the detection of these junctions is twofold. First, they can appear at arbitrary orientation and scale; second, biological micrographs are frequently contaminated by local variations in intensity and measurement noise. With the modified wavelet schemes presented above, one can design an efficient detector of the location and orientation of local symmetries.

From the circular harmonic wavelet representation, one constructs new steerable representations by using an orthogonal shaping matrix U to define new steerable-frame functions (Unser and Chenouard 2013). The role of the shaping matrix U is to give the wavelet functions a desired angular profile. The transform can be formulated as

$$\displaystyle{ \left [\begin{array}{c} \xi _{s,\boldsymbol{k}}^{(n_{1})} \\ \vdots \\ \xi _{s,\boldsymbol{k}}^{(n_{N})} \end{array} \right ] = \mathbf{U}\left [\begin{array}{c} \psi _{s,\boldsymbol{k}}^{(1)}\\ \vdots \\ \psi _{s,\boldsymbol{k}}^{(N)} \end{array} \right ], }$$

(3.34)

where $\psi _{s,\boldsymbol{k}}^{(n)} = \mathcal{R}^{n}\psi _{s,\boldsymbol{k}}$ is the nth-order circular harmonic wavelet at scale s and location $\boldsymbol{k}$, U is an orthonormal transformation matrix, and {$\xi _{s,\boldsymbol{k}}^{(n)}$} are the new wavelet channels at scale s and location $\boldsymbol{k}$. The number of channels corresponds to the number of harmonics. The new wavelets span the same space as the wavelet frame $\psi _{s,\boldsymbol{k}}^{(n)}$.

Specific detectors are designed by defining the right weights for U. Typically, the process involves an optimization problem, either in the space or in the Fourier domain. The list of coefficients to optimize can be expressed as a vector that takes the form $\boldsymbol{u} = (u_{1},\ldots,u_{N})$, with $\boldsymbol{u}\boldsymbol{u^{\mathrm{H}}} = 1$. One can specify a quadratic energy term to minimize in the space domain like

$$\displaystyle{ E[\xi ] = \mbox{ $\frac{1} {2\pi }$}\int _{0}^{\infty }\int _{ -\pi }^{\pi }\vert \xi (r,\theta )\vert ^{2}v(\theta )r\mathrm{d}\theta \mathrm{d}r, }$$

(3.35)

or (for symmetric patterns) in the Fourier domain like

$$\displaystyle{ E[\hat{\xi }] = \frac{1} {2\pi }\int _{0}^{\infty }\int _{ -\pi }^{+\pi }\vert \hat{\xi }(\rho,\phi )\vert ^{2}v(\phi )\rho \mathrm{d}\phi \mathrm{d}\rho. }$$

(3.36)

The angular weighting function $v(\theta ) \geq 0$ or v(ϕ) ≥ 0 should have minima on the unit circle that enforces the concentration of the energy along the desired pattern (for instance, symmetric shape or T-shape). Minimizing E will thus force the solution $\xi$ to be localized at the corresponding angles. Once the mother wavelet $\xi$ is found, its translates and dilates naturally share the same optimal angular profile around their center. By expanding $\xi$ as $\sum _{n}u_{n}\psi ^{(n)}$ and imposing unit norm on $\boldsymbol{u}$, this formalism leads to a quadratic optimization problem with quadratic constraints that can be solved through eigen decomposition. In Fig. 3.8 (ii) are shown the amplitude of three different detectors that one can design with the proposed method: corner, symmetric threefold junction, and T-junction detector. Key points in the image correspond to maxima in the response of the wavelet detector. The optimal steering angle can be determined by root finding, as presented in (Püspöki et al. 2013, Sec. 4.1). The rest of the detection scheme is achieved by traditional techniques which may combine the results at different scales, local maxima search, thresholding, among others. Detections in synthetic and microscopic data is visualized are Fig. 3.8 (iii), and (iv), respectively.

The construction presented here makes it possible to capture the local orientation of features in an image within a multiresolution hierarchy Püspöki et al. (2016). The relation between the Riesz transform and steerable filters is studied in (Felsberg and Sommer 2001). The properties of steerable filters using low-order harmonics are analyzed in Koethe (2006). The extension of the steerable wavelet design based on the Riesz transform for higher dimensions, along with potential biomedical applications, are presented in Chenouard and Unser (2012). Application of steerable wavelets in texture learning for the analysis of CT images are presented in Depeursinge et al. (2015, 2014b).

3.6 Conclusion and Outlook

We have presented a survey on the directional analysis of bioimages. We have discussed the benefits and drawbacks of classical gradient-based methods, directional multiscale transforms, and multiscale steerable filters. From a user perspective, we have identified steerable wavelets and shearlets as the most attractive methods. They unify high directional selectivity and multiscale analysis, which allows the processing of oriented patterns at different scales. There exist computationally efficient implementations of such schemes that are available for the public. Finally, they are still an active field of research, see the recent papers (Kutyniok 2014; Bodmann et al. 2015; Duval-Poo et al. 2015; Kutyniok et al. 2014) for shearlets and (Ward and Unser 2014; Pad et al. 2014; Depeursinge et al. 2014a; Dumic et al. 2014; Schmitter et al. 2014) for steerable filters and wavelets. Furthermore, the corresponding user packages and plugins are maintained and continuously extended.

The state of the art in the field will need to be adjusted to fulfill the upcoming needs of biomedical and biological imaging. Advances in microscopy and in some other measurement systems (CT, X-ray) will shape the future of research. Currently, microscopes are already routinely producing and storing large datasets (often several GBs per measurement) that have to be handled in a fast and efficient way. Moreover, the need to process spatio-temporal data (2D/3D images over time) is becoming unavoidable and is going to require the proper extension of current filter-based schemes. Along with efficiency, the robustness, the precision, and the depth of the extracted information can be improved. Another promising direction of future research is the recovery of directional phase information using complex-valued wavelet transforms such as the monogenic wavelets (Felsberg and Sommer 2001; Olhede and Metikas 2009; Held et al. 2010; Unser et al. 2009; Storath 2011a; Soulard et al. 2013; Häuser et al. 2014; Heise et al. 2014). Preliminary applications include equalization of brightness (Held et al. 2010), detection of salient points (Storath et al. 2015), enhancement of anisotropic structures in fluorescence microscopy (Chenouard and Unser 2012), and texture segmentation (Storath et al. 2014). Image-analysis tools based on monogenic wavelets are provided by the ImageJ/Fiji plugins MonogenicJ⁸ and Monogenic Wavelet Toolbox.⁹ A further possible direction is the extension of directional wavelet transforms to nonuniform lattices such as polar grids or general graphs (Hammond et al. 2011; Shuman et al. 2013; Sandryhaila and Moura 2013).