Automatic Segmentation of Neurons from Fluorescent Microscopy Imaging

Abstract

Automatic detection and segmentation of neurons from microscopy acquisition is essential for statistically characterizing neuron morphology that can be related to their functional role. In this paper, we propose a combined pipeline that starts from the automatic detection of the soma through a new multiscale blob enhancement filtering. Then, a precise segmentation of the detected cell body is obtained by an active contour approach. The resulted segmentation is used as initial seed for the second part of the approach that proposes a dendrite arborization tracing method.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

Thanks to the great advances in microscopy technologies and cellular imaging, we have many tools and techniques allowing to address fundamental questions in neuron studies. We can capture high-resolution images of single cells or neuron population that enable neurobiologists to investigate the neuronal structure and morphological development associated to neurological disorders.

Recent studies [1] claim that the morphology (i.e., size, shape and dendritic arborization) of a neuron is a key discriminant of its functional role. With different morphological features and different functional tasks, distinct neurons have diverse soma shapes and dendrite arborizations. Developmental abnormalities might lead to neuron malfunction and can be early signals for variety of neuropathies and neurological disorders.

To support neuroscientists in this study, fully automated tools for neuron detection and segmentation are required. Different approaches have been proposed [2, 3], however still to date, the state of the art is not still satisfactory given the complexity of the problem. For example, the manual interaction that some tools require [4] is time consuming, expensive and extremely subjective as it depends on the user expertise and diligence.

Moreover the task is quite complex for different reasons. First of all, neuronal samples are highly heterogeneous across different acquisitions. Images can be characterized by high cell density and shape variety and usually there is a low contrast at the neuron boundaries. Indeed, the fluorescence expressed is commonly non-uniform: it might present high intensity variability between soma, its border and the processes, leading to bad morphological segmentation of cell and significant fragmentation in dendrite appearance.

Traditional segmentation approaches that use basic method such as thresholding and morphological operators are not precise enough for the task at hand and lead to wrong segmentations. Learning approaches, nowadays broadly used in object segmentation also thanks to Deep Learning, are not suitable for these images because they require a huge amount of hand-labeled neuron samples for training [5].

Among deformable models, active contour models have demonstrated good performance in segmentation also in correspondence of challenging data [6, 7]. Their main issue is the high sensitivity to the initialization, which often requires user setting. To this aim, recent active contour models trying an hybrid approach to automate the initial mask have been introduced [8, 9].

Skeletonization is a global technique that extract the binary skeleton from a given neuronal structure [10, 11]. The key idea of these methods is an iterative erasure of voxels from the volume of the segmented object preserving the topology of the contained structure. Minimal path based tracing are other global approaches which aim at linking seed points through an optimization problem [12] or through Fast Marching algorithm [13]. Minimum Spanning Tree (MST) tracing deals with the link between detected points into a tree representation [14].

In our work, we present a combined and fully automatic approach in order to detect and segment the whole neuron. The first part [15] starts with soma detection using a multiscale blob enhancement filtering. Then, neuron bodies are segmented by an high performance active contour model. The resulting segmentation is used to initialize the second part of the method, concerning dendrite tree segmentation by hessian-phase based level set model.

The remainder of the paper is organized as follows. In Sect. 2 details on the adopted dataset are provided. We present the combined method in Sect. 3: for the detection and segmentation of cell bodies see Sect. 3.1 and for the dendritic tracing see Sect. 3.2. In Sect. 4 results are presented and conclusions are provided in Sect. 5.

Fig. 1.

Some images containing Retinal Ganglion Cells (RGCs) used for testing the proposed method [15]. The images show high variability across samples. In the bottom line, there is some magnified crops of the upper images, showing the complexity of images, where the analyzed structures are mixed with background and other structures.

Fig. 2.

Some samples from the second dataset imaged single neurons. Images shows, also for this dataset, the heterogeneity across samples and across microscopic acquisitions. In the bottom line there are some details of the correspondent top line images.

2 Materials

In this work, we use two different datasets: Mouse Retina [15] and Larva Drosophila [16]. Mouse Retina dataset shows populations of Retinal Ganglion Cells (RGCs) which play a central role into the complex and stratified structure of the retina. Retina samples were imaged using Leica SP5 upright confocal microscope. Images were acquired at (sub)cellular resolution and at high averaging number to reduce the noise level due to the limitation of light penetration in deep layers of the tissue where RGCs are located. A total of 5 images ($2048\,\times \,2048$ and $1024\,\times \,1024$ pixels), showing some hundreds of cells each, were selected from 3 distinct retina samples including: (i) three images obtained from samples with genetic fluorescence expression, (i.e., Im1 from PV-EYFP and Im2 and Im5 images from Thy1-EYFP staining), and (ii) two images from samples with immunofluorescence staining using the Calretinin calcium-binding protein (Im3 and Im4) (Figs. 1, 2 and 3). The samples were selected to best capture the variability in terms of fluorescence expression, cell and axonal bundle density and background.

Larva Drosophila dataset is acquired on some sensory neurons in wild-type larva Drosophila [16] studied over different development phases. This dataset shows single neurons including both cell body and dendritic tree. In our case, we study the 2D maximum projection value on xy plane across slices. Figure 2 shows some 2D maximum projection of the considered volumes.

Fig. 3.

Pipeline applied to two examples (from the top, Im1 (PV-EYFP) and Im2 (Thy1-EYFP)) with a crop in the central row, showing the difficulties caused by contiguous cells [15]. In column, starting from the left side: Original Fluorescent Microscopy Images; Results of the multiscale blob filter binarization; Results of the active contour segmentation in blue transparency over the original image for getting the suitable qualitative performance; Results of the watershed transform and of the final threshold. (Color figure online)

3 Method

To automatically study neuron morphology, we need to detect and segment the cell body(i.e. the soma) and trace its dendritic arborization. In this work, we describe the sequence of methods we used to solve the soma detection and segmentation task and the dendrite tracing.

3.1 Soma Detection and Segmentation

The pipeline for cells detection and segmentation is composed of three main steps. As shown in Fig. 3 a Blob-based Filtering (second column) is followed by an Active Contour (third column) and a Watershed Transform (last column).

The Multiscale Blob Enhancement Filtering is used to identify regions where neuronal cells are likely located. After the blob filtering approach, blob-shapes objects are binarized and used as initial ROIs for a Localizing Region-Based Active Contour [17] that traces cell borders. Due to the fuzzy cell boundaries and occlusions, multiple cells can be segmented all together as a unique entity. To cope this issue, we apply a watershed transform.

Blob-Based Filtering. The Multiscale Blob Enhancement Filtering improves the intensity profile of cell bodies and reduces the contribution of dendritic and axonal structures. The eigenspace of the Hessian matrix $\mathcal {H}$ is analyzed at multiscale to determine the local likelihood that a pixel belongs to a cell, i.e. to a blob profile. The proposed approach is inspired by the work of Frangi et al. [18] that defines a multiscale vessel enhancement filtering. The Frangi filter basically depends on the orientational difference or anisotropic distribution of the second-order derivatives to delineate tubular and filament-like structures.

For 2D images, the formulation of Frangi is defined as follows:

$$\begin{aligned} \mathcal {V}_{s}(x_o) = {\left\{ \begin{array}{ll} 0, \quad \text { if } \lambda _1^{x_o, s} < 0\\ e^{-\frac{1}{2{\upbeta }^2} \cdot \left( \frac{\lambda _2^{x_o, s}}{\lambda _1^{x_o, s}}\right) ^2} \cdot \left( 1 - e^{-\frac{\Lambda ^2}{2\upgamma ^2}} \right) , \quad \text {otherwise} \end{array}\right. } \end{aligned}$$

(1)

where $\lambda _1^{x_o, s}$ and $\lambda _2^{x_o, s}$ are the eigenvalues of the Hessian matrix at point $x_o$ and at scale s, $\upbeta$ and $\upgamma$ are two thresholds which controls the sensitivity of the line filter and S is defined as $\Lambda = \Vert \mathcal {H} \Vert _F = \sqrt{\sum _{j \le D}\lambda _j^2}$, where D is the dimension of the image. This measure is used for differing the structures from the background.

Starting from Frangi’s idea, we modify the filtering to filter-out line-like patterns in favor of blob-like structures (as [19]).

Instead of a vesselness measure, we define a blobness measure as follows [15]:

$$\begin{aligned} \mathcal {B}_{s}(x_o) = {\left\{ \begin{array}{ll} 0, \quad \text {if } \lambda _1^{x_o, s} < 0\\ e^{\frac{1}{2\upbeta ^2} \cdot \left( \frac{\lambda _2^{x_o, s}}{\lambda _1^{x_o, s}}\right) ^2}, \quad \text {otherwise} \end{array}\right. }\end{aligned}$$

(2)

where $\lambda _1^{x_o, s}$ and $\lambda _2^{x_o, s}$ are the eigenvalues of the Hessian matrix at point $x_o$ and at scale s. $\upbeta$ is a threshold which controls the sensitivity of the blob filter. In our experiments, both $\upbeta$ and the Hessian scale have been selected as the average neuron radius. Equation (2) computes the blobness in the case of bright objects over dark background. In case of dark structures, system conditions should be reversed.

The filter is computed at a multiscale level. The response of the blob filter would be maximum at scale s that is more suited to the diameter of the blob to detect. Our blob enhencement filtering is said multiscale because we combine the blob measure at different scales to obtain a final blobness estimation defined as:

$$\begin{aligned} \mathcal {B}_(x_o) = \max _{s_{min} \le s \le s_{max}} \mathcal {B}_{s}(x_o) \end{aligned}$$

(3)

where $s_{min}$ and $s_{max}$ are the minimum and maximum scales where we expect to find structures.

Active Contour. Within active contour models, we exploited Localizing region-based active contour [17], an improved version of traditional ones [6, 7]. The advantage of the proposed model is that objects characterized by heterogeneous statistics can be successfully segmented thanks to localized energies, where, instead, the corresponding global AC would fail. This approach allows to remove the assumption that foreground and background regions are distinguishable based on their global statistics. Indeed the improving hypothesis is that interior and exterior areas of objects are locally different. Within this framework, the energies are constructed locally at each point along the curve in order to analyze local regions. The localization radius is chosen following the size range of the objects to be segmented. In our case, for each image, we defined a radius equal to the average soma radius, depending on the image size and on the microscope lens used for the acquisition.

Thanks to this efficient technique, we obtain a segmentation mask which tightly fits real cell bodies.

Fig. 4.

Some cells are not easily visible to the human eye just visualizing the retina images, but they are discovered and segmented by our algorithm (for example, in this cropped figure, pink and blue cells were hardly detectable). Adding contrast to the image makes these somata clearer but it increases noise and cell heterogeneity [15]. (Color figure online)

Watershed Transform and Size Filter. Active contour can fail to separate groups of overlapping or contiguous cells. So, we exploit the simplicity and computational speed of the watershed transform, introduced by Beucher and Lantuéjoul [20], to split cells englomerates into groups of cells.

As a final step, we delete components which are too small or too large for being cell bodies (a given example is in Fig. 3, first line) applying a size filter to remove structures with size outside an acceptable range of soma dimensions. It is possible to fix this range by a statistical analysis of the dimensions removing the tails of the distribution.

Fig. 5.

Some example images from Larva Drosophila dataset. In column, from the left side: 2D projection of the original volume; soma detection and segmentation applying the first part of the proposed approach; whole neuron segmentation including dendrites.

3.2 Dendrite Segmentation

To reconstruct the dendrites, we exploit the soma segmentation as initialization seed to start a level set propagation with local phase and with hessian eigenspace information. The main idea is that local phase is extracted using quadrature filters and this allows to distinguish lines and edges in a image [21, 22]. In our case study, a dendrite can appear locally as a line or as an edge pair; then a multiple scale integration is used for capturing information about dendrites of varying width and contrast. Our novel idea is weighting this filter by the Hessian eigenspace that guarantees that only pixel belonging to filamentous structures contributes [18]. In particular we modify Eq. (4) in [21] weighting the evolution term with the first eigenvalue. The new evolving equation becomes:

$$\begin{aligned} \frac{\partial \Phi }{\partial t} \quad = \quad - | \lambda _1 | \,\, Re (\hat{q}(\sigma )) \,\, | \nabla \Phi | \,\, + \,\, \alpha k |\nabla \Phi | \end{aligned}$$

(4)

where $\lambda _1$ is the first eigenvalue computed in each pixel by the Hessian Matrix, $\hat{q}$ is the normalized phase function, $\alpha$ is a regularizer and k is the mean curvature. With this contribution, the background signal is omitted and $\lambda _1$ drives the level set only where the pixels belong to a structure. The result is a “local” filter which can drive a contour towards the dendrite arborization (see Fig. 7).

4 Results and Discussion

For the evaluation of soma detection and segmentation, we applied our pipeline to two different datasets, Mouse Retina [15] and Larva Drosophila [16]. For the evaluation of dendrite segmentation, we use Larva Drosophila dataset because only this one contains images with the complete dendritic tree labeled for each neuron. As previously described in Sect. 2, the first dataset, Mouse Retina, is composed of 5 different retinal images representative of possible variations on the retinal samples, such as brightness, intensity, size and number of cells, presence of axonal structures and processes, strong background signals, etc. These samples show images at the network scale of many dozens of RGCs with higher fluorescence expression into the soma. We generated the ground truth manually segmenting all cells in each image (around 280 cells in total). The second dataset, Larva Drosophila contains images made of 11 single neurons representative of spatially inhomogeneous signal-to-noise ratios. Also in this case, we manually segmented all the neurons (both soma and dendrites). For the gold dendritic tracing, we adopted Simple Neurite Tracer [4].

Fig. 6.

Variation of the $\%$ of detected cells in Mouse Retina dataset as a function of the $\%$ threshold of overlap between detected cell and the corresponding annotated ground truth [15].

4.1 Soma Detection and Segmentation Results

To give a qualitative evaluation, we report different examples of Mouse Retina in Figs. 3 and 4 and of Larva Drosophila in Fig. 5 (central column) where it is possible to see that our approach works in different sample conditions.

To quantify the performance of our method, we adopt the Dice Coefficient (DC), a widely used overlapped metric for comparing two segmentation. DC is defined as follows:

$$ DC = \frac{ 2 (A \cap B) }{(A +B)},$$

where A is the binary ground truth mask and B is the binary segmentation result. The DC value ranges between 0 (absence of agreement) and 1 (perfect agreement). A DC higher than 0.70 usually corresponds to a satisfactory segmentation [23].

Table 1. Results for soma segmentation on Mouse Retina samples [15]. Dice Coefficient is computed for all steps in the pipeline (Blob Filter, Active Contour and Watershed Transform) and it shows improvements after each step. For the final stage of the pipeline, there is also the percentage of detected cells computed assuming as detected a cell with minimum overlap $50 \%$ with ground truth fixed at $50\%$.

Table 2. Soma segmentation results on Larva Drosophila dataset. Dice Coefficient has been computed for each segmented soma.

Table 3. Dice Coefficient has been computed comparing our segmentation and Tuff segmentation with manual segmentation done by Simple Neurite Tracer [4].

Table 1 shows the quantitative results on our Mouse Retina samples. We compute the DC for each of the three steps in the pipeline (Blob-based Filtering, Active Contour and Watershed Transform). Each stage clearly improves the segmentation, reaching satisfactory results for all images. In Im3 (Fig. 1), the fluorescence is mainly expressed by the body cells; for this reason, we reach good scores right after the first two steps. The weaker DC values on images Im2 and Im5 are due to a strong presence of axonal structures which can be hardly removed. As an additional index of performance at the network scale, we also present the percentage of detected cells for each Mouse Retina image. Figure 6 shows the variation of the percentage of detected cells at different thresholds of overlapping between computer-aided segmentation with the ground truth to count a cell as detected. It can be observed that $50{\%}$ threshold is a good trade off between the certainty of a cell detection and a satisfactory retrieval. So, in Table 1, we consider a cell as detected if it is correctly segmented for more than $50\%$ of its total area, comparing the segmentation mask to the ground truth for each annotated RGC.

Fig. 7.

An example of the level set evolution starting from the soma segmentation as seed point in the Sample $\# 1$. The level set is shown at different evolution steps.

Table 2 reports the quantitative evaluation on the Larva Drosophila dataset. Worst values are obtained for Sample $\# 4$ (see Fig. 2, middle column) and $\# 6$, where background noise is strong and leads to confusing borders. In general, however, the values are significantly high with an average reaching 0.88.

4.2 Dendrite Segmentation Results

A qualitative evaluation of the dendrite segmentation starting from seed soma is shown in Fig. 7 (bottom right) and in Fig. 5. In particular Fig. 7 shows an example of level set initialization, evolution and result; Fig. 5 proposes some Larva Drosophila image results after the first part of the segmentation process (middle column) and at the final segmentation (column on the right side).

To quantitatively evaluate our neuron segmentation, we compare our method with a recent state-of-the-art automated approach proposed in [24], Tubularity Flow Field (Tuff). Tuff is a technique for automatic neuron segmentation that performs directional regional growing guided by the tubularity direction of neurites. We compute the DC on Larva Drosophila results for both methods and it can be observed that our method significantly outperform Tuff (Table 3).

5 Conclusion

We have presented a new approach for whole neuron segmentation in challenging fluorescent microscopy images. Our method is comprehensive of two main steps: soma detection and dendrite segmentation. In the first stage cell bodies are detected by a new blob filtering approach and segmented by an active contour model and a watershed transform. Then, a novel hessian-phase based level set has been developed allowing to segment the whole neuron morphology. Tests have been performed on large scale and single scale images. We obtained high results for both detection and segmentation of the soma and for the whole neuron reconstruction. Thanks to its generality and automation, this framework could be applied to similar images and it is easily reproducible for the full network reconstruction at the population level. Moreover, we could easily extend the method to 3D dimensions since our theoretical model adopted a general dimension formulation. Finally, it also opens new perspectives for the analysis and the characterization of neuronal cells.