1 Introduction

An annotated virtual 3D mouse brain populated with accurate neuronal reconstruction from In-Situ Hybridization (ISH) images is important for brain circuit mapping research [8, 14, 19]. The ability to reconstruct such virtual brain models and perform quantitative analysis on them requires automatic registration of thin, high-resolution, artifact-free mouse brain slice images [14]. However, brain slice images produced from conventional processing techniques are often present with severe histological artifacts, making it extremely difficult for further processing such as automatic alignment of adjacent slices and annotation of regions of the slices [20].

For common analyses of brain section images, we register these sections with a standardized reference atlas like the Allen Reference Atlas (ARA) maps. Such registration will also become difficult in the presence of histological artifacts introduced during manual sectioning of mouse brain tissues. All these artifacts can be broadly categorized either as global 3D deformations, which may happen during extraction of the brain from the skull, physical effects like gravity during mounting, etc. or as slice specific 2D deformations, which are very common tissue artifacts introduced during sample preparations including serial sectioning of the brain (shearing and tearing) and mounting slices on glass slides (tearing, folding, absence or displacement of small parts from some sections). Though most of the above artifacts have been addressed by complex non-linear registration techniques [18], slice-specific 2D artifacts such as tissue tears and tissue loss are extremely difficult to automatically detect and resolve [13]. Hence, slices with such artifacts are typically discarded, thereby, losing precious data.

There have been previous works on detection and correction of these slice-specific 2D artifacts. However, as most of them are either semi-automatic or use information from neighbouring slices, they are not scalable. For example, Qiu et al. [21] proposed to automatically detect slices with artifacts by looking for unexpected differences between a specified slice and its neighbouring slices. Hence, artifacts in an isolated slice cannot be detected and corrected. Further, such a method also requires slices to be close enough and the adjacent slice to be devoid of any artifacts, such that the difference between slices will imply the artifact. This sometimes poses restriction on the neuroanatomists who may want slices only from specific regions of the brain or want to slice the brain at larger intervals. Kindle et al. [13], on the other hand, proposed a semi-automatic method where they manually identify small tissue tears and fill them by warping neighboring regions around the tear. This approach only works well when the tear is small, horizontal and smooth. Moreover, one needs to be careful about obtaining undesirable warping effects while fixing these tears, especially when they are severe as shown in Fig. 2.

While the above techniques aim to detect and correct slices which have artifacts, many researchers try to overcome them. The most popular approach among these is performing cryosectioning of frozen mouse brain tissues [3, 7, 16]. The rationale behind it is that frozen tissues are much easier to slice into thin sections without tearing or significant deformation. Another technique often used is the introduction of quality control checks [16]. After sectioning the mouse brain, highly damaged slices are manually removed from the registration pipeline. Further, to aid in registration of such highly damaged slices, manual landmarks are often placed [7] or even manual initial registration is performed [25, 26]. All the above measures which mitigate the 2D slice-specific artifacts and help its registration, in addition to being time consuming and expensive, require a lot of planning of the process. Although, slicing thicker sections may be a plausible solution to avoid tissue tears [2], it constrains the subsequent staining and imaging procedures. One needs to ensure that the slicing thickness is in accordance with the penetration depth of the stain and depth of focus of the light microscope used. Serial two-photon tomography (STPT), though produces artifact-free, well-aligned, high-resolution 3D datasets, which makes the registration process much easier [14, 16, 22], neural circuit mapping based on conventional processed brain sections continues to have technical challenges in standardized registration with highly deformed and damaged brain slices. We present a method to automatically detect and handle damages in such mouse brain microscopic slice images in order to achieve an accurate registration. Furthermore, since we detect artifacts on individual slices without using information from neighbouring slices, our method can be easily scaled to handle very large datasets without imposing any restrictions to the conventional neuroanatomical procedures.

Fig. 1.
figure 1

Overview of our damaged region detection algorithm: (i). Dominant edges (MEI) extracted from the mouse brain microscopic slice image on the left. (ii). Outermost contour of MEI, which serves as the input to our algorithm. (iii). Constrained Delaunay Triangulation of vertices V & edges E using the outermost contour of MEI. (iv). Exterior Voronoi vertices (magenta) and edges (brown). (v). Three candidate damage regions whose medial axis (Voronoi edge sequence) length was above \(\alpha \). Points corresponding to only the 2nd candidate area were classified as damage region points as they were not vertically symmetric. (please zoom in for details) (Color figure online)

Full size image

In this paper we introduce a novel geometric algorithm to automatically detect major histological artifacts such as tears and tissue loss (missing data) in thin, high-resolution mouse brain slice images. We not only provide qualitative analysis by visual verification from subject experts but also perform quantitative evaluation of our method. We register 52 conventionally processed mouse brain slice images with major histological artifacts to their corresponding annotated atlas slice images from ARA with and without our damage detection algorithm and compare various registrations errors.

2 Proposed Method

Our algorithm to detect damage regions (tissue tears and tissue loss) in mouse brain slice images is motivated by two key observations. First, the contours of most of the damaged regions have long exterior medial axis creating deep concavity into the tissue (Fig. 2). It is quite rare that the tear happens in the interior of the tissue directly without affecting the boundary of the tissue.

Fig. 2.
figure 2

Results of our automatic damage region detection algorithm: A sample of eight high-resolution mouse brain slice images with single or multiple histological artifacts (tears and missing data) are shown in the first column. Exterior Voronoi vertices (magenta) and edges (brown) are shown in the middle column. Detected contours of the damage regions (yellow) in all eight images are shown in right column. All the sample images were obtained from different datasets spanning different regions of the mouse brain and we successfully identified all the damage regions in all the eight images. (please zoom in for details) (Color figure online)

Full size image

Second, the damage region exhibits vertical asymmetry between left and right regions of the mouse brain. It is also very rare that the same type and shape of tear or missing region happens on both lobes of the brain tissue slice.

Input. Given a high-resolution mouse brain microscopic slice image, we first compute a microscopic-edge image (MEI) by extracting the dominant edges using a variant of Canny edge detector. Our dominant edge detection algorithm automatically computes the threshold for hysteresis to suppress the edges with low gradient magnitude as shown in Fig. 1(i). The automatic threshold computation uses the idea of persistence of edges from the histogram of the gradient magnitude. We skip its details as its beyond the focus of this paper. We then compute the outermost contour of MEI, which serves as the input to our damage region detection algorithm (Fig. 1(ii)).

Construction of Constrained Delaunay Triangulation. Using the vertices V and edges E of the outermost contour of MEI, we first construct a Constrained Delaunay Triangulation (CDT) [5]. All edges of E are a part of this triangulation as shown in Fig. 1(iii). We then remove all the triangles lying inside the contour and retain only the exterior Delanuay triangles. As the outermost contour of MEI is a simple closed curve, we use the Jordan curve theorem to compute whether a triangle is inside or outside the contour [10]. If the winding number of a point inside the triangle is zero, the triangle lies outside the contour, else it lies inside. In order to obtain reliable Voronoi vertices and edges that can be used in computations downstream, we further clean the remaining exterior triangles by removing all “skinny” triangles – any triangle whose circumcenter does not lie within the triangle.

Computing Voronoi vertices and edges. From the vertices \(V^\prime \) and edges \(E^\prime \) of the remaining Delaunay triangles, we represent the exterior medial axis as the sequence of Voronoi edges that do not intersect the edges of the original contour of the image [1]. Since this would create many small medial axes as shown in Fig. 1(iv), we threshold them (remove edge sequence \(<\!\!\alpha \); we use \(\alpha =20\)) and retain only those medial axes corresponding to deep concavities. The vertices of the Delaunay triangles corresponding to the retained medial axis Voronoi vertices serve as candidates for the damaged regions as shown in Fig. 1(v).

Checking for Asymmetry and Damage Region Detection. There may be important features of the brain that may also have long medial axis, but these features are also symmetric on both sides of the brain. Hence, as the final step of our algorithm, we check whether the damage region candidate edge points are symmetric between the left and right half of the mouse brain. For this we first divide mouse brain into two halves by splitting its oriented bounded box (OBB) equally into left and right regions [9]. Popular methods for OBB estimation such as principal component analysis (PCA) [12] will fail when used on highly damaged microscope slices (Fig. 2) because the spurious edge points produced in damaged areas of the tissues images bias the PCA. Hence, we compute the convex hull of edge points in MEI and resample it such that we have a fixed number of points uniformly sampled along the convex hull. We then use PCA on the resampled convex hull curves to compute the OBB of the microscopic image slice. The combination of PCA on the resampled convex hull curve eliminates the edge effects including bias due to noise, tissue damage and other artifacts caused during sample preparation.

We then check for symmetry of the candidate edges of the damaged region by reflecting those edge points about the vertical axis dividing the OBB in half and using a small neighborhood to search for points having similar normal vectors around the expected region of symmetry. Normal vectors of edge points are computed using moving least squares [17] as it smoothly interpolates the normal vectors, diminishing the effect of noise, sharp features and topological foldings. Candidate edge points, which are asymmetric between the left and right regions of the brain are classified as damage region points. For example, as shown in Fig. 1, out of the three candidate damage regions, only the points corresponding to the 2nd candidate damage region were classified as damage region points. Points corresponding to the remaining two (1st & 2nd) candidate damage regions form important features of the mouse brain slice images and hence are also vertically symmetric.

Fig. 3.
figure 3

A. Correspondences used during feature based non-linear registration: Dense correspondences before (left) and after (right) our damage region detection algorithm between MEI (bottom) and its corresponding atlas from ARA maps (top). Correspondences are shown using similarly colored curve segments. B. Computation of registration errors: 20 manually selected points (magenta) are uniformly distributed and overlayed on the atlas from ARA maps (background). Error between their corresponding points in the matching microscopic slice is computed and reported in Table. 1. (please zoom in for details) (Color figure online)

Full size image

3 Results and Discussion

We evaluate our algorithm on 52 conventionally processed microscopic images of coronal mouse brain slices (5000\(\,\times \,\)8000 pixels) with a resolution of 0.6 \(\upmu \)m per pixel. These images were manually identified by subject experts from different mouse brain datasets to contain severe histological artifacts such as tears and missing data. These artifacts were produced either during serial sectioning of the mouse brain tissue or mounting of the thin slices on glass slides. Among the 52 slices, 45 slices had single tears or missing regions while 7 slices had multiple tears or missing regions. We ran our automatic damage region detection algorithm on all 52 microscopic images and successfully identified the damage regions in all of them as shown in Fig. 2. Our damage region detection results were found qualitatively quite accurate by the subject experts.

Fig. 4.
figure 4

Results from feature based registration techniques with and without our algorithm: The figure shows results of a standard feature based non-linear registration with corresponding ARA (overlayed in white) without (left) and with (right) our damage region detection algorithm. Incorrect registration is shown using yellow marked regions. (please zoom in for details) (Color figure online)

Full size image

We also quantitatively evaluate our algorithm by comparing results from standard feature based registration techniques when used with and without our damage region detection algorithm. Feature based registration algorithms have been quite popular in the past for registration of microscopic images [4, 6, 15, 23]. However, the presence of edges due to the damage regions misleads and corrupts the correspondence finding (Fig. 3A), resulting in bad registration. Hence, it is important to first accurately identify and remove points in the damage regions before performing feature based registration.

We perform an inter-stack feature based registration where we align all 52 microscopic images with their corresponding annotated atlases from ARA mapsFootnote 1 with and without damage region detection. In both cases, for registration, we first perform global affine alignment using a variant of iterative closest point (ICP) [24]. This is followed by a final non-linear alignment by solving the Laplace’s equation with Dirichlet boundary conditions [11] (Fig. 4). The only difference between the two cases is our damage region detection algorithm where the detected damage region points are excluded from the correspondence finding as shown in Fig. 3A. For statistical analysis we compared the root-mean-squared error (RMSE), the median error (MEE) and the maximal error (MAE) of 20 corresponding points which were manually picked and distributed uniformly in the microscopic and atlas image pair (Fig. 3B). Table 1 summarizes the registration errors after both affine and non-linear transformation with and without our damage region detection algorithm. We found lower registration errors when alignment was performed with our algorithm than without it.

Table 1. Comparison of registration errors (in pixels) with and without our damage detection algorithm after affine and final non-linear (affine+elastic) transformations.
Full size table
Fig. 5.
figure 5

Results of our automatic damage region detection algorithm on a synthetically damaged mouse brain slice image: To show the variety of damages our algorithm can handle, we ran our algorithm on a synthetically created damage slice image with four different types of tissue tears at different locations. (i). Original mouse brain microscopic slice image. (ii). Synthetically damaged slice image with four tissue tears. (iii). Exterior Voronoi vertices (magenta) and edges (brown). (iv). Detected contours of all the damage regions (yellow) overlayed on the damaged slice image. We successfully detected all the four different types of tears in the mouse brain slice image. (please zoom in for details) (Color figure online)

Full size image

We have developed and demonstrated a completely automatic algorithm to not only identify but accurately locate and handle slice-specific histological artifacts such as tissue tears and tissue loss in high-resolution microscopic mouse brain slice images. As these artifacts are very common in conventionally processed slices, our algorithm will have wide applicability and usefulness in broad range of experiments and neuroanatomical laboratories. We show results of one such application where we perform accurate registration of highly damage slices with their corresponding annotated atlas from ARA maps. Such applications play a vital role in reconstruction of mouse brain datasets. Another advantage of our algorithm is that it can locate multiple such artifacts that may be present in single slice images as shown in Fig. 2(vi) and (vii). This enables and facilitates extremely thin sectioning of the mouse brain tissue, which is necessary for an accurate 3D mouse brain model reconstruction. To further illustrate the difficulty and the effectiveness of our method, we also show the results of our damage region detection algorithm on a synthetically damaged slice image with more than three tissue tears, which are deep and randomly placed (Fig. 5).

However, there are still some extreme artifacts, which cannot be handled by our algorithm. For example, slices in which the tear goes all the way through will have more than one component. Such tissues are very difficult to mount since multiple components have to be accurately placed in their original positions onto the glass slide. In such cases, our algorithm will fail as it detects multiple components, and hence we do not process such slices any further. Few other extreme deformations are folding of the tissue and overlap of adjacent tissue regions. For such artifacts, a more complicated or semi-automatic approach might be helpful.

4 Conclusion

To the best of our knowledge the presented work is the first that automatically detects slice-specific histological artifacts such as tissue tears and tissue loss in high-resolution mouse brain slice images without using information from neighbouring slices. Our robust damaged region detection algorithm condones histological artifacts that occur in standard procedures to produce brain slice images. We believe that this work will have a major impact on brain circuit mapping by facilitating conventional neuroanatomical image registration and creation of 3-D whole brain map databases.