1 Introduction

Curvilinear structure detection (CSD) becomes an essential procedure in many practical tasks. Such as the fiber classification, river finding, blood vessel detection, free-way traffic patterns and so on. Silva, Martins [34] extracted the filament and tree like structures which forms the blood vessels in medical images based on the curvilinear structure detection. Shih and Kowalski [33] used curvilinear structures to detect the stars and galaxies formed filament-like structure in cosmological data. There are some other applications such as remote sensing and seismology. Martínez and Ludeña [27] and Stanford and Raftery [35] tried to detect curvilinear structures in noisy images. Einbeck and Dwyer [13] employed curvilinear structure identification to analyze the freeway traffic patterns. Dong and Mcavoy [11] applied curvilinear structure for processing monitoring. Su, Srivastava [36] used curvilinear structure identification for shape analysis in point clouds of 2D and 3D.

One of the most famous methods of CSD was proposed by Trevor Hastie [37]. The authors defined the “principal curves” as the middle axis, which passes the sample points in a specified way. However, some researchers thought the “principal curve” did not consider self-intersection and multiple curves, hence, scholars added strict assumptions to solve above problem. Typically, locally defined variants of principle curve was proposed by Ozertem and Erdogmus [29]. Furthermore, Pulkkinen [31] proposed a nonparametric estimation of marginal density method, and named it ridge-based curvilinear structure detection (abbreviated as RCSD).

In this paper, we make a tentative study of applying the newly-proposed RCSD to two applications: road detection and backbone detection, because of following reasons. (i) The roads from the remote sensing images may have multiple sub-roads; (ii) The backbone from dendrite image also have multiple dendrites; (iii) nonparametric is easy to implement and does not need to set hyperparameters. Therefore, we apply RCSD to these two fields.

The rest of the paper was organized as follows: Section 2 introduced the related work of road detection and the dendrite extraction. Section 3 described the method of RCSD. Section 4 described the road detection result by RCSD method from the remote sensing images. Section 5 described the backbone detection result by RCSD method from the neuron dendrite images. Section 6 provided the conclusion.

2 Related work

2.1 Road detection

With the development of the autonomous vehicles, the road detection has attracted more and more attentions from researchers. Recently, researchers have proposed various road detection methods.

For example: Barsi and Heipke [4] used a feed-forward neural network for road junction detection. They extracted raster and vector information as the features. The developed systems can automatically and accurately classify the samples as road junction or not. Kong, Audibert [20] presented a scheme to detect road from a single image. They firstly estimated the vanishing point and extracted main parts of the road. Then, the exact boundaries were obtained based on texture orientations and edge detection technique. Li, Zhang [22] put forward a system to detect road edges in remote sensing images of high resolution. Instead of revised parallel-beam radon transform and ridgelet transform, they used traditional edge detection operators. The proposed method can effectively detect straight road edges. A shadow-invariant feature space was proposed and a classification model was online trained to improve the adaptability of the system to the lighting and vehicles in the current images. Compared to the hue-saturation-intensity based algorithms, their method obviously has better performance. Guo, Mita [16] developed an advanced driver-assistance system. The road detection problem was regarded as a maximum posteriori problem based on Markov random field. They employed an optimization algorithm to label the samples as road or background. Álvarez, López [1] firstly estimated the priors to get the initial information about the road. Then, they extracted features from the image, including low-level cues and contextual cues. Finally, a generative model was trained to detect road borders. Byun, Seo [5] presented a road detection algorithm without the assumption that the surface of road is flat. They extracted spatial and visual context based on Markov random field, and converted the detection problem into a classification problem. Zhou, Kong [43] presented a road detection algorithm for unmanned aerial vehicle. Initially, the road region was obtained by graph cut based detection algorithm. Then, the road areas were tracked using a homograph based road tracking method. Li, Ding [23] proposed a road detection method for the autonomous navigation system. The proposed method first segment the rough road region based on the dark channel, then, three effective soft voting rules were used to distinguish the road from the segmented region.

However, those proposed method highly relied on the resolution of the image and can be easily affected by the noise. The survey is shown in Table 1. Therefore, in this paper, we proposed to use the ridge based curvilinear structure detection method to extract the road from the remote sensing images.

Table 1 Review of state-of-the-art road detection methods
Full size table

2.2 Backbone detection

For the neuron dendrite images, the backbone detection method can be summarized as two types: Semiautomatic method, which is based on software and needs to be revised manually in the final step, and another type is automatic method. For example, the SynD was proposed by for dendrite and synapse characteristics analysis in immune-fluorescence images. NeuronJ was proposed to trace the growing of the dendrite. NeuronIQ [28] was proposed for the co-focal multi-photon laser scanning.

Other automatic methods include follows: Koh, Lindquist [19] proposed to extract the dendrite backbone as the medical axis. Xu, Cheng [40] proposed a global threshold based method to segment the image, and then the centerline of the dendrites are extracted by the medical axis transform. Cheng, Zhou [10] used an adaptive threshold to the local contrast for the discrimination of the foreground which includes the spine and dendrite. Fan, Zhou [14] and Zhang, Zhou [42] used the second order derivative method to locate the medical axis of the dendrite backbone. Fan, Zhou [14] used an adaptive local binary fitting energy level set model to find the boundary of the dendrite. Their method is suitable for Alzheimer’s disease patients. Zhang, Zhou [42] build a classifier using Linear Discriminate Analysis (LDA) to classify the attached spines into valid and invalid types. Chen [6] proposed a three-step cascaded algorithm-RTSVM, which is composed of ridge detection and Twin Support Vector Machine (TSVM) classifiers for spine classification. Li and Cattani [21] proposed a method based on Wavelet Packet Entropy and Fuzzy Support Vector Machine. Fan, Zhou [15] gave a method based on curvilinear structure detector (CSD).

Those proposed methods for the dendrite location have good performance on the existed dendrite images. However, those methods either have strict assumptions of single dendrite or highly depend on the resolution of the images, as shown in Table 2. In order to overcome those limitations, we proposed a new method in this paper for the dendrite location, and for building the relationship between dendrite location and the road extraction from the remote sensing images.

Table 2 Review of state-of-the-art backbone detection methods
Full size table

3 Methodology of RCSD

In this paper, we proposed to use ridge-based curvilinear structure detection (RCSD) to estimate the possible generating function to describe the curvilinear structure from the original images. The advantages of RCSD are two folds: (i) It takes self-intersection and multiple curves into account. (ii) It is a nonparametric estimation of marginal density method. Mathematically, RCSD is described below.

The preprocessed gray image was normalized to the binary image. Then, we assumed that the observed samples either belong to background clutter or the main line, which is considered as curvilinear structures [31]. The sample points are expressed as random variables as formulation (1):

$$ R=\left\{\begin{array}{c}0\\ {}1\end{array},\right.\mathrm{the}\ \mathrm{observed}\ \mathrm{sample}\ \mathrm{belongs}\ \mathrm{to}\left\{\begin{array}{l}\mathrm{the}\ \mathrm{background}\ \mathrm{clutter}\\ {}\mathrm{the}\ \mathrm{curvilinear}\ \mathrm{structure}\end{array}\right. $$
(1)

where R represents for the sample points of the observed variable X in certain compact domain Ω∈Rd. The probabilities of the observed samples are shown as (2):

$$ {\displaystyle \begin{array}{l}P\left(R=1\right)=a\\ {}P\left(R=0\right)=1-a\end{array}} $$
(2)

a is set within the range of [0 1], and the sample points of the background clutter is thought as uniformly distributed in the domain of Ω∈Rd.

The generating function of the main line is described as:

$$ {\left\{{f}_i\right\}}_{i=1}^n $$
(3)

in which, n stands for the number of the main lines. The generating function is defined as a continuous mapping from the compact and connected domain Di ⊂ R. The function fi is set as a unit length parameterization for all i = 1, 2, …, n. When R = 1, the output of the function f will depend on three parameters i, Ψ, and b. i is used to figure out which line the sample points belong to. The variable Ψ represents for the coordinate along the specified lines. Considering that the additive noise of the generating functions, b is defined as the noise. We set

$$ P(i)={w}_i,{w}_i>0\ \mathrm{and}\ \sum \limits_{i=1}^n{w}_i=1 $$
(4)

and b ~ N(0,∂2), which means that the noise has zero mean and variance of ∂2. According to the above defined parameters, the output of X is expressed as:

$$ X\mid \left(T=1,i,\Psi =\varphi \right)=f\left(\varphi \right)+b $$
(5)

The density of x is:

$$ {p}_x\left(x|T=1,i,\Psi =\varphi \right)=\frac{1}{{\left(\sqrt{2\pi}\varphi \right)}^d}\exp \left(-\frac{{\left\Vert x-{f}_i\left(\varphi \right)\right\Vert}^2}{2{\varphi}^2}\right) $$
(6)

All the lines in an image are expressed as the set

$$ F=\left\{{f}_i\left(\varphi \right),i=1,2,\dots, n,\mathrm{and}\ \varphi \in {D}_i\right\} $$
(7)

However, for the practical applications, there is no prior knowledge about the data generating process available. Therefore, in this paper, we used the non-parametric estimation method, which was directly from the samples obtained from the observed variable X. It means that the density for X has very weak or no relationship with parameters i, Ψ, and b.

We employed the joint density, then via marginalizing. The obtained density will only depends on the variable X instead of i, Ψ, and b. Based on the relationship of the joint and conditional density, we can get (8).

$$ {\displaystyle \begin{array}{l}{P}_{x,T,i,}\left(x,R,i,\Psi \right)={P}_x\left(x|R=1,i,\Psi =\varphi \right){P}_{T,I,\Psi}\left(R=1,i,\varphi \right)\\ {}\kern6.25em ={P}_x\left(x|R=1,i,\Psi =\varphi \right){P}_{\Psi}\left(\varphi |i\right){P}_{T,I}\left(R=1,i\right)\\ {}\kern6.25em ={P}_x\left(x|R=1,i,\Psi =\varphi \right){P}_{\Psi}\left(\varphi |i\right)P(i)P\left(R=1\right)\end{array}} $$
(8)

for R = 0, we can get the density as

$$ {p}_{x,T,I,}\left(x,0,i,\Psi \right)={p}_x\left(x|T=0\right)\times p\left(T=0\right) $$
(9)

Here, px,R,I(x, 0, i, φ) means the joint density. Then the joint density is accumulated over on the domains of the variables R, φ and i.

As marginalizing the joint density can make the function in-depend on the prior parameters i, Ψ, and b, the marginal density is expressed as

$$ {p}_x(x)=\frac{\rho }{{\left(\sqrt{2\pi}\varphi \right)}^d}\sum \limits_{i=1}^n{w}_i{\int}_{P_i}\exp \left(-\frac{{\left\Vert x-{f}_i\left(\varphi \right)\right\Vert}^2}{2{\varphi}^2}\right){D}_{\Psi}\left(\varphi |i\right)\mathrm{d}\varphi +\frac{1-a}{V\left(\Omega \right)} $$
(10)

Here, V(Ω) is the volume of domain Ω.

The ridge curve is composed of ridge points, which are defined as local maximums on the cross section of the density with respect to the hyper-plane spanned by a subset of the hessian matrix. In order to implement the estimation of the marginal density, we used the kernel density estimation as it can work without prior knowledge about the data generating process. In this paper, we chose the Gaussian kernel density, which estimates \( \widehat{P} \) based on a set of samples

$$ Y={\left\{{y}_i\right\}}_{i=1}^N\subset {R}^2 $$
(11)

of the probability density P

$$ \widehat{P}=\frac{1}{N}\sum \limits_{i=1}^N{K}_H\left(x-{y}_i\right) $$
(12)

in which KH stands for the Gaussian kernel density with a positive and symmetric kernel bandwidth in the domain R2→ [0,∞], and H∈R2x2.

The ridges are separated by assuming that they only locate in the high probability density area. We set the threshold ς to separate the low probability density area and high probability density area. According to the above formulation, we need to first find modes of \( \widehat{P} \), then take the modes as the initial points to build the ridge point set to get the ridge curve based on these modes. According to the truth that the ridge points are those mode that lie on a ridge curve. The pseudo code of RCSD is shown in Table 3, and the flowchart is presented in Fig. 1.

Table 3 Pseudocodes of RCSD
Full size table
Fig. 1
figure 1

Flow chart of the ridge-based curvilinear structure detection

Full size image

As there are still some branch lines connecting to the main line, spurs are excluded based on the principle that there only exists one main connected ridge curve component passing through the intersection point. Therefore, the other branch lines passing through such an intersection point are split into different parts.

4 RCSD for road detection

The problem existing for the road detection based on the remote sensing images incudes that the sensor type can affect the image characteristic in some way. Meanwhile, weather, light, dust in the air and spectral can affect the remote image sensing. In order to reduce the impact from the sensors and improve the detection precision, we proposed a robust method based on the RCSD to extract the road. However, considering the noise during the imaging, we first employed the morphology trivial opening operation, which filters out nearly all the small clusters of noise and the small paths. The detail information of the morphology trivial opening operation is described as follows.

4.1 Preprocessing

The morphological opening operation can be expressed as:

$$ E\circ D=\left(E\ominus D\right)\oplus D $$
(14)

in which, E and D are sets in Z2. Equation (14) is termed of the dilation of the erosion of a set E by a structure element D. Erosion is expressed as (15)

$$ E\ominus D=\left\{x|{(D)}_x\subseteq E\right\} $$
(15)

in which, x stands for sets element. Erosion also can be expressed in the form as the duality of the erosion-dilation relationship as formulation (16).

$$ E\ominus D={\left({E}^c\oplus \widehat{D}\right)}^c $$
(16)

in which, Ec represents for the complement of set E.

Dilation is defined as

$$ E\oplus D=\left\{x|{\left(\widehat{D}\right)}_x\cap E\ne \varnothing \right\} $$
(17)

where \( \widehat{D} \) is obtained by inverting set D. Ø represents for empty set. Formulation (17) means the dilation of set E by D. Dilation can also be expressed as B is inverted first, then translated by x, which can make set E to intersect with \( \widehat{D} \) by at least of one element.

4.2 Experiments and results

Automatic road detection is important for the town planning, GPS navigation, traffic management, map updating, GPS navigation and autonomous vehicles. Currently, the road detection mainly relies on the manual extraction and semiautomatic method. However, it is a time-consuming operation and hard to repeat. Therefore, considering its importance, a lot of researchers contributed to find solutions of this problem. For the low-resolution images, the road is described more like lines. For the high resolutions images, the road looks like elongated regions with parallel boundaries. In this paper, we mainly focused on the remote sensing images, with the pipeline of our experiment shown in Fig. 2.

Fig. 2
figure 2

Pipeline of RCSD for road detection

Full size image

Figure 3 shows an example of high-resolution images. There are buildings and roads. First, we transferred the gray image into binary images, and then based on the ridge detection, the road is extracted from the remote sensing images as shown in Fig. 3(c). Figure 4 shows an example with lower resolutions, we can find that in Fig. 4(c) some pixels belonging to the road are missed as the pixel intensity is similar with the other components. As the resolution in Fig. 4 is much lower than Fig. 3, it is hard to distinguish the road pixel from the other buildings. Meanwhile, the road pixel is quite similar with the background information, therefore, the road pixels are taken as the background information when transferring to the binary images.

Fig. 3
figure 3

Road detection from high-resolution remote sensing images

Full size image
Fig. 4
figure 4

Road detection from low-resolution remote sensing images

Full size image

In order to test the precision of the proposed algorithm, we detected road from different resolution images, which include road in mountains and plain areas around cities as well as rural as shown in Figs. 3 and 4. The data includes 50 images with 19,465 road pixels.

Table 4 shows that there are 19,465 road pixels and 168,473 non-road pixels in total. There are 18,207 road pixels and 168,817 non-road pixels are correctly detected. We defined four parameters TP, TN, FP, FN as shown in Table 5 and employed the accuracy, specificity, sensitivity and precision to measure the performance of the proposed algorithm.

Table 4 The confusion matrix of road detection using our RCSD method
Full size table
Table 5 the physical meaning of the four basic parameters
Full size table

The accuracy (Acc), specificity (Spec), sensitivity (Sen) and precision (Prc) are expressed as Eq. (18)–(21).

$$ Acc=\frac{\mathrm{TP}+\mathrm{TN}}{\mathrm{TP}+\mathrm{FP}+\mathrm{FN}+\mathrm{TN}} $$
(18)
$$ Sen=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}} $$
(19)
$$ Spec=\frac{\mathrm{TN}}{\mathrm{TN}+\mathrm{FP}} $$
(20)
$$ Prc=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}} $$
(21)

The measurement of the road detection is shown in Table 6. The accuracy, specificity, sensitivity and precision achieved 99.10%, 99.53%, 98.08% and 93.50% respectively, which means that we correctly detected 18,207 pixels from 19,465 road pixels.

Table 6 The measurements of the road detection
Full size table

In order to test the performance of the proposed method, we compared our RCSD method with state of the art methods: Long, Shelhamer [25] proposed to apply the fully connected network for the pixel wise application. Badrinarayanan, Kendall [3] used the deep convolutional encoder-decoder method named as Segnet for the scene segmentation. Shi, Liu [32] proposed to use the Generative Adversarial Networks (GANs) for the road pixel detection.

The results are listed in Table 7 and Fig. 5. Obviously our RCSD method achieves the greatest value. The reason is: FCN [25] produced the segmentation map highly relied on the resolution. Segnet [3] and GAN [32] made more mistakes on the boundary on the road detection, since they are more suitable for classification other than segmentation.

Table 7 Comparison result of the state of the art methods by precision
Full size table
Fig. 5
figure 5

Comparison of methods of road detection

Full size image

5 RCSD for backbone detection

Extracting linear features has been widely applied for the roads, railroad, and river of satellite or aerial imagery. The structure of the backbone in neuron dendrite image is similar to the road from the remote sensing images. Therefore, we choose the neuron dendrite as the representation of the biomedical images. We proposed to extend the application to detect backbone from the neuron dendrite images. However, in order to explore the application of the RCSD, we will try more types of images in our future research, such as the blood vessel and so on.

5.1 Background

The dendrites are defined as the neuron branch [9]. It works as to transmit the electrochemical stimulation from other neural cells to the cell body or soma of their parent body. Dendrites received electronic stimulation from upstream neurons via the synapses. Dendrite works as a key role in integrating the inputs from the synapses and in make decision on the extent to which action potentials are produced by the neuron. The morphology of dendrites including the branch density and grouping patterns has a strong relationship with the function of the neuron. Furthermore, the impaired nervous system function is highly related with the malformation of dendrites. Therefore, the detection of the dendrite and quantity analysis is important for the statistics analysis for the biology analysis and build the function map of the function of the neuron and the morphology of the dendrite. Figure 6 provides the pipeline of our experiment applying RCSD for backbone detection.

Fig. 6
figure 6

Pipeline of RCSD for backbone detection

Full size image

The research shows that the dendrite growth is tightly correlated with the learning and memory formation. Figure 7 shows an example of one neuron dendrite image. It is composed of dendrite (the main axis from top left to bottom right) and the spine attached to the dendrite. In this paper, our research will focus on the backbone detection. The backbone is the skeleton of the dendrite.

Fig. 7
figure 7

Sample of a neuron dendrite image (Backbone detection is our aim)

Full size image

5.2 Experiments and results

Figure 8 shows different cases of the backbone location, including different numbers of the dendrite and whether has self-intersection. The result shows that the method used for the road detection from the remote sensing images can also be applied for the neuron dendrite detection.

Fig. 8
figure 8

Backbone detection results. (a) contains two nearly parallel dendrite branches. (b) has two non-parallel dendrite branches. (c) has a single main branch with a small subbranch. (d) presents a diverge takes place along a main branch. (e) shows a multi-branch dendrite. (f) gives two isolated dendrite branches

Full size image

In order to test the proposed algorithm, we proposed to compare our method used in this paper with the second order derivative (SOD) [42], conditional symmetric analysis (CSA) [9], and curvilinear structure detector (CSD) [15]. The comparison results are listed in Table 8. By the length of the detected dendrite, 1476.32 μm was obtained by RCSD, 1451.63 was obtained by CSA [9], 1435.81 was obtained by the second order derivative [42], and 1402.40 was obtained by CSD [15]. The relative difference (i.e., gap) was obtained for four algorithms: 3.42% by CSA [9], 4.45% by Second order derivative [42], 6.68 by CSD [15], and only 1.76% by proposed RCSD. It is obvious that the result of proposed RCSD is closer to the ground truth result. Figure 9 shows the bar-plot of the comparison in terms of length detected and gap.

Table 8 Different methods for the backbone detection from 15 sample (Unit: μm)
Full size table
Fig. 9
figure 9

Comparison of backbone detection in terms of relative difference

Full size image

Although this RCSD was developed for curvilinear structure detection, it may be used in other academic and industrial fields, such as traffic sign detection [30], pathological brain detection [2, 26], path planning [12, 38], hearing loss detection [7], tea category identification [17, 24, 39], breast cancer detection [8], multiple sclerosis detection [41], cerebral microbleeding identification [18], etc.

6 Conclusion

In this study, we proposed a method based on the ridge-based curvilinear structure detection (RCSD) to detect the road pixels from the remote sensing images, and to detect backbone areas from neuron dendrite images. We randomly selected 50 images, which includes 19,465 road pixels and 18,207 road pixels were corrected detected. The precision of RCSD achieved 93.5%. It proved that the proposed algorithm is effective for the road detection from the remote sensing images. Meanwhile, in order to extend the application of the RCSD, we tested the neuron dendrite images from the laser scanning microscopy. The experiments indicated the gap result of RCSD is only 1.76%, which is superior to state-of-the-art approaches.