3D Retinal Vessel Tree Segmentation and Reconstruction with OCT Images

Abstract

Detection and analysis of the arterio-venular tree of the retina is a relevant issue, providing useful information in procedures such as the diagnosis of different pathologies. Classical approaches for vessel extraction make use of 2D acquisition paradigms and, therefore, obtain a limited representation of the vascular structure. This paper proposes a new methodology for the automatic 3D segmentation and reconstruction of the retinal arterio-venular tree in Optical Coherence Tomography (OCT) images. The methodology takes advantage of different image analysis techniques to initially segment the vessel tree and estimate its calibers along it. Then, the corresponding depth for the entire vessel tree is obtained. Finally, with all this information, the method performs the 3D reconstruction of the entire vessel tree.

The test and validation procedure employed 196 OCT histological images with the corresponding near infrared reflectance retinographies. The methodology showed promising results, demonstrating its accuracy in a complex domain, providing a coherent 3D vessel tree reconstruction that can be posteriorly analyzed in different medical diagnostic processes.

1 Introduction and Previous Work

Nowadays, eye fundus is widely used in many analysis and diagnostic processes. Hence, the study of retinal images provide the specialists useful information that can be of a great utility to obtain accurate diagnosis in a large variability of pathologies. In that sense, Computer-Aided Diagnostic (CAD) systems in the ophthalmology field have widely spread over the years, as they facilitate specialists work, increasing their productivity and helping to establish preventive and therapeutic strategies.

The automatic detection and extraction of relevant structures of the retina help to significantly improve the procedure of clinical assessment. From all of them, the arterio-venular tree is of particular relevance. Retinal microcirculation is the easiest and less invasive way to access to the circulatory system in human body. Moreover, different studies established retinal vessel caliber as a relevant parameter in the evaluation of diabetic patients [1]. Smith et al. [2] indicated that small structural vessel changes in the retina can anticipate the development of severe hypertension. Some studies [3] also established calculations on the retinal microcirculation as possible biomarkers for cerebrovascular disease, or others [4, 5] that related the retinal vessels calibers to cardiovascular illnesses events. Besides the medical field, the retinal vessel tree was also applied with other purposes as, for example, the identification in biometric systems [6], similar to fingerprints or other biometric traits.

Given the relevance of this structure, many authors have faced this task over the years, proposing different methodologies for retinal vessel tree extraction. Classical acquisitions methods such as angiography or retinography depict vessel structure as a 2D projection of the real 3D layout. Thus, the vast majority of the approaches present in the literature are proposed in 2D representing partial information of the vasculature. Some representative examples are here presented. Yong et al. [7] proposed an adaptative threshold to localize and segment the vessels. Edge detectors were also employed, as in the work of Xiaolin et al. [8], where Canny was employed to detect the vessels and modified with a with a bilateral filter to remove the noise. Mendonça and Campilho [9] introduced an approach based on the combination of vessel centerline extraction and region growing to fill the resultant extracted vessels. Wavelet transform also demonstrated its utility in the problem, as Nayak et al. [10] indicated, being used in a method that extracts the vessel tree for diabetic retinopathy patients. Graph-cuts were also employed, as Chen et al. [11] proposed, in an automatic and unsupervised system was constructed using graph-cuts to identify the blood vessels. Artificial Neural Networks (ANNs) were commonly used in medical imaging methodologies, given their potential and robustness to provide accurate results in significantly adverse conditions. They were also used in this problem, as Chen et al. [12] implemented, where the method inputs image patches to the ANNs that differentiates the vessels from the rest of the retina.

In recent years with the appearance and popularization of the Optical Coherence Tomography (OCT) depth information in the retina can be analyzed, including presence and location of vessels. Some studies have been presented to undertake this task although most of them still present 2D projections of the vessel tree for this new modality. Guimaraes et al. [13] employed the OCT fundus images to locate the depth of the vessels, enclosed in the study of abnormal retinal vascular patterns. Pilch et al. [14] employed an statistical shape model created with a set of vessels that were manually segmented and used in a training phase. The model is then employed to segment the contours in axial direction. Wu et al. [15] designed a method using Coherent Point Drift to segment the retinal vessel point sets, that are posteriorly used as landmarks for image registration.

For that reason, in this work, we propose a 3D segmentation and reconstruction of the arterio-venular tree of the retina, taking advantage of the Optical Coherence Tomography (OCT) images that provide depth information in the eye fundus. Thus, it offers a complete set of characteristics of the vasculature that can permit to proceed with more reliable analysis and diagnostic processes involving the retinal microcirculation analysis.

2 Methodology

The input images of the method are obtained by the OCT technique that offers the near infrared reflectance retinographies and the corresponding histological sections. This technique permits to obtain tomographic images of the biological tissue with high resolution, performing consecutive measurements over the retina. Thus, these images compose the 3D visualization of the eye fundus of the patient. In Fig. 1 we can see an example of the OCT image of a patient. The input images include the selection of the region of interest (ROI), that indicates the part of the retinography corresponding with the histological sections, containing relevant structures to be analyzed, in particular in our case, the retinal vessels.

Fig. 1.

Example of near infrared reflectance retinography and histological section.

The methodology firstly extracts the vessel tree in the near infrared reflectance retinography, constructs a list of segments and corrects possible mistakes at the intersections. Then, the method estimates the diameter at each point of the vessels. Posteriorly, these segments and diameters are mapped in the histological sections to search for the z coordinate, that is, the depth at the points. Finally, with all this information, the 3D reconstruction of the vessel tree is achieved, interpolating with splines all the segments to obtain a smoother representation. Next subsection details each step.

2.1 2D Vessel Detection

Firstly, we segment the vessels in the 2D retinography, given its simplicity and well-established techniques. A segmentation based in morphological operators is applied [16] to obtain an initial representation of vessels. Segmentation approaches in 2D can not be directly used for reconstruction as they typically present cumulative errors due to misrepresentation of edges. Thus, a centerline-based approach is needed to correct deviations in the structure. The main idea is to have segments representing the approximate central line of vessels in order to get coordinates and to detect characteristic points (mainly bifurcations and crossings). This is computed by applying the approach proposed in [6] that exploits the idea that vessels can be thought of as creases (ridges or valleys) when images are seen as landscapes, curvature level curves are employed to calculate the creases (crest and valley lines). Among the different definitions of a crease, this work uses the one that is based on level set extrinsic curvature or LSEC, given its invariant properties. This method provides the vessel tree segmentation that is going to be the kernel structure to construct the entire 3D segmentation.

Creaseness measurement does not return a 1-pixel width segment as there are degrees of creaseness along the vessel. Hence, we label all the pixels with a tracking process that is going to check across the entire crease image, guaranteeing that all the pixels belong to any particular segment. At the end, we have a skeleton vessel tree structure where all the vessels are represented by one-pixel width segment with an initial and final point. Small segments are removed as they are obtained from other structures different to vessels or even noise in the image. Figure 2(a),(b) and (c) shows an example of vessel and segment extraction.

Fig. 2.

Example of the methodology in the 2D vessel tree extraction and refinement.(a) Input retinography.(b) 2D Vessel tree extraction.(c) Resultant Segments.(d) Entire set of end points.(e) Corrected bifurcations.(f) Corrected crossings.

2.2 Intersections Reconstruction

The main drawback of the crease method is the loss of vessel connectivity mainly in the intersection points as in these regions the direction of the crease can not be established as it can seen in Fig. 2. In order to correct miscalculations, a further analysis for those regions was designed to reconstruct the intersections. All the end points of the segments were analyzed for being distinguished as: real ends, bifurcations or crossings.

Bifurcations. When an end point is significantly close to any point of another segment. The method calculates for each end point its closest distance to any other segment, and defines a threshold to filter those that are at a distance we consider that belongs to a bifurcation. In that case, the end point is connected to the segment by interpolation of the continuity of its own segment.

Crossings. In this case, it is represented by a couple of end points that are close to other segment. If both end points are inside of a given threshold, a continuity among them is considered. Both points are again connected by interpolation joining in a single segment that crosses the other one.

The rest of the points are considered correct end points. This way, we have correctly characterized all the vessels in a set of segments, defined by a couple of begin and end points, and a list of consecutive pixels that represent the approximated center line of the vessel. Figure 2(d),(e) and (f) illustrates a process of end points analysis and intersection refinement.

2.3 Caliber Estimation

We also need to estimate the vessel caliber at each point of the segments, in order to correctly represent its appearance in the posterior 3D reconstruction. The orientation of each pixel is used in this estimation. An orientation $\theta$ is calculated by means of the angle between two consecutive points of the same segment $P_1(x_1,y_1)$ and $P_2(x_2,y_2)$ as:

$$\begin{aligned} \theta = arctan \left( \frac{y_2 - y_1}{x_2 - x_1} \right) \end{aligned}$$

(1)

The perpendicular of this orientation at the point indicates, therefore, the directions to compute the caliber of the vessel, as shown in Fig. 3(a). Hence, in both perpendicular directions we search for the edges that delimits the limits of vessel, calculating $r_1$ and $r_2$ as the length of the vessel in each direction. We search in both sides as the central point of the segment is not always exactly in the center of the vessel as Fig. 3(b) illustrates with an example. Therefore, the computed diameter, $d = r_1 + r_2$, is better adjusted to the real scenario.

Fig. 3.

Diameter estimation. (a) Representation of the analyzed direction at each point of a segment. (b) Example with the segments overlapping the vessels.

2.4 Vessel Mapping in the Histological Sections

The next phase is the estimation of the vessel depth, z, at all the detected points of the vessel tree. This can be analyzed in the OCT scans, where we can compute the exact depth in the retina where vessels are located.

In the OCT images, the retinal vessels are visualized as structures that block the transmission of light and leave a shadow, as presented in Fig. 4(a), with a width proportional to the vessel diameter, in the corresponding histological section where they are placed. As we know each histological section corresponds to a band in the 2D image, consequently we detect the intersection of this band in the 2D retinography with the detected segments. Then, these intersection points, and the corresponding diameters, are mapped in the histological section, constructing a rectangle with a width corresponding to the associated $r_1+r_2$. This way, we identify the projection zone of the vessel, that is, the region that corresponds to the vessel shadow, as shown in Fig. 4(b). In this projection zone, we can identify the location of the vessel which represents the depth z where the vessel is placed.

Fig. 4.

Vessel mapping in the histological sections. (a) Example of shadow projection of a set of vessels. (b) Example of mapping performed and construction of the associated rectangles covering the projection zones.

2.5 Depth Vessel Estimation

We obtain the corresponding z, depth, at each vessel point in 3 consecutive steps.

ILM and RPE layers segmentation. Firstly, we delimit the region where the vessel could be placed inside the projection zone to reduce the search space. The region of interest is delimited between the retinal layers Inner Limitant Membrane (ILM), the first intraretinal layer, and the Retinal Pigment Epithelium (RPE), formed by pigmented cells at the external part of the retina.

Canny edge detector is applied to find the limits of the layers considering that ILM and RPE contain the clearest edges due to the highest contrast of intensities of these layers. Then, in order to gain information, we apply the gradient to this image in horizontal direction, deriving strongest detections and guaranteeing a correct acquisition of both layers. Finally, the upper connected line of this detected edges corresponds to the limits of the ILM layer. The lower connected lines delimit de RPE layer, as shown in Fig. 5(a).

Vessel region detection. Each vessel is searched for within both layers, appearing as a small elliptic region with darker intensities due to its particular reflectance properties compared to retinal layers. Therefore, darkest neighbourhood inside the region of interest is identified as the center of the vessel, as illustrated in Fig. 5(b). Before that, a mean filter with a window of $3 \times 3$ is applied to smooth the ROI and avoid noisy detections.

Fig. 5.

Vessel detection in the histological sections. (a) Detection of ILM and RPE layers. (b) Detection of the vessel in the search region.

Depth calculation. After vessel location in OCT scans, their depth values z can be obtained. This is achieved by taking RPE layer lower limit as baseline and computing the height of the vessel center related to the baseline:

$$\begin{aligned} z = |C_v - P_i | \end{aligned}$$

(2)

where z is the distance that measure the depth value, $C_v$ indicates the y coordinate of the center of the detected vessel in the histological image, and $P_i$ indicates the y coordinate of the lower limit of the RPE layer.

2.6 3D Reconstruction

Having all the vessel information gathered, the 3D reconstruction of the arterio-venular tree can be performed at this stage. Each vessel is represented as a segment S, where each point $P_i$ of the segment S is represented by its spatial coordinates (x, y, z) and the diameter d. In order to get an smoother representation, an spline interpolation is applied to the set of points of S connecting them in a curve.

With all the curves of the segments, and given that the vessels are tubular structures, the 3D reconstruction is implemented as tubular shapes centered in the segment points $P_i$ of the constructed curves and with diameter d, indicating the caliber at this point of the vessel. This procedure is applied to all the interpolated curves of all the segments obtaining, therefore, the final 3D vessel tree reconstruction. Figure 6 presents a 3D representation of a crossing and a bifurcation.

Fig. 6.

Examples of 3D reconstructions. (a) A crossing. (b) A bifurcation.

3 Results

The proposed methodology was tested with a set of 196 OCT histological images that were taken with a confocal scanning laser ophtalmoscope, a CIRRUS™HD-OCT–Carl Zeiss Meditec, that also provides the near infrared reflectance retinographies. The scans are centered in the macula, from both left and right eyes of healthy patients, and with a resolution of $1520 \times 496$ pixels. Different tests have been designed to validate key steps of the methodology versus manual annotations of an expert.

Crossings and bifurcations. First of all, we analyzed the intersection reconstruction module. A total of 17 crossings and 26 bifurcations were manually annotated by a clinician expert and it was calculated the amount of them correctly identified and included by the intersection module. According to that, a percentage of success rate is shown in Table 1 as the ratio between them.

Table 1. Results obtained at different stages of the methodology.

Caliber estimation. In this case, we randomly selected 20 segments and for each segment 10 random points, summing a total of 200 points. Corresponding diameters at the selected points were also manually annotated by the clinician expert to compare with the method. At each point, the corresponding error is calculated as $d_e - d_c$ and the relative error as $\frac{d_e-d_c}{de}$, where $d_e$ is the expected diameter and $d_c$ is the diameter obtained by the method. Table 2 exposes the obtained results.

Table 2. Results obtained at the caliber estimation phase.

Fig. 7.

3D reconstruction of the vessel tree in a selected region of interest.

Vessels mapping in OCT. We also tested the accuracy of the mapping method. For all the images, points $P_e$ belonging to the projection zone $I_e$ in the OCT histological section are randomly extracted. By evaluating if the point $P_e$ is contained in the projection zone $I_c$ calculated by the proposed method a success rate can again be obtained. This was evaluated in a set of 607 mapped points. The results are shown in Table 1.

Depth calculation. We manually segmented the vessel region in the histological sections and defined as gold standard a success if the detection provided by the methodology falls in the manual detection. Table 1 shows the success rate in the analyzed images with a set of 704 annotated vessels, demonstrating that this strategy is robust in detecting the vessel location.

We finally include in Fig. 7 the 3D reconstruction of a vessel tree to show the capabilities of the method and the potential of the 3D extraction and reconstruction of the arterio-venular tree with respect to classical 2D identifications in further analysis and diagnostic and other applications.

4 Conclusions

In this paper, we propose a new methodology for 3D retinal vessel tree segmentation using OCT images that permits to obtain and represent the 3D model of a complex structure as is the vasculature of the eye. 3D segmentation approaches, instead of classical 2D ones, offer more information for the analysis of the retinal microcirculation, proven to be key for early assessment of several prevalent conditions such as hypertension, diabetes or cardiovascular risk.

The proposed methodology offers a complete analysis including a 2D retinography vasculature extraction, caliber estimation, mapping with the corresponding OCT histological sections and estimation of the depth coordinate, deriving the set of (x, y, z) coordinates and diameter d of the entire arterio-venous tree. The methodology was tested with 196 OCT histological sections and the corresponding near infrared reflectance retinographies, testing the different steps of the method, and providing promising results.

Future works plan to improve the phases of the method, in order to increase the success rates that were achieved. Moreover, a more complete validation is planned to do to strengthen the conclusions here reached. Finally, the methodology is planned for being expanded with the automatic classification of artery and vein, providing further information for analysis and diagnostic processes.