Keywords

1 Introduction

The eye allows the observation of human blood circulation in-vivo. Studies [1, 2] have shown that the arterio-venule diameter ratio (AVR) is associated with different risk factors such as hypertension, cardiovascular diseases, and diabetes. For that reason, assessing the AVR is of paramount importance in ocular examination.

Automatic assessment of AVR, or any other biomarker that manifests in the morphology, the tortuosity or the spatial configuration of the retinal vessels, require accurate segmentation of the whole retinal vascular network including veins and arteries.

Standard segmentation methods use the green channel in fundus images to extract the retinal vessel topology. The reason the green channel is usually used is that it provides the best contrast between the vessel structures and the background of the retina. However, most of these methods lack the capacity to segment the small vessels given that the contrast in the green channel is not sufficient for such segmentation. The width of these vessels and possible discontinuities in their curvature are also a major hindrance to their segmentation.

Hyperspectral imaging can potentially improve the segmentation of retinal blood vessels using images obtained at different wavelengths around the specific wavelength corresponding to the green channel. The hyperspectral imaging not only captures images of the retina at different wavelengths but also takes advantage of oximetry information which allows the varying oxyhemoglobin concentration or lack of it in a region of interest to be captured differently at each wavelength. For instance 569 nm captures a better contrast between the vessels and the retinal background while 600 nm is more sensitive to the higher level of HbO2 concentration level in the arteries.

Several methods have been proposed to automatically segment the retinal vessels from fundus images. These methods can be classified in two main categories. The first category of retinal vessel segmentation methods uses supervised algorithms [3,4,5,6]. The general idea is to train a classifier on local or global extracted features. The classifiers used in the literature include, among others, Bayesian classifiers [3], Neural Networks [4, 5] and Support Vector Machine [6]. These methods gained popularity due to their better performance when compared to their unsupervised counterparts. Their main drawback is the requirement of large datasets for the training phase. In this article, we investigate the potential advantage of using multispectral images for vessel segmentation and only a small dataset is currently available as the hyperspectral imaging is a new modality, which makes the use of supervised methods unpractical.

The second category consists of unsupervised methods [7,8,9,10,11,12,13,14] spanning a wide range of basic schemes. Jiang and Mojon [7] used multi-thresholding at different scales and combined the results of the segmentation of each level. Zana and Klein [8] used mathematical morphological operators to enhance and segment retinal vessels. Chaudhuri et al. [9] proposed a retinal vessel segmentation method based on matched filters. The authors assumed that the intensity distribution along the cross section of the vessels is Gaussian. For this purpose they applied a set of rotated Gaussian filters to detect the vessels. Several extensions were proposed to the initial matched filter segmentation method [10, 11]. However, all these methods assume that the cross-section intensity distribution follows a Gaussian profile, which is not always the case. Other works [12, 13] used tracking to trace vessels from a seed point to an extremity using local information to trace the vessels. The seed points can be either manually or automatically identified. The drawback of these approaches is that they tend to terminate at bifurcations and return an incomplete segmentation. In [14] the authors extended the vessel tracking method by using Tensor Voting to connect the fragmented vessel segments and reach the smaller sections of the vessels. Line detection was also proposed by Ricci and Perfetti [6]. Measuring the average intensity along a rotating line in a window centered at a given pixel, one can associate window size to the assumed range of vessel widths. A pixel is classified as a vessel pixel based on the line detector response. Nguyen et al. [15] expanded on this method and proposed a more generalized version. This version is dubbed the Multi-Scale Line Detector (MSLD). The MSLD algorithm proved to be a very robust algorithm when tested on the STARE [10] and DRIVE [4] datasets. Our work below is based on an extension of this approach. We chose this method for its simplicity and shorter execution time when compared to other highly achieving methods as in [14]. However, we should keep in mind that setting the parameters of the MSLD algorithm is not a trivial task that mostly affects the segmentation of the thinner blood vessels.

To the best of our knowledge, only one work in the literature [16] proposed a dual-wavelength retinal vessel segmentation. The main purpose of the article is to classify arteries and veins. For this purpose, the vessel structure was segmented using the algorithm described in [13]. However, no performance evaluation was provided for the segmentation algorithm.

In this study, we propose a method that extends the Multi-Scale Line Detector (MSLD) method [15] to segment retinal vessels in multispectral retinal images. Our contribution consists in demonstrating that the limitations of the MSLD segmentation algorithm in detecting small vessels could be compensated when multispectral images are available by combining the information from each of the segmentation results into a single vessel map.

2 Methodology Description

Retinal vessels are usually segmented from the green channel (one wavelength) in fundus image. Our method assumes that better segmentation can be achieved when more than one image obtained at different wavelengths are combined. For this reason, we use a sub-band of a hyperspectral retinal image. A hyperspectral retinal image is a 3D image that consists of several 2D images taken at different wavelengths of the same retina in the span of a second.

Each of the wavelength images was preprocessed to enhance its contrast. The vessel structure was then segmented in each of these images using the MSLD algorithm. Finally, a global segmentation map is reconstructed by adding the individual segmented vessel maps from each individual wavelength. The value of a pixel in the reconstructed image is the summation of the pixel intensities from each of the individual wavelength segmentation maps that are at the same position of that pixel. The reconstructed image is then thresholded in such a way that any value that is greater than zero is set to one. The regions wrongly detected as retinal vessels in that reconstructed image are cleared by deleting any connected region with a number of pixels smaller than a given threshold α.

3 Experimental Settings and Results

Our dataset was captured using a Metabolic Hyperspectral Retinal Camera (MHRC) developed by Optina Diagnostic (Montreal, Canada). An early prototype is described in details in [17]. Figure 1(a) shows the camera setup and Fig. 1(b) gives a rough description on the functionality of the apparatus. The camera sends a beam of monochromatic light through a tunable filter. The reflected light is collected on a 2D sCMOS sensor. The 2D images from different wavelength beams are stacked into one 3D cube to form the hyperspectral image. Compared to the previously described MHRC in [17], this version of the system included a sCMOS sensor which permitted acquisitions at a rate up to 100 frames per second.

Fig. 1.
figure 1

This picture was adapted from [17]

(a) Shows the MHRC set up (b) Shows the functionality of the system.

Full size image

The processed images are a sub-band of the original multispectral images that span the interval of 450 nm to 900 nm with a step of 5 nm. For our purposes, we considered the interval between 495 and 570 nm that spans the green region of the spectrum and presents the best contrast between the vessel structures and their environment. Thus, the dimensions of the processed images are 1536 × 1536 × 16 pixels.

The raw reflectance images were preprocessed using an in-house Matlab (The Mathworks, Natick, MA) tool. Preprocessing consists in normalization and registration of the acquired images. Normalization was necessary to account for spatial and spectral variations in the light source intensity and the system optics [17]. Registration was used to account for slight eye motion.

We set the threshold α on the small connected regions size to 800. This value was determined empirically and kept constant for all the subjects.

We processed multispectral images from six subjects including healthy controls and subjects followed at the ophthalmology clinic. We qualitatively compared the MSLD results for each of the individual wavelength images and the corresponding MSLD reconstructed image. Figure 2 shows a detailed case for subject 1. Only one case was detailed in this paper due to space constraints. In Fig. 2, the regions shown in the right columns are the enlarged detailed version of the contoured regions in one of the images that belongs to the same row. Each two corresponding regions are contoured with the same color.

Fig. 2.
figure 2

The upper 4 rows and left side 4 columns show the MSLD segmentation result for the individual images captured at the different identified wavelengths. The image in the lower left column is the preprocessed raw image at wavelength 530 nm. The middle image at the lowest row is the reconstructed segmented image. The right most column shows the enlarged contoured details at the image in the corresponding row. The yellow contour marks the same region in all involved images and the same is true for the green one. (Color figure online)

Full size image

To quantitatively evaluate the results of the proposed segmentation we built a ground truth by manually segmenting the vessels. Given that the higher wavelength images showed better contrast, the image at 570 nm was used to build the representative ground truth map for a given multispectral image. It is true that the degree of contrast in that image was not enough to permit an automatic segmentation of the thinner blood vessels. However, it was enough to allow a more precise manual segmentation. For instance, manual segmentation was able to account for discontinuities in the vessel structure in that image.

For each of the individual wavelength images and the reconstructed image, a score was calculated. The score is the normalized sum of the intersection foreground pixels in both the ground truth map and the automatically segmented map as described in Eq. (1).

$$ score = \frac{{sum\_pixels(GT_{foreground} \mathop \cap \nolimits AS_{foreground} ) }}{{sum\_pixels(GT_{foreground} )}} $$
(1)

Where \( GT \) refers to the ground truth map and \( AS \) refers to an automatically segmented map. The calculated scores for all 6 subjects are shown in Table 1. In this case the higher the score the more successful is the segmentation. Also the optical disk region is excluded from the score calculation.

Table 1. The calculated score for the reconstructed image (Recon.) and each of the wavelength images for the six subjects. The captured wavelength in subject 3 did not extend to 510 nm or lower.
Full size table

When examined, the high wavelength individual images appear to have most of the information. The vessels in these images are less fragmented than in the lower wavelength images. However, even the higher wavelength individual images, when taken separately, lack the complete information gathered in the reconstructed image. The fact that the vessels are less fragmented in the reconstructed image makes it easier to clean the final results of noisy patches. In fact, the noise was more likely to be distinguished from any fragmented vessel part. Furthermore, in the MSLD method the choice of the threshold value that provide an optimal vessel segmentation is not obvious and may depend on the intensity distribution in each image. The impact of this effect was minimized using several wavelength images. Even though the threshold value may not be optimal for some of the images, the combination of multiple images resulted in a better reconstructed segmentation map.

Figure 3 shows the results for subject 6 where the segmentation was not as successful as for the other subjects. The algorithm wrongly identified vessels in regions where anatomical features related to age-related macular degeneration (ARMD) such as the large hypopigmented region (clear zone) and the drusen (smaller white patches) were present.

Fig. 3.
figure 3

These are representative images from subject 6. The first and second images show the MSLD segmentation results at wavelength 525 nm and 565 nm respectively. The third image shows the preprocessed images captured at wavelength 530 nm where a multiple drusen and a large hypopigmented region are visible. The last image shows the reconstructed segmentation map.

Full size image

4 Conclusion

We proposed a multispectral retinal vessel segmentation technique that is based on the MSLD method. The proposed method combined information from the individual segmentation of each spectral image into a global reconstructed segmentation image. We tested our method on six different subjects. The preliminary results show the advantage of combining multispectral images to exploit more information than when using only one-wavelength image.

The current approach used to combine the information is not sufficient when signs of ARMD are present in the images. Thus, a combination method that is less sensitive to these confounding features is needed. Otherwise, a mid-processing step can be applied to the results of the individual MSLD images to exclude the wrongly-identified vessels before combination.

It would also be interesting to investigate the contribution of each wavelength image to the final reconstructed image. This can be used to determine if all the images are needed for the reconstruction.

An extensive validation is needed to understand better why the higher wavelength images gave more accurate segmentation results. We suspect the better contrast obtained for the hemoglobin and the choice of the MSLD threshold. Thus, it could be useful to adapt the threshold value for each wavelength. Also we will enhance the robustness of the method used to set the parameters of the algorithm once applied on a larger database.