Keywords

1 Introduction

Diabetic retinopathy (DR) is the most common complication of diabetes mellitus (DM) characterized by abnormal or damaged blood vessels in the retinal structure of the eye. It is one of the major causes of blindness in people of 20−65 years of age [1]. Approximately 382 million people across the world have been estimated to have DM in 2013 and this can rise to 592 million by 2035 [2]. After the onset of DM, there is increased chance for developing DR over the years. DR is asymptomatic and goes unnoticed until it reaches the advanced stage, and it is necessary to do a timely diagnosis with the help of better screening options and facilities [3]. Early diagnosis of DR helps in prevention of vision loss and impairment.

Based on the development of pathological features, DR is broadly classified into non-proliferative DR (NPDR) and proliferative DR (PDR). Various clinical features present through the different stages of DR are: Microaneurysm (MA), haemorrhages (HEM), hard exudates, soft exudates, neovascularisation and macular edema. NPDR occurs first and PDR is the advanced stage where there is development of new abnormal blood vessels. The treatment options available at the stage of PDR such as laser photocoagulation, anti-VEGF injection, intraretinal injection and vitrectomy are found to be less effective and do not provide the recovery of vision loss that has already taken place [2]. MAs are small protrusions within the capillary walls that appear as minute red dots on the retinal surface of the eye, and start to develop at the NPDR stage.

MAs are the first visible sign of DR [4]. MAs are low contrast circular structures with size ranging from 10 µm to 100 µm, usually less than 125 µm [5]. They share similar characteristics with other anatomical features such as HEM and blood vessels. It is necessary to extract MAs from other MA like structures for proper diagnosis and staging of DR. The severity of disease is indicated by the number of MAs as shown in Table 1.

Table 1. Grading of DR [5]
Full size table

It is evident from Table 1 that accurate detection of MA without overlooking them is essential for accurate staging of DR, which in turn is used for appropriate diagnosis and treatment options. Early detection of MAs can help in prevention of vision loss. People who are affected with DM must undergo regular screening to diagnose MAs at an early stage. For screening programs for a large population which involves relatively fewer expert ophthalmologists, a computer aided diagnosis (CAD) system can reduce the cost and workload involved. This works aims at the detection of MAs, using digital fundus photography (DFP) with emphasis on not missing out the difficult cases that include subtle MAs and those that are close to each other and proximal to the blood vessels.

This paper is organized as follows: In Sect. 2, recent work in detection of MAs in color fundus images is reviewed. In Sect. 3, the details of the proposed methodology for detection of MAs are presented. In Sect. 4, the results and analysis are presented. Conclusion and future scope are discussed.

2 Literature Review

Much of the related work on detection of MAs in DFP involves the following steps in common: The fundus images are first preprocessed to obtain better quality of the image and to highlight the necessary features in the image. Following preprocessing, the coarse segmentation of the fundus images is done to extract the candidate MAs. Subsequently, features are extracted from the candidate regions to distinguish false positives from true MAs. This step, called false positive elimination, is typically performed using a supervised classifier. This work addresses the coarse segmentation of MAs and hence review on literature relevant to this topic is presented below.

Spencer et al. [1], adopted subtractive shade correction and normalization for preprocessing fluorescein angiogram (FA) fundus images. Bilinear top-hat transformation was used to segment regions similar to MAs and Gaussian matched filter was employed to enhance them. Recursive region growing technique was used to extract the candidate MAs. This scheme had the disadvantage of not detecting low contrast and small MAs that were difficult to distinguish from the background. Moreover, those MAs that were conglomerated were also rejected.

In Walter et al. [6], the preprocessing was carried out on a green channel image, which provides high contrast background for dark lesions. Subtractive shade correction was carried out to alleviate non-uniform illumination in the image. Candidate MAs were detected by means of diameter closing and thresholding. In this study, an image set of 115 uncompressed digital images acquired after pupil dilation were considered were considered. The images are of size 640 × 480 with circular ROI. The major drawback in both [1] and [6] is the use of subtractive shade correction which resulted in degradation of images due to incorrect background approximation.

In Zhang et al. [7], an algorithm based on multi-scale correlation filtering and dynamic thresholding was done to extract MAs. The algorithm was evaluated on two databases namely ROC and DIARETDB1. In coarse segmentation, Gaussian kernels of different standard deviation (σ) were chosen to extract the ROIs. This was followed by adaptive thresholding to detect and eliminate the blood vessels. Higher σ value de-emphasizes sharp gradient changes in the image, thus making it more blurry.

Antal and Hajdu [8] employed dynamic selection of optimal combination of preprocessing steps and candidate extractor. Five preprocessing methods and five candidate extraction techniques were considered resulting in 25 combinations. The optimal selection of ensemble involved individual pairs to be evaluated and the final MAs were the fusion of MAs of each pair building up the optimal ensemble. Performance evaluation was tested on 199 images from three different databases namely ROC (Retinopathy Online Challenge), DIARET2.1 database and the database from Moorefields Eye Hospital, London, UK. The algorithm provided low false positive rate and low false negative rate with the use of optimal combinations, but with increased complexity and computational time taken for the system. Usage of combinational methods improves detection but with increased computational complexity.

In Zhang et al. [9], multi scale Gaussian correlation filtering (MSCF) followed by adaptive thresholding was used to locate all MA candidates. Region growing was performed on the extracted MAs and the resultant regions that were of size greater than 120 pixels were rejected. The algorithm for candidate MA detection was evaluated on the database ROC. MSCF involved the use of five different Gaussian kernels for matching MAs of various sizes. The coarse segmentation stage suffers from the disadvantage of having different scale selection which is not done automatically and could result in inaccurate detection of MAs. Increasing the number of Gaussian kernels further increases the complexity of the system.

Lazar and Hajdu [10] performed green channel extraction followed by local maximum region extraction by grayscale morphological reconstruction through breadth-first algorithm. Cross-sectional scanning was done on the image using larger cross sections of line operators. The method was tested on the ROC database. Elimination of optic disc and vessel bifurcation have not been addressed in this paper leading to false positives in the optic disc.

In Sopharak et al. [5], preprocessing was done on green channel image and denoising was done using median filter, followed by subtractive shade correction using averaging filter and contrast enhancement using contrast-limited adaptive histogram equalization (CLAHE). Then, detection and elimination of exudates and vessels were performed. Coarse segmentation of MAs was performed by using extended minima transform and diameter closing. This algorithm was also adopted by Aishwarya et al. [11] and validated on DIARETDB1 database. Subtractive shade correction resulted in incorrect background approximation. Other demerits of this algorithm were its inability to detect too small MAs and those MAs that were located near to the blood vasculature. Faint vasculatures were also left undetected.

In Tavakoli et al. [12], top-hat transform was used to decrease background variation. In order to remove small MA-like noise, averaging was done. The preprocessed image was then subdivided into several subimages. The vascular tree was then detected and eliminated by using Radon transform in all the subimages obtained, resulting in coarse segmentation of MAs. Performance evaluation was done on three different retinal image databases, the Mashhad database with 120 FA images, a local database with 50 FA images and ROC (Retinopathy Online Challenge) with 22 images. Some MAs that were located near to each other and too big MAs were wrongly detected as blood vessels.

Rosas-Romero et al. [4], computed the ratio of green to red channel for shade correction. This was followed by median filtering for denoising and pointwise pixel transformation for spatial normalization. The ROIs were extracted using bottom-hat transformation and thresholding techniques which are also adopted in [13]. This was followed by hit or miss transformation to segment the MA candidates. Too small MAs that were close to each other and conglomerated MAs were found to get eliminated in the hit or miss transformation stage. Faint MAs were left undetected due to low contrast image.

The proposed method involves the use of a simple yet robust method namely for accurate extraction of optic disc and blood vessels simultaneously, in a single step. This is done by employing bottom-hat transformation which extracts only dark regions and also performs optic disc elimination at the same time, resulting in improvement of processing speed and reduced complexity [4]. Further, it alleviates false positives resulting from improper segmentation of optic disc. Shade correction using green to red channel ratio was done as a replacement to background approximation, resulting in better image quality. In almost all related work, MAs that are close to each other and to the blood vessels were not detected properly. This paper aims at improving the detection of MA candidates through the use of modified contrast enhancement technique using morphological operations, and multiple SEs in the coarse segmentation stage.

3 Methodology

The overall flow of the proposed method is illustrated in Fig. 1. Broadly the steps involved include preprocessing, candidate extraction for dark object filtering and finally segmentation of candidate MAs.

Fig. 1.
figure 1

Framework for coarse MA detection from color fundus images

Full size image

3.1 Image Preprocessing

3.1.1 Shade Correction

The fundus images are affected by non-uniform illumination that results from factors including curvature of retina, misalignment of patient’s eye and fundus camera, ocular opacities, improper dilation of pupil, poor focus of camera and inadequate illumination. This causes gradual decrease in illumination from the region of optic disc towards the periphery. The red and green channels of a fundus image contain most of the image information. The green channel of the fundus image provides the highest contrast for all blood-filled structures while red channel exhibits highest reflectance of red color and appears bright. On the contrary, the blue channel contains the least informative content as blue is absorbed by most parts of the eye. Reducing non-uniform illumination by the popular subtractive shade correction has its own demerits in choosing the appropriate size of averaging filter for background approximation. Therefore, reduction of non-uniform illumination has been performed by the red and green channels exploiting the fact that the ratio of green to red channel is a constant independent of illumination. This is computed in accordance with Eq. 1.

$$ {\text{F}}_{\text{s}} \,\left( {{\text{r,}}\,{\text{c}}} \right)\,{ = }\,\left[ {{{{\text{f}}_{\text{G}} \,\left( {\text{r, c}} \right)} \mathord{\left/ {\vphantom {{{\text{f}}_{\text{G}} \,\left( {\text{r, c}} \right)} {{\text{f}}_{\text{R}} }}} \right. \kern-0pt} {{\text{f}}_{\text{R}} }}\left( {\text{r, c}} \right)} \right] $$
(1)

where Fs (r, c) is the shade corrected image, fG (r, c) is the green channel component at the row r and at column c, fR (r, c) is the red channel component at the row r and column c.

The shade corrected image is shown in Fig. 2(b)

Fig. 2.
figure 2

(a) Original image and (b) Shade corrected image by employing green to red channel ratio

Full size image

3.1.2 Denoising

The common types of noise that affect the fundus images are salt and pepper noise, shot noise and Gaussian noise. In order to eliminate the effect of noise on retinal images, the shade corrected image is denoised using a combination of median and Gaussian filter. Median filter has been proved to be effective in removal of salt and pepper noise with edge preservation while Gaussian filter provides effective noise attenuation for Gaussian noise and Poisson noise. The result of denoised image after performing shade correction is shown in Fig. 3(b)

Fig. 3.
figure 3

(a) Shade corrected image and (b) Denoised image using median-Gaussian filter

Full size image

3.1.3 Illumination Normalization

For reducing the inter-image illumination variations, which could arise due to diversity in ethnicity, illumination normalization of the image is performed using the pixel transformation using Eq. 2 [4].

$$ \text{I}_{\text{a}} \text{(r, c) = }\frac{{\upsigma_{\text{n}} }}{{\upsigma_{\text{I}} }}\text{ }\left( {\text{I}_{\text{b}} \left( {\text{r, c}} \right){ - \mu }_{\text{I}} { + 2\sigma }_{\text{I}} } \right){ + \mu }_{\text{n}} { - 2\sigma }_{\text{n}} $$
(2)

where \( \text{I}_{\text{a}} \text{(r, c) } \) and \( \text{I}_{\text{b}} \left( {\text{r, c}} \right) \) are the image grayscale values at position (r, c) after and before normalization, \( \upsigma_{\text{I}} \) is the standard deviation of the image, \( \upsigma_{\text{n}} \) is the reference standard deviation, median of standard deviation of all images,\( \upmu_{\text{I}} \) is the mean of the image, \( \upmu_{\text{n}} \) is the reference mean, median of mean of all images.

In this step, the mean and standard deviation of all the images get transformed to the reference mean and standard deviation value. Normalization of grayscale content plays an important role during thresholding. Proper normalization helps in choosing a single threshold value for all images. The images before and after normalization are shown in Fig. 4.

Fig. 4.
figure 4

(a) Denoised image and (b) Normalized image

Full size image

3.2 Bottom-Hat Transformation

The algorithm utilizes morphological techniques to perform dark region extraction. The dark regions present in the retinal images are MAs, blood vessels, HEM and noise. The goal of the first step is that the red regions corresponding to the local minima of original image should be enhanced, while the bright regions like the optic disc corresponding to the local maxima namely should be eliminated. Bottom-hat transformation otherwise called black top-hat has been used in the proposed method for extraction of dark regions. Bottom-hat operation involves subtraction of input image from the morphologically closed image.

$$ {\text{f}}_{\text{bh}} = [({\text{f}} \bullet {\text{b}})- f] $$
(3)

where f is the input image to this stage, b is the structuring element used for closing operation, \( \bullet \) is the closing operator, fbh is the bottom-hat transformed image.

The closing operation is performed using an SE of disk size ‘9’, which is chosen empirically as the appropriate size of MAs. Choosing the appropriate size of SE is important in extracting all MAs. Performing two-dimensional bottom-hat transform results in isolation of certain regions of blood vessels which may be misclassified as MAs. In order to reduce the occurrence of false positives, 1D bottom-hat operation is performed row-wise and column-wise, the results of which will be combined using logical AND operation in the final step. The results of bottom-hat transformation are presented in Fig. 5(a−c)

Fig. 5.
figure 5

(a) Results of 2D bottom-hat, (b) 1D bottom-hat over every column and (c) 1D bottom-hat over every row, (d–f) Results of contrast enhanced image for corresponding 2D and 1D bottom-hat results using morphological enhancement.

Full size image

3.3 Contrast Enhancement

To further enhance the faint MAs, contrast enhancement is performed using morphological techniques. CLAHE proves to be inefficient since it does not pick up many MAs which are of low contrast. To improve the sensitivity, a combination of top-hat and bottom-hat transform is used for enhancement which retains almost all MAs in the thresholding stage. Enhancement is performed using the expression in Eq. 4.

$$ {\text{A}}_{\text{E}} \,{ = }\,{\text{A}}\,{ + }\,\text{A}_{\text{TH}} \, - \,{\text{A}}_{\text{BH}} $$
(4)

where A is the input image (result of previous processing step), ATH is the top-hat transformed image and ABH is the bottom-hat transformed image (result of employing different size of SE), AE is the enhanced image.

This technique improves the contrast selectively for dark lesions and blood vessels for an SE of size chosen to be 9 and 20 pixels.

3.4 Thresholding

Following enhancement, thresholding is performed. Binarization is applied to both 2D and 1D bottom-hat results using Otsu’s method. Empirical choice of threshold value being employed is avoided using Otsu’s thresholding. The results of thresholding operation on these outputs are presented in Fig. 6(d−f), respectively.

Fig. 6.
figure 6

(a-c) Results of enhanced bottom-hat transformed images and (d-f) corresponding thresholded images.

Full size image

3.5 Hit or Miss Transformation

The thresholded image contains all the dark candidates that include MAs, blood vessels and HEMs. Appropriate methods have to be used to detect only the candidate MAs and eliminate other non-MA structures. This two-step procedure is performed using a single technique, hit or miss transformation on both 2D and 1D images separately. Hit or miss transformation is a morphological technique that can extract specific shapes of interest. Blood vessel removal is automatically achieved by the hit or miss transform, due to its ability to discriminate structures based on shape. Though HEMs are also roughly circular, they are much larger and the size-based discrimination of hit-or miss transformation is capable of eliminating HEMs in the detection process. The proposed algorithm uses SEs to exactly match the size of MAs while removing other non-MA structures and noise simultaneously, resulting in detection of MA candidates alone. Circular SEs are built using inner and outer disk structures separated by a black ground with a small white region located inside the inner disk as shown in Fig. 8. The size of inner white region is limited to a radius of 3 pixels so as to discard regions smaller than this, which correspond to noise. Don’t care condition is introduced inside the inner disk for providing flexibility to match all MAs with varying sizes and irregular shapes and also outside the outer disk to detect other neighbourhood MAs. The black background helps in removal of blood vessels and in detection of two or more MAs as separate structures. Unlike MAs, blood vessels do not have the black background and thus gets eliminated in the process. MAs near to each other and close to the vasculature often get missed out. Those MAs which are clubbed together and are of size larger than the SE are also not detected. Therefore, selection of single SE cannot detect those MAs which are of large and small sizes when compared to the typical size and also those which are clubbed together or overlapped with each other.

To improve sensitivity of MA detection, SEs of different sizes are chosen to accommodate all possible MA candidates that do not fit in the particular size of SE. MAs are irregular shaped structures that are approximately 9 pixels in size. Thus the optimum size of circular SE is chosen with an inner radius of 9 pixels and outer radius of 11 pixels with a 2 pixel gap for the black background. Thus the lower limit on the detected regions of 3 pixels is imposed by the white region while the don’t care region imposes an upper limit on the inner radius of up to 9 pixels. Similarly, other sizes of SEs are chosen with inner and outer radii of 6 and 7 pixels, 13 and 15 pixels, 18 and 20 pixels respectively. Smaller SE with 6 ad 7 pixels radii is chosen to detect smaller MAs and those that were partially detected in the binarization stage, also retaining MAs that are close to each other providing a gap of 1 pixel. The large radii SE of 13 and 15 pixels are used to detect larger MAs, and those MAs that were clubbed together in thresholded image are detected using SE of 18 and 20 pixels radii. By adopting various SEs, a significant increase in the detection results was achieved. The optimum size SE (9 and 11 radii) is shown in Fig. 7. The corresponding candidate MA extracted is shown in Fig. 8. The resulting images of different SEs are combined using logical OR operation for their respective 2D and 1D images in the latter stage.

Fig. 7.
figure 7

Optimum size of SE

Full size image
Fig. 8.
figure 8

(a) Results of hit or miss transformation with 2D thresholded image, (b-c) with 1D thresholded image.

Full size image

3.6 Extraction of Connected Components

The extracted candidate MAs do not cover the entire region encompassed by the MA. To recover the entire shape of MA and to eliminate those regions of blood vessels that are still detected in the hit or miss transformation stage, extraction of connected components as in Eq. 5 is performed with 3 iterations on the binary image resulting from hit or miss transform.

$$ {\text{X}}_{\text{K}} \, = \,\left( {{\text{X}}_{\text{K - 1}} \oplus \,{\text{b}}} \right) \cap \,\text{f} $$
(5)

where XK-1 is the image dilated with the structuring element ‘b’, until the complete shape of the component is extracted,

  • ⊕ is the dilating operator,

  • b is the suitable 5 × 5 square SE for performing dilation operation,

  • f is the thresholded image used for extraction of connected components,

  • XK is the extracted component image.

The images after extraction of connected components contain portions of blood vessels along with extracted MAs which are eliminated by performing logical AND operation on 2D and 1D images. The 1D image along vertical column will recover only vertically oriented blood vessels and 1D image along every row will recover only horizontal blood vessels and 2D image extraction will result in recovering both horizontal and vertical blood vessels. Hence the common portions of blood vessels are alone extracted by performing AND operation of images, reducing the occurrence of false positives. The results of extraction of connected components after performing logical AND operation is shown in Fig. 9.

Fig. 9.
figure 9

(a) Candidate MAs after extraction of connected components, (b) Fundus image with the detected MAs after coarse segmentation.

Full size image

4 Results and Conclusion

4.1 Database Description

The color fundus images considered in the study were taken from DIARETDB1 database. All the images were captured with the 50° field-of-view digital fundus camera. There are totally 89 images which were taken in the Kuopio University hospital. Out of the 89 images, 84 contain at least mild non-proliferative signs (MA) of the diabetic retinopathy and 5 are considered as normal which do not contain any signs of diabetic retinopathy. Ground truth images annotated by expert groups are provided for reference.

4.2 Coarse Segmentation Results

The performance of MA detection using the proposed method is analyzed using FROC curve obtained using varying the threshold value in the binarization stage. The FROC curve plots average number of false positives to the sensitivity obtained for varying values of threshold. A set of 20 images each with approximately 30 to 50 MAs were considered. The FROC curve for the proposed method was obtained with the use of contrast enhancement through modified morphological enhancement, also employing the use of combination of all four SEs whose size and role are tabulated in Table 2. This plot was compared to the FROC curve obtained through the use of only the optimum size of SE in the hit or miss transformation stage, with no contrast enhancement. The results are shown in Fig. 10. It can be observed from the results that there is drastic up shift in the curve obtained through the proposed method, verifying that the sensitivity of MA detection has been improved. This improvement in MA detection rate is because of the ability of the proposed work to capture the difficult cases including small MAs, faint MAs, and MAs close to each other and to the vasculature, which are overloaded by the existing systems. In future, this work could be extended by incorporating fine segmentation to eliminate false positives. Increase in false positives in the attempt to increase the sensitivity of the system can therefore be addressed by the fine segmentation stage.

Table 2. Various structuring elements used to perform hit or miss transformation
Full size table
Fig. 10.
figure 10

FROC plots comparing the coarse segmentation results of the proposed method (curve marked in blue) with the previously used method (curve marked in red). (Color figure online)

Full size image

4.3 Conclusion

The result of coarse segmentation stage has not been reported in many papers. Only the performance of the classifier employed in the fine segmentation stage (false positive eliminated) has been discussed much. As a result, there is no true picture on the number of missed out MAs during coarse segmentation. This paper has successfully reported the results of coarse segmentation thus presenting the true sensitivity rate. From the coarse segmentation results obtained, it can be seen that the proposed method has achieved higher sensitivity by detecting almost all MAs that were difficult to detect otherwise. The contrast enhancement using modified morphological enhancement improved the detection of faint MAs that were difficult to distinguish from the background. Choosing and employing different SEs for detection of MAs greatly improved sensitivity by picking up the MAs that were getting missed out in other methods of detection process. Improving MA detection rate in the coarse segmentation is important to achieve an overall high sensitivity. Further reduction of false positives is carried out in the fine segmentation stage.