1 Introduction

Retinal images are widely used in screening eye diseases, such as the diabetic retinopathy, glaucoma, and age-related macular degeneration [1, 2]. High quality image is a guarantee of success for the diagnostic purpose. However, the image quality is not always satisfactory due to major reasons like acquisition process and eye diseases, and low contrast and insufficient illuminance are common problems [3,4,5].

Using a high-performance camera is a direct way to improve the image quality, but a high-performance camera is often expensive, which is hard to wide spread in less developed regions. Besides, improving the professional skills of photographers can also be useful. But these two means are not helpful if the image quality decrease is caused by eye diseases.

The eye diseases that cause retinal image degradation include small pupil, corneal edema, uveitis, or vitreous hemorrhage, etc. Among these causes, cataract plays a leading role. Cataract is defined as lenticular opacity, and it reduces the transparency of lens [6]. The good visual perception of retinal image is difficult to be obtained, since the contrast of retinal structure is reduced during the imaging process.

The blurriness of retinal image can be classified into four grades [7], and four examples are shown in Fig. 1a–d. Figure 1a is a clear image, in which the optic disc, macula, main vessels, and even capillary vessels can be clearly seen. Figure 1b is a slightly blurred retinal image, in which the main vessels and optic disc are visible, while the capillary vessels are only faintly visible. Figure 1c shows a moderately blurred image, and only main blood vessels and optic disc are visible. In Fig. 1d, almost no retinal structures can be observed. In addition, some retinal images not only suffer low contrast but also are underexposed. Two examples are shown in Fig. 1e, f, and both of the two blurry images are with insufficient illuminance compared with Fig. 1a–d.

Fig. 1
figure 1

Retinal images. a Clear image; b a slightly blurred image; c a moderately blurred image; d a severely blurred image; e, f two blurry images with insufficient illuminance

Full size image

Compared with the clinical notes the advantages of using photographs are noticeable, because some abnormalities in retinal images are difficult to describe [8, 9]. If the image is blurry or has insufficient illuminance, the alteration of disease that occurs over time cannot be disclosed in time. Besides, with the rapid development of automatic image processing [1, 10,11,12,13,14,15,16,17,18], the automatic retinal image processing has been extensively researched in the past two decades, such as eye disease detection [11, 12], vessel segmentation [16, 17], and vessel tracking [10, 18], etc. However, the reliability of automatic retinal image processing is easily affected by the poor image quality [15].

Image enhancement is an essential step to better identify the retinal structures in degraded retinal images. In the enhancement of blurry images, contrast improvement is the primary factor. Both of the retinal structures and pathological changes should be more visible after enhancement. In addition, the illuminance of underexposed image is weak, in which the retinal structures are hard to be recognized. Thus, the illuminance improvement for such images should be taken into account as well.

Based on the above analysis, we developed two solutions for the enhancement of blurry retinal images with different illuminance, and our contributions can be summarized in three aspects.

  • We propose a novel contrast stretching strategy to improve the visibility of retinal structures. In this strategy, the relatively low parts of pixel values are termed as the base-intensity and removed, and the non-uniform addition is further performed to improve the contrast.

  • An effective intensity transfer strategy is introduced, which can provide required illuminance for a single channel after contrast stretching. A guided image is needed in this process, which can be the original image or another image. The color correction can be achieved if the intensity transfer is performed on three channels.

  • On the basis of the contrast stretching and intensity transfer, we develop two solutions for enhancing blurry retinal images with different illuminance.

2 Related works

The specialization and modularization based methods are required for the enhancement of retinal images, because retinal images have special texture characteristics and color. Four state-of-the-art methods that are highly related to our proposed solutions are reviewed [19,20,21,22]. The three methods in [19, 21, 22] are developed for enhancing retinal images. The method in [20] is developed for the enhancement of low-light outdoor images, which can also be used to improve the quality of retinal images with insufficient illuminance.

It is essential to note that the classical image enhancement methods, such as the contrast limited adaptive histogram equalization (CLAHE) and multi-scale retina cortex (MSR) are not reviewed. That is because CLAHE has already been employed in [19, 22], and the authors of [21] claimed that their performance is superior to CLAHE. The MSR is assessed in [20], and its performance is not as good as the multi-layer image enhancement model. Thus, the CLAHE and MSR are not mentioned.

2.1 Contrast improvement–based methods

The main purpose of methods in [19, 21] is to improve the sharpness of retinal structures. The method of [19] achieves this goal in three steps. The authors assume that the opacity medium exists in low frequency component, and the low pass filtering in frequency domain is used to obtain the opacity medium. After removing opacity medium, the CLAHE is performed. Finally, a round of linear contrast stretching is operated on green channel to further improve the contrast. Different from the method in [19], the authors of [21] estimate the amplification coefficients for each pixel based on the image formation model. Results show that both of the two methods can obviously improve the contrast of retinal structures.

2.2 Illuminance improvement–based method

Different from methods in [19, 21], the illuminance improvement is taken into account as well [22]. The Gamma map is used to enhance the global illuminance, since it can effectively increase the dynamic range of low gray level pixels, and image details in dark areas can be better observed. After that, the CLAHE is applied on the lightness channel, and the contrast of retinal structures is significantly improved. The theory of this method is simple, but it is stable and with high efficiency.

2.3 A multi-layer model

The authors focus on the quality improvement for low-light outdoor images [20]. They assume that the image details exist in each frequency band, and a multi-layer image enhancement model is derived to adjust the illuminance and contrast in each frequency band. In this process, the multi-layer lightness statistics of high-quality images are incorporated into the multi-layer enhancement model. Results show that the multi-layer model works well on low-light images, and the illuminance improvement and contrast stretching are achieved at the same time.

According to the review of related works, we realized that the contrast improvement is crucial for both blurry retinal image and underexposed retinal image. For the retinal images with insufficient illuminance, the dark area will become even darker if only the contrast is improved, and the over enhancement will be introduced. Thus, the illuminance improvement should be considered for such images as well. Accordingly, the blurry retinal images with different illuminance should be treated differently.

3 Methodology

The contrast stretching and intensity transfer are the main steps in this work. The reason of introducing the intensity transfer is because the color of retinal image often changes obviously after contrast stretching. Thus, the intensity transfer is further developed, since it can be used for the color restoration.

3.1 The proposed contrast stretching strategy

3.1.1 Theoretical analysis

The proposed contrast stretching strategy is inspired by the cross sections of retinal structures. Three cross sections of the blood vessel, optic disc, and macula from the green channel of Fig. 1a are taken as examples. According to Fig. 2, the blood vessel and macula have much lower intensity than the surrounding pixels, while the pixels of optic disc usually have the highest intensity. The intensity difference in the three cross sections are 65, 102, and 42, and the retinal structures are clear.

Fig. 2
figure 2

Cross sections of different retinal structures. a Vessel; b optic disc; c macula; d, e, f the pixel values of the cross sections

Full size image

Think about that a simplest contrast stretching method can be realized by multiplying the three sets of pixel values by two. After that, the intensity difference will naturally increase which doubles the previous, and the sharpness of visual effect will be improved. However, the assumption is not valid, since the pixel values are easy to saturate which will exceed 255.

In Fig. 2, the minimal values on the three cross sections are 89, 136, and 38, respectively. The interval between 0 and the minimal value only raises the intensity in the multiplication process, which has no contribution in improving the contrast in local areas. We called this interval as the base-intensity in this study, which should be removed before the contrast stretching.

3.1.2 Base-intensity estimation

The illuminance of a retinal image is uneven, in which the macula is dark while the optic disc is often the brightest part. The base-intensity is different in different local areas. Instead of using a fixed value, the Gaussian function is used to estimate the base-intensity in local areas. A two-dimensional Gaussian function is expressed as

$$ G\left(x,y\right)=a{e}^{-\frac{{\left(x-\mu \right)}^2+{\left(y-\mu \right)}^2}{2{\sigma}^2}}, $$
(1)

where x and y denote the coordinates of pixels, and μ is the central point in the square kernel. σ determines the falling gradient of the Gaussian function. In order to guarantee the integral G(x, y) is 1, a is always set as

$$ a=\frac{1}{2\pi {\sigma}^2}. $$
(2)

In order to obtain the lower parts of pixel values, the Gaussian kernel is constrained by a coefficient θ which is shown as

$$ a\hbox{'}=\theta \cdotp \frac{1}{2\pi {\sigma}^2}. $$
(3)

In formula (3), θ is used to decrease the weights of each coefficient in Gaussian kernel. The value of θ should be less than 1, only then can the estimated base-intensity be meaningful. Effectively estimating the base-intensity is difficult, since there are always pixels whose estimated base-intensity is greater than the intensity of itself. Thus, the negative values are inevitably generated after subtracting the base-intensity. To guarantee the effectiveness of contrast stretching and good visual perception, we allow to generate a small percentage of negative pixel values in removing the base-intensity. The value of θ is set adaptively for each image according to the percentage of negative pixel values. We use formula (4) to process these negative values. Suppose Ir is the image after base-intensity removal, and formula (4) is expressed as

$$ {\mathrm{out}}_p=\frac{{\mathrm{in}}_p-{\min}_p}{T_p-{\min}_p}\cdotp \left({T}_p-{\min}_p\right)\cdotp {T}_p, $$
(4)

where inp denotes the pixels that have less intensity than Tp in Ir, and minp denotes the minimal value of Ir. For example, if Tp is set as 30, it means that these negative pixels are remapped to (0, 30). This is a compression process, which ensures no information loss in the process of base-intensity removal.

3.1.3 Non-uniform addition

After removing the base-intensity, the non-uniform addition is used to enhance the contrast. The principle of addition is that the higher the pixel value, the higher the added intensity. Thus, the contrast can be better improved. The value of any pixel should not exceed 255 after addition. To satisfy these conditions, the compressed Gamma map is used to generate intensity curve for addition. Different from its classical usage, the Gamma map does not directly encode the input pixel values, but it is compressed for addition in this approach. The Gamma map is a non-linear function which is expressed as

$$ G\left(i,j\right)=F{\left(i,j\right)}^{\gamma }, $$
(5)

where F(i, j) and G(i, j) denote the input and output pixels, and γ is the scale coefficient. The shape of Gamma map is convex when γ is less than 1. Although it meets the requirement of contrast stretching, the efficiency is low. That is because the added intensity for dark pixel is relative high, and the intensity difference is hard to be fully stretched. Thus, the shape of Gamma map should be concave, i.e., the γ value should be greater than 1. When γ is set as 1.5, the Gamma map and the diagram of compression are shown in Fig. 3.

Fig. 3
figure 3

Addition curve based on the compressed Gamma map

Full size image

In Fig. 3, the left part denotes the standard Gamma map. The horizontal axis is the range of pixel values, and the vertical axis represents the encoded values by using formula (5). The curve of Gamma map cannot be directly used for addition, because the pixel value is easy to exceed 255. Thus, the compression is needed as shown in the right part of Fig. 3. A and B denote the minimal pixel value and maximal pixel value in Ir. The lower limit of the compressed curve should be the same with A (A ≥ 0). The upper limit is defined as (255–B). That is, the range of horizontal axis is compressed from (0, 255) to (A, B), and the vertical axis is compressed from (0, 255) to (A, 255–B). The compressed curve is taken as the look up table for addition. In this manner, there will be no pixel value exceeding 255 after addition.

3.1.4 A complete round of contrast stretching

A complete process of contrast stretching based on a single channel is drawn in Fig. 4. Before contrast stretching, a round of padding is needed. It aims to avoid the over enhancement of retinal boundary. The base-intensity is removed based on the constrained Gaussian kernel, and formula (4) is used to remap negative values to positive. After that, the non-uniform addition and padding removal are performed. Up to now, a complete round of contrast stretching is realized.

Fig. 4
figure 4

A round of contrast stretching based on a signal channel

Full size image

Figure 4 shows the process of one round contrast stretching on a single channel, and the contrast improvement can be visually observed. Based on Fig. 4, the contrast stretching is performed on all three channels, and the result is shown in Fig. 5. It is observed that the sharpness of stretched retinal image is obviously improved. However, the color of retinal image changes obviously after contrast stretching. Since the color of retinal image is useful in clinical diagnosis, the color correction should be considered.

Fig. 5
figure 5

Performance of contrast stretching on a color image. a Original image; b the contrast stretched image

Full size image

3.2 Color correction based on intensity transfer

Based on the assumption of base-intensity, the contrast improvement of retinal structure is obvious, but the color of retinal image is easy to change. That is because the base-intensity removal and non-uniform addition will change the original intensity proportion of the three channels. Thus, the intensity transfer is further developed, and the color can be corrected after the intensity transfer is performed on three channels. The proposed intensity transfer is based on the Gamma map. If we directly perform the adjustment on the original image, the contrast will be reduced again. Because the Gamma map will compress the bright pixels when γ is less than 1 or compress the dark pixels when γ is greater than 1.

Instead of adjusting the intensity on original image, we choose to modify the low-frequency component, because the low-frequency component is much smoother than the original image, which reflects the mean intensity in local areas. The process is based on formula (6), and the performance of three channels’ superposition is illustrated in Fig. 6.

$$ \mathrm{I}\left(x,y\right)=\mathrm{Low}\left(x,y\right)\cdotp \mathrm{Coff}\left(x,y\right), $$
(6)

where Low(x, y) denotes the low frequency component, and original image I(x, y) is the multiplication of low frequency and coefficients. By only modifying the intensity level of low-frequency component, the original intensities of three channels can be corrected. In Fig. 6, the original blurry image is used as the guided image for intensity transfer.

Fig. 6
figure 6

Diagram of intensity transfer

Full size image

In Fig. 6, the guided information include the average pixel values (APV), the range of pixel values (RPV), and the minimal pixel value (MinPV), and they are all calculated from the low frequency component of a single channel. The low frequency component is obtained using Gaussian filter. Combined with the Gamma map, the intensity of each channel is adjusted according to the guided information. The flow chart of the intensity adjustment based on a single channel is presented in Fig. 7.

Fig. 7
figure 7

Flow chart of intensity adjustment based on a single channel

Full size image

In Fig. 7, APV1 denotes the average pixel value calculated from the current input channel, and APV is calculated from the guided image. γ is the index of Gamma map. If APV1 is less than APV, γ should be set to a value less than 1, so that the intensity of input channel will increase after the Gamma map. If APV1 is greater than APV, the value of γ should be greater than 1. It ensures that the intensity of input image moves towards the guided image. In Fig. 7, formula (7) is used to adjust the range of pixel values, and formula (8) can nonlinearly increase or decrease the pixel values. They are expressed as

$$ {\mathrm{C}}^{"}\left(\mathrm{x},\mathrm{y}\right)=\left(\frac{{\mathrm{C}}^{\prime}\left(\mathrm{x},\mathrm{y}\right)-\min \left({\mathrm{C}}^{\prime}\left(\mathrm{x},\mathrm{y}\right)\right)}{\max \left({\mathrm{C}}^{\prime}\left(\mathrm{x},\mathrm{y}\right)-\min \left({\mathrm{C}}^{\prime}\left(\mathrm{x},\mathrm{y}\right)\right)\right)}\cdotp \mathrm{RPV}+\mathrm{MinPV}\right), $$
(7)
$$ {\mathrm{C}}^{\prime}\left(\mathrm{x},\mathrm{y}\right)={\left(\mathrm{C}\left(\mathrm{x},\mathrm{y}\right)\right)}^{\upgamma}, $$
(8)

where C(x, y) denotes the channel under processing, and C ' (x, y) is the result after a round of Gamma map. RPV and MinPV are the range of pixel values and minimal pixel value in the low frequency component of guided channel. C '  ' (x, y) is the updated channel after a round of intensity adjustment. When the difference of APV1 and APV is less than the defined threshold, the iteration is terminated, and the threshold selection is described in discussion.

3.3 Two solutions for blurry retinal image

3.3.1 Blurry retinal image with sufficient illuminance

The framework of enhancement solution for blurry retinal image with sufficient illuminance is shown in the upper part of Fig. 8. The contrast stretching is performed on each channel in a color retinal image, and then the intensity transfer is conducted by using the original image as guided image.

Fig. 8
figure 8

Block diagram of the two solutions

Full size image

Clearly identifying the hidden retinal structures is the main purpose to enhance the blurry retinal image with sufficient illuminance. Thus, we directly perform the contrast stretching on the three color channels. Besides, the image color should not be changed after enhancement, and therefore, the original image is used as the guided image.

3.3.2 Blurry retinal image with insufficient illuminance

This solution also aims to keep the original color after enhancement, and this process is realized by only modifying the non-color channel. The HSI (hue, saturation, and intensity) color space is selected in this solution, as shown in the bottom part of Fig. 8. For the contrast stretching, a round of Gamma map is firstly operated, since the Gamma map can increase the contrast of relatively dark pixels when γ is less than 1 [22]. Thereafter, the proposed contrast stretching strategy is performed. At last, another image with required illuminance is selected as the guided image in intensity transfer, and the guided image should be converted to HSI color space as well.

This solution is inspired by the color space transformations, which include three widely used color spaces. They are the spaces of LAB (luminosity, A: red to green, B: yellow to blue) [22], HSI (hue, saturation, and intensity) [19] and HSV [20]. Although enhancing the non-color channel can improve image quality without introducing obvious color variation, the performance difference among the three color spaces is rarely mentioned. Thus, the color space associating with the optimal enhancement performance should be obtained.

By using the same operations and guided image, two sets of enhanced images based on the three color spaces are shown in Fig. 9, and only the non-color channels are processed. The contrast of the two retinal images is improved by using each of the three color spaces. However, the color is degraded obviously by using LAB space, since the enhanced images become grayish. The results based on HSV and HSI have good color fidelity. As for HSV color space, the contrast and illuminance are more obvious, but the noise is enlarged as well.

Fig. 9
figure 9

Performance comparison based on different color spaces. a Original images; b enhanced images based on LAB color space; c enhanced images based on HSV color space; d enhanced images based on HSI color space

Full size image

More concretely, the channels of hue and saturation in HSI and HSV color spaces are the same, and the only difference is the non-color channel. The I channel is the mean of the red, green, and blue channels, while the V channel of HSV color space represents the maximal pixel value of the red, green, and blue channels. Compared with the maximal value, the average value can better represent the illuminance level. Thus, the HSI color space is used.

4 Experimental results and discussion

For the retinal images of different blurriness, there is no need to enhance the clear images as the details can be clearly seen. As for the severely blurred images, there is no need to process as well, because almost all the retinal structures have disappeared. Thus, the testing set is consisted of slightly blurred ones and moderately blurred ones. A total of 200 blurry retinal images with sufficient illuminance and 100 blurry retinal images with insufficient illuminance are tested, and these images are all collected from cataract patients visiting Beijing Tongren Hospital. The capability of intensity transfer is firstly assessed. Thereafter, the subject analysis and quantitative analysis are followed. These four state-of-the-art methods reviewed in section 2 are all employed for the comparison study. The method in [21] is our former research, and the code of [20] is provided by the authors. The two methods of [19, 22] are implemented according to their publications.

4.1 Capability of intensity transfer

For retinal images with insufficient illuminance, the intensity of each channel is relatively weak, and therefore the original image is not suitable to be the guided image. In this situation, the guided image can be another image. To test the capability of intensity transfer when using a different image as the guided image, we select one image from the DRIVE database [23] as the guided image for this test. The DRIVE database is established to enable comparative studies in retinal images, and researchers usually test their algorithms on this database and share the results. This database includes 40 standard retinal images which have similar color. The color change is easier to observe than the intensity change in a single channel, so the test is performed in three channels. As shown in Fig. 10, the colors of the three images are all corrected after the intensity transfer, which demonstrates the effectiveness of the proposed intensity transfer strategy.

Fig. 10
figure 10

Capability of intensity transfer using a different guided image. a, b, c Three blurry retinal images with insufficient illuminance; d, e, f the enhanced images by using the guided image for intensity transfer

Full size image

4.2 Visual assessment of enhancement

The two methods of [19, 21] only focus on the contrast improvement, while [20, 22] also improve the illuminance. Thus, the methods of [19, 21] are compared with solution 1, and [20, 22] are compared with solution 2.

4.2.1 Blurry retinal images with sufficient illuminance

Four blurry retinal images and their corresponding enhanced images are shown in Fig. 11. In the four sets of images, label (a) denotes the original image. The labels of (b), (c), and (d) represent the results of [19, 21] and the proposed solution 1, respectively. It can be observed that the visibility of retinal structures like blood vessels, optic disc, and macula is improved by all of the three methods.

Fig. 11
figure 11

Visual assessment for images with sufficient illuminance. a Original images; b results of [19]; c results of [21]; d results of the proposed solution 1

Full size image

In Fig. 11, the color of image 2(b) changes obviously compared with the original image. That is because a linear stretching is operated on the green channel in [19]. When encountering such an image that its intensity of green channel is low, the intensity of green channel will be increased after stretching. Accordingly, the color of enhanced image is changed. The limitation of [21] is over enhancement, such as the bright retinal boundary in image 1(c), 2(c), and especially 4(c). As a result, the visibility of retinal structures around the boundary is affected. The reason is because there is no padding before enhancement, and therefore the pixels around the boundary are over stretched. In image 3(c), the macula and blood vessels in optic disc are also over enhanced, and the macular becomes much darker, which may be mistaken as an abnormality. As for the proposed solution 1, it performs well in contrast improvement and color fidelity. Compared with [21], the visual contrast improvement is similar, while the over enhancement is significantly avoided.

4.2.2 Blurry retinal images with insufficient illuminance

This solution also aims to retain the original color, and it is realized by only operating the non-color channel while keeping the color related channels unchanged. In this test, Fig. 5a is selected as the guided image to provide the intensity information for I channel. Four sets of enhanced images by using [20, 22] and the proposed solution 2 are shown in Fig. 12.

Fig. 12
figure 12

Visual assessment for images with insufficient illuminance. a Original images; b results of [22]; c results of [20]; d results of the proposed solution 2

Full size image

In Fig. 12, the results of [22] own the highest contrast, while the noise is greatly enlarged at the same time, and the images look very unnatural. Noise extensively exists in retinal images, and the noise in dark areas is inevitably enlarged after enhancement, owning to the contrast stretching and intensity improvement. The useful texture information may be destroyed if the noise is heavy. Thus, the capability of noise suppression should be taken into account for retinal images with insufficient illuminance. Image 4 is the darkest in the four original images, in which the noise is the most difficult to suppress. For this image, the introduced noise is also obvious by the proposed solution 2 and [20], but both of them have better noise suppression than [22]. In addition, the choroids in image 2 almost disappeared after the enhancement by the method of [22]. For the results of [20], the enhanced images have similar visual effect compared with the proposed solution 2. This is because [20] and the proposed solution 2 only enhance the non-color channels, while keep the channels of hue and saturation unchanged. A tiny visual difference is that the noise introduced by [20] is a little more obvious than the proposed solution 2, especially for image 3.

4.3 Quantitative analysis

4.3.1 Overall quality

We use the overall quality to assess the enhanced images with sufficient illuminance. Since the contrast improvement and color fidelity are the main purposes for these images, the overall quality is defined as

$$ \mathrm{Overallquality}=\frac{\mathrm{STD}}{\mathrm{colordifference}}, $$
(9)

in which the STD is the standard deviation. It measures the dispersion degree for a set of pixels, and a higher STD value indicates that the data distribute in a wider range [19]. According to the research of [24], the color difference (CD) can be calculated by

$$ \mathrm{CD}=\frac{\sqrt{{\left({\mathrm{OR}}_{\mathrm{i},\mathrm{j}}-{\mathrm{ER}}_{\mathrm{i},\mathrm{j}}\right)}^2+{\left({\mathrm{OG}}_{\mathrm{i},\mathrm{j}}-{\mathrm{EG}}_{\mathrm{i},\mathrm{j}}\right)}^2+{\left(\left({\mathrm{OB}}_{\mathrm{i},\mathrm{j}}-{\mathrm{EB}}_{\mathrm{i},\mathrm{j}}\right)\right)}^2}}{3\ast \mathrm{N}} $$
(10)

in which ORi, j, OGi, j, and OBi, j are pixels of red, green and blue channels of original image. ERi, j, EGi, j, and EBi, j denote the corresponding pixels from enhanced image. N is the total number of pixels. That is, a higher score of overall quality indicates that the contrast is better stretched, and the color difference is small. Otherwise, the contrast is not well stretched, or the color changes obviously. Based on 200 blurry images, the average scores of overall quality are shown in the left part of Table 1, and the proposed solution 1 has the highest score of overall quality. Specifically, compared with the results by [19, 21], there are 169 and 132 images that achieve higher overall quality scores based on the proposed solution 1.

Table 1 The average overall quality scores and LIP scores
Full size table

4.3.2 Lightness inequality preservation

For blurry retinal images with insufficient illuminance, the original illuminance and color saturation are inevitably modified after enhancement. Instead of using overall quality, the lightness inequality preservation (LIP) is utilized for quantitatively assessing the enhanced retinal images with insufficient illuminance. The LIP is based on the statistic of pixel pairs, and the process is shown in formula (11)–(13). O(x1, y1) and O(x2, y2) denote a pixel pair in original image, and E(x1, y1) and E(x2, y2) denote the same pixel pair after the enhancement. When the pixel pair satisfies formula (12), the LIP score is 1. Otherwise, the LIP score is 0. It can be considered that the original texture structure is modified if the inequality of pixel pair is reversed after enhancement [20].

$$ \mathrm{LIP}\ \mathrm{score}=\left\{\begin{array}{c}1\kern0.5em \mathrm{if}\kern0.75em (12),\\ {}0\kern0.5em \mathrm{if}\kern0.75em (13),\end{array}\right. $$
(11)
$$ {\mathrm{O}}_{\mathrm{I}}\left({\mathrm{x}}_1,{\mathrm{y}}_1\right)>{\mathrm{O}}_{\mathrm{I}}\left({\mathrm{x}}_2,{\mathrm{y}}_2\right)\mathrm{and}{\mathrm{E}}_{\mathrm{I}}\left({\mathrm{x}}_1,{\mathrm{y}}_1\right)>{\mathrm{E}}_{\mathrm{I}}\left({\mathrm{x}}_2,{\mathrm{y}}_2\right), $$
(12)
$$ {\mathrm{O}}_{\mathrm{I}}\left({\mathrm{x}}_1,{\mathrm{y}}_1\right)>{\mathrm{O}}_{\mathrm{I}}\left({\mathrm{x}}_2,{\mathrm{y}}_2\right)\mathrm{and}{\mathrm{E}}_{\mathrm{I}}\left({\mathrm{x}}_1,{\mathrm{y}}_1\right)<{\mathrm{E}}_{\mathrm{I}}\left({\mathrm{x}}_2,{\mathrm{y}}_2\right). $$
(13)

In this test, we randomly selected 2000 pairs of pixels from each image and counted the pixel pairs whose lightness inequality is not changed. Based on the 2000 pixel pairs, the mean proportion of pixel pairs that preserve the inequality are shown in the right part of Table 1. That is, the proposed solution 2 also takes the best in LIP score comparison.

4.4 Improvement on automatic retinal image processing

In addition to the visual effect improvement, the enhancement can also improve the reliability of automatic retinal image processing. The automatic vessel tracking [18] and vessel segmentation [23] are selected to demonstrate the improvement.

Vessel tracking is useful in measuring the curvature or the width of blood vessel, but the reliability is often reduced due to the low contrast. Based on [18], two sets of vessel tracking results using the retinal image before and after enhancement are shown in Fig. 13. At the beginning of tracking, a pair of marginal points and the tracking direction should be determined. Then, the locations of marginal points can be constantly confirmed [18]. In Fig. 13, the first column is the tracking result before enhancement, and second column is the result after enhancement. The last column is acquired from the enhanced images, which are used as references. The red arrow indicates the tracking direction. It is observed that the tracking before enhancement are all wrong, while the two sets of tracking turn correct after the enhancement.

Fig. 13
figure 13

Performance improvement for vessel tracking. a Tracking results before enhancement; b tracking results after enhancement; c references

Full size image

Vessel segmentation is also important, since some eye diseases can be detected according to the shape of vessel tree. Based on the multi-scale linear segmentation [23], two blurry images and the corresponding confidence maps of vessels are shown in Fig. 14. Confidence map reflects the probability of vessel pixel, and it is an intermediate result for the final segmentation. The first column is the original image, and the second column is the confidence map using multi-scale linear detector directly. The third column is the result based on solution 1 and multi-scale linear method. It is observed that many vessel segments are not detected before enhancement, and the width of blood vessel is obviously less than normal. As shown in the third column, these two problems are solved by performing the enhancement before segmentation. Although the parapapillary atrophy around optic disc is highlighted, the enhancement is still meaningful since more pixels of blood vessels are detected. Thus, the enhancement can be used as a pre-processing step when segmenting blurry retinal images.

Fig. 14
figure 14

Performance improvement for vessel segmentation. a Original image; b confidence maps using multi-scale linear detection; c confidence maps using solution 1 and multi-scale linear detection

Full size image

5 Discussion

We discuss two aspects associating with the enhancement. One is the necessity of developing different solutions for retinal images with different illuminance and how to choose a more adequate solution in a real scenario. The other aspect is the parameter settings.

For retinal images, the existence of blurry retinal images with insufficient illuminance is an objective fact. By performing the method of [21] on the image with insufficient illuminance, the result is shown in Fig. 15. The macula and pixels surrounding the optic disc become very dark which appear like abnormalities. In addition, the illuminance of optic disc is too strong to see the small vessels. In a real scenario, the statistics of the standard DRIVE database can be used to determine whether the illuminance of a retinal image is sufficient. The image with the minimal average pixel value in DRIVE database is used as the criterion. For blurry retinal image which has lower average intensity than that image, both of the contrast and illuminance should be improved, and another image is used as the guided image to provide the required illuminance (i.e., solution 2). Otherwise, just the contrast improvement is needed, and the original image is used as the guided image to achieve the fidelity of color and illuminance (i.e., solution 1).

Fig. 15
figure 15

Result of performing the method of [21] on a retinal image with insufficient illuminance. a Original image; b enhanced image

Full size image

In this work, there are three parameters which significantly affect the enhancement performance. (1) We firstly discuss the threshold in intensity transfer, and it denotes the difference of average pixel values between the stretched image and guided image. The lower the threshold, the closer the color is to the guided image. The threshold is set to 1 in this work, and the color of stretched image can be very close to the guided image. (2) The kernel width of the contrasted Gaussian function is also important. If it is set to a small value (e.g., the minimum value 1), the base-intensity will be the weighted original image. In this manner, the relative differences between any two pixels are not changed, and therefore the lightness inequality can be well preserved, while the contrast is reduced after removing the base-intensity because the absolute differences between pixels become smaller. To keep the absolute differences between pixels as much as possible, the base-intensity should be smooth. For retinal image, the kernel size should be close to the optic disc, because the optic disc is usually the largest retinal structure, and its diameter is about 1/6–1/7 of the retina diameter. If the retina diameter is W, the kernel width is set as W/7 in this work. (3) For the negative pixel values generated in removing the base-intensity, if we do not allow to generate negative values, the base-intensity will be very small, and the space for contrast improvement will be affected. Thus, we allow a small number of negative pixel values in removing the base-intensity. The number of negative pixel values is set to 0.05% of the total number of pixels in retina, and θ is adaptively adjusted according to this percentage.

6 Conclusion

We have presented two solutions for enhancing retinal images with different illuminance. The two solutions are developed based on the proposed contrast stretching and intensity transfer. For the contrast stretching, we assume that the intensity of pixel is not fully meaningful in forming the texture information, in which the relative low part mainly supports the illuminance. This part is called as the base-intensity in this research, and it is estimated by the constrained Gaussian function. After removing the base-intensity, the non-uniform addition is used to increase the contrast. Thereafter, an effective intensity transfer strategy is further developed, which can be used to provide required illuminance for a single channel. Moreover, the color correction can be realized if the intensity transfer is performed on all the three channels. A guided image is needed in this process, which can be the original image or another image. Results show that the proposed solutions can obviously improve the visibility of hidden retinal structures, and the enhancement is also helpful for automatic retinal image processing. Future work will focus on the quality assessment methods for retinal images.