1 Introduction

The low-light image usually suffers from low contrast and poor visibility [1, 2]. Besides, many vision tasks such as autonomous driving, remote sensing, and railway disaster prevention surveillance would suffer from performance degradation due to the bad quality of low-light images [3]. Therefore, the LLIE has been extensively studied and achieved remarkable progression. For example, histogram equalization (HE) algorithms [4,5,6], the Retinex-based algorithms [7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29], the de-hazing algorithms [30], and the data-driven learning algorithms [31,32,33,34,35] have been proposed. However, it is still a challenge to enhance the low-light image captured from complex light conditions

This paper briefly reviews the main methods and technologies of low-illumination image enhancement.

HE-based methods are the simplest method to enhance the low-light image [4,5,6]. However, it may lead to over-enhancement and color distortion. Some improved methods [36,37,38,39,40,41] have been proposed to overcome the above problems. In recent work, Lee et al. [42] proposed a novel contrast enhancement algorithm based on the layered difference representation of 2D histograms. However, this method results in loss of image details and noise amplification.

Retinex-based algorithms regarded the image as the product of the illumination map and the reflectance map [7,8,9,10,11,12,13]. For single-scale Retinex (SSR) [12] and multi-scale Retinex (MSR) [13], treat the reflectance map after removing the illumination map from the input image as the final enhanced result. However, the methods may lead to halo artifacts and color distortion in the low-light image enhancement. [14,15,16,17] proposed an adaptive filtering algorithm to reduce the color distortion and the halo artifacts of the image. Petro et al. [18] proposed a multi-scale Retinex chrominance preservation algorithm (MSRCR) to enhance details and contrast with color correction. Besides, [19,20,21,22,23] further improve the performance of MSRCR. However, these methods may cause over-enhancement [23]. Fu et al. [24] proposed adjusting the illumination map by fusing multiple derivations of the initially estimated illumination map (MF). Since the illumination map structure is blind, MF may lose the realism of the texture-rich regions. Guo et al. [25] proposed a method of low-light image enhancement via illumination map estimation to refine the initial illumination map by imposing a structure-aware prior. However, there is the lack of constraints on the reflectance map, and the noise in the image may be amplified. Wang et al. [26] proposed a naturalness-preserving enhancement algorithm (NPEA), used a bright-pass filter to estimate the illumination, and presented a bi-log transformation to adjust the illumination map to avoid over-enhancement. Kimmel et al. [27] proposed a Retinex-based variational framework to transform the illumination map estimation into an optimal quadratic programming problem. Although the algorithm is highly complex, the enhancement result is promising. Fu et al. [28] proposed a weighted variational model (SRIE) that can estimate the reflectance map accurately while suppressing noise and processing the illumination map to enhance the low-light images. The algorithm shows impressive results. However, it does not consider the noise.

The dehazing-based method [30] tried to treat the inverted low-light image as a hazy image. However, the low-light image enhancement method based on dehazing lacks the necessary physical explanation [30].

The significant effect of deep convolutional neural networks (CNNs) leads to the development of data-driven image enhancement algorithms [31]. These methods usually require paired images to train the model. However, obtaining images of different luminance in the same place is a challenge [31].

The low-light enhancement methods tend to improve visibility by enhancing image luminance. However, this way may cause the visual effect to be unnatural. Deep Retinex-Net combines deep learning with Retinex to decompose images into illumination and reflectance maps using the neural network and gets good low-light image restoration results [32]. Chen et al. [33] develop the processing of low-light images, based on end-to-end training of a fully convolutional network. The network replaces much of the traditional image processing method, which performs poorly in such data. Shen et al. [34] propose a semantic-guided zero-shot low-light enhancement network for low-light images, which is trained in the absence of paired images, unpaired datasets, and segmentation annotation, and the model outperforms the previous state-of-the-art qualitatively and quantitatively.

In low-light image enhancement, the enhancement may cause dark areas to become brighter, which is an unnatural visual effect for the human visual system [26], as shown in Fig. 1b.

Fig. 1
figure 1

The over-enhancement result of the weak light image. (a) Weak light image. (b) Over-enhance results

Full size image

In this paper, different from the existing Retinex-based low-illumination image enhancement methods, we propose a restoration algorithm based on enhanced reflectance maps for low-illumination images. It is well known that the reflectance map contains rich texture and color information. Therefore, we believe that directly enhancing the reflectance map is more effective than adjusting the illumination map. Adelson's shadow experiments show that reducing the intensity of the illumination map can improve the contrast of the image. Accordingly, we reduce the intensity of the illumination map to get more color and structure information from the reflectance map, based on the Retinex model. The details are described in Sect. 2.

The contributions of this paper are as follows. (1) Our scheme is the first to use the enhanced reflectance map to recover low-light images. (2) We propose a novel gamma correction to maintain the naturalness of the image. (3) We created a dataset of nighttime railroad scenes. (4) Compared with the reference method, our method has a faster processing speed.

The rest of this paper is organized as follows. Section 2 describes the algorithmic details of RME. Performances of the proposed method are evaluated both quantitatively and qualitatively in Sect. 3. We conclude in Sect. 4.

2 Method

We propose an effective algorithm for low-illumination image enhancement. Unlike existing algorithms, our algorithm recovers the low-illumination image by enhancing the reflectance map. We reduce the intensity of the illumination map to get an enhanced reflectance map based on Adelson’s checker shadow experiment and the Retinex model. Then, we apply an adaptive gamma correction to the illumination map and the reflection map, respectively. Finally, the gamma-corrected reflectance map and illumination map are fused to get the enhanced results. The experimental results show that our method achieves impressive results. The overall framework is shown in Fig. 2.

Fig. 2
figure 2

The model of low-light image enhancement is proposed in this paper

Full size image

2.1 Retinex model and Adelson checkerboard shadow experiment

2.1.1 Retinex model

In the Retinex model, the input image can be considered as the product of the reflectance map and the illumination map, as shown in Eq. 1:

$$ S = R{\bullet}L $$
(1)

where S is the input image, and operator ‘•’ denotes element-wise multiplication. R is the reflectance map that represents the characteristics of texture and structure. L is the illumination map that represents the distribution of light intensity in the image. Most of the existing Retinex-based methods are get enhanced results of low-light images by adjusting the illumination map. However, these methods may lead to image enhancement results with over-enhancement, color distortion, noise, and unnatural visual effects. It is well known that reflectance maps contain more information than illumination maps. Therefore, it is more effective to process the reflectance map than the illumination map.

2.1.2 Adelson checkerboard shadow experiment

In the Adelson checker shadow experiment, for most people, square A seems to be darker than square B, as shown in Fig. 3a. However, in Fig. 3b, two vertical reference bars reveal that square A and square B are the same, which is S(A) = S(B). We noticed that square B is in the cylinder's shadow, so the luminance intensity of square A is higher than square B, which is L(A) > L(B). According to the equation of Retinex \(S = R \bullet L\), it can derive the reflectance map of square A is smaller than square B, which is R(A) < R(B).

Fig. 3
figure 3

Adelson’s checker shadow illusion. (a) The original image of the illusion. The squares marked A and B are the same shade of gray, yet they appear different. (b) The original image plus two stripes. By joining the squares marked A and B with two vertical stripes of the same shade of gray, it becomes apparent that both squares are the same

Full size image

Adelson's experiments show that reducing light levels can improve the contrast of images. The core of our proposed algorithm is built on this basis.

2.2 Reflectance map enhancing (RME)

The core idea of this paper comes from the Adelson experiment and the Retinex model. The algorithm recovers a low-light image by processing the reflectance map.

2.2.1 Solving reflectance and illumination map

This paper uses a multi-scale Gaussian low-pass filter to extract the illumination map from the input image. The reflectance map is got by \({\varvec{R}}={\varvec{S}}/{\varvec{L}}\), based on the Retinex model.

The illumination map L is formulated as follows:

$$ L_{i} \left( {x,y} \right) = \sum\limits_{n = 1}^{N} {W_{n} \cdot \left\{ {\left. {\log \left[ {S_{i} \left( {x,y} \right) * F_{n} \left( {x,y} \right)} \right]} \right\}} \right.} $$
(2)

where Li (x, y) is the estimated value of the illumination map, Si (x, y) is the ith channel two dimensional matrices of the input image, mark \(*\) denotes the convolution operator, n varies from 1 to N different scales, and \(W\) represents the weighting parameters for each scales. \(F_{n} \left( {x,y} \right)\) represents the Gaussian surround function. It can be defined as:

$$ F_{n} (x,y) = K_{n} \exp \left[ {\frac{{ - (x^{2} + y^{2} )}}{{\sigma_{n}^{2} }}} \right] $$
(3)

where Kn is selected so that \(\iint {F_{n} (x,y)dxdy} = 1\), \(\sigma\) is the standard deviation of Gaussian function.

The different scale filters can extract diverse feature information on the input image. In this paper, we set five scales low-pass filter to get more information from the input image.

To get an enlarged reflection map, we reduce the luminance intensity of the illumination map, as shown below:

$$ L_{w} = \lambda_{1} \times L, \, s.t. \, 0 < \lambda_{1} < 1 $$
(4)

where \({L}_{\omega }\) is the illumination map with reduced bright intensity, and \({\lambda }_{1}\) is the weighting coefficient. The value \({\lambda }_{1}\) is proportional to the brightness of the input image.

To reduce the computational cost, we estimate only the illumination map. The reflectance map is solved according to Eq. 1, as follows.

$$ R = {S \mathord{\left/ {\vphantom {S {L_{w} }}} \right. \kern-\nulldelimiterspace} {L_{w} }} $$
(5)

The low-light images are often got under complex lighting conditions, such as non-uniform lighting. This imposes higher requirements on the algorithm. Therefore, we further improve the method by introducing luminance compensation and gamma correction.

2.2.2 Adaptive gamma correction

We propose an adaptive gamma correction method based on the radiation intensity of the image environment and introduce the principle of the proposed adaptive gamma correction, taking the gamma correction of the reflectance map as an example. The formula is as follows:

$$ R_{g} = \left( R \right)^{\gamma } , \, \gamma { = }\left( {\lambda_{2} } \right)^{{\frac{{f_{mean} \left( V \right) - \left( {R * 255} \right)}}{{f_{mean} \left( V \right)}}}} $$
(6)

where Rg denotes the gamma-corrected reflectance map, \({f}_{mean}\left(V\right)\) is the average of the V channel luminance in HSV space, and \({\lambda }_{2}\) is the intensity control factor for gamma correction, and we set \({\lambda }_{2}\) as greater than 1.

It is well known that the better result of ambient light enhancement can significantly improve the visual effect of low-light images. Therefore, the difference between ambient light and reflectance map is used as an adaptive variable to control the gamma factor in our algorithm. The gamma factor is proportional to the difference in brightness between the light and dark areas of a non-uniform low-light image. Our proposed adaptive gamma correction takes \({f}_{mean}(V)\) as the ambient light intensity of the image, and the dark and bright areas away from the \({f}_{mean}(V)\) are the areas where the gamma correction focuses on adjusting. We use the ambient light intensity as the reference point for image brightness restoration to better maintain the original naturalness of the image. Therefore, we use the difference between the \({f}_{mean}(V)\) and the ‘R’ map as a control \(\gamma \)-value adaptive factor to balance the illumination of light and dark areas of low-light images.

In Fig. 2b, a schematic of our gamma correction curve is shown.

In Fig. 4a, b, the black dashed line indicates the gamma correction for \(\gamma =1\), and the blue solid line in Fig. 4a indicates the gamma correction for \(\gamma <1\). The solid green line in Fig. 4b indicates our gamma correction curve, which is equivalent to translating the gamma correction curve for γ > 1 in Fig. 4a.

Fig. 4
figure 4

Schematic of our gamma correction curve. (a) The blue curve indicates the gamma correction commonly used in existing methods for LLIE. (b) The green solid line indicates our gamma correction curve

Full size image

It improves the overall brightness of low-light images, avoids overexposure of bright areas, and suppresses noise amplification in dark areas.

2.2.3 Image naturalness preservation

As we know, the illumination map is essential for image naturalness preservation [2, 3]. We apply the same adaptive gamma correction on the illumination map to ensure it is consistent with the reflectance map. This way maintains the naturalness of the image and improves the visual effect. The formula is as follows:

$$ L_{g} = 255\left( \frac{L}{255} \right)^{\gamma } , \, \gamma { = }\left( {\lambda_{2} } \right)^{{\frac{{f_{{{\text{mean}}}} \left( V \right) * I_{e} - L}}{{f_{{{\text{mean}}}} \left( V \right) * I_{e} }}}} {\text{, s}}{\text{.t}}{\text{. 0 < I}}_{e} < 1 $$
(7)

where \(L\) is the estimated illumination map, \({I}_{e}\) is the ambient light intensity coefficient, and \({I}_{e}\) is inversely proportional to the ambient light intensity of the image. The curvature of the curve for gamma correction will affect the light–shadow relationship of the enhanced image. In this paper, the value of \({\lambda }_{2}\) is close to 1 to maintain the naturalness of the image.

In this paper, the enhanced result \({S}_{e}\) is obtained by fusing the reflectance map Rg and the illumination map Lg. The formulation \({S}_{e}\) is given as follows:

$$ S_{e} = L_{g} \bullet R_{g} $$
(8)

Finally, we convert Se back to RGB space to get the final image enhancement result. Figure 5b shows enhancement results of the low-light image under non-uniform luminance. The dashed box represents the results in the bright and dark areas of the image with an enlarged display, respectively. As you can see, the method improves the contrast and visibility of the image. Significantly, the dark area noise of the image is not amplified, and the bright area is not over-enhanced.

Fig. 5
figure 5

The result of keeping the image natural and enhancing the brightness. (a) Non-uniform and low-light image. (b) Our method enhances the results of (a). Some areas are highlighted by red and green rectangles, and zooming-in is commended for better visualization and comparison

Full size image
figure a

2.2.4 Ablation study

In this section, we show the intermediate results of our algorithm and verify the utility of each part of the algorithm through ablation experiments.

Figure 6b shows the reflectance map R, and Fig. 6(c) shows the reflectance map Rg via adaptive gamma correction by our algorithm. One can see that the Rg better balances the illumination of light and dark areas of the image, and the visualization of image details is better than R.

Fig. 6
figure 6

Intermediate results and effectiveness of the algorithm. (a) Input image. (b) The reflectance map was obtained via our algorithm. (c) The reflectance map Rg was obtained by adaptive gamma correction

Full size image

As shown in Fig. 7, the middle row, one can find that the bright and dark areas of the image appeared overexposed and underexposed, respectively. Comparing the middle row and the bottom row in Fig. 7, we can find that the visual effect of the bottom row is significantly better than that of the middle row.

Fig. 7
figure 7

Top row: Input low-light images. Middle row: Enhanced results of performing adaptive gamma correction on reflectance maps only. Bottom row: The results enhanced by our method

Full size image

In conclusion, the above analysis proves that our proposed method is effective for adaptive gamma correction of reflectance and illumination maps.

3 Experiments

In this paper, we compare 6 representative methods including LDR [36], MSRCR [20], NPEA [26], MF [24], SRIE [28], and LIME [23]. We select images with challenging from the LOL and the self-created dataset, forming an image testing dataset that includes 150 low-light images under different luminance conditions, such as backlit, non-uniform light, and extremely dark. The experiment results reveal the advantage of our method in comparison with other state-of-the-art methods.

The results of these methods are produced by performing codes in MATLAB on a PC with Intel i5-6500, 8 GB RAM, and these codes are obtained from the authors’ websites. To ensure fair comparisons, we set the parameters of these methods to be the recommended ones.

The complexity of the proposed algorithm is O (\(m\times n\times {\rm I}_{\mathrm{max}}\)), where \(m,n\) are sizes by pixel of the width and the height of an image, \({\rm I}_{\mathrm{max}}=255.\) Table 1 shows the average run times of testing 150 low-light images for the various methods. The underlined Bold and Bold numbers in the table indicate the optimal and suboptimal running times, respectively.

Table 1 Average running time of relevant methods
Full size table

To test the robustness of the algorithms in complex luminance conditions, we tested the performance of the algorithm in different low-illumination scenarios, such as extreme low illumination, backlit, non-uniform illumination, and actual-world scenes, as in Figs. 8, 9, 10, 11.

Fig. 8
figure 8

The experiment results of the color distortion detection. (a) Test image. (b) LDR [36]. (c) MSRCR [20]. (d) NPEA [26]. (e) MF [24]. (f) SRIE [28]. (g) LIME [23]. (h) Proposed RME

Full size image
Fig. 9
figure 9

Visual results of different methods of the backlit illumination image. (a) Test image, (b) LDR [36], (c) MSRCR [20], (d) NPEA [26], (e) MF [24], (f) SRIE [28], (g) LIME [23], and (h) our RME are presented

Full size image
Fig. 10
figure 10

Visual results of different methods of the non-uniform illumination image. (a) Test image. (b) LDR [36]. (c) MSRCR [20]. (d) NPEA [26]. (e) MF [24]. (f) SRIE [28]. (g) LIME [23]. (h) Proposed RME

Full size image
Fig. 11
figure 11

Visual results of different methods on the self-created Real-World Dataset. (a) Test image. (b) LDR [36]. (c) MSRCR [20]. (d) NPEA [26]. (e) MF [24]. (f) SRIE [28]. (g) LIME [23]. (h) Proposed RME. The red and green boxes in the image correspond to the visual effects of the dark and bright areas, respectively

Full size image

3.1 Color distortion of LLIE

In this section, the group experiment aims to reveal the effect of color distortion in the low-light image enhancement field. As Fig. 8 shows, our algorithm restores the original color and gets a clear image enhanced result. It is better than comparison algorithms.

3.2 LLIE in backlit

The second group of experiments demonstrates the low-light image enhancement effect of backlit scenes under different light intensities. As shown in Fig. 9g4, one can find that the stronger the backlit of a low-illumination image, the more obvious the over-enhancement in its dark areas, and the naturalness of the enhancement result will decrease, as shown in Fig. 9c, d, e and g. In Table 2, we show the performance of the relevant methods in maintaining the naturalness of the images. The underlined symbol Bold is the optimal and symbol Bold is the suboptimal result in each assessment metric.

Table 2 Quantitative performance comparison in Fig. 9 in terms of NIPE
Full size table

The Natural Image Quality Evaluator (NIQE) is used to detect the naturalness and distortion of an image. A smaller NIQE indicates a higher quality of enhancement results. As shown in Table 2, our algorithm performs well on the NIQE metric.

3.3 LLIE in non-uniform lighting

The third group of experiments demonstrates the low-light image enhancement effect of non-uniform luminance scenes under different light intensities. It is a challenging task that the existing methods usually cause over-enhancement in the bright area and noise amplification in the dark zone.

It can be seen in Fig. 10 that some images show noise amplification, artifact, over enhancement, and color distortion. Specifically, the enhanced effects of LDR and MSRCR show significant color distortion, as shown in Fig. 10b, c. Image distortion can lead to damage to the information in the image, as shown in Fig. 10d, e, and g. However, RME can generate more visually appealing results. In Table 3, we show the performance of the relevant methods in maintaining the information of the images.

Table 3 Quantitative performance comparison on Fig. 10 in terms of ENTROPY (Underlined Bold and Bold represent optimal and suboptimal results, respectively)
Full size table

3.4 LLIE in real-world scene

We note that the input images in the LOL dataset are relatively noise-free. However, real-world images of low-light scenes have a variety of noise. The fourth group of experiments demonstrates the influence of real-world scenes on low-light image enhancement under different light intensities.

We created a dataset of the low-light images, named Railway at Night, to verify the performance of the algorithms. This paper selects four low-light images of different light conditions from our dataset to verify the performance of the related algorithms. Figure 11a1–a4 shows four images from our dataset, including noise, weak-light, backlit, and non-uniform illumination, respectively. In Fig. 11, the dark and light areas of the image are shown enlarged by red and green rectangles, respectively, for better viewing and comparison. All methods enhanced brightness and restored the image details to a certain extent. However, there also appear some unsatisfied visual effects. For example, in Fig. 11d1, e1, e4, and g1, the results make noise amplification. In Fig. 11d2, g2, c4, and g4, it is easy to find that the results are over-enhanced. In Fig. 11c3, d3, e3, and g3, it can be seen that there are obvious halo artifacts in the enhanced result of the image. Fortunately, our results effectively avoid the problems mentioned above. Our visual effects are better than the control group. In summary, experiments verify that our method is effective in real-world scenes.

In Table 4, we show the performance of the relevant methods in maintaining the PSNR of the images.

Table 4 Quantitative performance comparison on Fig. 11 in terms of PSNR (Underlined Bold and Bold represent optimal and suboptimal results, respectively)
Full size table

3.5 Quantitative assessments

Image quality assessment (IQA) is divided into two categories. One is a reference assessment, and the other is a no-reference assessment. In this paper, we adopt peak signal-to-noise ratio (PSNR), structural similarity (SSIM), information entropy, Natural Image Quality Evaluator (NIQE), perception-based image quality evaluator (PIQE), and luminance order error (LOE) to assess the quality of the enhanced image.

We selected 150 low-light images of different luminance from LOL and self-created datasets for extended experiments to verify the performance of the low-light image enhancement algorithm in different scenes. As shown in Table 5, the quantitative scores of RME are better than other methods in 4 terms of quantitative evaluation metrics, such as SSIM, ENTROPY, NQIE, and LOE. In the other two-term quantitative evaluation metrics, the scores of RME are suboptimal solutions.

Table 5 Average of the quantitative performance on the test image dataset (Underlined Bold and Bold represent optimal and suboptimal results, respectively)
Full size table

Due to the box-plot and the inter-quartile range (IQR) having advantages in identifying data outliers and determining data bias, we introduce box-plot and IQR to test the stability of the related algorithm in processing low-light images of different scenes, as shown in Fig. 12.

Fig. 12
figure 12

The box-plot and the inter-quartile range evaluation results for (a) PSNR, (b) SSIM, (c) ENTROPY, (d) NIQE, (e) PIQE, and (f) LOE

Full size image

To unify the evaluation standard, the IQR value is normalized. In Table 6, the RME model gets the best values on SSIM, NIQE, and AVERAGE, while gets suboptimal on PSNR, entropy, PIQE, and LOE. SRIE model obtained optimal on PIQE. In Table 6, underlined symbol Bold and symbol Bold represent optimal and suboptimal results, respectively. It is worth noting that the RME model gets the lowest AVERAGE, which indicates that RME has strong stability, robustness, and generalization ability.

Table 6 Robustness comparison of related algorithms in terms of IQR
Full size table

In summary, our quantitative assessment of ours achieves or exceeds state-of-the-art algorithms. The experimental results show that our method is effective, while the enhanced results for our algorithm are consistent with a quantitative and qualitative evaluation of image quality.

4 Conclusion

In this paper, we employ the low-light image with a variety of complex lighting conditions to test the effectiveness of our proposed algorithm. The experimental results show our algorithm has achieved impressive results in both qualitative and quantitative evaluation metrics. We notice that noise in the low-light images has a significant impact on performing computer vision tasks. Thus, we will further investigate the high-noise level low-light image enhancement algorithm.