1 Introduction

The purpose of image enhancement is to make the useful information in the obtained image clearer and clearer, and provide good precondition for subsequent processing. Image enhancement is a preprocessing algorithm for a series of intelligent operations. The enhancement of the effect and the integrity of the information such as detail edges are to ensure target recognition, target tracking, feature point matching, image fusion, super-resolution reconstruction [3] and other images. They are the important conditions for processing the algorithm. With the iterative update of the captured electronic products, although the image pixels obtained by people are getting higher and higher, the picture quality is getting better and better, but due to environmental reasons such as light and fog, the image will not be able to achieve the desired effect. Therefore, It is necessary to enhance the image.

Retinex theory is the theory of the retinal cerebral cortex, which was proposed by Edwin. H. Land in 1963. After decades of development, it has evolved from SSR to a multi-scale weighted average Retinex algorithm (MSR), and its improved Retinex algorithm MSRCR, and others. Derived algorithms and various algorithms combined with classical algorithms. Retinex theory plays an important role in image enhancement technology due to its constant color thinking. However, many experimental results show that image enhancement based on Retinex theory still suffers from defects such as “halo artifact”, image over-enhancement and low computational efficiency [22]. For the enhancement of low-illumination color images, Jang CY et al. proposed an adaptive weighted MSR algorithm for lighting and object contrast edges. This algorithm can keep the color constant and have a better dynamic range of the image, but it has been over-enhanced [5]. Kwon HJ et al. proposed a brightness adaptive transform image enhancement algorithm based on the luminance component function. The algorithm uses the constructed brightness function to enhance the image. The enhanced image has better contrast and saturation, but for the low brightness image, the enhancement effect is not obvious [8].

In recent years, due to the progress of deep learning algorithm, low illumination image enhancement also realizes deep processing. Literature [9] is an article using deep learning methods to complete low-light image enhancement tasks. It proves that a stacked sparse denoising autoencoder based on synthetic data training can enhance and denoise low-light noisy images. Model training is based on image patches, using sparsity regularized reconstruction loss as the loss function. The main contribution is to propose a training data generation method (ie Gamma correction and adding Gaussian noise) to simulate low-light environments. Two types of network structures are explored: LLNet, which learns contrast enhancement and denoising at the same time; S-LLNet, uses two modules to perform contrast enhancement and denoising in stages. Experiments were performed on real low-light images, which proved the effectiveness of the model trained with synthetic data. The paper [14] regards the traditional multi-scale Retinex (MSR) method as a feedforward convolutional neural network with different Gaussian convolution kernels, and demonstrates it in detail. Then, following the MSR process, they proposed MSR-net, which directly learns the end-to-end mapping from dark images to bright images. The paper [1] mainly focuses on Single Image Contrast Enhancement (SICE), which is aimed at a problem of low contrast under underexposure and overexposure. Its main contribution is to build a multi-exposure image data set, including low-contrast images with different exposures and corresponding high-quality reference images. A two-stage enhancement model is proposed. In the first stage, the weighted least squares (WLE) filtering method is used to decompose the original image into low-frequency components and high-frequency components, and then the two components are enhanced separately; in the second stage, the enhanced low-frequency and high-frequency components are merged, and then Enhance again and output the result. In order to avoid the phenomenon of color cast, the two-stage structure is designed. The article is explained in this way to balance the enhancement effect of the smooth component and the texture component of the image. The Decomposition step of the first stage of the model uses the traditional method, and the Retinex-Net described later is implemented using CNN. The training data uses high-quality images adjusted with PS and corresponding synthetic low-light images (randomly reduced brightness, contrast, and Gamma correction). There are many low-light image algorithms based on deep learning algorithms. They are all done by adding deep learning ideas on the basis of traditional image algorithms. Although deep learning is good, most of its algorithms rely heavily on data sets. Even in literature [2], a low-light image enhancement network that does not rely on paired or unpaired training data is proposed. The network design is sophisticated and innovative, the performance is very strong, but generalization is also difficult to achieve. The cost and requirements of various aspects of common deep learning algorithms are extremely high, so traditional methods still have high research value.

Qi Mou et al. Made two-step improvements on the results of MSR processing of low illumination images: first, guided filtering and high-frequency lifting were used to enhance the details of the image; then, low rank decomposition was used to eliminate the noise in the image processing process, effectively avoiding the impact of noise on image vision [12]. However, the processing effect of image silhouette is not very good. Jingwei Dong et al. Paper [6] used the improved homomorphic filtering algorithm to enhance the RGB components of low illumination image, and then transformed the RGB image into HSV color space to perform adaptive nonlinear stretching on saturation component; At the same time, the brightness component is enhanced by the improved MSR algorithm, the illumination component is corrected by Gamma transform, and the reflection component is processed by sigmoid function. Finally, the image is converted to RGB space. Wei Lu et al. Proposed a low illuminance image enhancement algorithm based on the fusion idea under the Retinex theory. The enhanced image obtained from the single-scale Retinex algorithm based on bilateral filtering was weighted and fused with the image obtained from the MSR enhancement algorithm of the luminance component Y in the YCbCr color space. However, the phenomenon of over enhancement of color information and distortion of the sky color background still existed [15]. Similarly, in YCbCr color space, Huijuan Tian et al. Extracted Y component from space to construct the original illumination image component L1, and Gamma corrected L1 to obtain the enhanced illumination image component L2. The Reinex model was used to obtain the enhanced image R. the multi-scale detail enhancement method was used to enhance the image r to obtain the final enhanced image re [4], Compared with the other six algorithms, it is faster. However, subjective evaluation has no obvious advantage in image brightness enhancement.

In order to overcome the shortcomings of image enhancement algorithms based on Retinex theory, this paper proposes a Gabor-filtered single-scale Retinex algorithm (SSR). The luminance I component is extracted in the HSI color space, and the MSRCR enhancement is performed on the luminance component with color recovery, and the enhanced luminance component and the color restored image are obtained, and the image is subjected to Gabor filtering-based single-scale Retinex enhancement in the RGB color space. Obtaining an enhanced image with better texture detail information; then weights and merging the enhanced two images to obtain a final enhanced image.

2 Experiments and methods

2.1 Image enhancement algorithm based on Retinex theory

The establishment of Retinex theory is based on the combination of the retina + cortex of the human visual system, the retinal cerebral cortex theory. Edwin. H. Land’s Retinex model is based on the following three points. First, there is no color in the real world. The color that people perceive is the result of the interaction between light and matter. The colorful soap bubbles are generated by the interference of light on the surface of the film. Second, each color region is composed of three primary colors of red, green and blue of a certain wavelength. The third and third primary colors determine the color of each unit area. That is, Retinex theory is a computational theory of color perseverance.

The image formation in Retinex theory is shown in Fig. 1. The incident light L can be light in any direction. As long as it can irradiate the visible light of an object, its frequency must be within the frequency range of visible light. When it irradiates object R, it forms reflected light through the reflection of the object, and the reflected light enters the eye of the observer to form an image S. The reflectivity R is determined by the object itself [21], and is not changed by the incident light L. The basic content of Retinex theory is that the color of an object is determined by the ability of the object to reflect light from long (red), medium (green) and short (blue) light, rather than the absolute value of the intensity of the reflected light; It is not affected by the illumination heterogeneity and has consistency. That is, the Retinex theory is based on the consistency of color perception (color constancy).

Fig. 1
figure 1

Principle of image formation in Retinex theory

Full size image

Since the image is composed of a plurality of pixels, the image S(x, y) can be decomposed into a Eq. (1) between the illumination component L(x, y) and the object reflection amount R(x, y) for calculation. After performing logarithmic transformation using Eq. (2), it is possible to calculate by Eq. (3) based on the arithmetical properties of the logarithm.

$$ S\left(x,y\right)=L\left(x,y\right)R\left(x,y\right). $$
(1)
$$ \lg \left[S\left(x,y\right)\right]=\lg \left[L\left(x,y\right)R\left(x,y\right)\right]=\lg \left[L\left(x,y\right)\right]+\lg \left[R\left(x,y\right)\right]. $$
(2)
$$ \lg \left[R\left(x,y\right)\right]=\lg \left[S\left(x,y\right)\right]-\lg \left[L\left(x,y\right)\right]. $$
(3)

(x, y) are the coordinates of the image pixel in the two-dimensional space, respectively. The specific work process is shown in Fig. 2.

Fig. 2
figure 2

Retinex theory workflow

Full size image

2.2 Image enhancement based on SSR, MSR and MSRCR algorithms

The SSR algorithm is an improved algorithm for the Retinex algorithm proposed by Jobson et al. [7]. The algorithm is also called the single-scale Retinex algorithm. In general, a low-pass filter (usually a Gaussian low-pass filter) is used to convolve with the original image as the illuminance component estimator. The original image is subtracted from the illuminance estimation image to obtain a reflection image, which is calculated by the Eq. (4).

$$ \lg \left[R\left(x,y\right)\right]=\lg \left[S\left(x,y\right)\right]-\lg \left[L\left(x,y\right)\right]=\lg \left[S\left(x,y\right)\right]-\lg \left[S\left(x,y\right)\ast G\left(x,y\right)\right]. $$
(4)

Where * is a convolutional symbol, G(x, y) is a Gaussian filter function, which is calculated by Eq. (5), and σ is a scale parameter. Different enhancement effects can be obtained by adjusting the magnitude of σ. When the value of σ is large, the color of the enhanced effect image remains good, but the texture details are not clear; when the value of σ is small, the details of the enhanced image are preserved, but severe color distortion is generated.

$$ G\left(x,y\right)=\frac{1}{2{\pi \sigma}^2}\exp \left(-\frac{x^2+{y}^2}{2{\sigma}^2}\right). $$
(5)

In order to solve the shortcomings of SSR, a multi-scale Retinex algorithm, namely MSR algorithm, is proposed based on the single-scale Retinex algorithm. The MSR combines the enhanced graphs processed by SSR algorithms of different scales to obtain enhanced images by weighted average combination.

$$ \lg \left[R\left(x,y\right)\right]=\sum \limits_{j=1}^n{w}_j\left\{\lg \left[S\left(x,y\right)\right]-\lg \left[S\left(x,y\right)\ast {G}_j\left(x,y\right)\right]\right\}. $$
(6)

In Eq. (6), Gj(x, y) is a Gaussian filter function, the scale parameter is σj, n is the number of scale parameters, usually takes 3; wj is the weight corresponding to the jth filter function and satisfies the condition \( {\sum}_{\mathrm{j}=1}^{\mathrm{n}}{\mathrm{w}}_{\mathrm{j}} \), usually \( {\mathrm{w}}_1={\mathrm{w}}_2={\mathrm{w}}_3=\frac{1}{3} \).

For color images, color distortion is prone to occur after the image is enhanced using the MSR algorithm. Since the color image is processed, it is necessary to separately perform MSR enhancement on the three components R, G, and B, and then combine the three enhanced components into the final enhanced image. This kind of operation breaks the inherent proportional relationship between the three components of R, G, and B, so the enhanced image is prone to color distortion. In order to overcome the shortcomings of MSR algorithm which is easy to produce color distortion, a color recovery factor is introduced based on MSR algorithm, and the MSR algorithm with color recovery factor is proposed, which is MSRCR algorithm [17, 18]. They can be calculated with Eqs. (7) and (8).

$$ {R}_{{MSR CR}_j}={C}_j{R}_{MSR_j}. $$
(7)
$$ {C}_j\left(x,y\right)=\beta \lg \left(\frac{\alpha {S}_j\left(x,y\right)}{\sum \limits_{j=1}^3{S}_j\left(x,y\right)}\right). $$
(8)

In Eq. (8), Cj is the color restoration coefficient of the jth color component, j = 1, 2, 3 represent the three color components of R, G and B, respectively; in addition, \( {\mathrm{R}}_{{\mathrm{MSR}}_{\mathrm{j}}} \) is the enhanced result obtained by the j-th MSR algorithm; α is the adjustment parameter factor of the nonlinear intensity; β is the gain constant. SSR, MSR and MSRCR all use Gaussian filter as the filter function to estimate the illuminance component, lack of boundary preservation characteristics, resulting in unclear image edge details and halo artifacts [10].

2.3 Low illumination color image enhancement based on Gabor filtering and Retinex theory

For some shortcomings of the classical Retinex algorithm, a low illumination color image enhancement algorithm based on Gabor filtering and Retinex theory is proposed. The algorithm mainly includes three parts: The first one is to use the MSRCR to enhance image with color recovery of luminance component I in HSI space, and obtains brightness-enhanced image with good color and saturation without color distortion. The second one is to use the SSR of the Gabor filter to enhance image based on RGB space, then obtains the enhanced image with better edge texture details. The third part is weighted fusion operation of the enhanced images obtained from the first two parts with performing in the RGB space, then obtains the finally enhanced image. The specific flow chart of the algorithm is shown in Fig. 3.

Fig. 3
figure 3

The flow chart of the algorithm in this paper

Full size image

The pseudo code of the algorithm in this paper is as follows:

figure a

2.3.1 Gabor filter

The Gabor filter is a Fourier transform with a Gaussian window function. For an image, the window function can obtain local information of the image, so the information of each position can be obtained by moving the center of the window. In addition, since the Muse function is still a Gaussian function after Fourier transform, the transform is still local in the frequency domain [23]. Therefore, when analyzing the texture image, the Gabor transform can simultaneously satisfy the requirements of localization in the spatial domain and the frequency domain. The imaginary number expression of the two-dimensional Gabor function can be obtained by Eq. (9).

$$ g\left(x,y;\lambda, \theta, \psi, \sigma, \gamma \right)=\mathit{\exp}\left(-\frac{x^{,2}+{\gamma}^2{y}^{,2}}{2{\sigma}^2}\right)\mathit{\exp}\left(i\left(2\pi \frac{x^{\prime }}{\lambda }+\psi \right)\right). $$
(9)

Where x′ = x cos θ + y sin θ, y′ =  − x sin θ + y cosθ.λis the wavelength of the sine function, and its value is specified in pixels, usually greater than or equal to 2, but not greater than 1/5 of the input image size. Θ is the direction of the Gabor kernel function (filter). This parameter specifies the direction of the parallel stripes of the Gabor function, and its value is 0 to 360 degrees. ψ refers to the phase shift, the phase shift of the tuning function, and the value ranges from −180 to 180. σ is the bandwidth, the standard deviation of the Gaussian function, usually 2π. γ is the aspect ratio of the space, which determines the ellipticity of the Gabor function shape. When γ = 1, the shape is round; when γ < 1, the shape elongates along the direction of the parallel stripes. Usually this value is 0.5.

2.3.2 Retinex image enhancement based on Gabor filtering

Due to the characteristics of the Gabor filter, it can provide both local information of the signal and the overall information of the signal. For image enhancement, this can be the premise of texture detail retention. In addition, since the Gabor filter utilizes the window filter function, it has the advantages of better enhancement of Gaussian filtering and overcomes the disadvantage of loss of texture details. Its fundamental idea is to divide the signal into a number of small time intervals as the target of the Fourier transform analysis to obtain the frequency that the signal has at this time interval [11]. The operation method is to introduce a sliding window to the specific function f(t) and then perform Fourier transform [20]. In order to preserve the texture details of the image, the use of the Gabor filter is a guarantee for the experimental results. The Gabor function is expressed by the Eq. (10).

$$ G\left(x,y\right)=\frac{1}{\sqrt{2{\pi \sigma}_x{\sigma}_y}}\exp \left[-\left(\frac{x^2}{{\sigma_x}^2}+\frac{y^2}{{\sigma_y}^2}\right)/2\right]\exp \left(-2\pi jfx\right). $$
(10)

Where, σx and σy are Gaussian envelope constants, that is, Gaussian function diffusion factors, which are mathematically standard deviations. f is the frequency parameter of the filter. Retinex image enhancement based on Gabor filter, that is, the Gauss kernel function of Retinex is replaced by Gabor filter function to complete the image enhancement.

2.3.3 MSRCR image enhancement based on HSI space

Color images have three color channels of R, G, and B. Retinex theory is needed to enhance the three channels separately. However, due to the strong correlation between the three channel components of RGB, the enhancement of the three channels respectively causes their correlation to be destroyed, resulting in image color distortion. But in the HSI color space, where H is the hue; S is the saturation; I is the intensity or brightness. The enhancement of the entire image can be achieved only by the enhancement of the luminance I component, and it is relatively easy to convert the RGB and HSI color spaces, and the program is easy to implement. The conversion relationship between RGB and HSI is calculated by Eq. (11).

$$ {\displaystyle \begin{array}{l}H=\left\{\begin{array}{l}\theta, G\ge B\\ {}2\pi -\theta, G<B\end{array}\right.,\theta ={\cos}^{-1}\left[\frac{\left(R-G\right)+\left(R-B\right)}{2\sqrt{{\left(R-G\right)}^2+\left(R-B\right)\left(G-B\right)}}\right],\\ {}S=1-\left[3\min \left(R,G,B\right)\right]/\left(R+G+B\right),\\ {}I=\left(R+B+G\right)/3.\end{array}} $$
(11)

The HSI model is composed of three sectoral regions: RG, GB, and BR. Then the conversion relationship of the three regions is as shown in Table 1.

Table 1 Conversion relationship between HSI and RGB
Full size table

2.3.4 Enhanced image weighted fusion

Two enhanced images are obtained by the above two image enhancement algorithms, and the two enhanced images are merged with the Eq. (12) to obtain an enhanced image with better detail enhancement and edge preservation.

$$ {R}_i\left(x,y\right)={mR}_{1i}\left(x,y\right)+\left(1-m\right){R}_{2i}\left(x,y\right),i=R,G,B. $$
(12)

Where, Ri(x, y) is the value of R, G, B of the fused image; R1i(x, y) is the value of R, G, B of the image after Gabor filtering enhancement; R2i(x, y) is MSRCR The values of R, G and B of the image after the algorithm is enhanced; m is the weight coefficient and satisfies 0 ≤ m ≤ 1. It needs many experiments to determine the specific m value. After many experiments, we have taken the value of m and obtained the fusion image with the best visual effect.

3 Results

In order to verify the effectiv1eness of the proposed algorithm, the simulation environment selected was Matlab 2015a. We selected five images (a-e) in the data set for experimental analysis. Most of them are from the network and have different sizes, as shown in Fig. 4. In order to select the appropriate fusion control coefficient m to achieve better image fusion effect, we conducted a lot of experiments, and finally chose m = 0.4.

Fig. 4
figure 4

The above five images are color images with low illumination

Full size image

The classical single scale SSR, multi-scale MSR and MSRCR with color recovery algorithm have poor color fidelity and low contrast for low illumination images, and are prone to over enhancement and color frame loss. The experimental results of the image enhancement algorithm in this paper are compared with the SSR algorithm based on Gamma correction [4], the MSR algorithm based on bilateral filtering and the experimental results of the MSRCR image enhancement algorithm in reference [16]. The effect comparison chart is shown in Fig. 5. In the experiment, three scale parameters σ1 = 16, σ2 = 80, σ3 = 200, spatial geometric standard deviation σd = 5σ, brightness standard deviation σr = 0.001.

Fig. 5
figure 5

The above four groups of pictures are visual contrast pictures of the processing effect of low illumination color image a to e. they show the effect comparison pictures of the algorithm in this paper, SSR algorithm based on Gamma correction, MSR algorithm based on bilateral filtering and MSRCR algorithm based on reference [16]

Full size image

Compared with the MSR algorithm based on bilateral filtering, the effect is the worst. Although the edge contour is kept well, the pixel uniform position appears the phenomenon of over enhancement distortion. Maybe my Sigma value is not suitable. After many times of debugging, it shows a better result. The color fidelity of MSRCR algorithm in reference [16] is better than that of traditional MSRCR, and the edge texture details and color fidelity of SSR image enhancement algorithm based on Gamma correction are very good. But compared with the Retinex theoretical algorithm based on Gabor, the algorithm in this paper is more desirable in detail. It can be seen from the visual effect, the algorithm in this paper is better.

In order to objectively evaluate the processing results, this paper selects mean square error, information entropy value and average gradient for comparison [14]. Mean square error (MSE) is a commonly used method in image evaluation. Which can be calculated by Eq. (13). Information entropy (IE) is mainly an objective evaluation index that measures how much information an image contains [19], which represents the richness of information in the image. The information entropy of the image can be defined by Eq. (14). The average gradient represents the degree of change in the gray value of the image, and is one of the criteria for judging the processing of image details [16]. The average gradient (AG) can be defined by Eq. (15).

$$ MSE=\frac{\sum \limits_{x=1}^M\sum \limits_{y=1}^N{\left[R\left(x,y\right)-I\left(x,y\right)\right]}^2}{M\times N}. $$
(13)

Among them, R (x, y) and I (x, y) respectively represent the gray value of the original image and the enhanced image at the spatial position (x, y). The smaller the MSE, the better the image.

$$ IE=E\left[{\log}_2P{(X)}^{-1}\right]=\sum \limits_jP\left({X}_j\right)\ast {\log}_2{X}_j. $$
(14)

In Eq. (14), the neighborhood gray value of the image is selected as the spatial feature quantity of gray distribution, and the pixel gray level of the image forms a feature binary group, denoted as (i, j), where i represents the gray level of the pixel value (0 ≤ i ≤ 255), j represents the mean gray value of the neighborhood (0 ≤ j ≤ 255). It can reflect the comprehensive characteristics of the gray value of a certain pixel position and the gray distribution of surrounding pixels, where f(i, j) is the frequency of the feature two-tuple (i, j), and N is the size of the image.

$$ AG=\frac{1}{m\times n}\sum \limits_{i=1}^m\sum \limits_{j=1}^n\sqrt{\frac{{\left(\frac{\partial f}{\partial x}\right)}^2+{\left(\frac{\partial f}{\partial y}\right)}^2}{2}}. $$
(15)

Where,m × n is the size of the image, \( \frac{\partial f}{\partial x} \)is the horizontal gradient of the image, and \( \frac{\partial f}{\partial y} \) is the vertical gradient of the image.

The performance analysis indicators of each algorithm: mean square error (MSE), information entropy (IE) and average gradient (AG) comparative analysis results are shown in Tables 2, 3 and 4.

Table 2 Measurement of results of mean square error (MSE)
Full size table
Table 3 Measurement of results of information entropy (IE)
Full size table
Table 4 Measurement of results of Average Gradient (AG)
Full size table

The smaller the MSE, the better the image. We can see from Table 2 that the mean square error of the image obtained by the algorithm in this paper is the smallest in general, so we can conclude that the effect of this algorithm is the best.

Taking the higher the image information entropy, the higher the amount of information contained in the image as the standard, we can get from Table 3: the information entropy of images a, c, e obtained by the algorithm in this paper is the largest, but for images c and d, the image information entropy of SSR based on Gamma correction is larger, but overall, the image effect obtained by this algorithm is still good.

The average gradient refers to the obvious difference of gray level near the edge or shadow line of an image, that is, the change rate of gray level is large, which can be used to represent the image clarity. It reflects the change rate of the contrast of the tiny details of the image, that is, the rate of the density change in the multi-dimensional direction of the image, and represents the relative clarity of the image. Generally speaking, the larger the evaluation gradient, the more levels of the image, the clearer it will be. Therefore, it can be seen from Table 4 that the average gradient of images a, c, e processed by the enhancement algorithm in this paper is larger than that of the other three algorithms, but the average gradient of images b and d after SSR based on Gamma correction and MSR algorithm based on bilateral filtering is larger respectively, but in general, the algorithm in this paper performs well.

In conclusion, after many images verification, the Retinex theory algorithm based on Gabor filter can enhance the low illumination image. By comparing with other algorithms proposed in other references, the algorithm in this paper is slightly better than other algorithms in the case of certain visual fidelity and brightness improvement and detail retention.

4 Discussion

This paper proposes a low-light color image enhancement algorithm based on Gabor filtering and Retinex theory. The algorithm executes the MSRCR image enhancement algorithm in the HSI color space to obtain a brightness image with better hue saturation and contrast. A single-scale SSR image enhancement algorithm based on Gabor filtering is implemented in RGB color space to better enhance edge texture information retention. Finally, the images are weighted and fused to obtain the final enhanced image. From the subjective vision and objective standards, the results of the algorithm are compared with some traditional image algorithms in the past two years. Multiple image data tests show that the algorithm can effectively remove halos and over-enhancement when the visual effect is good, and obtain rich details of edge texture, color and hue, thereby obtaining an image with moderately enhanced brightness.

However, due to the limitations of classic image algorithms, image data sets cannot be enhanced on a large scale, so image enhancement processing can be performed in batches. In the future, we will optimize the parameters of the algorithm adaptively, hoping to better adapt to the parameters of different images, and to expand the deep learning algorithm for image enhancement, and look forward to better results in the future.