1 Introduction

The human visual system can discriminate between the contents in the images, but that becomes a challenge when the images appear very dark with non-uniform illumination and backlighting. Apart from the low contrast images, the presence of simultaneously very low and very high-intensity regions in the image results in a backlit image. The edge and texture information about the objects in backlit images are poor as compared to the ones in visual images. Hence the enhancement of backlit images is required. This becomes essential in many applications to enhance the image as a pre-processing step in different applications as discussed in [10, 13, 17]. The problem is challenging because the objects of interest are sometimes completely not visible at all or dominated by the backlighting conditions during the generation of the image. Therefore, understanding the objects in a better way, to improve the object detection and recognition tasks, enhancement of backlit images becomes a mandatory step for an overall gain to such processes.

A variety of proposed approaches are available in the literature for the enhancement of uniform or non-uniform illumination. Many techniques that adapt enhancement in the spatial domain are based on the neighborhood of each pixel and obtain satisfactory results [15, 33]. However, they produce halos or produces a very high amount of enhanced noise and image artifacts. Therefore, finding better visibility for the objects in dark regions of the backlit image becomes challenging and interesting. There exists pervasive treatment in the literature regarding image enhancement over the years. For backlit images, the most promising and popular techniques are a combination of different features and fusion-based methods.

1.1 Related work

Multiple approaches to address the issue of undesired illuminations have been studied and proposed in the literature to improve the image quality. We briefly reviewed major approaches for the enhancement of undesired illumination conditions, as well as their limitations.

The histogram equalization-based method to study the undesired illumination was proposed in [27]. Histogram equalization-based methods assume that the histogram is distributed uniformly. Wang et al.  [37] discussed a detailed experiment-based review summarizing the widely used classes of low-light image enhancement algorithms. Also, the camera response models are studied for low-light image enhancement in [41]. Ren et al. [28] studied the retinex model for the naturalness preserving of low-light image enhancement. Research for nighttime image development, a new illumination boost algorithm is proposed in [2]. This algorithm requires basic exponential, logarithmic functions, and linear scaling functions, etc. Wang et al. [39] proposed a novel absorption of light scattering-based model (ALSM) for the enhancement of low light images using the response characteristics of cameras. A pair of complementary gamma functions (PCGF) is proposed by [19], they obtained an underexposed version of the original low-light image, and an overexposed version. Srinivas et al. [33] proposed spatial contextual similarity histogram (SCSH) based on the spatial contextual similarities between the neighboring pixels.

A detailed experiment-based review summarizing the widely used classes of low-light image enhancement algorithms is presented in [37]. Camera response models are studied for low-light image enhancement [41] and combines the retinex model for the naturalness preserving of low-light image enhancement [28]. Research for nighttime image development, a new illumination boost algorithm is proposed [2]. The newly developed algorithm requires basic exponential, logarithmic functions, and linear scaling functions, etc. Wang et al. [39] proposed a novel absorption of light scattering-based model (ALSM) for the enhancement of low light images using the response characteristics of cameras. A pair of complementary gamma functions (PCGF) is proposed by [19], to obtain an underexposed version of the original low-light image, and an overexposed version can be obtained. Srinivas et al. [33] proposed spatial contextual similarity histogram (SCSH) based on the spatial contextual similarities between the neighboring pixels.

Contrast enhancement of images using spatial information of pixels is also studied in the literature. The discrete entropy measures for gray level distribution are studied in [5,6,7] to achieve contrast enhancement for low contrast images. This is particularly valid when the amount of specific image histogram intensity values remains the same pre and post enhancement, as with most histogram-based contrast enhancement methods. Niu et al. [26] proposed a novel approach based on tone preserving discrete entropy maximization and finds its prospect. They combined the preprocessing of image restoration with their algorithm to make it more robust. Li et al. [18] proposed a global and adaptive contrast enhancement algorithm for low illumination gray images, it reduces the uneven illumination and lowers the overall contrast of the gray image. Zarie et al. [42] studied the triple partitioning of the histogram and histogram clipping to control the enhancement ratio. Shin et al. [31] proposed an approach to increase the visibility of still images automatically. This approach operates without any threshold or magic value by analyzing the image data alone. The fusion of images using a bracketed exposure sequence to obtain a high-quality image is proposed by Mertens et al. [24]. This study suggests the blending of multiple exposures guided by contrast and saturation. However, the exposure fusion has certain limitations as discussed in [12]. Multi-exposure-based image enhancement method is performed for detail preserving based on the tone mapping curves and exposed regions as discussed in [21]. Singh et al. [32] studied the detail enhancement by implementing the exposure fusion based on nonlinear translation-variant filters. Wang et al.  [36] studied the multi-layer lightness statistics of high-quality outdoor images with a combination of contrast enhancement and naturalness preservation. Jha et al. [16] proposed a novel technique using dynamic stochastic resonance in the discrete cosine transform (DCT) domain. Chouhan et al. [8] presented a study on noise enhanced iterative processing. This is based on Fourier coefficients for enhancement of low contrast images using Iterative scaling of DCT coefficients. Martorell et al. [23] proposed a novel algorithm for multi-exposure fusion by decomposing the patches of the image with DCT transform. Morel et al. [25] proposed a method of simple gradient-domain to eliminate the effect of non-uniform illumination by preserving the details in the image. CNN-based Unet model is proposed by Huang et al. [14], they proposed the mix loss functions to enhance the low-illumination images. The contrast and brightness are broadly over-enhanced using these approaches. In the final steps, enhancing a backlit image require multiple images with different exposure information.

Kim et al. [17] proposed a block-based backlit region detection and enhancement for consumer devices. The method partitions a backlit image into multiple rectangular blocks and performs the classification and contrast enhancement processes. This idea becomes a building block in the selection of ROI in backlit images in our proposed. Dhara et al.  [9] studied the method for backlit image restoration. This method involves the human visual system (HVS) sensitivity-based approach using a multi-scale fusion of multiple pseudo-exposed images and the information for backlit restoration. There exist content-aware algorithms that perform enhancement of dark images, sharpening of edges by simultaneously revealing the details present in textured regions, also by preserving the smoothness of flat regions as discussed in [29]. Buades et al. [4] proposed a method for backlit image contrast enhancement using multiple tone mappings and adjustment of different regions in the image. The results obtained are a combination of all these processing and image fusion algorithms. Zhao [43] proposed a K nearest neighbors (KNN)-based matting for dark foreground extraction in backlit images and logarithmic transformation to enhance the backlit region. This method employs only logarithmic transform to enhance the foreground region. A simplistic backlit image enhancement technique realized by intensity conversion with preservation of hue and saturation is proposed in [34]. Multi-step methods for enhancement of backlit images are studied based on transmission coefficients computation, multiple exposures generation based on transmission coefficients, and image fusion in [15]. Fu et al.  [11] proposed a fusion of derived inputs with corresponding weights in multi-scale mode to enhance the given input. Wang et al. [35] proposed a novel approach for the enhancement of region of interest-based on the multi-scale-based fusion strategy for single backlit image enhancement. Contrast measure (CM) [4] based on the local variance of the image is considered to be one of the suitable parameters for the validation of results obtained. Another measure to observe the change in the texture of the image in literature is the discrete entropy (DE) [30] of pixels [1, 3, 38]. The DE calculation can reliably represent the change in contrast and reveals the textural content improvement in the enhanced image. A balanced mean magnitude of relative error (BMMRE) [40] measure is also evaluated for the DE values.

As many results suggest, a trade-off between the detail enhancements, local contrast, and naturalness of the image always exists. A novel approach for the enhancement of region of interest in backlit images is proposed in this work, primarily focusing on enhancing features of ROI of backlit images. Therefore, in light of the foregoing discussions, there exists a need to overcome the limitations of existing techniques for the enhancement of backlit images. As observed from the literature, the following research gap is observed by the authors. First, there is a need to develop a method for backlit image enhancement that focuses on contrast enhancement of regions that are affected due to backlighting conditions. Second, preserving and enhancing the textural content of the dark object regions in the backlit images. The author’s contribution of this work is to address the above research gaps, which are as follows. First, to develop a novel method that generates multiple exposure images exploiting the global tone mapping functions (GTMF). Second, gradient map-based information for preserving and enhancing the textural contents of ROI of backlit image. A novel approach for the enhancement of ROI of backlit is discussed by primarily focusing on the multiple features such as GTMFs, gradient map, exposure map, etc. is proposed in the present paper.

The paper is organized as follows. Section 2 details the proposed methodology, Sect. 3 presents the experimental results and discussion, and Sect. 4 finally discusses the conclusions.

Fig. 1
figure 1

a, b, c, d, and e represents the Original input backlit image by FBSB [35] and respective ROI inside the yellow marker

Full size image
Fig. 2
figure 2

a, b, c, d, and e represents the Original input backlit image by LBRB [20] and respective ROI inside the yellow marker

Full size image

2 Proposed methodology

The two important aspects of backlit images are lack of luminance and degraded contrast in the underexposed regions [35]. Therefore,in the proposed approach, the problems associated with backlit images is divided into three parts mainly: luminance and contrast enhancement technique, preserving boundary information, and fusion of all features information. The fundamental steps are build by separation of the three-channel images into Red (R), Green (G), and Blue (B) for individual treatment. Then the results of specific feature operations and exposure maps are merged into a single one. Figure 3 represents detailed structure framework diagram representing multiple features generated during different stages in the process. All three input channel images work as inputs and operated in layers. At the final stage, all the feature results are combined to generate the enhanced version of the image.

Fig. 3
figure 3

Detailed structure framework diagram representing multiple features at different stages

Full size image

2.1 Tone mappings

There are very low and very high-intensity regions in the backlit images, which requires treatment of low luminance regions and dynamic range compression. First, the input backlit image is divided into three-channel images—R, G, and B respectively. For simplicity, two global tone mapping operations viz. gamma correction and logarithmic transformation are selected. Both of them have wide acceptance in the literature. The gamma correction and logarithmic transformation are applied over all the channel images. Gamma correction and logarithmic transformation are defined in Eqs. (1) and (2) respectively.

$$\begin{aligned}&\text {Gamma}(\gamma )(I)=255 \times \left( \frac{I}{255}\right) ^{\gamma }, \gamma \in \{0.4,0.8,2\} \end{aligned}$$
(1)
$$\begin{aligned}&\text {Log}(\alpha )(I)=\frac{255 \times \log (\alpha I+1)}{\log (255\alpha +1)}, \alpha \in \{0.1,0.3,0.5\} \end{aligned}$$
(2)

where, I represents the image, \(\gamma \) and \(\alpha \) coefficients represent the adaption factor and brightness factor respectively.

Exposure measure defines the closeness to the mid-intensity value by using a Gaussian curve [24] as given in Eq. (3),where \(\sigma _{i}\) is a parameter that the authors fix to 0.2.

$$\begin{aligned} wt(I)=\exp {\left( -\frac{(I/255 -0.5)^{2}}{2\sigma _{i}^{2}}\right) }, \sigma _{i}=0.2 \end{aligned}$$
(3)

2.2 Gradient mapping and filtering

To observe the directional change and edge information, a gradient map is used which is defined in Eq. (4). This preserves the edge information in the channel images. Every pixel measures the change in intensity of the same point as in the original channel image, in a particular direction.

$$\begin{aligned} \nabla f={ \begin{bmatrix}g_{x}\\ g_{y} \end{bmatrix}} = {\begin{bmatrix} \partial f/\partial x \\ \partial f/\partial y \end{bmatrix}} \end{aligned}$$
(4)

\(\partial f/\partial x\) represents the derivative with respect to x.

\(\partial f/\partial y\) represents the derivative with respect to y.

Fig. 4
figure 4

a and d represents ROI of original image and corresponding histogram from Fig. 2, b and e represents the enhanced ROI obtained by LBRB [20] and corresponding histogram, c and f represents the enhanced ROI by the proposed method and corresponding histogram

Full size image

In a similar context, the image is smoothened using order statistics filtering. Median filtering has been employed for this purpose. It is defined in Eq. (5). It replaces each entry with the median of neighboring values. Median filtering helps to obtain a smoothed version of the image.

$$\begin{aligned} z[r,s]=\text {median}\{ q[i,j], (i,j) \in w \} \end{aligned}$$
(5)

where w represents local neighborhood centered around location [r, s] in the image. The size used in this paper is \(5 \times 5\).

2.3 Fusion

All the color channel images-based tone mappings, exposedness, gradient etc. obtained by applying (1), (2), (3), (4) and  (5) are combined using image fusion algorithm based on Mertens et al. [24] and Ma et al. [22]. The algorithm is designed to merge several feature inputs of the processed channel images as per equation (6).

$$\begin{aligned} F(i)=\sum _{k=1}^{N}wt_{k}(i)I_{k}(i) \end{aligned}$$
(6)

where, \(wt_{k}(i)\) represents the weight map based on well-exposedness generated from Eq. (3), N represents the number of images to fuse, and \(I_{k}(i)\) represents the different feature inputs of the image.

3 Experiments and discussion

A comparative study has been presented based on the state-of-the-art Fusion-based method for single backlit image (FBSB) [35] and Learning-Based Restoration of Backlit Images (LBRB) [20] to assess the proposed algorithm’s performance. The experiments were conducted on 82 test images from the backlit image database shared by; 64 backlit image database from the Li et al. [20] and 18 backlit image database from the Wang et al. [35]. Experiment over all the images are processed on a PC with an Intel i5-3230M based processor with a base clock of 2.6 GHz and 8 GB RAM. For the evaluation purpose we have presented validations for a number of test image results of the proposed method with the state-of-the-art in Tables 1 and 2.

3.1 Subjective evaluation

The ROI of the original image is reproduced in Figs. 1 and 2 which is taken from FBSB [35] and LBRB [20], respectively. In Figs. 4, 5 and 6, we present the subjective evaluation of a total of ten ROI for detailed analysis. It can be observed that FBSB and LBRB are not able to enhance the ROI sufficiently. In most cases, results of FBSB and LBRB suffer from certain low contrast and textural information. In terms of visibility of ROI of backlit images, LBRB and our proposed method are better than FBSB. But LBRB suffers from over-exposedness and low textural detail enhancement in some cases. Figure 4 shows such an example where the visibility difference of ROI from the Girl image can be observed. The corresponding enhanced ROI of the proposed result Fig. 4e provides better visualization as compared to the original ROI and LBRB as in Fig. 4a, c respectively. The same can be observed in Figs. 5 and 6 where the visibility and textural content of the ROI of the backlit image is enhanced significantly using the proposed method. The output results by the proposed method are shown in Figs. 4, 5 and 6. A three-column-based representation is used for showing the comparison in Figs. 4, 5 and 6. The first column represents the ROI of the original image; the second column shows the results obtained from LBRB [20] in Fig. 4, FBSB [35] in Fig. 5 and LBRB [20] in Fig. 6. The third column shows the results of the proposed method in respective figures.

Fig. 5
figure 5

a ROI of original image from Fig. 1, b the enhanced ROI by FBSB [35] and c the enhanced ROI by the proposed method

Full size image
Fig. 6
figure 6

a ROI of original image from Fig. 2, b the enhanced ROI by LBRB [20] and c the enhanced ROI by the proposed method

Full size image

3.2 Quantitative evaluation

For the quantitative evaluation, we considered contrast measure (CM) [4], discrete entropy (DE) [30], and balanced mean magnitude of relative error (BMMRE) [40] to quantify the degree of enhancement. CM [4] is used to observe the amount of change in contrast that has been improved in the image. It is defined in Eq. (7).

$$\begin{aligned} \text {CM} = \frac{1}{P}\sum _{k=1}^{P}\frac{\text {Var}_{k}(\text {processed})}{\text {Var}_{k}(\text {original})} \end{aligned}$$
(7)

where \(\text {Var}_{k}\) is the variance of the intensity values in a patch of size \(16\times 16\). \(\text {Var}_{k}(\text {processed})\) and \(\text {Var}_{k}(\text {original})\) represent that variance is computed on both the original and processed images. P represents the total number of patches.

We consider DE [30] as the second measure to evaluate the information present in the output. It is used to estimate the amount of texturedness present in the image. This measure is appropriate as it reveals the textural content improvement in the enhanced version of the image. The higher the value indicates, the richer the textural content of the enhanced image. It is defined in Eq. (8).

$$\begin{aligned} \text {DE} =-\sum _{i=0}^{255}h_{i}\log (h_{i}) \end{aligned}$$
(8)

\(h_{i}\) represents the pixels having intensity i in-between the number of different intensity values, i.e., 256 for 8-bit images.

BMMRE [40] measures the balanced mean magnitude of relative error between the original and resultant DE values. The higher the value of BMMRE of DE indicates the more significant textural change in the enhanced image. A balanced mean magnitude of relative error measure of DE is obtained as follows:

$$\begin{aligned} \text {BMMRE}=\frac{1}{n} \sum _{i=1}^{n} \frac{\left| y_{i}-\hat{y}_{i}\right| }{{\text {Min}}\left( y_{i}, \hat{y}_{i}\right) } \end{aligned}$$
(9)

\(y_{i}\) and \(\hat{y}_{i}\) represents the ith DE values of original and enhanced image respectively. n is the total number of images.

A detailed analysis-based evaluation of results for CM is presented in Table 1. Similarly, analysis-based evaluation of results for DE and BMMRE of DE are presented in Table 2 respectively. The results of the proposed approach are compared with the results obtained by [20, 35]. A comparison of the contrast measure is given in Table 1. The contrast measure of the enhanced ROI of the proposed method is high, indicating the increase in contrast as evident in Table 1. Table 2 represents the comparison of improved DE results used to estimate the growth in texturedness in the ROI image. Figure 4 column-wise represents the ROI and corresponding histogram of the original, LBRB, and proposed approach, respectively. A stretch of histogram in Fig. 4f can be observed very clearly as compared to the distributions in Fig. 4b, d respectively. The enhanced ROI image obtained by the proposed model, which is shown in the third column of Fig. 5 and Fig. 6 is found better in terms of CM, DE, and BMMRE of DE, respectively.

The CM values are given in Table 1 for output results of FBSB [35], LBRB [20] and the proposed method, respectively. The results in the proposed section of Table 1 show an overall gain of the measure concerning the contrast enhancement measure. Results with higher CM values have shown a good trade-off between the textural enhancement and naturalness of the image results. For example, the enhanced ROI in case of Board, Hills, Tower, Cycle, Girl, Bracelet and Fat Man are enhanced significantly with CM values 6.70, 6.18, 13.26, 17.29, 18.11, 30.91 and 17.80 respectively over the LBRB [20] and FBSB [35] results of the same. A bar chart-based representation for Comparison of CM results is shown in Fig. 7. This may be inferred as the trade-off between the exposedness, and textural details are small for higher CM values up to 2–3 times. For very high CM values, i.e., up to 4–5 times, an increase in the trade-off between the exposedness and textural details is observed. From Table 1 the CM values of enhanced ROI of the proposed method proves to be superior over the enhanced ROI by LBRB and FBSB. This is evidenced by the average CM results of Enhanced ROI (proposed) 12.06 as compared with 5.92 and 3.10 of LBRB [20] and FBSB [35] respectively.

Table 1 Comparison of contrast measure (CM)
Full size table
Table 2 Comparison of discrete entropy (DE) and BMMRE of DE
Full size table

The DE evaluation can reliably represent the change in textural detail but not the enhancement of contrast. The DE values of the original ROI, enhanced ROI by the LBRB [20] and FBSB [35] and the proposed method are given in Table 2. Table 2 represents the comparison of improved DE results used to estimate the growth in texturedness in the image. As shown in comparison Table 2 Board, Hills, Building, and Cycle gain an edge over the enhanced ROI by FBSB in terms of the DE measure. The DE values of the proposed method for Board, Hills, Cycle, Girl, Bald Man, and Fat Man are 6.78, 7.35, 6.72, 7.46, and 7.26 respectively. A bar chart-based representation for Comparison of DE results is shown in Fig. 8. From Table 2 the DE values of enhanced ROI of the proposed method gains an edge over the individual enhanced ROI by LBRB and FBSB. This is evidenced by the average DE value of Enhanced ROI (proposed) 6.44 as compared with 6.09 and 6.29 of LBRB [20] and FBSB [35] respectively. A drop in the proposed DE measure of enhanced ROI in Table 2 for Tower, Building, Girl, Bracelet, and Hairy Man is observed due to the trade-off in textural content enhancement and contrast. Presence of optimal tone mapping and complex pyramid-based operations in FBSB and LBRB, respectively. Although in the case of Tower, the DE values are almost the same. This is observed due to the trade-off in textural content enhancement and contrast, minor artifacts in the edges of results.

Fig. 7
figure 7

Bar chart representation for comparison of contrast measure (CM)

Full size image
Fig. 8
figure 8

Bar chart representation for comparison of discrete entropy (DE)

Full size image

BMMRE of DE evaluation in Table 2 for LBRB, FBSB and proposed approach obtained is 0.177, 0.241 and 0.254 respectively. BMMRE shows a gain of 0.013 in the value. A higher BMMRE of DE signifies more enhanced textural content in the ROI of backlit images. The perceptual color reproduction in the enhanced ROI of proposed results has better visualization as in Figs. 5c and 6c.

Based on the above evaluations and discussion, it is evident that the proposed method performs better than others for enhancing the ROI of backlit images, subjectively and quantitatively. The evaluation parameters show an overall contrast and textural enhancement in the ROI of backlit image for the proposed approach concerning the FBSB [35] and LBRB [20] outputs.

4 Conclusion

A simple and novel approach is presented in this paper for the contrast enhancement of ROI of backlit images. The contrast stretching is performed using Global tone mapping functions. Multiple techniques such as gradient map, filtering, and exposure maps are employed to enhance multiple features of ROI such as edges, textural information, exposure, and minimization of noise. However, a trade-off between the precise boundary and color enhancement is observed in some cases. The validation of the proposed method based on CM and DE values for all the images are discussed along with BMMRE of DE. The evaluation of BMMRE of DE strengthens the claim for higher DE in support of better textural information in enhanced ROI. Further, the issues associated with the trade-off in the color enhancement of over-exposed images are the subject of future research.