Adaptive histogram equalization based on modified probability density function and expected value of image intensity

Abstract

Most of the adaptive histogram equalization methods enhanced the image locally instead of global enhancement. In this paper, a global enhancement method is proposed which is based on modified probability density function and expected value of image intensity. Adaptiveness is introduced here in the form of expected value of image intensity. This method can be very well utilized in all the display devices. The proposed method is compared with the state-of the-art other contrast enhancement techniques. Experimental results show that the proposed method surpasses the other techniques both quantitatively and qualitatively.

1 Introduction

Image enhancement is considered to be the most functional area of image processing. The application domain consists of several fields such as medical, security, space, military, remote sensing and graphic art [1, 2]. An image can be degraded due to different reasons like if the quality of the gadget used for imaging is very poor, if there are extreme environment conditions or if the person clicking the image is not expertized. Hence, contrast enhancement for both images and videos can effectively improve the image visual quality for human perception and automatic recognition.

A number of contrast enhancement methods have been proposed since last two decades. Basically these enhancement techniques can be categorized in two groups: spatial domain methods and transform domain methods [3, 4]. Spatial domain-based techniques change the contrast of an image by directly manipulating the intensity values of an image. Histogram specification, histogram modification, histogram equalization, filtering techniques, log transformation and power law transformation come under spatial domain methods. In transform domain methods, image intensity is not directly manipulated; instead, orthogonal transform is changed by altering the frequency content of an image. Discrete Fourier transform (DFT), discrete cosine transform (DCT) and discrete wavelet transform (DWT) fall under transform domain techniques.

In this paper, an adaptive histogram equalization using modified PDF is proposed for contrast enhancement. Basically it is comprised of two steps:

• Modification of the image histogram probability distribution function (PDF) adaptively.

• Reconstruction of the image using cumulative distribution function (CDF).

The proposed method is based on spatial domain contrast enhancement. It can maximize the information as well as make the enhancement level appropriate. The proposed algorithm could be utilized in the area of consumer electronics, such as TV, smartphones, all such display devices.

The rest of the paper is organized as follows: Sect. 2 presents the related works. Detailed analysis of the proposed method is described in Sect. 3. Evaluation parameters are explained in Sect. 4. Experimental results are discussed in Sect. 5. Finally, the paper is concluded in Sect. 6.

2 Related work

This section presents the spatial domain-based state-of-the-art techniques utilized for image contrast enhancement as our work is based on spatial domain technique. Histogram equalization (HE) [2] is one of the most simplest and widely used methods. It reorganizes the pixel intensity values of an image uniformly throughout the range. If I is an image having L discrete intensity levels as $(I_{0}, I_{1},\dots I_{L-1})$, then transformation function for HE is given as:

$$\begin{aligned} f(I_k) = I_{0}+(I_{L-1}-I_{0})*C(I_k) \end{aligned}$$

(1)

where $C(I_k)$ is the CDF up to $k\mathrm{th}$ intensity level $(I_{k})$ and L refers to maximum value of intensity. HE fails to preserve the mean brightness of the image and also results in over-enhancement due to which the image loses its natural look. To deal with these limitations and issues, many HE variants are proposed.

Contrast enhancement using brightness preserving bi-histogram equalization (BBHE) [5] method was proposed by Kim in 1997. BBHE separates the image histogram in two parts based on its mean, and then each histogram is equalized independently. Though this method conserves the mean brightness, it introduces artefacts in the image and also over-enhances the image.

Similar to BBHE, a method based on equal area dualistic sub-image histogram equalization (DSIHE) [6] was proposed in 1999, where image histogram is divided into two equal parts based on median. This method also preserves the brightness of the image but loses the natural appearance of the image.

Adaptive image contrast enhancement using generalizations of histogram equalization (AHE) [7] was proposed in 2000. In this particular method, local cumulative functions are modulated for contrast enhancement, but AHE fails to bring out the details of the image.

Minimum mean brightness error bi-histogram equalization (MMBEBHE) [8] was proposed in 2003. MMBEBHE calculates the histogram bisection threshold in a way that the brightness difference between the input image and the output image is minimized. MMBEBHE preserves maximum mean brightness, but the overall output image is distorted.

Recursive mean separate histogram equalization

(RMSHE) [9] and recursive sub-image histogram equalization (RSIHE) [10] were proposed in 2003 and 2007, respectively. Both these methods divide the image histogram recursively on the basis of mean and median, respectively, but to decide an optimal value of iteration factor is a difficult task. Recursive separated and weighted histogram equalization mean (RSWHE-M) [11] was given by Kim and Chung after further extending RMSHE and RSIHE methods, for brightness preservation. It is a time-consuming method.

Exposure-based sub-image histogram equalization (ESIHE) [12] was proposed in 2014. In this method, histogram division is based on the multi-exposure process of camera. The contrast of the image is enhanced very well, and over-enhancement problem is also resolved by applying the clipping process. But the image brightness is not preserved.

Intensity exposure-based bi-histogram equalization

(IEBHE) [13] was proposed in 2016. In this method, histogram is segmented based on intensity exposure. Sometimes this method results in under-enhancement .

In 2017, a method low-light image enhancement using variational optimization-based retinex model (LLVORM) [14] was proposed. Basically this method uses spatially adaptive $l_2$-norm-based retinex model for the enhancement of low-light images.

Another method Bi-histogram equalization using modified histogram bins (BHEMHB) [15] was proposed in 2017. This particular method divides the histogram on the basis of image median brightness and also varies the bins of the histogram.

The proposed technique presents efficient algorithm for image contrast enhancement and when compared to above techniques provides better results in terms of image quality, details of the image and level of contrast enhancement. It is different from other AHE methods as this method is globally adaptive.

3 Proposed method

The flowchart of the proposed method is shown in Fig. 1.

Fig. 1

Flowchart of the proposed method

3.1 Generation of image histogram and clipping threshold

The first step of the method is to generate histogram of the image. Let an gray image (I) have intensity from $I_0, I_1\dots I_{L-1}$. Its histogram is calculated and represented as $H(I_k)$. Now, the clipping process is applied on the image histogram to avoid the over-enhancement of the image which is generally observed in the enhanced images generated by other HE techniques. The clipping threshold is calculated as the median value of the intensity level occurrences in the image histogram.

$$\begin{aligned} T_\mathrm{clipp}=\frac{n(i)+1}{2} \end{aligned}$$

(2)

where n(i) is the number of gray levels of image histogram. Clipped histogram is calculated as:

$$\begin{aligned} H_{C}(I_k)= {\left\{ \begin{array}{ll} T_\mathrm{clipp}, &{}\quad \text {if } (H(I_k)\ge T_\mathrm{clipp})\\ H(I_k), &{}\quad \text {otherwise} \end{array}\right. } \end{aligned}$$

3.2 Modification of the image histogram PDF

After calculating the clipped histogram as above, now the second step is to obtain the PDF of clipped histogram and to modify that PDF. The PDF of $k\mathrm{th}$ intensity level $p(I_k)$ is given as

$$\begin{aligned} p(I_k)=\frac{H_{C}(I_k)}{M \times N} \end{aligned}$$

(3)

where $M \times N$ is the total number of pixels in the image. To modify the PDF, two individual steps are followed here:

3.2.1 Calculation of expected value of image intensity $(E[I_k])$

The normalized expected value of image intensity $E[I_k]$ is a adaptive parameter in our proposed method. It directly effects to the enhancement results.

$$\begin{aligned} E[I_k]= \frac{\sum ^{L-1}_{I_k=0} I_{k}*p(I_k)}{\mathrm{max}(I_k)} \end{aligned}$$

(4)

where L is the total number of intensity levels and $\mathrm{max}(I_k)$ is the maximum intensity value of image. The $E[I_k]$ is adaptive in the sense as it will change depending on average brightness of the image. As HE is good for low-brightness image, for such image we will get a low value of $E[I_k]$ and for high-brightness image $E[I_k]$ will be higher. Thus, accordingly PDF will be modified for proper image enhancement.

Fig. 2

Modified PDF graph for F16 image with different $E[I_k]$ values

3.2.2 Calculation of average absolute error metric (e)

Another parameter which is calculated in this paper for PDF modification is average absolute error metric (e). Basically, we are finding the difference between two PDF’s, and hence, it is termed as error metric. Here, PDF of the image is divided recursively based on each intensity value.

• 1. Let PDF of image be a row matrix of size $1\times L$, where $P_0$, $P_{1}\dots P_{L-1}$ represent the probability density value at particular gray level.

• 2. This PDF is divided in two sub-PDF’s, i.e., $\mathrm{PDF}_1$ and $\mathrm{PDF}_2$. $\mathrm{PDF}_1$ will vary from $P_0$ to $P_{i}$. $\mathrm{PDF}_2$ will vary from $P_{i+1}$ to $P_{L-1}$.

• 3. Let mean of $\mathrm{PDF}_1$ and $\mathrm{PDF}_2$ be $\mathrm{PDF}_\mathrm{1avg}$ and $\mathrm{PDF}_\mathrm{2avg}$, respectively.

• 4. We generated two identity matrix $I_1$ of size $\mathrm{PDF}_1$ and $I_2$ of size $\mathrm{PDF}_2$.

• 5. Now, we multiply $\mathrm{PDF}_\mathrm{1avg}$ with $I_1$ and $\mathrm{PDF}_\mathrm{2avg}$ with $I_2$ and store these values as $M_1$ and $M_2$ correspondingly where $M_1$ is of dimension $\mathrm{PDF}_1$ and $M_2$ is of dimension $\mathrm{PDF}_2$. M consists of $M_1$ and $M_2$.

• 6. Next, we find the absolute difference between PDF and M which is given as $D_i$. Again, we find the mean value of $D_i$, i.e., $D_\mathrm{im}$.

• 7. This whole process from step 2 is repeated for all the intensity values from $i =0$ to 255 and corresponding $D_\mathrm{im}$ values are stored for each iteration. The final matrix F is obtained after storing all values of $D_\mathrm{im}$.

The normalized value of F is given as:

$$\begin{aligned} F_\mathrm{norm} = F/\mathrm{max}(F) \end{aligned}$$

(5)

Final error metric (e) is given as:

$$\begin{aligned} e= F_\mathrm{norm}/\mathrm{sum}(F_\mathrm{norm}) \end{aligned}$$

(6)

The modified PDF ($\mathrm{PDF}_\mathrm{modified}(I_k)$) is

$$\begin{aligned} \mathrm{PDF}_\mathrm{modified}(I_k)=\frac{\mathrm{PDF}(I_k)+E[I_k]*e}{\mathrm{sum}(\mathrm{PDF}(I_k)+E(I_k)*e)} \end{aligned}$$

(7)

3.3 Reconstruction of image back with CDF-based transformation

The modified PDF obtained in Eq. (6) is now transformed as in the HE technique to get the enhanced image. For this, we calculated the CDF as:

$$\begin{aligned} \mathrm{CDF}(I_k)=\sum _{k=0}^{L-1}\mathrm{PDF}_\mathrm{modified}(I_k) \end{aligned}$$

(8)

The transformation function is:

$$\begin{aligned} F_L(I_k)= I_0+(I_{L-1}-I_0)*\mathrm{CDF}(I_k) \end{aligned}$$

(9)

$F_L(I_k)$ is the final enhanced image.

3.4 Impact of $E[I_k]$ on image enhancement

The expected value of image intensity $E[I_k]$ is key adaptive parameter in our proposed method, since this directly effects on the enhancement results. The reason behind calculating the $E[I_k]$ value according to Eq. (3) is due to two cases.

Case 1: When the value of $E[I_k]$ is very low

If we select $E[I_k]$ value to be too small like 0.001, then the factor ($E[I_k]*e$) in Eq. (6) will be neglected and it will convert to

$$\begin{aligned} \mathrm{PDF}_\mathrm{modified}(I_k)= \frac{\mathrm{PDF}(I_k)}{\mathrm{sum}(\mathrm{PDF}(I_k))} \end{aligned}$$

(10)

and since

$$\begin{aligned} \sum ^{L-1}_{k=0} \mathrm{PDF}(I_k) =1 \end{aligned}$$

(11)

Therefore, the modified PDF will be equal to the original PDF.

$$\begin{aligned} \mathrm{PDF}_\mathrm{modified}(I_k)= \mathrm{PDF}(I_k). \end{aligned}$$

(12)

Figure 2 shows the modified PDF graph with different values of $E[I_k]$. It can be observed from this graph that when the $E[I_k]$ value is 0.001, it overlapped the original PDF . Yellow line overlapped the blue line.

Also the enhanced image results in a similar image to HE image. Figure 3 shows the results for F16 enhanced image with different values of $E[I_k]$. As can be seen, Fig. 3d is similar to Fig. 3b.

Case 2: When the value of $E[I_k]$ is very high

If $E[I_k]$ value is selected to be too large like 100, Eq. (6) will become

$$\begin{aligned} \mathrm{PDF}_\mathrm{modified}(I_k)= \frac{E[I_k]*e}{\mathrm{sum}(E[I_k]*e)} \end{aligned}$$

(13)

From Fig. 2, it can be seen that when $E[I_k]$ is 100, the generated PDF is almost flat. Magenta color line is for $E[I_k] = 100$, and the image is converted back to original image. This is happening so because since the PDF is flat, the CDF is consecutively increasing in point steps and the values are approximately equal to ($\frac{1}{255}$, $\frac{2}{255}$, $\frac{3}{255}\dots$ and so on) and it is almost a straight line. Hence, the pixel intensity values in enhanced image are nearly equal to original image. As can be seen, Fig. 3g, h is almost similar to original image.

In Fig. 2, the red line is the modified PDF with the $E[I_k]$ value calculated according to Eq. (3). As we can see, it altered the graph very well enhancing the very low peaks and suppressing the very high peaks of the original PDF. Figure 3c is the enhanced image with reference $E[I_k]$ value, i.e., 0.77. It can be observed that it is the best image among all.

Fig. 3

Enhancement results of F16 image with different $E[I_k]$ values: a original image, b HE image, c enhanced image with reference $E[I_k]$ value, d enhanced image with $E[I_k] = 0.001$, e enhanced image with $E[I_k] = 0.1$, f enhanced image with $E[I_k] = 1$, g enhanced image with $E[I_k] = 10$, h enhanced image with $E[I_k] = 100$

4 Performance evaluation measures

To compare the discussed image enhancement methods quantitatively, four parameters are tested for quantitative assessment which are entropy, peak signal-to-noise ratio (PSNR), gradient magnitude similarity deviation (GMSD) and histogram utilization efficiency ($\mathrm{UE}_\mathrm{hist}$).

Entropy [9] is the measurement of information content present in the image. Entropy is calculated with the help of PDF as

$$\begin{aligned} \mathrm{Entropy}=-\sum _{k=0}^{L-1}p(I_k)\mathrm{log}_2 p(I_k) \end{aligned}$$

(14)

where $p(I_k)$ is the probability value of the $k\mathrm{th}$ intensity level. The higher value of entropy indicates more details in an image.

Even though human eye requires that the image should be enhanced, it should also preserve its natural appearance. PSNR is a good metric [10] and is mainly employed to calculate the quality gain between the input and processed images. A high value of PSNR indicates better results.

$$\begin{aligned} \mathrm{PSNR}= 10 \mathrm{log}_{10}\frac{(L-1)^2}{\mathrm{MSE}} \end{aligned}$$

(15)

where MSE is mean square error.

GMSD [16] is a very productive and logical quality index measurement. Basically it tells the amount of distortion present in the image. So, the value of GMSD should be as low as possible. Mathematical calculation for GMSD is given by:

$$\begin{aligned} \mathrm{GMSD} = \sqrt{\frac{1}{YZ}\sum _{i}\sum _{j}(\mathrm{GMS}(i,j)-\mathrm{GMSM})^2} \end{aligned}$$

(16)

where $Y \times Z$ is the size of the image I.

$\mathrm{UE}_\mathrm{hist}$ is another significant metric that can be calculated to interpret the property of the image histogram [17]. For a good contrast enhancement, the image must possess regularly scattered histogram throughout the complete area without changing the primary features of the histogram too much. $\mathrm{UE}_\mathrm{hist}$ is given as:

$$\begin{aligned} \mathrm{UE}_\mathrm{hist} = \frac{\mathrm{NB}_\mathrm{e}}{\mathrm{NB}_\mathrm{o}} \end{aligned}$$

(17)

where $\mathrm{NB}_\mathrm{e}$ and $\mathrm{NB}_\mathrm{o}$ represent the number of nonzero bins (utilized gray levels) of enhanced and original images correspondingly. Since the number of nonzero bins in enhanced image is comparatively lower than in original image, the value of $\mathrm{UE}_\mathrm{hist}$ is less than 1. But, if the value is much less than 1, it destroys the image quality. Hence, it should be closer to 1.

Table 1 Various parameters measures

Table 2 Various parameters measures

5 Experimental results and analysis

The proposed method has been applied to several gray images, and the performance is compared with different techniques ranging from 1997 to 2017. Those are HE, BBHE, AHE, RSIHE, RSWHE-D, MMBEBHE, ESIHE, IEBHE, LLVORM and BHEMHB. The test images are taken from the standard Computer Vision Group (CVG-UGR) database and Kodak lossless true color image suite.

5.1 Quantitative assessment

Tables 1 and 2 give the results for various test images obtained after applying different methods. Two best values are highlighted in the tables. The values of entropy acquired by the proposed method for different images is highest as compared to other methods and also the closest values to input image. The average value 6.37 is also just next to 6.38. Hence, it can be concluded that the proposed method provides maximum details in the image. So, the proposed method can be very well utilized where the details of the image are important such as medical images.

Next, if we analyze the PSNR value in Table 1, only plane and girl images give the highest value for the proposed method and the rest of images have less values as compared to the other methods. However, the average value is highest for our technique. This shows that the proposed method is robust and maintains the quality of different types of images. Hence, this method can be used for variety of images from different fields.

As discussed in Sect. 4, GMSD value for an image must be as low as possible, and from Table 2, it can be observed that GMSD values for F16, fish, plane, couple, tank, girl and U2 are minimum for the proposed method. Also the average value is minimum as compared to all the other techniques. This shows that proposed method can enhance the image with very less distortion.

From Table 2, it can be noticed that $\mathrm{UE}_\mathrm{hist}$ values for all the test images are highest for our technique and also the average value is highest. From this, it can be concluded that the proposed method can enhance the contrast very well together with preserving the basic features and characteristics of the image.

5.2 Visual assessment

Visual assessment is also called as qualitative assessment of images. It is very essential to study the visual results for an image, and as until and unless the quality of an image does not satisfy the human eye, we cannot say that the image is perfect.

Fig. 4

Enhancement results of Einstein image for different techniques: a original image, b HE, c BBHE, d AHE, e RSIHE, f RSWHE-D, g MMBEBHE, h ESIHE, i IEBHE, j LLVORM, k BHEMHB, l proposed

Fig. 5

Enhancement results of lady image for different techniques: a original image, b HE, c BBHE, d AHE, e RSIHE, f RSWHE-D, g MMBEBHE, h ESIHE, i IEBHE, j LLVORM, k BHEMHB, l proposed

Fig. 6

Enhancement results of U2 image for different techniques: a original image, b HE, c BBHE, d AHE, e RSIHE, f RSWHE-D, g MMBEBHE, h ESIHE, i IEBHE, j LLVORM, k BHEMHB, l proposed

Fig. 7

Enhancement results of girl image for different techniques: a original image, b HE, c BBHE, d AHE, e RSIHE, f RSWHE-D, g MMBEBHE, h ESIHE, i IEBHE, j LLVORM, k BHEMHB, l proposed

Fig. 8

Enhancement results of window image for different techniques: a original image, b HE, c BBHE, d AHE, e RSIHE, f RSWHE-D, g MMBEBHE, h ESIHE, i IEBHE, j LLVORM, k BHEMHB, l proposed

Number of images were tested, but here the visual results are shown only for five images due to space limitation.

Figure 4 shows the result for Einstein image. HE, BBHE and AHE methods have over-enhanced the image making the image too intense. Also it can be seen that the histograms of these method have lost their original characteristics. RSIHE method has made the Einstein face unnaturally bright creating artefacts in the image. RSWHE-D has blurred the image. MMBEBHE has introduced distortions in the enhanced image. ESIHE, IEBHE and LLVORM have enhanced the image up to good extent. The image obtained by BHEMHB is enhanced, but it is not very smooth. Finally, the proposed method result for Einstein image is best among all. Image is very well enhanced with clear view.

Figure 5 shows the result for lady image. HE, BBHE and AHE provided nearly same images after enhancement resulting in over-enhancement. RSIHE and RSWHE-D introduced artefacts. MMBEBHE provided good result, but it also does not preserve the natural features of the input image as depicted from the histogram. IEBHE resulted in dull image. ESIHE, LLVORM and BHEMHB well enhanced the image with appropriate features. Now, if we look at the proposed method image, it is enhanced very well giving a very pleasant appearance. Similarly, Fig. 6 can also be analyzed.

Figures 7 and 8 show the results for color images. Figure 7 shows the result for girl image. As it is clear from the figure, HE, BBHE, AHE and MMBEBHE produced distortions and noise in the image. RSIHE enhanced the image, but again there is some distortion introduced in the front portion of the image. RSWHE-D blurred the image. ESIHE provided good enhancement result. Image generated by IEBHE is too much bright. LLVORM and BHEMHB enhanced the image, but the color is distorted. The proposed method enhanced the image very well with fine details.

Figure 8 shows the results for window image. Original image is a dull image. Image generated by HE, BBHE and AHE appears to very bright and clear, but if it is clearly seen, all the three images have lost their natural features making the image unrealistic. RSWHE-D and MMBEBHE introduced distortions in the image. IEBHE, LLVORM and BHEMHB resulted in very bright images. RSIHE, ESIHE and proposed methods provided well-enhanced images while preserving the natural look.

6 Conclusion

In this paper, a new technique of image contrast enhancement is proposed which modifies PDF of the image adaptively based on the expected value of image intensity. This method can efficiently resolve the artefacts present in the other HE-based techniques. Since PDF of the image is changed for complete image, it is global adaptive method. The proposed method outperformed the state-of-art existing methods in terms of entropy, PSNR, GMSD and histogram utilization efficiency. The visual analysis also showed the robustness of the method and superiority on the existing methods for different types of general-purpose images.