1 Introduction

The image enhancement process changes the pixel’s intensity in order to produce esthetically pleasing image to human eye. Histogram equalization (HE) [2] is generally used due to its simple and efficient approach for image enhancement. It basically redistributes the image intensity values uniformly which gives a higher contrast. In HE, the contrast of the image is increased by taking transformation function as cumulative density function (CDF) of the image. However, there are some drawbacks when HE is applied. The image mean brightness is disturbed and some artifacts may be introduced in the enhanced image.

Bi-histogram equalization (BBHE) technique [6] is introduced to conserve mean brightness of the input image. In BBHE image histogram is sub divided based on the input image’s intensity mean value. After that these sub-images are equalized independently. In dualistic sub-image histogram equalization (DSIHE) [20], the input image is divided into two histograms based on its median gray level, by keeping CDF = 0.5 or with the help of equal area property. Alex Stark proposed Adaptive image contrast enhancement using generalizations of histogram equalization (AHE) in 2000 [19] which had two variable parameters. Histogram equalization results were modulated by one of the parameter and other parameter regulated the image contrast.

In recursive mean sub-image histogram equalization (RMSHE) [14], the mean based histogram sub-division is done more than once. Mean brightness is preserved as we increase the number of partitions. The RMSHE preserved the image brightness as well as enhanced the contrast better than the above three methods. In recursive sub-image histogram equalization (RSIHE) [14] histogram separation is done based on the median more than once. RSIHE preserves mean brightness similar to RMSHE. This method produces better image quality as compared to previously discussed methods.

RSWHE-M (Recursive Separated and Weighted Histogram Equalization Mean) method was given in 2008 after further extending RMSHE and RSIHE methods. It maintains the image brightness as well as enhances the contrast of image. In RSWHE-M [7], histogram segmentation is done in the same way as in RMSHE, and then an accumulative probability weighted factor αx is calculated for each sub-histogram. RSWHE-D (Recursive Separated and Weighted Histogram Equalization Median) is similar to RSWHE-M except that histogram segmentation is done with the help of median. The major drawback of the recursive methods is that they are very time consuming as the division process is done more than once.

In minimum mean brightness error bi-histogram equalization (MMBEBHE) [1] the image histogram is divided into sub histograms on the basis of minimum mean brightness error and thereafter each sub-divided histogram are equalized independently with HE. But image get over enhanced or under enhanced if PDF of image is very narrow in shape.

Survey of contrast enhancement techniques based on histogram equalization was presented in 2011 [5] which discussed the comparative study of different HE techniques. Similarly Survey on histogram equalization method based image enhancement techniques was presented that also compared the various HE based methods [10]. Contrast enhancement techniques for images also showed the different methods of enhancement [13].

Exposure based sub Image Histogram Equalization(ESIHE) [15] was proposed in 2014. This is very effective for over enhanced images and provides maximum entropy along with enhancement. In this technique, one clipping threshold is calculated and the histogram is clipped according to this clipping threshold. This clipping process prevents the over enhancement problem. ESIHE control only over enhancement [4, 16], it does not preserve mean brightness.

Contrast enhancement using entropy-based dynamic sub-histogram equalization (CEDHE) [11] proposed in 2016 used a iterative division of the histogram depending on the entropy of the sub histograms. Finally the sub-histograms were equalized to obtain the enhanced image. This technique enhanced the contrast of the image but the resultant image is not smooth as well as the image is over enhanced.

In the proposed method unsharp masking technique is first used to sharpen the image, then the image is clipped and equalized based on the histogram equalization. The over and under enhancement problem is avoided by clipping. Finally the resultant image is again smoothed and sharpened by the filter employed in unsharp masking. This technique is very effective for general-purpose images. The objective results of this method are much better as compared to all the previous methods as well as the subjective quality is also highest among all.

Rest of the paper is organized as follows. Section 2 discuss about motivation for this work. Proposed work is presented in the Section 3. In Section 4 evaluation parameters are discussed which are used in results to verify the proposed method justification. Results are discussed in Section 5, which consist of three subsections to verify the proposed method differently. Finally conclusions held in Section 6.

2 Motivation

The main purpose behind introducing this method is to basically avoid the over and under enhancement problem which is observed in all the histogram equalization techniques [3] discussed above. This method can well preserve the brightness of the image as well as prevents artifacts in the enhanced image unlike the other HE methods. The other reason to put forward this method is that, it can very well bring out the details of the image while maintaining the natural appearance of the image. Since, the filter is being employed in this method, the variable parameter of the filter makes it suitable to be used for different types of images. If compared to Recursive methods and MMBEBHE method as discussed in Section 1 the proposed method is also very fast.

In HE techniques, gray values are transformed to a higher or lower gray levels depending on probability density function (PDF) of image. Figure 1a shows the original Einstein image and histogram equalized Einstein image is shown in Fig. 1b. Figure 1c shows change in the intensity level after HE technique. The x-coordinate is showing the input intensity and the y-coordinate is the increment or decrement of each intensity level. Figure 1d shows the modified values of intensity by using the curve illustrated in Fig. 1c. Figure 1b shows that the face portion of Einstein has became extra white, this shows high change in intensity and that is also clear from Fig. 1d. Figure 2a and b shows the F16 image and histogram equalization of F16 image respectively. Figure 2c again shows change in the intensity level after HE technique. Figure 2d shows the modified values of intensity by using the curve illustrated in Fig. 2c. Figure 2b shows that HE makes more dark then original image, this shows high change in intensity and that is also clear from Fig. 2d. A higher change in gray levels leads to over enhancement or under enhancement of the enhanced image. If graph of HE technique is much above the reference line then output image gets over enhanced as in Fig. 1b and if the graph of HE technique is much below reference line, output image is under enhanced as in Fig. 2b. In the proposed technique the over and under enhancement of images is avoided along with the sharpening. of images.

Fig. 1
figure 1

a Einstein image b Histogram equalized Einstein image c Intensity levels after HE d High variation of gray levels in positive direction

Full size image
Fig. 2
figure 2

a F16 image b Histogram equalized F16 image c Intensity levels after HE d High variation of gray levels in positive direction

Full size image

Figure 3 shows the histogram results of Girl image at different stages. The idea behind combining Sharpening process with HE is explained from this figure. Figure 3a is the input Girl image. Figure 3b is the histogram of input image. Figure 3c is obtained after applying HE to input image. Figure 3d histogram is the difference between Fig. 3c and b. Figure 3e is the FFT generated for the Fig. 3d.

Fig. 3
figure 3

Girl image: a Input image b Histogram of input image c Histogram of enhanced image with HE d Difference between (c) and (b) (e) FFT of (d) f Histogram of sharpened image g Histogram of enhanced sharped image combined with HE h Histogram of final enhanced image after applying filter i Difference between (h) and (b) (j) FFT of (i)

Full size image

Figure 3f is the histogram of sharpened image. Figure 3g is obtained after applying HE to the sharpened image. Figure 3h histogram is final enhanced image histogram obtained after applying filter again. It can be clearly seen that sharpening process only sharps the image but, if we combine it with HE than it can also increase the dynamic range of the image providing over all enhancement as can be seen in Fig. 3h. Figure 3i histogram is the difference between Fig. 3h and b. Now, if we compare Fig. 3i and d the change in number of pixels in y-direction is smooth in Fig. 3i rather than Fig. 3d. In Fig. 3d there are more oscillations produced in the number of pixels. This can also be observed from Fig. 3e and j which are the FFT of Fig. 3d and i respectively. In Fig. 3e the oscillation is too high as compared to Fig. 3j. Due to this reason the enhanced image obtained through HE method has arte-facts. Hence, this was the motivation for proposing this particular technique.

3 Proposed method

The steps of the proposed algorithm is explained by the flowchart below (Fig. 4).

Fig. 4
figure 4

Flowchart of the proposed method

Full size image

3.1 Image sharpening

Basically sharpening is a method to improve the apparent sharpness of an image. It describes the information present in the image, especially those which are missed out by the viewer [17]. Here, Unsharp masking technique is used for sharpening. Basically a blurred (unsharp) copy of the image is subtracted from the original image to find out all the edges. Using this edge details a mask is created. The edges contrast is then improved and the overall effect is made applicable to the original image.

The specimen image is spatially filtered with a Gaussian filter to produce a unsharp mask [8, 12]. This filter can be treated as a convolution operation of an image with a kernel mask giving a two-dimensional Gaussian function hg(x,y) and is given as:

$$ h_{g}(x,y)=\frac{1}{2\pi\sigma^{2}}.e^{\frac{-(x^{2}+y^{2})}{2\sigma^{2}}} $$
(1)
$$ U(x,y)= \frac{h_{g}(x,y)}{{\sum}_{x}{\sum}_{y}h_{g}} $$
(2)

The parameterσ defines the size of the Gaussian kernel mask, and the dimension of the kernel mask decides the domain of frequencies that are rejected by the Gaussian filter. The original image subtracts the unsharp mask according to the equation:

$$ F(x,y)=\frac{c}{2c-1}.I(x,y)-\frac{1-c}{2c-1}.U(x,y) $$
(3)

In (3), the brightness value of a pixel at the coordinate (x,y) in the filtered image is given by the function F(x,y), and I(x,y) and U(x,y) represent the brightness values of the corresponding pixels in the original and unsharp mask (blurred) images, respectively. The constant c controls the relative weightings of the original and blurred images in the difference equation. The benefit of using this method over other sharpening filters is the flexibility of control due to adjustable parameter. In this filter, there are two variable parameters i.e. one is standard deviation σ as mentioned in (1) and the other is c mentioned in (3). Generally the range of c lies between 0.5 to 1 and the value of σ is calculated in pixels.

3.2 Clipping threshold

After obtaining the Sharpened image by above procedure the histogram of the image is now generated and clipped to avoid the over and under enhancement caused by the regular HE techniques. Here, the clipping threshold is computed as the mean of gray level occurrences. The reason for selecting this particular method of histogram clipping is due to its efficiency and fast speed. Let F is the filtered image, range of the gray values lies between 0 to L-1(F0,F1,...,Fi,...,FL− 1). H(F) = (n0,n1,...,ni,...,nL− 1) is the histogram of the filtered image, where ni is the number of pixels with i gray level. Let N is total number of pixels in image. Clipping threshold (CT) is calculated as:

$$ C_{T}=\frac{{\sum}^{L-1}_{i = 0}.H(i)}{L} $$
(4)

Clipped Histogram is calculated as:

$$H_{C}(F)= \left\{\begin{array}{ll} C_{T},& \text{if } (H(F)\geq C_{T})\\ H(F), & \text{otherwise} \end{array}\right. $$

Figure 5 shows the clipped and un-clipped histogram for the F16 image. It can be observed from the figure that the part of the histogram which has peaks is smoothed by the clipping process and the rest part is overlapped.

Fig. 5
figure 5

Histogram of the F16 image before and after clipping

Full size image

3.3 Segmentation threshold

Clipped histogram (HC(F)) is now bisected on the basis of the mean intensity value like in BBHE technique. Let Fm denotes the mean intensity value of the clipped histogram. Hence, histogram subdivision is done with Fm and this process generates two sub images Fl and Fu as

$$ H_{C}(F)= F_{l}\cup F_{u} $$
(5)

where

$$ F_{l}=(F(x,y)\leq F_{m}) $$
(6)
$$ F_{u}=(F(x,y)> F_{m}) $$
(7)

The sub image Fl composed of (F0,F1,...,Fm) and the other sub image Fu composed of (Fm+ 1,Fm+ 2,...FL− 1).

3.4 Equalization process

After the segmentation of the clipped histogram the histogram equalization process is applied to the two sub images. Probability density function(PDF) of the sub images is calculated as:

$$ p_{L}(F_{i})=\frac{{n^{i}_{L}}}{n_{L}} $$
(8)

where i = 0,1,...,m and nL is the total number of pixels from F0 to Fm intensity levels.

$$ p_{U}(F_{i})=\frac{{n^{i}_{U}}}{n_{U}} $$
(9)

where i = (m + 1),(m + 2),...,(L − 1) and nU is the total number of pixels from Fm+ 1 to FL− 1 intensity levels. Cumulative density functions (CDF) are then defined as

$$ C_{L}(F_{m})=\sum\limits^{m}_{i = 0}p_{L}(F_{i}) $$
(10)
$$ C_{U}(F_{L-1})=\sum\limits^{L-1}_{i=m + 1}p_{U}(F_{i}) $$
(11)

Transform functions in terms of cumulative density functions:

$$ T_{L}(F_{i})=F_{0} +(F_{m}-F_{0})*C_{L}(F_{i}) $$
(12)
$$ T_{U}(F_{i})=F_{m + 1} +(F_{L-1}-F_{m + 1})*C_{U}(F_{i}) $$
(13)

Transform function of the image is given by TF

$$ TF=T_{L}(F_{i})\cup T_{U}(F_{i}) $$
(14)

The TF image obtained above is again passed through a gaussian filter for sharpening purpose as in Section 3.1 and a final enhanced image is produced.

4 Evaluating parameters

In literature two parameters Absolute Mean Brightness Error (AMBE), [18, 21] and Entropy are widely-used as a quantative assessment parameters for image enhancement.

ABME is used to measure the brightness difference between the original and the enhanced image and is defined as

$$ AMBE(I,Ie)=|I_{m} - Ie_{m}|, $$
(15)

where Im is mean intensity of the input image and Iem is mean intensity of enhanced image. For better preservation of mean brightness, the value of AMBE is much less, means the value of Im and Iem are close to each other. Im and Iem can be calculated as:

$$ I_{m}= \frac{{\sum}^{M}_{i = 1}{\sum}^{N}_{j = 1} I(i,j)}{M \times N} $$
(16)
$$ Ie_{m}= \frac{{\sum}^{M}_{i = 1}{\sum}^{N}_{j = 1} Ie(i,j)}{M \times N} $$
(17)

where, I(i,j) and Ie(i,j) are the input and enhanced image respectively which contains M * N pixels.

Entropy is simply the amount of information (in the Shannon sense) that would be needed to specify the details of the image [9].

$$ E= -\sum\limits^{L-1}_{i = 0}p(I_{i})log_{2}p(I_{i}) $$
(18)

where p(Ii) is the probability value of the ith intensity level. The higher value of entropy indicates a image with high contrast.

Histogram utilization efficiency (Effhist) [11] is one more important parameter which can be tested to analyze the image histograms property. It is given as:

$$ Eff_{hist} = \frac{NZ_{be}}{NZ_{bo}} $$
(19)

where NZbe and NZbo represents the number of non-zero bins(utilized gray levels) of enhanced and original image correspondingly. (Effhist) value should be approaching to 1.

5 Results and discussion

In this section, first of all the results of proposed method is analyzed at each step in Figs. 6, 7, 8, 9 and 10. Next, proposed technique is applied over some standard images like F16, couple, lena, einstein, lady, U2, girl, Sphinx and Cactus and its performance is compared with some HE based methods including BBHE, DSIHE, RSIHE,RSWHE-D, MMBEBHE, ESIHE, CEDHE. Quantitative evaluation is done with the help of three parameters i.e. Entropy, AMBE and Histogram Utilization Efficiency.

Fig. 6
figure 6

Enhancement result of Girl image: a Original, b Sharped, c After applying HE to (b), d After applying filter to (c)

Full size image
Fig. 7
figure 7

Enhancement result of F16 image: a Original, b Sharped, c After applying HE to (b), d After applying filter to (c)

Full size image
Fig. 8
figure 8

Enhancement result of Einstein image: a Original, b Sharped, c After applying HE to (b), d After applying filter to (c)

Full size image
Fig. 9
figure 9

Enhancement result of Couple image: a Original, b Sharped, c After applying HE to (b), d After applying filter to (c)

Full size image
Fig. 10
figure 10

Enhancement result of Lady image: a Original, b Sharped, c After applying HE to (b), d After applying filter to (c)

Full size image

Figure 6 shows the enhancement result for Girl image at different stages of the proposed method. Figure 6a is the original image. Figure 6b is obtained after applying initial unsharp mask filter to the input image. As we can see that the girl features are sharped at this step. Figure 6c is obtained after applying HE to the sharped image. We can observe that image is overall enhanced together with sharpened features. Figure 6d is final enhanced image obtained after applying filter again to Fig. 6c. If carefully observed Fig. 6d is more pleasing than Fig. 6c. Every details of the image is very sharp and clear. Specifically, if we see the girl’s hair, every curls of hair is clearly visible.

Similarly we can analyze Fig. 7. Figure 7a is the original F16 image. In Fig. 7b it can be seen that plane and the surrounding mountain linings are very sharped. In Fig. 7c the overall image is enhanced with very good contrast. Finally in Fig. 7d the image looks very clean and sharp with enhanced contrast, brightness as well as detail richness. Similar analysis can be done for other images as well (Figs. 8, 9 and 10).

5.1 Quantitative assessment

Table 1 shows the results for different images and different techniques. The best and second best values are marked in bold. The higher value of entropy indicates more detail in an image. It can be observed that the proposed method achieves higher entropy values for all images as compared to other techniques. Hence, the details richness is best in proposed method. However for other HE techniques the entropy values are very less than the original image entropy values.

Table 1 Various parameters measures
Full size table

The value of AMBE must be small for better brightness preservation. From table, it is clear that RSWHE-D and MMBEBHE are preserving maximum brightness. Proposed method also have lower AMBE compared to HE, BBHE, DSIHE, ESIHE and CEDHE. Hence it can be concluded that proposed method preserve the brightness up to a certain level that helps to avoid the artifacts present in the HE method.

The Histogram utilization efficiency (Effhist) values obtained by proposed method for F16, Couple, Lena, Einstein, Lady, U2 and Girl images are 0.97, 0.99, 0.98, 0.96, 0.97, 0.95 and 0.98 respectively, which are the highest values among all. CEDHE method achieves second highest values for Effhist.

5.2 Performance analysis based on visual quality

Proposed method provide control on under and over enhancement and maintains the brightness of image and give natural appearance compared to image generated using HE, BBHE, DSIHE, RMSHE, RSIHE, RSWHED, MMBEBHE, ESIHE, CEDHE techniques.

In proposed technique, sharpening process is used to sharpen the edges, as it can be seen that all the edges around the plane in Fig. 11j are highlighted. Due to under enhancement problem in most of the HE techniques natural clouds become more darker so the high gray level are transformed in low gray level. Proposed method provide proper enhancement without under enhancement.

Fig. 11
figure 11

Enhancement result of F16 image: a Original b HE c BBHE d DSIHE e RSIHE f RSWHE-D g MMBEBHE h ESIHE i CEDHE j Proposed

Full size image

For Einstein image in Fig. 12, the face portion is over enhanced in HE, BBHE, DSIHE and RSIHE methods while in weighted technique the image looks like blur image. Although the AMBE values is less in RSWHE-D but the visual appearance of the image is not good. MMBEBHE ESIHE and CEDHE images are also not very much appealing as compared to proposed technique.

Fig. 12
figure 12

Enhancement result of Einstein image: a Original b HE c BBHE d DSIHE e RSIHE f RSWHE-D g MMBEBHE h ESIHE i CEDHE j Proposed

Full size image

In Fig. 13 the background of the girl image has artifacts in all the HE techniques except ESIHE and CEDHE. Also the face of the girl is distorted in all the techniques except ESIHE technique. The image enhanced by CEDHE method is not smooth and clear. The proposed technique has control on enhancement and gives the proper result compared to all the methods.

Fig. 13
figure 13

Enhancement result of Girl image: a Original b HE c BBHE d DSIHE e RSIHE f RSWHE-D g MMBEBHE h ESIHE i CEDHE j Proposed

Full size image

Figure 14 shows the result for couple image. As we can see that original image is dark image, HE method enhanced the image but it has over enhanced it making the image too bright. Result for BBHE, DSIHE and RSIHE for couple image looks nearly same but these methods introduced arte-facts in the image. Some parts of the image are blurred in the RSWHE-D and MMBEBHE enhanced image. Image obtained by CEDHE is not very smooth. ESIHE produced good contrast enhancement result. Proposed method enhanced the image very well with good contrast and brightness. It gives much better visual quality than ESIHE as well.

Fig. 14
figure 14

Enhancement result of Couple image: a Original b HE c BBHE d DSIHE e RSIHE f RSWHE-D g MMBEBHE h ESIHE i CEDHE j Proposed

Full size image

Figure 15 shows the result for Lady image. HE, BBHE and DSIHE methods made the Lady face and background too much white. RSIHE and MMBEBHE techniques also produced arte-facts in the enhanced image. The image obtained by RSWHE-D is completely distorted as we can see from the figure. ESIHE enhanced the image upto some level but it also made face of the Lady bright in comparison to original image. CEDHE resulted in somewhat dull image. The image enhanced by proposed method looks visually very well with sharp front and background features.

Fig. 15
figure 15

Enhancement result of Lady image: a Original b HE c BBHE d DSIHE e RSIHE f RSWHE-D g MMBEBHE h ESIHE i CEDHE j Proposed

Full size image

Figures 16 and 17 shows the result for color images. It can be observed that the HE and BBHE enhanced images have distortion in the background area. DSIHE and RSIHE images are almost same as the original image. MMBEBHE and RSWHE-D again produces the artifacts. ESIHE and CEDHE has good contrast enhancement, but the proposed method gives the best image with the clear background.

Fig. 16
figure 16

Enhancement result of Cactus image: a Original, b HE, c BBHE, d DSIHE, e RSIHE, f RSWHE-D, g MMBEBHE, h ESIHE, i CEDHE j Proposed

Full size image

In Fig. 17 the HE, BBHE and MMBEBHE makes the image too much bright. DSIHE, RSIHE and ESIHE enhances the contrast upto some level. RSWHE-D has artifacts in the background portion. CEDHE and proposed technique enhances the image very well.

Fig. 17
figure 17

Enhancement result of Sphinx image: a Original, b HE, c BBHE, d DSIHE, e RSIHE, f RSWHE-D, g MMBEBHE, h ESIHE, i CEDHE j Proposed

Full size image

A mean opinion score (MOS) test was conducted on test images shown above to support the visual assessment results. The opinion was taken from 20 people and the image ratings were given from 1–5, where 1 means worst and 5 corresponds to best. Table 2 shows the MOS results for HE, BBHE, DSIHE, RSIHE, RSWHE-D, MMBEBHE, ESIHE, CEDHE and proposed method. In the table it can be very well observed that the MOS value for the proposed technique is highest for all the images except the sphinx image.

Table 2 MOS results
Full size table

5.3 Discussion on over enhancement and under enhancement

HE techniques are used to enhance the image but some time image gets over enhanced or under enhanced by some of HE technique and creates artifacts because of over enhancement and objects are not clearly distinguished with background because of under enhancement. Figures 18 and 19 shows the transformed intensity comparison between the proposed method and the other techniques. Figure 18 shows the over enhancement of the different techniques for the couple image. As it can be seen from Fig. 18a to h that the intensity curve of enhanced image is far away from the reference line in all the methods except the proposed one. Figure 19 shows both over and under enhancement for the girl image. Again it can be seen that the intensity curve of enhanced image both in upper part and in lower part is far away from the reference line in HE, BBHE, DSIHE, RSWHE-D, MMBEBHE and CEDHE. Here RSIHE and ESIHE are close to proposed technique.

Fig. 18
figure 18

Transformed intensity comparison in proposed and others techniques for couple image

Full size image
Fig. 19
figure 19

Transformed intensity comparison in proposed and others techniques for girl image

Full size image

6 Conclusion

In this paper a new method of histogram equalization with the unsharp masking technique and controlled enhancement is proposed that can effectively solve the problem of artifacts and visual effect. It prevents the image to be over enhanced and under enhanced and also gives the highest information present in it. Proposed method outperformed state of art existing methods in terms of entropy, Histogram utilization efficiency and mean opinion score. The visual assessment results also shows the robustness of the method and superiority on existing methods for different variety of general-purpose images. In future this work can be extended for enhancement of satellite and agriculture based images by utilizing and analyzing the variable parameters of the filter.