Recursive Median and Mean Partitioned One-to-One Gray Level Mapping Transformations for Image Enhancement

Abstract

This paper presents two novel recursive partitioned one-to-one gray level mapping (RPOGM) algorithms, viz., recursive median partitioned one-to-one gray level mapping (RMDPOGM) and recursive mean partitioned one-to-one gray level mapping (RMPOGM). The proposed RPOGM methods serve multiple objectives and address the issues such as (i) intensity saturation, (ii) intensity compression and (iii) ensure uniform degree of enhancement of all gray levels and thus result in overall enhancement of the processed image. In RMPOGM, image/histogram is partitioned recursively, (recursion level restricted to two, resulting in four sub-histograms) based on mean. RMDPOGM is similar to RMPOGM except histogram partitioning is done based on median. In RPOGM methods, image-dependent weights for each sub-histogram are calculated separately. Later, these weights are used for transformation. Finally, all the transformed sub-images are combined to get the processed image. As the images processed by these methods are not having any loss of details, it results in retaining the structural details of the objects and hence preserves fine contours even after enhancement. This results in low gradient magnitude similarity deviation (GMSD) between the processed image and input image. Experimental results show the superiority of the proposed methods over the state-of-the-art histogram equalization methods in terms of preserving entropy, preserving mean brightness and having low GMSD.

1 Introduction

Image enhancement is widely used to process surveillance, military, satellite, medical and geophysical images to get the visually important details. Due to simplicity and less computational cost, conventional histogram equalization (HE) [5] is initially preferred. HE produces better image if all the gray levels occupy frequencies above some threshold. But in real scenario, images may not satisfy this condition. In such cases, HE is having the problem of producing artifacts such as noise amplification and patches in the processed image. This is due to saturation of intensities (narrow range of peaks or high frequency gray levels in the input histogram being spread to wide range in the output histogram) in histogram of processed image. This causes large brightness deviation in the output image and produces large absolute mean brightness error (AMBE). Preserving brightness is most important in consumer electronic products. To cater to the needs of consumer electronic products, new techniques are derived. These are, modifications applied to input histogram, before conventional HE is applied on them. Conventional HE operates on whole image, whereas the later techniques operate on sub-images. First such modification proposed to preserve the brightness is, brightness preserving bi-histogram equalization (BBHE) [9]. BBHE bisects the image based on mean intensity. Other modifications to partition the histogram of image are minimum mean brightness error bi-histogram equalization (MMBEBHE) [3], dualistic sub-image histogram equalization (DSIHE) [21], recursive mean separate histogram equalization (RMSHE) [4] and recursive sub-image histogram equalization (RSIHE) [15]. DSIHE separates the histogram based on median intensity, to ensure that there is equal number of pixels in each sub-histogram. RMSHE adopts the features of BBHE to separate each sub-histogram recursively. DSIHE features are recursively applied in RSIHE for histogram separation. But in both RSIHE and RMSHE, the partitions are restricted to power of two. The problem of these methods is that too many recursive steps lead to no enhancement of the image [8], and rather leads to original image and also optimum recursion point to stop the histogram partition is not mentioned in these methods. MMBEBHE carries recursive operation of BBHE for finding separation point, which gives processed image’s mean brightness closer to input image mean brightness. It is having high computational cost to find separation intensity, especially for images having more gray levels [2, 13]. Although these methods preserve the brightness to considerable amount, they suffer from intensity saturation and intensity compression in some sub-histogram regions, and fail to give noise-free and naturally enhanced image.

Unlike previous recursive partitioned histogram equalization (RPHE) methods: HE, BBHE, MMBEBHE, DSIHE, RMSHE and RSIHE which use statistical parameters in splitting histogram, dynamic histogram equalization (DHE) [1] and brightness preserving dynamic histogram equalization (BPDHE) [8] methods use local minima and local maxima in separating the histogram, respectively. In DHE, 3 × 3 averaging filter is applied on histogram to avoid insignificant minima. Later, local minima are found out to carry out histogram separation. If there is any dominant portion in the sub-histogram, recursive operation is carried out till dominant-free sub-histograms are achieved. These separated sub-histograms are assigned to new dynamic range according to their occupied range before they are equalized by conventional HE. One problem associated with DHE is that it produces large AMBE as sub-histograms are assigned to new ranges. To address this problem, BPDHE was developed. BPDHE uses Gaussian filter to smooth the histogram, and later local maxima are used for separating the sub-histograms. These separated histograms are assigned to new dynamic range, and conventional HE is applied to equalize. Finally, brightness normalization is carried out to preserve the mean brightness. One problem associated with this is selection of normalization factor. If normalization ratio is less than 1, it gives poor contrasted image, and if greater than 1, it results in some of the intensities rounded to the maximum gray level [12]. Both the methods produce the images with some aspects of HE, especially if the separating point between two minima and two maxima is large. In this scenario, the sub-histogram falling between these minima or maxima is assigned to very large new dynamic range according to their proportion, which results in intensity saturation.

Kim et al. [10] proposed two approaches, viz., recursively separated weighted histogram equalization based on mean (RSWHE-M) and recursively separated weighted histogram equalization based on median (RSWHE-D) to preserve mean brightness well and to improve the image quality. RSWHE-M is similar to RMSHE in partitioning the histogram into sub-histograms, and these are restricted to four. These sub-histogram probability density functions (PDF) are modified through weights, based on normalized power law function. Modified sub-histograms are processed by the conventional HE for enhancement. RSWHE-D is similar to RSWHE-M, but histogram separation is based on median. Author claims that RSWHE-M gives better results than the others. But, RSWHE-M suffers from intensity saturation and intensity compression in some of the sub-histograms, giving patchy look in some portions of the image [16].

To control the over enhancement, hence intensity saturation, clipped histogram equalization (CHE) methods, viz., median-mean based sub-image-clipped histogram equalization (MMSICHE) [16], bi-histogram equalization with plateau limit(BHEPL) [13], weighted threshold histogram equalization (WTHE) [20], quadrant dynamic histogram equalization(QDHE) [12] and exposure-based sub-image histogram equalization(ESIHE)[17], adaptive image enhancement based on bi-histogram equalization (AIEBHE) [19] with plateau limit, adaptive contrast modified histogram equalization (ACMHE) [14] and recursively separated exposure-based sub-image histogram equalization (RS-ESIHE) [18] were proposed to control the degree of enhancement. BHEPL uses mean to separate histogram into two. Plateau level is set to clip the histograms, and these clipped histograms are equalized by HE to get final image. In WTHE, weighted and threshold PDF is used to alter the histogram. It restricts the modified probability (frequencies) to be within the upper and lower thresholds. Finally, HE is performed on the modified histogram. QDHE adopts the features of both RSIHE and DHE. It uses feature of RSIHE in separating histograms, and these four sub-histograms are restricted to clipped threshold, calculated by averaging the intensity frequencies. It uses feature of DHE in allocating the clipped sub-histograms to new dynamic range. Finally, conventional HE is applied on them to get processed image.

ESIHE uses exposure threshold [7] gray level to split the histogram into two. These sub-histograms are limited to clipped threshold, calculated by averaging histogram frequencies. Then, conventional HE is applied on clipped histograms to get over all equalized images. MMSICHE uses median intensity to divide the histogram. Then, each of the sub-histogram is divided into two based on mean intensities. Median of occupied gray level frequencies is used as a clipped threshold to clip all sub-histograms. Conventional HE is applied on these clipped histograms to get enhanced image.

AIEBHE adopts the feature of DSIHE in separating sub-histograms. Later, unlike all CHE methods, AIEBHE uses the minimum value among: gray level frequencies, mean and median of image as clipping threshold. After clipping, sub-histograms are processed by conventional HE to get output image.

Adaptive contrast modified histogram equalization (ACMHE) mimics the features of RSIHE in partitioning histogram, feature of QDHE in setting plateau level, feature of QDHE with added enhancement rate parameter to adjust dynamic range allocation of sub-histograms. Later, HE is performed on these modified and re-allocated sub-histograms. Finally, enhancement level is adjusted to get better image.

RS-ESIHE uses the recursion version of ESIHE to enhance the low contrasted images. Here, ET gray is used to bisect the histogram. Later, the two sub-histograms are bisected to two more based on lower ET and upper ET. Later, plateau level is calculated by averaging gray level frequencies and is applied to clip all four sub-histograms. Finally, HE is applied to these sub-images and combined to get the processed image. Even though it results in better image than the ESIHE-processed image, it also results in gray level loss and improper utilization of gray band due to fall of exposure threshold in separating sub-histograms [6].

Advanced HE variants, named clipped histogram equalization with multi-plateau levels (CHEWMP), have been developed in recent years. In 2015, Lim et al. [11] proposed a new histogram equalization method for digital image enhancement and brightness preservation. This work adopts the feature of BBHE in separating sub-histograms and feature of QDHE in deciding clipping level. Later, these sub-histograms are altered through three clipping levels (these are calculated as mean of global and local sub-histograms), and HE is applied in the final stage to get processed image. In 2017, bi-histogram equalization using two plateau limits (BHE2PL) [2] is proposed. This uses mean as in BBHE to divide the histogram into two. Later, nonzero frequency gray levels of these sub-histograms are restricted to two clipping levels. Finally, HE is applied to the sub-histograms and combined to get final image.

Even though the above techniques have improved performance over conventional HE, these methods suffer from gray level loss. These lost gray levels get overlapped with existing gray levels affecting many gray levels. The objects corresponding to these affected gray levels loose much of the information and do not guarantee natural look and natural enhancement. The gray level loss causes a decrease in output image entropy. Also, these methods enhance some of the objects well at the cost of others, as they are having non-uniform degree of enhancement of gray levels.

To address the above issues, two new techniques are proposed, viz., RMDPOGM and RMPOGM, that use data-dependent weights for transformation. RMDPOGM and RMPOGM adopt the feature of RSIHE and RMSHE, respectively, in partitioning the histogram into sub-histograms. For the sub-histograms, the corresponding weights are calculated. Each weight is dependent on a number of nonzero frequency gray levels in that sub-histogram. Later, these weights are used to map the gray levels. The main idea behind this is to utilize the gray band (0–255) efficiently and uniformly. It ensures the gray levels being uniformly enhanced in each sub-histogram. This makes the corresponding objects of the gray levels being enhanced uniformly (no saturation of intensities), leading to naturally enhanced image. As the transformation is strictly monotonic, it guarantees preservation of gray levels and thus preserves entropy after enhancement. These methods not only address the saturation of intensities, but also avoid intensity compression. These attributes result in maintaining fine contours of the objects in the processed image. This results in naturally enhanced image with all structural details, leading to low gradient magnitude similarity deviation (GMSD) [22] between input and output images. They also preserve mean brightness well, and thus are best suited for consumer electronic products.

This paper is organized as follows: The “details and commonalities of different HE methods” that depend on cumulative distribution function (CDF)-based transformation is discussed in Sect. 2. Section 3 explains the proposed RPOGM transformations. Section 4 gives simulation results. Concluding remarks of the paper are given in Sect. 5.

2 Image Enhancement Using CDF-Based Transformation Function

Let us consider an image, F(x, y) captured under low contrast environment. The most commonly used method to improve the contrast is HE. The gray level of an image at pixel location (x, y), given by r, is denoted as

$$ F\left( {x, y} \right) = r;\quad r \in \left[ {0,L - 1} \right] $$

(1)

where L is total number of gray levels, equal to 256.

Collection of all these frequencies with respect to gray levels is the image histogram. A vector representation of this is

$$ \varvec{H} = \left[ {f_{0} ,f_{1} , \ldots f_{L - 1} } \right] $$

(2)

where f_r is the frequency of the rth gray level.

The HE maps input gray level r ∊ [0, L − 1] to the output gray level s ∊ [r_minimum, r_maximun] using CDF, C_r, as

$$ s = T\left( r \right) = r_{\text{minimum}} + \left( {r_{\text{maximum}} - r_{\text{minimum}} } \right)C_{r} . $$

(3)

Various modifications to this fundamental HE are proposed. However, all the CDF-based transformations share some or all commonalities as listed below.

• 1. Transformed functions, while mapping, exhibit steep rise in certain gray levels. Steep rise causes saturation of intensities, results in noise amplification. More the steep rise, more the intensity saturation.

• 2. Flat regions are due to many-to-one mapping, causing gray level loss.

• 3. More the range of flat region, more the compression of gray levels. This results in false contours of the objects that belong to the compressed gray levels.

• 4. Inverse mapping is not possible due to gray level loss and gray level overlap.

To overcome these issues, RPOGM techniques are proposed. The merits of these are:

• 1. No steep rise in the transformation function, balanced gray level distribution in output image’s histogram as it relies on data-dependent weights.

• 2. No flat regions in the transformation function as the mapping is one to one.

• 3. Mapping is reversible as there is no loss of gray levels.

Along with these features, the proposed RPOGM techniques give better image quality than the state-of-the-art HE variants, especially for the images that are having large uniform background and small non-uniform foreground (non-UFG), such as the objects that are taken in clear sky under day light conditions, the images of small objects on the sea or any image that is having large uniform background (UBG).

The images with large uniform background result in histogram having few peaks that are concentrated with a little span in the gray band [0–255]. Here, the span refers to the range of gray levels placed between first peak and last peak. As these peaks are related to uniform background, they should not be enhanced or should not be spread in the processed histogram so as to have a noise-free image.

This motivates us to choose recursive mean/median partitioning of the histogram, as starting, middle and ending sub-histograms cover these peaks if image has dark/black, gray and bright/white backgrounds, respectively.

This kind of statistical partitioning gives scope to enhance the non-peaks that correspond to small objects in the foreground (local features).

This has been verified for the girl and jet images. For the other images, the proposed methods are giving comparable/better results than the HE variants. But, the proposed algorithms have limitations in processing the images having large non-uniform foreground with small uniform background or those having widely occupied gray levels in gray band.

The formulation of the proposed RMPOGM and RMDPOGM methods is discussed in the following section.

3 Proposed RPOGM Transformations and their Computational Complexity

In RPOGM transformations, the number of input gray levels is preserved. Two transformation functions, viz., RMPOGM and RMDPOGM, are developed based on histogram vector H.

3.1 The RMPOGM Transformation

In this, transformation functions are developed based on mean gray level of F(x, y). From the statistics, mean gray level r_mean can be obtained as

$$ r_{\text{mean}} = \frac{{\mathop \sum \nolimits_{r = 0}^{L - 1} rf_{r} }}{{\mathop \sum \nolimits_{r = 0}^{L - 1} f_{r} }} $$

(4)

This mean can be used to divide the histogram of image into two parts H_L and H_U, given by

$$ \varvec{H} = [\varvec{H}_{\text{L}} ,\varvec{H}_{\text{U}} ] $$

(5)

where

$$ H_{\text{L}} \left( i \right) = f_{i} \quad {\text{where}} \quad {i} = 0, 1, 2, \ldots r_{\text{mean}} $$

(6)

$$ H_{\text{U}} \left( i \right) = f_{{i + r_{\text{mean}} + 1}} \quad {\text{where}} \quad {i} = 0, 1, 2, \ldots L - r_{\text{mean}} - 2 $$

(7)

The lower and upper sub-histograms, H_L and H_U, are further divided into two, viz., H_Ll, H_Lu and H_Ul, H_Uu based on the means r_meanl and r_meanu, respectively, as

$$ r_{\text{meanl}} = \frac{{\mathop \sum \nolimits_{r = 0}^{{r_{\text{mean}} }} rf_{r} }}{{\mathop \sum \nolimits_{r = 0}^{{r_{\text{mean}} }} f_{r} }} $$

(8)

$$ r_{\text{meanu}} = \frac{{\mathop \sum \nolimits_{{r = r_{\text{mean}} + 1}}^{L - 1} rf_{r} }}{{\mathop \sum \nolimits_{{{\text{r = r}}_{\text{mean}} { + 1}}}^{L - 1} f_{r} }} $$

(9)

Then, histogram H can be partitioned into four parts $\varvec{H}_{\text{Ll}} ,\varvec{H}_{\text{Lu}} ,\varvec{H}_{\text{Ul}}$ and $\varvec{H}_{\text{Uu}}$ given by

$$ \varvec{H} = [\varvec{H}_{\text{Ll}} , \varvec{H}_{\text{Lu}} ,\varvec{H}_{\text{Ul}} ,\varvec{H}_{\text{Uu}} ] $$

(10)

where

$$ H_{\text{Ll}} \left( i \right) = f_{i} \quad {\text{where}}\quad i = 0, 1, 2, \cdots r_{\text{meanl}} $$

(11)

$$ H_{\text{Lu}} \left( i \right) = f_{{i + r_{\text{meanl}} + 1}} \quad {\text{where}}\quad i = 0, 1, 2, \ldots r_{\text{mean}} - r_{\text{meanl}} - 1 $$

(12)

$$ H_{\text{Ul}} \left( i \right) = f_{{i + r_{\text{mean}} + 1}} \quad {\text{where}}\quad i = 0, 1, 2, \ldots r_{\text{meanu}} - r_{\text{mean}} - 1 $$

(13)

$$ H_{\text{Uu}} \left( i \right) = f_{{i + r_{\text{meanu}} + 1}} \quad {\text{where}}\quad i = 0, 1, 2, \ldots L - r_{\text{meanu}} - 2 $$

(14)

The occurrences of f_r = 0 for r ∊ [0, r_meanl], r ∊ [r_meanl + 1, r_mean], r ∊ [r_mean + 1, r_meanu], r ∊ [r_meanu + 1, L − 1] can be separated, and nonzero frequency gray levels are obtained in each partition.

Let C_Ll, C_Lu, C_Ul and C_Uu denote number of nonzero frequency gray levels in $\varvec{H}_{\text{Ll}} ,\varvec{ H}_{\text{Lu}} ,\varvec{H}_{\text{Ul}}$ and $\varvec{H}_{\text{Uu}}$, respectively. It is easy to note that the number of zero frequency gray levels in $\varvec{H}_{\text{Ll}} ,\varvec{ H}_{\text{Lu}} ,\varvec{H}_{\text{Ul}}$ and $\varvec{H}_{\text{Uu}}$ are Z_Ll, Z_Lu, Z_Ul and Z_Uu, respectively, as

$$ Z_{\text{Ll}} = r_{\text{meanl}} + 1 - C_{\text{Ll}} $$

(15)

$$ Z_{\text{Lu}} = r_{\text{mean}} - r_{\text{meanl}} - C_{\text{Lu}} $$

(16)

$$ Z_{\text{Ul}} = r_{\text{meanu}} - r_{\text{mean}} - C_{\text{Ul}} $$

(17)

$$ Z_{\text{Uu}} = L - 1 - r_{\text{meanu}} - C_{\text{Uu}} $$

(18)

New vectors $\varvec{N}_{\text{Ll}} ,\varvec{N}_{\text{Lu}} ,\varvec{N}_{\text{Ul}}$ and $\varvec{N}_{\text{Uu}}$ are developed by removing zero frequencies from $\varvec{H}_{\text{Ll}} ,\varvec{ H}_{\text{Lu}} ,\varvec{H}_{\text{Ul}}$ and $\varvec{H}_{\text{Uu}}$. If jth nonzero frequency gray level is present at ith position of vector $\varvec{H}_{\text{Ll}}$, we can relate $\varvec{N}_{\text{Ll}}$ and $\varvec{H}_{\text{Ll}}$ as

$$ N_{\text{Ll}} \left( j \right) = H_{\text{Ll}} \left( i \right)\quad {\text{where}}\;H_{\text{Ll}} \left( i \right) \ne 0\;{\text{and}}\;j = 0, 1, 2, \ldots C_{Ll} - 1 $$

(19)

Similarly, $\varvec{N}_{\text{Lu}}$ and $\varvec{H}_{\text{Lu}} ,\varvec{N}_{\text{Ul}}$ and $\varvec{H}_{\text{Ul}} ,\varvec{N}_{\text{Uu}}$ and $\varvec{H}_{\text{Uu}}$ are related as

$$ N_{\text{Lu}} \left( j \right) = H_{\text{Lu}} \left( i \right)\quad {\text{where}}\quad H_{\text{Lu}} \left( i \right) \ne 0\;{\text{and}}\;j = 0, 1, 2, \ldots C_{\text{Lu}} - 1 $$

(20)

$$ N_{\text{Ul}} \left( j \right) = H_{\text{Ul}} \left( i \right)\quad {\text{where}}\;H_{\text{Ul}} \left( i \right) \ne 0\;{\text{and}}\;j = 0, 1, 2, \ldots C_{\text{Ul}} - 1 $$

(21)

$$ N_{\text{Uu}} \left( j \right) = H_{\text{Uu}} \left( i \right)\quad {\text{where}}\;H_{\text{Uu}} \left( i \right) \ne 0\;{\text{and}}\;j = 0, 1, 2, \ldots C_{\text{Uu}} - 1 $$

(22)

N, new indexed histogram, is generated by cascading $\varvec{N}_{\text{Ll}} ,\varvec{ N}_{\text{Lu}} ,\varvec{N}_{\text{Ul}}$ and $\varvec{N}_{\text{Uu}}$ as

$$ \varvec{N} = \left[ {\varvec{N}_{\text{Ll}} ,\varvec{N}_{\text{Lu}} ,\varvec{N}_{\text{Ul}} ,\varvec{N}_{\text{Uu}} } \right] $$

(23)

The weights corresponding to sub-histograms are given by

$$ \alpha_{\text{Ll}} = \frac{{1 + r_{\text{meanl}} }}{{C_{\text{Ll}} }} $$

(24)

$$ \alpha_{\text{Lu}} = \frac{{r_{\text{mean}} - r_{\text{meanl}} }}{{C_{\text{Lu}} }} $$

(25)

$$ \alpha_{\text{Ul}} = \frac{{r_{\text{meanu}} - r_{\text{mean}} }}{{C_{\text{Ul}} }} $$

(26)

$$ \alpha_{\text{Uu}} = \frac{{L - 1 - r_{\text{meanu}} }}{{C_{\text{Uu}} }} $$

(27)

The gray levels in $\varvec{N}_{\text{Ll}}$ are transformed by the above data-dependent weight α_Ll as

$$ s = T_{\text{Ll}} \left( j \right) = \left( {j + 1} \right)\alpha_{\text{Ll}} \quad {\text{for}}\; j = 0,1, \ldots C_{\text{Ll}} - 1 $$

(28)

Similarly, gray levels in $\varvec{N}_{\text{Lu}} , \varvec{N}_{\text{Ul}}$ and $\varvec{N}_{\text{Uu}}$ are transformed as

$$ s = T_{\text{Lu}} \left( j \right) = r_{\text{meanl}} + \left( {j + 1} \right)\alpha_{\text{Lu}} \quad {\text{for}}\;j = 0, 1, \ldots C_{\text{Lu}} - 1 $$

(29)

$$ s = T_{\text{Ul}} \left( j \right) = r_{\text{mean}} + \left( {j + 1} \right)\alpha_{\text{Ul}} \quad {\text{for}}\;j = 0, 1, \ldots C_{\text{Ul}} - 1 $$

(30)

$$ s = T_{\text{Uu}} \left( j \right) = r_{\text{meanu}} + \left( {j + 1} \right)\alpha_{\text{Uu}} \quad {\text{for}}\;j = 0, 1, \ldots C_{\text{Uu}} - 1 $$

(31)

The above method is summarized in the following steps

• 1. Calculate $r_{\text{mean}} ,r_{\text{meanl}}$ and r_meanu using Eqs. (4), (8) and (9), respectively.

• 2. Partition H as $\varvec{H}_{\text{Ll}} , \varvec{H}_{\text{Lu}} ,\varvec{H}_{\text{Ul}}$ and $\varvec{H}_{\text{Uu}}$ using Eqs. (11), (12), (13) and (14) using r_meanl, r_mean and r_meanu

• 3. Generate $\varvec{N}_{\text{Ll}} ,\varvec{ N}_{\text{Lu}} ,\varvec{ N}_{\text{Ul}}$ and $\varvec{N}_{\text{Uu}}$ from $\varvec{H}_{\text{Ll}}$,$\varvec{ H}_{\text{Lu}} ,\varvec{H}_{\text{Ul}}$ and $\varvec{H}_{\text{Uu}}$ using Eqs. (19), (20), (21) and (22), respectively.

• 4. Derive weights $\alpha_{\text{Ll}} , \alpha_{\text{Lu}} , \alpha_{\text{Ul}}$ and α_Uu using Eqs. (24), (25), (26) and (27).

• 5. Apply weights on $\varvec{N}_{\text{Ll}} ,\varvec{N}_{\text{Lu}} ,\varvec{N}_{\text{Ul}}$ and $\varvec{N}_{\text{Uu}}$ to get output as in Eqs. (28), (29), (30) and (31).

3.2 The RMDPOGM Transformation

The RMDPOGM is similar to RMPOGM except finding the median gray level of image, r_median. This is used to separate the image into two as lower half F_L(x, y) and upper half F_U(x, y) that ranges $[0, r_{\text{median}} ]$ and $[r_{\text{median}} + 1,L - 1]$, respectively. These two images are further divided into two of each as $F_{\text{Ll}} \left( {x,y} \right)\; {\text{and}}\;F_{\text{Lu}} \left( {x,y} \right)$, F_Ul(x, y) and F_Uu(x, y) by medians as r_medianl, r_mediani, which are median intensities of $F_{L} \left( {x,y} \right)$ and $F_{u} \left( {x,y} \right)$, respectively. The histograms of $F_{\text{Ll}} \left( {x,y} \right), F_{\text{Lu}} \left( {x,y} \right)$, $F_{\text{Ul}} \left( {x,y} \right)$ and $F_{\text{Uu}} \left( {x,y} \right)$ are $\varvec{H}_{\text{Ll}} ,\varvec{ H}_{\text{Lu}} ,\varvec{H}_{\text{Ul}}$ and $\varvec{H}_{\text{Uu}}$ with ranges $\left[ {0,r_{\text{medianl}} } \right]$, [r_medianl + 1, r_median], $\left[ {r_{\text{median}} + 1,r_{\text{medianu}} } \right]$ and $\left[ {r_{\text{medianu}} + 1,L - 1} \right]$, respectively.

This method is briefed as follows.

• 1. Calculate r_median, r_medianl and r_medianu.

• 2. Use r_medianl, r_median and r_medianu to partition H as $\varvec{H}_{\text{Ll}} , \varvec{H}_{\text{Lu}} ,\varvec{H}_{\text{Ul}}$ and $\varvec{H}_{\text{Uu}}$ using Eqs. (11), (12), (13) and (14).

• 3. $\varvec{N}_{\text{Ll}} ,\varvec{ N}_{\text{Lu}} ,\varvec{N}_{\text{Ul}}$ and $\varvec{N}_{\text{Uu}}$ are generated from $\varvec{H}_{\text{Ll}} ,\varvec{ H}_{\text{Lu}} ,\varvec{H}_{\text{Ul}}$ and $\varvec{H}_{\text{Uu}}$ using Eqs. (19), (20), (21) and (22).

• 4. Equations (24), (25), (26) and (27) are used to derive the $\alpha_{\text{Ll}} , \alpha_{\text{Lu}} , \alpha_{\text{Ul}}$ and α_Uu by replacing r_mean by their corresponding r_median.

• 5. Transformations $\varvec{T}_{\text{Ll}} ,\varvec{T}_{\text{Lu}} ,\varvec{T}_{\text{Ul}}$ and $\varvec{T}_{Uu}$ are derived by applying above weights on $\varvec{N}_{\text{Ll}} ,\varvec{N}_{\text{Lu}} , \varvec{N}_{\text{Ul}}$ and $\varvec{N}_{\text{Uu}}$ as in Eqs. (28), (29), (30) and (31).

Following observations can be made from RPOGM transformations.

3.3 Computational Complexity of the Proposed Techniques

For an image having P rows and Q columns, histogram computation of image requires O(PQ) time. O(L) time is required for the computation of mapping function. To enhance the image by mapping function requires another O(PQ) time. So, total time required for HE is O(2PQ + L). Compared to HE, BHE requires O(L) more time for mean computation. So, total time required for BHE is $O\left( {2PQ + 2L} \right)$. Similarly RMSHE, DSIHE, RSIHE, BHEPL, QDHE, ESIHE, MMSICHE, AIE-BHE, RSESIHE and BHE2PL require $O\left( {2PQ + 3L } \right), O\left( {2PQ + 2L } \right), O\left( {2PQ + 2L} \right), O\left( {2PQ + 2L + 2 } \right), O\left( {2PQ + 3L + 8 } \right), O\left( {2PQ + 1 + 2L } \right), O\left( {2PQ + 3L + 3 } \right), O\left( 2PQ + 3L \right), O\left( {2PQ + 3L + 3 } \right)\;{\text{and}}\; O\left( {2PQ + 2L + 6 } \right)$, respectively.

For RMDPOGM, computation time is similar to RSIHE, but it requires O(4) time in addition to RSIHE for calculating four weights for four sub-histograms. So, total time required for RMDPOGM is O(2PQ + 2L + 4). Computation time for RMPOGM is similar to RMDPOGM, but it requires an additional O(L) for lower and upper mean findings for bisecting sub-histograms. So, total time required for RMPOGM is O(2PQ + 3L + 4).

Note Compared to order of image size PQ, L = 256 is a small value. So, all the above methods are real-time implementable as they require computation time of O(PQ) to get enhanced image.

4 Simulation Results

Experimental results of the proposed RMDPOGM and RMPOGM methods are compared with the old-to-recent existing histogram equalization methods, viz., HE, BBHE (1997), RMSHE (2003), DSIHE (1999), RSIHE (2007), BHEPL (2009), QDHE (2010), ESIHE (2014), MMSICHE (2014), AIEBHE (2014), RSESIHE (2015) and BHE2PL (2017) (HE-BHE2PL). To test the efficiency of these algorithms, two widely used image quality measures (IQM), viz., entropy and AMBE, are used. Fifteen test images viz., girl, baboon, kodim1, kodim2, kodim3, kodim4, copter, hands, Lena, sail boat, jet, air plane, Zelda, field and fish images are considered and IQM are tabulated. We also used gray level analysis tables along with these measures for better understanding of causes for image quality degradation in HE-based images. To find out the structural similarity of the enhanced image with input image, gradient magnitude similarity deviation (GMSD) [22] is employed.

4.1 Entropy Assessment

Information content of an image depends on entropy. The more the entropy, the more the details in the image. Entropy is calculated by

$$ {\text{Entropy}} = - \mathop \sum \limits_{l = 0}^{L - 1} p\left( l \right)\log p\left( l \right) $$

(32)

The entropy values of different images, processed by various methods, are tabulated in Table 1. Gray level analysis of test images, viz., girl and jet, are listed in Tables 2 and 3, respectively. These tables give the information about number of gray levels lost, how many numbers of gray levels are getting overlapped in the existing output gray levels and the total number of gray levels affected. The number of gray levels affected is sum of number of gray levels lost and number of gray levels overlapped in output image.

Table 1 Entropy values of different test images

Table 2 Gray level analysis for girl image

Table 3 Gray level analysis for jet image

From the above tables, it can be observed that in all the HE variants except BHE2PL, many gray levels are lost (lost gray levels have overlapped with existing gray levels) reducing the output image’s entropy. In BHE2PL a few gray levels of some of the images are being affected. The affected gray levels lead to loss of many fine details in the image. The loss of details can be observed in HE-processed images (Figs. 1b–m, 4b–m) of girl and jet images, respectively, at marked areas of Figs. 1p and 4p. For girl image, the lost details can be noticed in face and hair. Similarly, the loss of details can be visually noticed at the objects [objects that are marked with numbers 1 and 2 of jet image (Fig. 4p)] of images in Figs. 1b–m and 4b–m. The loss of details are proportional to number of gray levels affected as listed in Tables 2, 3, 4 and 5. It can be noticed that the loss of details in different HE-processed images are proportional to the number of gray levels affected. For example, in girl image (Fig. 1a), it can be observed from Table 2 that DSIHE-processed image (Fig. 1e), is having the highest number of gray levels affected (114 out of 136), resulting in highly degraded image. BHE2PL-processed image (Fig. 1m), is having the least number of gray levels affected (3 out of 136), resulting in quality image compared to all other processed images (Fig. 1b–l) of HE-RSESIHE methods. The images processed by RMDPOGM and RMPOGM do not have any gray level loss. As the proposed methods retain the number of gray levels even after enhancement, they preserve the entropy. These methods are giving natural enhancement, as can be observed in the images (Figs. 1n, o, 4n, o) of girl and jet images, respectively.

Fig. 1

a Original girl image, b HE, c BBHE, d RMSHE, e DSIHE, f RSIHE, g BHEPL, h QDHE, i ESIHE, j MMSICHE, k AIEBHE, l RSESIHE, m BHE2PL, n RMDPOGM, o RMPOGM processed images of girl, p marked image of girl

Fig. 2

a HE, b BBHE, c RMSHE, d DSIHE, e RSIHE, f BHEPL, g QDHE, h ESIHE, i MMSICHE, j AIEBHE, k RSESIHE, l BHE2PL, m RMDPOGM, n RMPOGM processed transformations of girl image

Fig. 3

a Original histogram of girl, b HE, c BBHE, d RMSHE, e DSIHE, f RSIHE, g BHEPL, h QDHE, i ESIHE, j MMSICHE, k AIEBHE, l RSESIHE, m BHE2PL, n RMDPOGM, o RMPOGM processed histograms of girl, p marked image of (a)

Fig. 4

a Original image of jet, b HE, c BBHE, d RMSHE, e DSIHE, f RSIHE, g BHEPL, h QDHE, i ESIHE, j MMSICHE, k AIEBHE, l RSESIHE, m BHE2PL, n RMDPOGM, o RMPOGM processed images of jet, p marked image of (a)

Fig. 5

a Original histogram of jet, b HE, c BBHE, d RMSHE, e DSIHE, f RSIHE, g BHEPL, h QDHE, i ESIHE, j MMSICHE, k AIEBHE, l RSESIHE, m BHE2PL, n RMDPOGM, o RMPOGM processed histograms of jet, p marked image of (a)

Table 4 GMSD values of processed images

Table 5 AMBE values of processed images

4.2 Assessment of GMSD Through Gray Level Analysis

The perceptual similarity between the processed and input image is assessed by using GMSD. An efficient and simple whole reference image quality assessment method is GMSD, which performs better than the other state-of-the-art whole image quality assessment methods, viz., peak signal-to-noise ratio (PSNR), information fidelity criteria (IFC), geometric structure distortion (GSD), gradient-structure similarity (G-SSIM), structure similarity (SSIM), visual information fidelity (VIF), most apparent distortion (MAD), multiscale-SSIM (MS-SSIM), gradient similarity (GS), gradient magnitude similarity mean (GMSM), information weighted-SSIM (IW-SSIM) and feature similarity (FSIM) [22]. Even though GMSD has high computational complexity compared with PSNR and GMSM, it has better correlation with human visual perception [22]. GMSD has less computational complexity and better performance than the other metrics, viz., IFC, GSD, VIF, MAD and SSIM and all hybrids of SSIM [22].

Pixel-wise gradient magnitude similarity (GMS) is given by

$$ {\text{GMS}}\left( i \right) = \frac{{2m_{r} \left( i \right)m_{d} \left( i \right) + c}}{{m_{r}^{2} \left( i \right) + m_{d}^{2} \left( i \right) + c}} $$

(33)

where m_r and m_d are gradient magnitudes of reference and distorted images, respectively. Here, the input image is considered as reference image and enhanced image as distorted image. The constant c is set as 0.0026 [22]. From the gray level analysis Tables 2, 3, it is observed that the HE methods [HE, BBHE, RMSHE, DSIHE, RSIHE, BHEPL, QDHE, ESIHE, MMSICHE, AIEBHE and RSESIHE (HE-RSESIHE)] are having many gray levels affected. These affected gray levels affect the objects to which they belong. More the gray levels affected, more the structural deviation of the output image. As most of the HE methods are having many gray levels affected, it is reflecting in structural changes in corresponding output image.

As BHE2PL is having very few gray levels affected, it results in better image with structural details than others. Standard deviation of GMS is GMSD, given by

$$ {\text{GMSD}} = \sqrt {\frac{1}{PQ}\mathop \sum \limits_{i = 1}^{PQ} \left( {{\text{GMS}}\left( i \right) - \frac{1}{PQ}\mathop \sum \limits_{i = 1}^{PQ} {\text{GMS}}\left( i \right)} \right)^{2} } $$

(34)

The lower the GMSD, better the similarity of input and output images. It can be observed from the entropy Table 1, gray level analysis Tables 2, 3 and GMSD Table 4 that they are closely correlated with each other. The lesser the number of gray levels affected, lower the loss of details, better the entropy and less loss of structural details. This reflects as lower GMSD. So, to have low GMSD, the image should have a smaller number of gray levels affected with high entropy.

The HE methods are having many gray levels affected, high GMSD and are producing false contours at the marked objects (Figs. 1p, 4p) of Figs. 1b–m and 4b–m. It can also be observed that CHE methods, viz., BHEPL, BHE2PL, are having a smaller number of gray levels affected compared to HE-RSIHE methods, resulting in relatively better image having lower GMSD than HE and RPHE methods. Even in CHE methods, BHE2PL is having very low GMSD (refer GMSD Table 4 as average of 15 test images, 0.0350) for test images, resulting in better structural detailed image.

From Tables 2, 3, it can be observed that the proposed methods, viz., RMDPOGM and RMPOGM, are having zero gray levels affected; thus, they are preserving the entropy (refer Table 1). This is leading to very low GMSD for the RPOGM-processed images compared to HE-, RPHE- and CHE-processed images, Table 4. (This can also be observed in the marked objects (Figs. 1p, 4p) of images (Figs. 1n, o, 4n, o) of well-enhanced girl and jet images, respectively.)

4.3 Brightness Preservation

Absolute mean brightness error (AMBE) is a good estimate of brightness preservation. Brightness preservation is the most important parameter in consumer electronic products. AMBE is calculated as

$$ {\text{AMBE}} = \left| {r_{\text{meanx}} - r_{\text{meany}} } \right| $$

(35)

where r_meanx is the mean of input image and r_meany is the output image mean. We can observe from Table 5 that the numbers marked as bold represent low AMBE. It can be observed that proposed methods are having low AMBE for most of the images and for other images also the values are closer to the low AMBE (marked bold). Average value of AMBE shows that both the proposed methods are preserving brightness, thus best suited for consumer electronic products.

4.4 Visual Quality

Visual quality is an important parameter in analyzing noise amplification, over enhancement, artifacts such as patches and unnatural look in the processed images. Visual quality is assessed by analyzing the input, output histograms and its intensity transformation. Various HE methods are compared with RMDPOGM and RMPOGM methods.

To test the robustness of the proposed algorithms, we have taken two images with different foregrounds and different backgrounds, viz., girl (Fig. 1a) and jet (Fig. 4a) images. The marked objects of these images are shown in Figs. 1p, and 4p, respectively. Visually noticeable loss of details can be found at these marked objects in the processed images (Figs. 1b–m, 4b–m) of HE, BBHE, RMSHE, DSIHE, RSIHE, BHEPL, QDHE, ESIHE, MMSICHE, AIEBHE, RSESIHE and BHE2PL (HE-BHE2PL), respectively.

Let us consider the girl image (Fig. 1a) for analysis. HE-, BBHE-, DSIHE- and RSIHE-processed images (Fig. 1b, c, e, f), respectively, have noise amplified in the background. This is due to steep rise in transformations (Fig. 2a, b, d, e). Also, these methods have darkened the hair and brightened the face, due to intensity compression on left and right side of the histograms, respectively [refer callouts (Fig. 3b) for detailed description] (Fig. 3b, c, e, f).

BHEPL- and ESIHE-processed images (Fig. 1g, i), have low noise in background, as there is small steep rise region in transformations (Fig. 2f, h). These methods have introduced some loss of details on face and have small patchy look. This is due to small flat region in transformations of BHEPL and ESIHE.

There is no background noise in RMSHE-, QDHE- and MMSICHE-processed images (Fig. 1d, h, j), which are looking better than all aforementioned HE methods. This can be observed as there is no intensity saturation in histograms (Fig. 3d, h, j) and no steep rise in the transformations (Fig. 2c, g, i). But, small flat region in transformations is causing face to lose some of the details and looks slightly brighter and patchy.

All the HE-based methods suffer from gray level loss and gray level overlap with existing gray levels. This leads to many-to-few/many-to-one gray level mapping (flat region) in the transformations. As MMSICHE and AIEBHE are having minimum number of gray levels lost, it has less loss of details and is looking better than all other HE-based methods (Fig. 1j, k). This can also be observed from gray level analysis that only 36 gray levels are affected by these methods. RSESIHE even though seems to be visually good (Fig. 1l), a careful observation reveals that there is a loss of details on hair and face.

Also, RSESIHE-processed girl image is having noise in the background. From the output histogram, it can be noticed that gray levels have not utilized gray band efficiently. [Output histogram occupies gray from 30 to 234 only as lower ET and upper ET fall at 30 and 234, respectively. The reason behind this is as there is no gray from 0–30 and above 234 in input histogram, there is no scope to enhance them.] BHE2PL-processed image (Fig. 1m), is giving pleasing results than all other HE variants (HE-RSESIHE), except that very few details on hair have been lost.

RMDPOGM- and RMPOGM-processed images (Fig. 1k, l) are giving good enhancement with natural look. It is not having patches, noise and loss of details as there is neither steep rise nor flat region in the transformations (Fig. 2j, k). This results in no intensity saturation and intensity compression in histograms, respectively (Fig. 3k, l).

Next, we consider jet image (Fig. 4a) for analysis. Pixels belonging to the background of image falls to the gray levels from 153 to 216. This gray level range is marked with red thick line in histogram (Fig. 5m). This narrow range of marked gray levels is spread to a wide range in HE-, BHE-, RMSHE-, DSIHE-, RSIHE-, BHEPL-, QDHE- and ESIHE-processed histograms (Fig. 5b–i), marked as a thick red line. Among the red marked gray levels, the contribution of black marked gray levels is large in noise enhancement. The background noise is directly proportional to range of intensity spread marked with black. In HE-, BHE-, DSIHE- and RSIHE-processed histograms (Fig. 5b, c, e, f), the spread is more compared to RMSHE-, BHEPL-, QDHE- and ESIHE-processed histograms (Fig. 5d, g, h, i). Hence, HE-, BHE-, DSIHE- and RSIHE-processed images (Fig. 4b, c, e, f), respectively, have relatively more noise than that of Fig. 4d, g, h, i. In the marked areas numbered 3 and 4 (ref Fig. 4p), there is a noticeable loss of details for all HE-based images.

Fig. 6

a HE, b BBHE, c RMSHE, d DSIHE, e RSIHE, f BHEPL, g QDHE, h ESIHE, i MMSICHE, j AIEBHE, k RSESIHE, l BHE2PL, m RMDPOGM, n RMPOGM processed transformations of jet image

AIEBHE-processed image (Fig. 4k) has better details in object 1, but has loss of details in object 2 and has darkened the object. Also, this is introducing background noise due to spread of peaks. ESIHE- and RSESIHE-processed images (Fig. 4i, l), are giving better details than all others at objects 1 and 2, but they are having loss of details and noisy in other parts of jet and background. Also, like girl image, RSESIHE-processed jet image is also having inefficient utilization of gray band. [Output histogram occupies gray from 9 to 255, as lower ET falls at 9. As there is no gray from 0 to 9 in input histogram, there is no scope to enhance any gray.]

MMSICHE-processed image (Fig. 4j), is having better details and pleasing look of total jet except at objects 1 and 2. But as like others, it is also having low noise in background, due to low saturation of intensities in histogram (Fig. 5j). BHE2PL-processed image (Fig. 4m), even though having pleasing details at objects 1 and 2 along with other parts of jet is introducing low noise at background.

Among all HE methods (HE-BHE2PL), BHE2PL is having low noise in the background. This is due to low spread of peaks in the gray space (Fig. 5m). RMDPOGM and RMPOGM-processed images (Fig. 4n, o) are giving natural enhancement in the objects 1 and 2 along with other parts of jet with all details and are not darkening the objects Also, they are not producing background noise as there is no intensity saturation in histograms (Fig. 5n, o) and no steep rise in transformations (Fig. 6m, n).

Gray level analysis Tables 2, 3 of girl and jet images reveal the fact that proposed methods are retaining all the gray levels in the processed image and hence are able to retain the entropy and structural details even after enhancement. From this, it can be said that flattening the output histogram does not guarantee good image quality.

4.5 Observations from the Visual Analysis

• 1. The HE and RPHE methods (BBHE, DSIHE, RSIHE) methods are having high intensity saturations in histograms (girl and jet images) or steep rise in transformations, leading to noisy output images. Among RPHE methods, RMSHE is having low noise in jet image as there is low intensity saturation in output histogram.

• 2. Even though CHE (BHEPL, QDHE, ESIHE, MMSICHE and AIEBHE) and CHEWMP (BHE2PL) methods are intended to avoid intensity saturation and are better than HE and RPHE methods, they are introducing low noise for jet and girl images. This is due to low intensity saturation of gray in output histogram.

• 3. The proposed methods have neither intensity saturation nor intensity compression in histograms and have neither steep rise nor flat region in transformation. This results in neither gray level overlap nor gray level loss in histograms. This assures uniform degree enhancement of gray levels. It results in good overall image quality with all details as observed in RPOGM-processed images of girl and jet images.

5 Conclusion

Two novel approaches for gray level mapping, viz., RMDPOGM and RMPOGM for image enhancement, are proposed in this paper. All the conventional HE methods are having the problem of gray level loss after transformation, as they are dependent on CDF. Also, these methods are having non-uniform degree of enhancement of gray levels, resulting in some of the objects being enhanced well at the cost of others. As the frequency of gray levels in the processed histograms is altered due to gray level overlap, the entropy gets reduced. This results in loss of details in the enhanced image, introducing deviated structural details in the output image and results in high GMSD.

The presented work has addressed the above issues and is having salient features, viz., (i) assures uniform degree of enhancement of gray, (ii) effectively addresses intensity saturation and compression problems, (iii) ensures highest entropy as the mapping is strictly monotonic, and (iv) ensure lower structural deviations of objects in the processed image, hence are free from false contouring. It has been seen that the RPOGM methods are giving good results for images with large uniform background and small non-uniform foreground. As future work, we are looking to implement an ideal foreground enhancement (where background gray enhancement is zero, i.e., background noise is zero) via recursive RPOGM techniques.