Adaptive Enhancement of Underwater Images

Abstract

Underwater images usually suffer from low contrast and non-uniform lightning. To overcome this histogram equalization is the basic technique for enhancement due to its simple function and effectiveness. This method tends to change the brightness of an image and therefore, not suitable for consumer electronic products, where preserving the original brightness is essential to avoid annoying artifacts. A number of techniques have been developed over a period of time to overcome these undesirable effects. But none of the technique is found suitable for enhancement of image under poor illumination conditions, which preserve the brightness of the original image. In this article we present a survey of different techniques that are based on histogram equalization, and also applied those techniques on underwater images. We also made a comparison of the processed images by a set of ten quality metrics.

1 Introduction

Under water imaging requires that vehicles carry their own light sources with them as ambient light is nonexistent. The images are affected with the transmission of limited range of light, disturbance of lightening, low contrast and blurring. Research work is in progress to compensate these artifacts and improve the visibility of underwater images [1–4]. In this paper we have proposed few image enhancement techniques based on histogram equalization. The effectiveness of various algorithms is accessed using set of quality metric parameters.

In conventional methods of image enhancement, human viewers choose the method of enhancement that is most appropriate for a given input image. That is to say, the method is chosen in an ad-hoc manner. The assessment of the algorithm is done in a subjective manner by human beings. While a human image processing expert may select the best method on a case-to-case basis based on visual inspection, such a human intervention may not be feasible in practice. For applications in which images are ultimately to be viewed by human beings, the only “correct” method of visually quantifying the image is through subjective evaluation. In practice however, subjective evaluation is usually too inconvenient, time-consuming unreliable and expensive. It is therefore necessary to devise methods to automatically select the suitable enhancement routine for any given input image. The adaptive methods of enhancement are modification of classical methods. Depending on the characteristics of the input image, adaptive method decides whether to increase the dynamic range of the image or to enhance the details of the dark regions of the image without affecting mid and bright pixels.

Image enhancement techniques have been used in various applications where the subjective quality of images is very important. Contrast is an important factor in image quality estimation. The term refers to the amount of gray scale differentiation that exists between various image features while working on gray scale images. Images having higher contrast level display a larger grayscale difference than those of lower contrast. The contrast variations affect the ultimate form of the image. There are many approaches for enhancing the contrast of images. Techniques using histograms are most common in the Contrast problem. Among these, Histogram Equalization (HE) is the classical method due to its simplicity and effectiveness. HE uses histogram information of the image and turns them into images with uniform histogram distributions. However, this technique is less effective when the contrast characteristics vary drastically across the image as in backlight conditions. That is the bright area becomes saturated in the resultant image due to the compensation taken place in the dark area. Moreover, the resultant image may have regions of decreased local contrast. This is because HE only uses global information (from the whole image) and does not consider local information of luminance variation within neighborhoods of each pixel. To overcome this many researchers have proposed techniques [5–15] for histogram equalization using local information.

In this article we compare each of these different techniques of histogram equalization and their effectiveness while processing the underwater images. We have used some of the quality parameters to analyze the results while finding the suitability of the algorithm.

2 Techniques Based on Histogram Equalization

In general, a histogram is the estimation of the probability distribution of a particular type of data. An image histogram represents a graphical tonal distribution of the gray values in a digital image. By viewing the image’s histogram, we can analyze the frequency of appearance of the different gray levels contained in the image. A good histogram is that which covers all the possible values in the gray scale used. This type of histogram suggests that the image has good contrast and that details in the image may be observed more easily.

2.1 Histogram Equalization

Histogram equalization is a straightforward enhancement technique to achieve better quality images in grey scale. The histogram equalization redistributes intensity values along the total range of values in order to achieve higher contrast. This method is especially useful when an image is represented by close contrast values, such as images in which both the background and foreground are bright at the same time, or else both are dark at the same time [5].

2.2 Brightness Preserving Bi-Histogram Equalization

Brightness Preserving Bi-Histogram Equalization (BBHE) has been proposed by Kim [6]. The BBHE firstly decomposes an input image into two sub-images based on the mean of the input image. One of the sub-images is the set of samples less than or equal to the mean whereas the other one is the set of samples greater than the mean. Then the BBHE equalizes the sub-images independently based on their respective histograms with the constraint that the samples in the formal set are mapped into the range from the minimum gray level to the input mean. The samples in the latter set are mapped into the range from the mean to the maximum gray level.

2.3 Recursive Mean-Separate Histogram Equalization

Recursive Mean-Separate Histogram Equalization (RMSHE) has been proposed by Chen and Ramli [7]. This method is a generalization of BBHE referred to as to provide not only better but also scalable brightness preservation. BBHE separates the input image’s histogram into two based on its mean before equalizing them independently. While the separation is done only once in BBHE, this method proposes to perform the separation recursively; separate each new histogram further based on their respective mean. Besides, the recursive nature of RMSHE also allows scalable brightness preservation, which is very useful in consumer electronics.

2.4 Bilinear Interpolation Dynamic Histogram Equalization

Bilinear Interpolation Dynamic Histogram Equalization (BIDHE) has been proposed by Xu and Liu [8]. First, the original image is divided into some same size sub-images. Second, the histogram of each sub-image is partitioned into sub-histograms without domination. Third, the new dynamic ranges are allocated for sub-histograms. Finally, HE and Bilinear Interpolation are respectively implemented to the image.

2.5 Brightness Preserving Dynamic Histogram Equalization

Brightness preserving dynamic histogram equalization (BPDHE) has been proposed by Ibrahim and Kong [9], which is an extension to HE that can produce the output image with the mean intensity almost equal to the mean intensity of the input, thus fulfill the requirement of maintaining the mean brightness of the image. First, the method smoothes the input histogram with one-dimensional Gaussian filters, and then partitions the smoothed histogram based on its local maximums. Next, each partition will be assigned to a new dynamic range. After that, the histogram equalization process is applied independently to these partitions, based on this new dynamic range. For sure, the changes in dynamic range, and also histogram equalization process will alter the mean brightness of the image. Therefore, the last step in this method is to normalize the output image to the input mean brightness.

2.6 Multipeak Histogram Equalization with Brightness Preserving

The Multipeak histogram equalization (MHE) has been proposed by Wongsritong et.al. [10]. Here, each detected peak of histogram is independently equalized. The effect of brightness saturation can be defeated, as the perceptibility can be improved. By using histogram equalization method, the output image can be preserved the mean value brightness of the input image.

2.7 Minimum Mean Brightness Error Bi-Histogram Equalization

Minimum Mean Brightness Error Bi-Histogram Equalization (MMBEBHE) has been proposed by Chen and Ramli [12] to provide maximum brightness preservation. BBHE separates the input image’s histogram into two based on input mean before equalizing them independently. This method proposes to perform the separation based on the threshold level, which would yield minimum Absolute Mean Brightness Error (AMBE—the absolute difference between input and output mean). An efficient recursive integer-based computation for AMBE has been formulated to facilitate real time implementation.

2.8 Dualistic Sub-Image Histogram Equalization

Dualistic Sub-Image Histogram Equalization (DSIHE) has been proposed by Yu Wan et al. [11]. Here, first, the image is decomposed into two equal area sub-images based on its original probability density function. Then the two sub-images are equalized respectively. At last, we get the result after the processed sub-images are composed into one image.

2.9 Adaptively Modified Histogram Equalization

Adaptively Modified Histogram Equalization (AMHE) is proposed by Hyoung-Joon Kim et al. [13] which is an extension of typical histogram equalization. To prevent any significant change of gray levels between the original image and the histogram equalized image, the AMHE scales the magnitudes of the probability density function of the original image before equalization. The scale factor is determined adaptively based on the mean brightness of the original image.

2.10 Weighted and Threshold Histogram Equalization

Weighted and threshold histogram equalization (WTHE) is proposed by Wang and Ward [14]. In this method, the Probability distribution function of an image is modified by weighting and thresholding before the histogram equalization (HE) is performed. This method provides a convenient and effective mechanism to control the enhancement process while being adaptive to various types of images.

2.11 Average Luminance with Weighted Histogram Equalization

Average Luminance with Weighted Histogram Equalization (ALWHE) has been proposed by Tai et al. [15] for Low Dynamic Range Images (LDRI). Automation of parameter selection can easily be done by iteratively using the proposed method. The proposed method overcomes the disadvantage of HE like luminance shifting and washed-out looking.

3 Quality Metrics

Image quality measurement is crucial for most image processing applications. The best way to assess the quality of an image is perhaps visual observation called subjective quality measurement, since human eye is the ultimate receivers in most image processing environment. The subjective quality metric, Mean Opinion Score (MOS), although used for many years, found to be too inconvenient, slow and expensive for practical usage. The objective image quality metrics can predict perceived image quality automatically using a set of Quality Metric Parameters.

In this paper, we have proposed a number of quality metric parameters and used them to study the performance of the proposed enhancement algorithms for a set if input images.

The quality parameters are: Entropy, Entropy Error Rate (EER), Grey level Energy (GLE), Spatial Frequency (SF), Global Contrast (GC), Quality Index (QI), Absolute Mean Brightness Error (AMBE), Relative Entropy (RE), Structural Content (SC) and Maximum Difference (MD).

A brief description of each of these metrics is given below:

3.1 Entropy

The entropy [16] is the measure of information content in an image and is given by

$$ {\text{Entropy = }} - \sum\limits_{{{\text{k}} = 0}}^{ 2 5 5} {p(k)\mathop {\log }\nolimits_{2} } (p(k)) $$

(1)

If the enhanced image is higher value of entropy than the input image, then we can say that the image is truly enhanced. Hence, the entropy ratio for image enhancement is given by

$$ {\text{ER}} = \frac{\text{Entropy(OutputImage)}}{\text{Entropy(InputImage)}} $$

(2)

3.2 Entropy Error Rate

The entropy error rate (EER) [17] is a measure of distribution of image information. EER of an image is given as

$$ {\text{EER}} = \frac{{\overline{\text{HD}} - \overline{\text{HB}} }}{\text{S}} $$

(3)

where ${\bar{\text{H}}}_{\text{D}}$ and ${\bar{\text{H}}}_{\text{B}}$ are the average entropy for the darker pixels and brighter pixels, and S is a statistic that estimates the relative position of the mean within the intensity histogram. If EER of an image has a relatively large positive value, then the image is enhanced.

3.3 Gray Level Energy

The grey level energy (GLE) [16] indicates how the grey levels are distributed. If an image has a GLE value approaching to 1, then the image is said to be enhanced. The value of GLE is given by the following expression:

$$ {\text{E(x) = }}\sum\limits_{ 0}^{ 2 5 5} {\mathop {\text{p(i)}}\nolimits^{ 2} } $$

(4)

where E(x) refers to the gray level energy with 256 bins and P(i) refers to the probability distribution function of the histogram.

3.4 Spatial Frequency

The spatial frequency (SF) measure indicates the overall activity level in an image [18]. SF is defined as follows:

$$ {\text{SF}} = \sqrt {\mathop R\nolimits^{2} + \mathop C\nolimits^{2} } $$

(5)

where

R is column frequency and is given by ${\text{R}} = \sqrt {\frac{1}{MN}*\sum\limits_{j = 1}^{M} {\sum\limits_{k = 2}^{N} {\mathop {(\mathop X\nolimits_{j,k} - \mathop X\nolimits_{j,k - 1} )}\nolimits^{2} } } }$

C is column frequency and is given by ${\text{C}} = \sqrt {\frac{1}{MN}*\sum\limits_{k = 1}^{N} {\sum\limits_{j = 2}^{M} {\mathop {(\mathop X\nolimits_{j,k} - \mathop X\nolimits_{j - 1,k} )}\nolimits^{2} } } }$

Xj, k denotes the pixel intensity values of image, M and N are number of horizontal and vertical directions. The value of spatial frequency is high for a good enhanced image.

3.5 Global Contrast

The global contrast (GC) [19] value of an image is defined as the second central moment of its histogram divided by the total number of pixels in the image and is given by

$$ {\text{Global Contrast}} = \frac{{\sum\limits_{i = 0}^{L} {({\text{i }} - \, \mu )2*Hist(i)} }}{\text{N}} $$

(6)

where μ is the average intensity of the image, Hist (i) is the number of pixels in the image with intensity value i, L is the highest intensity value.

3.6 Quality Index

The Image quality index (QI) is suggested according to the statistical features gray level histogram of an image and is given as

$$ {\text{Q}} = \frac{1}{\Updelta f}\sqrt {\frac{{\sum\limits_{i = 0}^{L} {({\text{p}}_{\text{j}} { - }\overline{\text{p}} )^{2} } }}{\text{L}}} $$

(7)

where Q is image quality, $\Updelta {\text{f}}$ is the dynamic range, ${\text{p}}_{\text{j}}$ is the histogram, ${\bar{\text{p}}}$ is the average of the histogram and L is the number of gray levels.

3.7 Absolute Mean Brightness Error

Absolute Mean Brightness Error (AMBE) [20] measures the deviation of the processed image mean from the input image mean. The AMBE value provides a sense of how the global appearance of the image has changed with respect to lower values. The expression for AMBE is given as:

$$ {\text{AMBE}} = \left| {\mu {\text{p}} - \mu {\text{i}}} \right| $$

(8)

where ${{\upmu}}_{\text{p}}$ is the processed image and ${{\upmu}}_{i}$is the input image.

3.8 Relative Entropy

The relative entropy (RE) of probability distribution of the input image with respect to the probability distribution of the output image is defined as the summation of all the possible states of the system. The lower value of RE indicates good enhanced image. The expression for RE is given as:

$$ {\text{RE}} = - \, \sum\limits_{{{\text{k }} = { 0}}}^{ 2 5 5} {{\text{p(k)log}}_{ 2} \left( {\frac{\text{p(k)}}{q(k)}} \right)} $$

(9)

3.9 Structural Content

Structural content (SC) of an image is defined as the ratio between the sum of the pixels in the original image at location (m, n) to the sum of the pixels in the processed image at the same location (m, n) and is given as:

$$ {\text{SC}} = \frac{{\sum\limits_{m = 1}^{M} {\sum\limits_{n = 1}^{N} {X(m,n)^{2} } } }}{{\sum\limits_{m = 1}^{M} {\sum\limits_{n = 1}^{N} {Y(m,n)^{2} } } }} $$

(10)

where M is the number of columns, N is the number of rows, x(m, n) denote the pixel values of the original image at location (m, n), y(m, n) denote the pixel values of the processed image at location (m, n).The value of structural content will be high for a good enhanced image.

3.10 Maximum Difference

Maximum difference (MD) of an image is defined as the absolute difference between the original image at location (m, n) and the processed image at the same location (m, n). The value of MD will be high for a good enhanced image. The expression for MD is given as:

$$ {\text{MD}} = {\text{Max }}\left( {\left| { \, X(m,n) - Y(m,n) \, } \right|} \right) $$

(11)

where X(m, n) denote the pixel values of the original image at location (m, n), Y(m, n) denote the pixel values of the processed image at the same location (m, n).

4 Discussion

HE [5] is a technique commonly used for image contrast enhancement, since HE is computationally fast and simple to implement. HE performs its operation by remapping the gray levels of the image based on the probability distribution of the input gray levels. Brightness preserving Bi-Histogram Equalization (BBHE) [6], Recursive Mean Separate HE (RMSHE) [7], Brightness preserving Dynamic Histogram Equalization (BPDHE) [9] and AMHE [13] are the variants of HE based contrast enhancement techniques. BBHE divides the input image histogram into two parts based on the mean of the input image and then each part is equalized independently. This method tries to overcome the problem of brightness preservation. RMSHE [7] is an improved version of BBHE. However, it is also not free from side effects. For this purpose, Ibrahim and Kong proposed Brightness Preserving Dynamic Histogram Equalization (BPDHE) [9], this method partitions the image histogram based on the local maxima of the smoothed histogram. It then assigns a new dynamic range to each partition. Finally the output intensity is normalized to make the mean intensity of the resulting image equal to the input one. Adaptively Modified Histogram Equalization [13] scales the magnitudes of the probability density function of the original image before equalization. To resolve the fog-degraded image problem, fog removal method based on bilinear interpolation dynamic histogram equalization (BIDHE) [8] was proposed. The original image was split into some sub-images with same size, and the histogram of each sub-image was partitioned by the local minima. Minimum Mean Brightness Error Bi-Histogram Equalization (MMBEBHE) [11] is a variation of BBHE which perform the separation based on the threshold level, which would yield minimum Absolute Mean Brightness Error. The effect of brightness saturation can be defeated using Multipeak Histogram Equalization [10] which is based on global histogram equalization technique. However, luminance shifting and washed out looking can be occurred on the images after processing by this histogram based methods. To overcome this Average Luminance with Weighted Histogram Equalization (ALWHE) has been proposed by [15] where the automation of parameter selection can easily be done by iteratively using the proposed method.

5 Results and Analysis

A set of three underwater images were taken for the above study. The enhancement algorithms proposed in the Sect. 2 of the paper is applied to all the images and the results were analyzed using the set of quality parameters described in Sect. 3. Figures 1, 2 and 3 shows the results of enhancement using each of these algorithms. Tables 1, 2 and 3 shows the analysis using quality parameters. The quality parameters across each algorithm for a given input image are studied. Since the parameters GLE and QI do not vary much across different algorithms, we have discarded them for further analysis.

Fig. 1

Image courtesy of Bazeille et al. [12]. From left to right input Image, HE, BBHE, RMSHE, MMBEBHE, DSIHE, AMHE, WTHE, ALWHE, BIDHE, BPDHE, MHE

Fig. 2

Image courtesy of Bazeille et al. [12]. From left to right input Image, HE, BBHE, RMSHE, MMBEBHE, DSIHE, AMHE, WTHE, ALWHE, BIDHE, BPDHE, MHE

Fig. 3

From left to right input Image, HE, BBHE, RMSHE, MMBEBHE, DSIHE, AMHE, WTHE, ALWHE, BIDHE, BPDHE, MHE

Table 1 Results for image 1

Table 2 Results for image 2

Table 3 Results for image 3

A study of the above three sets of results show that the few algorithms of adaptive enhancement gives the desired value of quality parameters. Analysis show that HE, BBHE, DSIHE and BIDHE gives comparatively higher value of Entropy, Entropy Error Rate, Spatial Frequency, Structural Content, Maximum Difference and lower value of Relative Entropy.

6 Conclusion

The paper discusses a number of techniques for enhancement of underwater images, all of which, in principle, are based on classical histogram equalization techniques. Based on results obtained, the methods have proven to improve several distinctive obstructions in the underwater images. The analysis of the results concludes that HE, BBHE, DSIHE and BIDHE can preserve the mean brightness of underwater images better in comparison to the other methods. The study also shows that only some of the quality parameters can be chosen for the purpose of comparison of different algorithms.

Besides the good features of the algorithms, there still exist some issues and areas of improvement for the proposed techniques. First, the good performance of the algorithm depends on the wise selection of few input parameters. It might be possible to find a set of parameters that optimize the performance however, it requires extensive tests and experiments with different inputs. At the moment, the method for removing outliers is not stable because sometimes it removes too much information which is not outliers and sometimes it misses some artifacts. In terms of algorithms, we can always update with new techniques of image enhancement. There are more and more interest in underwater imaging and many good techniques have been proposed in recent years.