A natural-based fusion strategy for underwater image enhancement

Abstract

Underwater images generally are characterized by color cast and low contrast due to selective absorption and light scattering in water medium. Such degraded images reveal some limitations when used for further analysis. To overcome underwater image degradation, various enhancement techniques are developed. Especially, the fusion-based methods have made remarkable success in this filed. However, there are still some defects in the fusion of input images and weight maps, which cause their results to be unnatural. In this paper, we propose a novel and effective natural-based fusion method for underwater image enhancement that applies several image processing algorithms. First, we design an adaptive underwater image white balance method motivated by our statistical prior to mitigate the impact of color deviation of underwater scenes. We then derive two inputs that represent local detail-improved and global contrast-enhanced versions of the color corrected image. Instead of explicitly estimating weight map, like most existing algorithms, we propose a naturalness-preserving weight map estimation (NP-WME) method, which models the weight map estimation as an optimization problem. Particle swarm optimization (PSO) is used to solve it. Benefiting a proper weighting, the proposed method can achieve a trade-off between detail enhancement and contrast improvement, resulting a natural appearance of the fused image. Through this synthesis, we merge the advantages of different algorithms to obtain the output image. Experimental results show that the proposed method outperforms the several related methods based on quantitative and qualitative evaluations.

1 Introduction

The quality of visual information of underwater images plays an important role in many ocean engineering applications and scientific researches, such as aquatic robot inspection [53], underwater saliency detection [29]. However, underwater images are usually plagued by color cast and low contrast owing to the presence of selective absorption and light scattering. For underwater scenario, The selective absorption leads to the serious color cast and image information loss [2]. The light scattering blurs image features and reduces the contrast [9]. Therefore, underwater image enhancement techniques are desired in both scientific researches and computer applications.

Several attempts have been made to enhance the visual manifestation of such degraded underwater images. On the whole, the early methods mainly make use of multiple images [42, 47] or additional information [30] to restore the underwater image. Nevertheless, these algorithms are generally limited in practical applications. In recent years, researchers focus on haze removal from a single image, most of which have been made using the image formation model (IFM). The core idea of these IFM-based dehazing methods depends on various hypothesis or priors. However, for underwater scenes, these extra information may not always hold, due to the unbalanced attenuation caused by the selective absorption in water. Subsequently, with the popularity of Graphics Processing Unit (GPU), the deep learning approach has become the most advanced solution in the image dehazing field. However, these methods introduce unsatisfactory results since the lack of dataset, which contains both the real-world underwater images and their corresponding ground truth for different water types. Although various methods are proposed to imitate underwater image, there still exists a gap between synthetic and real-world underwater images [36].

Noticing these problems, several methods [3, 4, 7, 39] based on fusion have been proposed. In [3], Ancuti et al. proposed a multi-scale fusion method that combined the result images of color correction and contrast enhancement by four weight maps about laplacian contrast, local contrast, saliency and exposedness. Recently, [4] modified this previous work without calculating the exposedness weight map, to avoid amplifying some artifacts and improve the color appearance of resultant images. Although these methods make valuable achievements. But it still suffers from artifacts and unnatural appearance, because it limits itselves to these several weight maps only, without fully considering the information balance among different derived inputs version. In addition, these methods estimate weights based on single pixel, which is susceptible to noise and easy to produce visual artifacts in the fused image.

To solve the aforementioned problem, we propose a novel enhancement method that reasonably combines several algorithms to obtain final enhanced result. The specific contributions are as follows:

• We propose an effective natural-based fusion method to enhance underwater image, which simultaneously solves the dehazing problem. Extensive experiments show that the proposed method has better results than other related methods.

• To remove the color casts of underwater images, we propose an adaptive underwater image white balance (AUWB) method inspired by our statistical prior that the green channel is relatively well preserved. By considering the significant attenuation difference between the green channel and the degraded channels into a gain factor, the proposed AUWB method can effectively compensate various color distorted underwater images.

• To improve the naturalness of the fused image, we design a novel naturalness-preserving weight map estimation (NP-WME) algorithm, which is formulated as an optimization problem. Particle swarm optimization (PSO) provides a helpful support to solve it. Compared with previous fusion-based algorithms, our enhanced image achieves a good trade-off on the transfer of contrast and details without the significant amplification of visual artifacts.

The rest of this paper is organized as follows: Section 2 introduces the work related to underwater dehazing. The proposed method is described in Section 3. Section 4 evaluates and compares experimental results. Section 5 concludes the paper.

2 Related work

The existing image dehazing methods can be roughly divided into the following categories: traditional methods [25, 62] and deep-learning based methods [32, 59]. The traditional methods are further classified into underwater image restoration methods and underwater image enhancement methods. Here is a brief overview of all the above methods.

2.1 Underwater image restoration method

Underwater image restoration methods [54, 55, 60] use the optical model parameters derived from prior knowledge to restore the image by reversing the degradation process. Most of these technologies are based on image formation model (IFM). To estimate the derived parameters, the dark channel prior (DCP) [24] is used. Since the DCP has been effectively proven, many researchers have proposed various underwater restoration methods based on DCP. For example, Drews Jr et al. [16] proposed the underwater DCP (UDCP) from the traditional DCP by excluding the red channel in producing the prior. Peng et al. [44] proposed a generalized dark channel prior (GDCP), which estimated ambient light using the depth-dependent color, and also used adaptive color correction incorporated into the IFM. Apart from the DCP-related priors, there are also useful priors proposed by other underwater image restoration works. Galdran et al. [20] proposed a red channel prior method to recover the lost contrast in underwater images. This prior reversed the red channel to compensate the strong attenuation of red light in the water medium. Li et al. [34] represented an effective underwater image dehazing method built on a minimum information loss principle. And then, a histogram distribution prior was proposed to increase the contrast and brightness of underwater images. Peng and Cosman [45] proposed a depth estimation method for underwater scenes based on image blurriness and light absorption, which can be used in the IFM to restore underwater images. Hou et al. [26] proposed a variational model based on non-local different operators, which successfully integrated the underwater image formation model into the variational framework.

2.2 Underwater image enhancement method

On the other hand, underwater image enhancement methods [23, 37] improve the image quality by changing the intensity values of the RGB color channels according to certain rules. For example, in [11], Chambah et al. proposed an automatic color equalization (ACE) model to correct strong and nonuniform color casts in underwater images. In [27], Iqbal et al. used histogram stretching method to restore the color balance and improve the contrast of underwater images. This approach is easy to apply and can successfully improve underwater image quality. Fu et al. [19] proposed a two-step enhancement method, which carries out a color correction process followed by a contrast enhancement process. Mi et al. [40] proposed a multi-purpose oriented method, which decomposes the input into illumination and reflectance layer. Brightness compensation is performed on the illumination layer, while enhancement of color correction and contrast are carried out on the reflectance layer through a multi-scale processing strategy. Zhang et al. [61] introduced a method called LAB-MSR, which is implemented by extending the Retinex algorithm to CIEL^∗a^∗b^∗ color space.

2.3 Deep-learning based method

More recently, deep-learning based dehazing network is designed for underwater images. Fabbri et al. [17] proposed UGAN, which uses gradient penalty as the soft constraint on the output image. In letter [35], Li et al. proposed a weakly supervised color transformation technique inspired by cycle-consistent adversarial networks (CCAN) to correct color. Most recently, UWCNN [32] is proposed, which uses a CNN architecture trained on the underwater scene prior based underwater images. Islam et al. [28] presented a supervised enhancement model called FUnIE-GAN for estimating clear image. Guo et al. [21] proposed a multi-scale dense block, called DenseGAN that employs the use of dense connections, residual learning and multi-scales network for enhancing underwater image.

3 The proposed method

In this section, the proposed method is deliberated for generating a haze-free underwater image characterized by natural appearance and good visibility. An overview of the proposed method is shown in Fig. 1. The different steps involved in the proposed method are as follows:

Fig. 1

Overview of the proposed method. AUWB: Adaptive underwater image white balance. SLIC: Simple linear iterative clustering. NP-WME: Naturalness-preserving weight map estimation

3.1 Adaptive underwater image white balance

In terms of underwater image enhancement, it is necessary to correct the unwanted color appearance caused by selective absorption of colors with depth. In water, the unbalance attenuation caused by the selective absorption varies with geographical, seasonal, and climate related events [8]. For example, visible light with the longest wavelength is absorbed at the highest rate, appearing deep-blue or blue-green to the eye, when travling in clear open waters. In near-shore waters, sea water contains more suspended particles and organic matter, which strongly absorb the shortest wavelength, resulting in yellowish or greenish appearance.

When focusing on the issue of color correction, existing white balance methods are considered to search for a proper solution, and the visual results are shown in Fig. 2. However, most traditional methods perform poorly due to not considering the selective absorption. They fail to remove the color shifting, and generally look bluish. The Gray World is the best method to remove blue tone, but it suffers from severe red artifacts. Recently, some methods [4, 5, 12] have been proposed to reduce the effect of selective absorption. In [4], a constant proportional compensation model is developed to make up the difference between the degraded channel and other channels. The global visual performance can be improved. However, it obtains the degraded channels by artificially observing the tones of the image, which reduces their practical applicability. Therefore, how to effectively compensate for the light attenuation is still a challenge.

Fig. 2

Comparative results of different white balance methods. From left to right are original images that are examples of the four color tones, and the results generated by Grey Edge [58], Shades of Grey [18], Max RGB [31], Gray World [10], and AUWB method. One to three rows are selected from UCCS [38]. The set contains three 100-image subsets of “Blue”, “Green-blue”, and “Green” tones according to the degree of color casts. Fourth row is selected from UIEBD [36], This subset contains 10 typical yellow images

To solve the problems mentioned above, an adaptive underwater image white balance (AUWB) approach is proposed. Firstly, we have noticed that images captured under water usually show four color tone types, namely, blue, blue-green, greenish and yellow. To analyze the selective absorption of each color tone, we gather an underwater image dataset from UCCS [38] and UIEBD [36]. The corresponding example images are shown in the first two columns in Fig. 2. Table 1 calculates the mean (μ) and standard deviation (σ) of the three channels in each color tone type. The bold value means the maximum value of the R-G-B channel. From Table 1, the following observations can be obtained :

Table 1 The mean (μ) and standard deviation (σ) of each subdataset

1) Generally, the red component is attenuated faster than blue/green ones, which is only suitable for images with bluish and blue-green tones in clear water.

2) When the images appear greenish or yellow, the mean value of the blue channel is much smaller than the green. It reveals the fact that, the blue channel may be significantly attenuated, due to the absorption by organic matter when traveling in sea water.

3) The green channel is relatively well preserved under water, compared to the red and blue ones.

Following the above statistical prior, we design a gain factor Z to carry information about the difference in average pixel value between the green channel and the degraded channels. This information is critical for controlling the appropriate amount of pixel values that should be added to the degraded channels. The gain factor Z is defined as:

$$ Z^{c}=\left | \frac{\mu \left (I^{g} \right ) -\mu \left (I^{c} \right )}{\mu \left (I^{g} \right )+\mu \left (I^{c} \right )} \right |,c\in \left \{ r,g,b \right \} $$

(1)

where $\mu \left (I^{c} \right )$ is the average of color component I^c.

And then the underwater image white balance function is defined as:

$$ W\left (I^{c}\left (x \right ) \right )=I^{c}\left (x \right )+Z^{c}I^{g}\left (x \right ) $$

(2)

where $W\left (\cdot \right )$ is the white balance function.

To adaptively correct various color cast images, we add a threshold judgment (τ), and $\tau =\lambda \sum \mu \left (I^{c} \right )$. If $\mu \left (I^{g} \right )-\mu \left (I^{c} \right )\geq \tau$, then the AUWB compensates for the attenuation of degraded channels. Otherwise, it means the image has less color cast and white balance may causes unnecessary artifacts. In practice, our tests reveal that λ = 0.1 is appropriate for various illumination conditions and acquisition settings.

Overall, we present an effective and automatic AUWB method, which is able to selectively respond to color casts. For a given underwater image I(x), the color corrected image $I_{WB}\left (x \right )$ is obtained as:

$$ I_{WB}^{c}\left (x \right )=\left\{ \begin{array}{ll} W\left (I^{c}\left (x \right ) \right ) , & \mu \left (I^{g} \right )-\mu \left (I^{c} \right )\geq \tau , \\ I^{c}\left (x \right ) , & otherwise. \end{array} \right. $$

(3)

To further analyze the robustness of the AUWB, Fig. 3 shows the comparisons of several white balance techniques to measure the color corrected accuracy of images taken at different depths. Our assessment considers the CIEDE2000 [49], which measures the color difference between the reference ground truth Macbeth Color Checker and the corresponding color patch, manually located in each image. As can be seen, the AUWB approach as a pre-processing step performs better than other methods in most of cases. In addition, compared to Ancuti [4], the proposed method is adaptive and can effectively handle multiple underwater environments.

Fig. 3

Comparisons of several well-known white balance approaches (Grey Edge [58], Shades of Grey [18], Max RGB [31], Gray World [10], and Ancuti’s white balance method [4]). The original images (selected from [46]) are captured at the depths of 3.26m, 6.06m, 8.98m, 12.25m, and 15.11m. The color checker is used as a reference for calculating CIEDE2000 [49]. The range of CIEDE2000 is [0,100], a smaller value indicates a small color difference. In other words, the smaller the metric, the better

3.2 Natural-based fusion

In this section, our goal is to develop an effective natural-based fusion method that blends the advantages of different mature image processing techniques. Specifically, the proposed technique could be divided into the following four steps:

3.2.1 Inputs

A natural and intuitive way to enhance the degraded images is simply to adjust illumination since it is linked to the naturalness of the image [57]. Therefore, in this section, the color corrected image is converted into CIEL^∗a^∗b^∗ color space. Then, the L channel is extracted as the illumination information. Our two inputs are derived from the estimated illumination, which allows us to avoid distortion. The first input I_C is designed to address the global contrast in order to clarify the dark regions of the image. Many techniques can be used to enhance global contrast, such as sigmoid function and gamma correction. In this step, we obtain the first input by applying the Contrast-Limited Adaptive Histogram Equalization (CLAHE), which useful for expanding the global contrast between adjacent structures. Since enhancing the global contrast is at cost of blurring the visual information. Thus, the second input I_L is derived to improve local detail by using Laplacian filter.

In order to avoid the artifacts caused by the pixel-wise weight map estimation, we use SLIC (Simple Linear Iterative Clustering) [1] to divide the input image into N non-overlapping patches. To achieve the division of the input images, the color corrected image is divided into non-overlapping N image patches (Mask) by using SLIC. To ensure that the segmentation results of two input images are consistent, we directly apply the mask to the input images to obtain final input images ${I}_{C} = {I}_{C}^{n}$ and ${I}_{L}= {I}_{L}^{n}$, where $n=\left (1,2,{\cdots } ,N \right )$.

3.2.2 Weights

Estimating the weight map is an important step of image fusion. Existing fusion-based methods often introduce unnatural appearance, due to the inaccurate weight map estimation. To fill this gap, the NP-WME method is proposed in this work. It is designed as an optimization problem, defined as:

$$ W=argmin\left \{ \sum\limits_{c}\mathcal{L}\left (f\left (GF(W) , I_{C},I_{L} \right ) \right ) \right \} $$

(4)

where c indicates each channel, W is the initial weight map, $GF\left (\cdot \right )$ is refinement function, $f\left (\cdot \right )$ is the abbreviation of fusion function, ${\mathscr{L}} \left (\cdot \right )$ is the objective function. Note that $f\left (\cdot \right )$ is a function of three parameters, W is the only unknow component.

To estimate the value of W with high accuracy, PSO is selected to solve this optimization problem. Since PSO has been proven to be effective in image enhancement field [41, 48]. Furthermore, it is uncomplicated to hybridize PSO with other algorithms and can be used in various application environments with a little modification [56].

Particle swarm optimization (PSO) is a random search algorithm based on swarm cooperation [22], which is developed by simulating the foraging behavior of birds. Each bird is regarded as a “particle”. The position of each particle represents a solution to the problem. The algorithm randomly initializes a group of particles, then iterates to find the optimal solution, and uses fitness to determine the quality of the solution. Each iteration of the particle tracks the personal best (P_best) and the global best (G_best) to dynamically update its velocity and position. It specifically includes the following steps:

Parameter Initialization. :

Initializing the beginning parameters, such as the maximum number of iterations t_max, total population of particles m, inertia weight ω, learning factor c₁ and c₂, and the random numbers r₁ and r₂.

Updating the Particles. :

The velocity of each particle is updated based on the summation of parameters, which is defined as:

$$ v_{i}^{t+1}=\omega {v_{i}^{t}}+c_{1}r_{1}\times \left (P_{best}-{x_{i}^{t}} \right )+c_{2}r_{2}\times \left (G_{best}-{x_{i}^{t}} \right ) $$

(5)

where $i=\left (1,2,{\cdots } ,m \right )$, v is velocity, ω is an inertia factor. c₁ and c₂ are non-negative constants and learning factors, r₁ and r₂ are two random numbers. t represents the number of current iterations. In addition, the personal best (P_best) is the current optimal position searched by the single particle, and the global best (G_best) is the current optimal position searched by all particles. Both P_best and G_best are updated at every iteration.

Then, the particle’s new velocity $v_{i}\left (t+1 \right )$ is used to update the position x of particle defined as:

$$ x_{i}^{t+1}={x_{i}^{t}}+v_{i}^{t+1} $$

(6)

In the velocity update formula (5) , the first item is the product of the inertia weight and the current velocity of the particle, which represents the trust of the particle in its current motion, and is based on the original velocity’s inertial movement. The second term represents the state of self-consciousness, which is the particle’s judgment of its own history. The third item indicates social consciousness and is the mutual cooperation and information sharing of the various particles in the group.

Objective Function. :

The fitness of the particle’s current position is measured by the objective function. Once the inputs of the natural-based fusion process are obtained, it is possible to estimate fused image using enhancement properties: contrast and information loss. The objective function is formulated to maximize the contrast and minimize the information loss of the fused image, defined as:

$$ \mathcal{L}=\mathcal{L}_{con}-\mathcal{L}_{inf} $$

(7)

where ${\mathscr{L}}_{con}$ and ${\mathscr{L}}_{inf}$ are defined as:

$$ \mathcal{L}_{con}=\sum\limits_{c} \sum\limits_{i\in {\varOmega}_{n}} \frac{G_{c}\left (i \right )-\mu \left (G_{c} \right )}{\left | {\varOmega}_{n} \right |} $$

(8)

$$ \mathcal{L}_{inf}=\sum\limits_{c}\sum\limits_{i\in {\varOmega}_{n}} \left (max\left (0,J_{c}\left (i \right ) -255\right )^{2}+min\left (0,J_{c}\left (i \right ) \right )^{2} \right ) $$

(9)

where c indicates each channel, $\left | {\varOmega }_{n} \right |$ is the cardinality of the n th patch Ω_n, G_c is a gray-scale layer of fused image J_c, and the intensity range is set 0 to 255, and $\mu \left (G_{c} \right )$ is the average intensity of G_c. The information loss can be counted by the sum of enhanced pixel values that are out of bounds (0,255).

In Eq.(7), the first term ensures the fused image J has rich contrast and texture information, and the second term is used to measure the lost information cost. A good fused result is described by the large value of ${\mathscr{L}}$. In other words, our weight maps can be obtained when the ${\mathscr{L}}$ achieves the largest value.

Fig. 4 shows the different weights obtained by fusion-based method [3], fusion-based method [4] and the proposed method. It can be observed that the visual effect of the proposed method is pleasing, while the contrast is well enhanced. The results of [3] and [4] lose the global lightness order and have the certain lightness distortion. Therefore, it proves that the proposed natural-based fusion method has better flexibility and can well transform the contrast and details into the fused image. For clarity, the entire procedure of calculating weights $(W^{\prime })$ is outlined in Algorithm 1.

Fig. 4

Different weight maps obtained by fusion-based method [3], fusion-based method [4] and the proposed method

3.2.3 Refinement

To ensure spatial consistency, we refine the weight maps of input images with guided filter. Guided filter can maintain the edges while smoothing the image. The gray-scale version of the input image is selected as the guided image. The refined weight maps can be written as:

$$ W_{y}^{\prime}=GF\left (I_{gray},W_{y},r,\varepsilon \right ), y=\left \{ C,L \right \} $$

(10)

where $GF\left (\cdot ,\cdot ,\cdot ,\cdot \right )$ represents the guided filter operation. W is the weight map of each component of the n th image. $W^{\prime }$ is the refined weight map. r is the size of the filter window. ε denotes the regularization parameter, which determines the blur degree of the filter. According to the experiments, r = 3 and ε = 0.15.

3.2.4 Fusion framework

The final enhanced image version J is obtained by fusing the defined inputs with the weight measures at every pixel location, which is defined as:

$$ J = f\left ({W}^{\prime},I_{C},I_{L} \right )= W_{C}^{\prime}\cdot I_{C}+W_{L}^{\prime}\cdot I_{L} $$

(11)

where I_C and I_L are the input images, $W_{C}^{\prime }$ and $W_{L}^{\prime }$ are the weight maps. The weights are normalized, and $W_{C}^{\prime }+W_{L}^{\prime }=1$. For clarity, the detailed calculation steps of the proposed method are outlined in Algorithm 2.

4 Experimental results

In this section, we test various underwater images to examine the effectiveness of the proposed approach. Test images are selected from underwater image enhancement benchmark dataset (UIEBD) [36] and real-world underwater image enhancement (RUIE) [38] dataset. The UIEBD contains 890 degraded underwater images and 60 challenging data. RUIE is composed of three subsets, which contains a total of 3930 challenged images of marine life captured in real marine aquaculture environment.

To verify the performance of the AUWB method, we conduct comparative experiments with several traditional color constancy techniques, namely, Grey Edge [58], Shades of Grey [18], Max RGB [31] and Gray World [10]. Furthermore, the performance of the natural-based fusion method is compared with that of several underwater image enhancement/restoration methods, namely, UDCP [15], Histogram prior [33], Haze Line [6], UIBLA [45], GDCP [44], fusion-based (Ancuti2012) [3], fusion-based (Ancuti2018) [4], UGAN [17], UWCNN [32], FUnIE-GAN [28].

4.1 Parameters

4.1.1 Parameter settings

In this work, several parameters are initialized manually. They are the image patches N, the number of particles m, the maximum number of training iterations t_max, the inertia factor ω, the positive constants c₁ and c₂, the random numbers r₁ and r₂.

In the analysis done by Bratton and Kennedy [14], it was reported that under testing, a population size between 20-100 particles produced better results for the test problems. Further, in [41], this argument was also confirmed. So in this work, m is set to be 30.

The inertia weight ω is added to obtain a better balance between global exploration and local exploitation of PSO approach. A larger value of ω promotes a global exploration (searching new areas), while a smaller value encourages a local exploitation [51]. It was found that linearly decreasing the inertia weight from 0.9 to 0.4 produces good results [50]. In this work, ω is set to be 0.8.

c₁ and c₂ are learning factors. The cognitive acceleration (c₁) starts with a higher value and linearly decreases, while the social acceleration (c₂) starts with a lower value and linearly increases. The ranges of the values are the following: c₁ decreases linearly from 2.5 to 0, and c₂ increases linearly from 0 to 2.5. In this work, both c₁ and c₂ are set to be 2. r₁ and r₂ are random numbers in [0,1].

N is the number of image patches. When an image is divided into 400 patches, the fused image shows a pleasant result. So in this work, N is set to be 400.

4.1.2 Parameter analysis

The maximum number of iterations is highly problem-dependent. 400 patches of one original image have been tested to determine the appropriate iteration value. It can be seen from Fig. 5, the algorithm is converged before 30 times. So in this work, t_max is set to be 30.

Fig. 5

The iteration curve of average fitness value on image with 400 patches. The x-axis represents the number of iteration, the y-axis represents the average fitness value (£) of all patches

4.2 Underwater dehazing evaluation

4.2.1 Qualitative comparison

Figure 6 presents the results obtained on seven different representative underwater images in the UIEBD through several recent dehazing methods. Experiments show that the effects of attenuation can be successfully eliminated by proposed method and the visibility of raw underwater images is effectively improved. Although qualitative comparison also shows that most of the above methods can well complete the task of underwater image enhancement, but due to the accurate estimation of the weight maps and the excellent performance of the white balance, the proposed technique can restore most scene details and maintain a natural appearance.

Fig. 6

Subjective comparisons on underwater images from UIBED [36] testing set. From left to right are original images, and the results generated by UDCP [15], Histogram Prior [33], Haze Line [6], UIBLA [45], GDCP [44], fusion-based (Ancuti2012) [3], fusion-based (Ancuti2018) [4] and the proposed method

As shown in Fig. 6, the result of fusion-based [3] method introduce a significant color shift, due to the inaccurate color correction. The Histogram Prior [33] method produces unsatisfying results with reddish color shift and artifacts. The Haze Line [6] often fail to remove color cast. All the IFM-based methods (e.g., UDCP [15], histogram prior [33] and UIBLA [45]) fail to estimate transmission and veiling light accurately, which results in visually unpleasing restoration results.

4.2.2 Quantitative comparison

For quantitative evaluation, full-reference criteria: peak signal to noise ratio (PSNR), patch-based contrast quality index (PCQI) [52], non-reference criteria: Blur [13], underwater image quality measure (UIQM) [43] are used. For full reference evaluation, we regard the reference images supported by UIEBD as basic facts, and calculate the PSNR and PCQI between the restored result and the corresponding reference.

A higher PSNR value indicates that the result is closer to the reference in terms of image content. The Blur metric [13] evaluates the image quality in terms of blur perception, and the blur metric ranges from 0 to 1, indicating the best to the worst.

PCQI [52] method is used to predict the human perception of contrast variations. A higher PCQI score, the better image contrast. PCQI can be described as:

$$ PCQI=\frac{1}{M}\sum\limits_{i=1}^{M}q_{i}\left (x_{i},y_{i} \right )q_{c}\left (x_{i},y_{i} \right )q_{s}\left (x_{i},y_{i} \right ) $$

(12)

where M means the total number of the patches in the image, q_i, q_c and q_s are three comparison functions.

The UIQM metric [43] is designed specifically to quantify the colorfulness(UICM), sharpness(UISM) and contrast(UIConM) that characterize underwater images. A greater value of the UIQM reprents higher image quality. UIQM can be described as:

$$ UIQM=c_{1}\times UICM+ c_{2}\times UISM+ c_{3}\times UIConM $$

(13)

where c₁, c₂ and c₃ are the scale factors, we set c₁ = 0.0282, c₂ = 0.0953, and c₃ = 3.5753 as the original paper.

Tables 2, 3, 4 and 5 represent the evaluation values of the enhanced results shown in Fig. 6 in terms of PSNR, Blur, PCQI and UIQM, respectively. The red value indicates the best result, the blue value indicates the second best. As described in these four tables, the proposed method performs better in most cases.

Table 2 Underwater dehazing evaluation based on PSNR

Table 3 Underwater dehazing evaluation based on Blur

Table 4 Underwater dehazing evaluation based on PCQI

Table 5 Underwater dehazing evaluation based on UIQM

4.3 Naturalness-preserving analysis

For underwater image enhancement based on fusion, the fused results with naturalness are important and highly desired for many practical applications (such as object recognition, object tracking, etc.). However, The fused results obtained by using existing algorithms often have artifacts. In this section, we compare the fused results of the proposed technique with previous state-of-the-art fusion-based methods on the several challenging underwater images, as shown in Fig. 7.

Fig. 7

Experimental results about naturalness preserving among the images yielded by different fusion-based mathods. (a) Original image, (b) Fusion-based method (Ancuti2012) [3], (c) Fusion-based method (Ancuti2018) [4], (d) Global-based fusion, (e) Ours

From Fig. 7, it can be seen that the proposed technique is characterized by higher robustness for extreme underwater cases (such as non-uniform artificial illumination and turbid sea water). Fusion-based methods [3, 4] tend to introduce artifacts, over-enhancement and color cast (for example, reddish color). Global-based fusion results are obtained by setting the number of patches N to 1. The results show that the global-based fusion method has also been over-enhanced. In contrast, as displayed in enlarged regions, our algorithm thoroughly removes haze and naturally restores contrast.

5 Conclusion

In this paper, we propose an efficient natural-based fusion method for underwater image dehazing. Firstly, the adaptive underwater image white balance (AUWB) method is employed to reduce the color deviation, which is based on a statistical prior that the green channel is relatively well preserved under water. Secondly, to solve the problems of low contrast, a novel natural-based fusion method is proposed, which combines the result images of global contrast-enhanced and local detail-improved. And in this procedure, we modeled weight map estimation as an optimization problem. Particle swarm optimization (PSO) is used to solve it, which promotes more accurate transfer of contrast improvement and detail enhancement to the fused image. The qualitative and quantitative results demonstrate that the proposed method achieves better natural visual quality with more valuable information and higher contrast. In addition, the proposed technique still suffers from low computational efficiency, and we will further explore the solutions to this drawback in future work.