1 Introduction

In underwater image acquisition process [35], some quality problems generally occur in captured images, which are derived from many factors. For example, the light direction inevitably varies because scattering always exists. Likewise, the light energy level can not keep on an even keel due to a portion of light is absorbed by objects. In addition, the maximum attenuation of red light in the underwater environment always results in color distortion [1]. Degraded underwater images have many limitations for oceanographic survey, ocean environments exploitation, and underwater objection detection [24, 33]. Therefore, underwater image processing is full of challenging owing to the specific physical characteristics of underwater environment. It is hence essential to develop an effective technology which allows for efficient and robust restoration for underwater images.

Many attempts to automate underwater image enhancement and restoration were proposed in recent decades. Some traditional image enhancement algorithms, such as histogram equalization [17] and probability-based method [10] can efficiently improve visual quality. However, these methods are not relying on the  underwater image formation model (IFM) and the principle of underwater image quality degradation. Thus, the restored results can not meet our special demands under different challenge scenes. Among most of the IFM based algorithms, the dark channel prior (DCP) proposed by He et al. [16] was originally applied to estimate the transmission map and global background light. Afterward, various researches for underwater image dehazing [15, 4, 20, 29, 26] derived from DCP were published. Unfortunately, the traditional DCP is not completely suitable for underwater environment due to the maximum attenuation of red light. In [16], He et al. used the dark channel to obtain the air light, and selected the top 0.1% pixels with the largest brightness in the dark channel. However, the global background light of underwater image was easily interfered by white objects in water, which made the background light value estimated too high. To address this problem, Wen et al. [44] introduced an underwater dark channel prior (UDCP), which only need the information from blue and green channels. While DCP and UDCP could increase the effect of green and blue and deepen the background. Galdran et al. [11] inverted red channel and proposed a red channel method to recover color associated to short wavelengths. However, the Galdran’s method based algorithms didn’t pay attention to denoising task and even increased the noise level in results. This leaded to different levels of noise remained in the restored results, which made it more difficult for image analysis.

It is well known that the estimations of transmission map and global background light play crucial roles in obtaining a haze-free underwater image. Carlevaris-Bianco et al. [2] proposed a simple and effective maximum intensity prior (MIP) to remove the effects of light scattering. They estimated the depth scene by exploiting maximum intensity in attenuation between the three RGB channels. Peng et al. [32] employed image blurriness and light absorption to improve its accuracy in estimating the transmission map and background light. All these method could recover images with good performance validated by objective evaluations, but without any consideration about the texture and edge information of the image. Moreover, most underwater restoration methods only focused on how to improve the estimation of transmission map and background light, while ignored the influence of noise appeared in the underwater image, resulting in noise amplification and valuable information loss. Therefore, techniques in terms of underwater image dehazing and denoising should be an important tool in underwater applications.

Recently, variational methods based on partial differential equations and differential geometry have been widely and excellently applied in image processing, especially in image denoising [45, 3, 47, 43], dehazing [12, 42, 8, 19], segmentation [39, 41, 7, 28], and inpainting [48, 6]. The initial variational model was proposed by Tikhonov [40] for noise reduction, which was composed of data term and regularization term. The Data term indicated the relationship between denoising image and original image, and the regularization term can control the smoothness on the restored image by introducing the first or second derivative. In 1992, Osher et al. [34] improved the Tikhonov model by employing total variation (TV) term as the regularizer that can produce excellent edge preservation results. In [23], Kimmel et al. proposed a variational Retinex framework motivated by Retinex theory. The proposed formulation unified the previous different Retinex methods. Fang et al. [8] proposed a new fast variational approach for atmospheric images dehazing and denoising. They utilized the window adaptive algorithm to estimate the transmission map and further employed multiple-splitting Chambolle-Pock algorithm without inner loop to boost its efficiency. The TV regularizer approximated the piecewise constant function with a good performance in noise suppression, but the problem of ‘staircase effect’ existed in image region. To resolve this limitation, Shu et al. [36] presented a high-order total generalized variation-regularized variational (TGV) model. Although TGV alleviates the weakness of TV, it may blur the details of the inflection point. Hou et al. [18] solved the problem of non-uniform illumination utilizing the nonlocal differential operator, their proposed L1-based nonlocal total variational (NLTVL1) model was derived from variational framework for Retinex. In [38, 37, 49], the curvature regularizer was successfully used to maintain the inflection point feature. The current variational restoration methods are all based on the propagation process of light in the atmosphere.

Motived by total variational methods [42, 8] for dehazing fog-degraded image, we propose a novel variational approach with total curvature and total variation based on underwater image formation model. The main contributions of the proposed method include:

  1. (i)

    We successfully incorporate the underwater image formation model into variational image restoration framework;

  2. (ii)

    The data term and the regularization term of the unified variational model are designed deriving from variational Retinex model;

  3. (iii)

    The alternating direction method of multipliers (ADMM) is further intentionally designed for the main challenge of the non-smooth problem and computation efficiency.

The rest of our work is organized as follows. The underwater optical image formation model is briefly introduced in Section 2. Section 3 describes the proposed model and its numerical solution of ADMM algorithm. The qualitative and quantitative evaluation results are given in Section 4. Concluding remarks are presented in Section 5.

2 Underwater optical image formation model

The proposed method is based on the simplified underwater optical image formation model [30, 21], which is given as

$$ I\left(x,y\right)=J\left(x,y\right)\cdot t\left(x,y\right)+B\cdot \left(1-t\left(x,y\right)\right) $$
(1)

where I(x, y) represents the observed underwater image, J(x, y) is the recovered image, t(x, y) is the underwater transmission map, and B denotes the underwater global background light. Given a known underwater image I(x, y), we need to obtain the information about the underwater transmission map (UTM) and underwater global background light (UGBL) to restore the latent image.

Empirically, the value of underwater global background light B is usually replaced with the brightest pixel in the image I(x, y). However, in most cases, the brightest pixel in foreground will be mistaken as the UGBL. In [25], Li et al. employed a hierarchical searching technique based on quad-tree subdivision [22] to estimate the UGBL. Unfortunately, in fact the estimated UGBL is not the exact brightest pixel in the image since some pixels with smaller intensity values will be included in the candidate area. In order to tackle this problem, we first pick top 0.1% brightest pixels in each dark channel as the candidate pixels, next the value with the largest difference between red and blue dark channels are selected as the underwater global background light, which ensures its robustness and accuracy. That's because the energy of red light is absorbed largely, followed by the green and blue light.

After the UGBL is determined, the recovered underwater image subsequently depends on the estimation of underwater transmission map. In most cases, although DCP method is the most popular and effective method to estimate UTM, it is not suitable for underwater scene. Fortunately, a novel underwater dark channel prior (UDCP) is specially designed for underwater image dehazing. The definition of UDCP is expressed as

$$ {J}^{udark}(x)=\underset{y\in \varOmega (x)}{\min}\left(\underset{c\in \left\{G,B\right\}}{\min}\left({J}^c(y)\right)\right) $$
(2)

where Ω(x) refers to a window centered on the pixel x, c represents the color channels c ∈ {G, B}.

Practically, red color associated to shortest wavelength have the fastest attenuation rate. In our work, we employ a red channel technique [11] to first estimate the UTM of red channel, the expression is as follow.

$$ t{(x)}^R=1-\omega \underset{y\in \varOmega (x)}{\min}\left(\min \frac{I_R}{B_R}\right) $$
(3)

where R denotes the red channel, ω is a constant (0 <ω≤ 1) controlling the level of the dehazing. Then we can obtain the UTMs of green and blue channels. The light is set to be uniform based on the Lambert-Beer law [13], the underwater transmission map t is rewritten as

$$ {t}_{\lambda }(x)={e}^{\left(-{\beta}^cd(x)\right)} $$
(4)

where βc is the attenuation coefficient of different channel, d(x) is the depth distance.

According to [14], the linear relationship between attenuation coefficient and wavelengths of light can be expressed as

$$ \Big\{{\displaystyle \begin{array}{l}\frac{\beta^G}{\beta^R}=\frac{\beta^R\left(-0.00113{\lambda}^G+1.62517\right)}{B^G\left(-0.00113{\lambda}^R+1.62517\right)}\\ {}\frac{\beta^G}{\beta^R}=\frac{\beta^R\left(-0.00113{\lambda}^G+1.62517\right)}{B^G\left(-0.00113{\lambda}^R+1.62517\right)}\end{array}} $$
(5)

where \( \frac{\beta^G}{\beta^R} \) and \( \frac{\beta^B}{\beta^R} \) represent the G-R and B-R attenuation coefficient ratios, respectively. In general, the values of βR, βG and βB are empirically selected with 620 nm, 540 nm, and 450 nm, respectively. Combining with Eq. (4) and Eq. (5), we can obtain that

$$ \Big\{{\displaystyle \begin{array}{l}{t}^G(x)={t}^R{(x)}^{\frac{\beta^G}{\beta^R}}\\ {}{t}^B(x)={t}^R{(x)}^{\frac{\beta^G}{\beta^R}}\end{array}} $$
(6)

3 A new formulation with developed ADMM algorithm

A novel variational model with curvature regularization integrated with the underwater image formation model will be described in this section. This section starts with how we achieve this novel model under classic variational framework by reformulating variables based on underwater optical image formation model, followed by implementation procedure of the ADMM algorithm.

Obviously, Eq. (1) can be easily rearranged as

$$ \left(B-J\left(x,y\right)\right)=\frac{1}{t\left(x,y\right)}\cdot \left(B-I\left(x,y\right)\right) $$
(7)

In order to change product form into add form, Eq. (7) can be converted to

$$ \log \left(B-J\left(x,y\right)\right)=\log \frac{1}{t\left(x,y\right)}+\log \left(B-I\left(x,y\right)\right) $$
(8)

For clarity, we set s = log(B − J(x, y)), \( r=\log \left(\frac{1}{t\left(x,y\right)}\right) \), l = log(B − I(x, y)), then Eq. (8) can be rewritten as

$$ s=r+l $$
(9)

In [42, 8], the variational model has been successfully combined with the foggy image formation model, inspired by them, we minimize the following energy functional to recover the image s.

$$ E\left(r,s\right)=\alpha {\int}_{\varOmega}\mid \nabla \cdot \frac{\nabla r}{\mid \nabla r\mid}\mid dx+\beta {\int}_{\varOmega}\mid \nabla s\mid dx+\frac{1}{2}{\int}_{\varOmega }{\left|s-l-r\right|}^2 dx+\frac{\lambda }{2}{\int}_{\varOmega }{\left|r-{r}_0\right|}^2 dx $$
(10)

where curvature item \( \nabla \cdot \frac{\nabla r}{\mid \nabla r\mid } \) and TV item ∣ ∇ s∣ are the regularization items, which keep the details of underwater recovered image while removing noise, ∣r − r0∣ control the similarity of r and initial r0. The data term |s − l − r|2 maintains the fidelity between the restored image and original degraded image. α, β, λ are positive parameters wherein α and β determine the degree of denoising, their values are empirically selected with α, β ∈ [1 − 10], λ plays a binding role on r and the larger values indicate more important on the corresponding items, its value is application-based. After testing experiment, the value of λ ∈ [0.8 − 1.5] can satisfy the various underwater challenge scenes. \( {r}_0=\log \left(\frac{1}{t_0}\right) \) represents the initial value of r.

The problem of Eq. (10) is mathematically ill-posed, which can be resolved by using the ADMM algorithm. Its implementation for the proposed variational model is given below.

First, we introduce the auxiliary variables \( \overrightarrow{p}=\nabla r \),\( \overrightarrow{n}=\frac{\overrightarrow{p}}{\mid \overrightarrow{p}\mid } \),\( q=\nabla \overrightarrow{n} \),\( \overrightarrow{w}=\nabla s \),\( \overrightarrow{m}\approx \overrightarrow{n} \). Based on this observation, Eq. (10) can be transformed into the following iterative optimization form:

$$ \left(r,s,\overrightarrow{p},q,\overrightarrow{n},\overrightarrow{m},\overrightarrow{w}\right)=\mathrm{argmin}\left\{\begin{array}{l}\alpha {\int}_{\varOmega}\mid q\mid dx+\frac{1}{2}{\int}_{\varOmega }{\left|s-l-r\right|}^2 dx+{\int}_{\varOmega }{\sigma}_1^k\left(|\overrightarrow{p}-\overrightarrow{p}\cdot \overrightarrow{m}|\right) dx\\ {}+\frac{\mu_1}{2}{\int}_{\varOmega }{\left(\overrightarrow{p}-\overrightarrow{p}\cdot \overrightarrow{m}\right)}^2 dx+{\int}_{\varOmega }{\overrightarrow{\sigma}}_2^k\left(\overrightarrow{p}-\nabla r\right) dx+\frac{\mu_2}{2}{\int}_{\varOmega }{\left(\overrightarrow{p}-\nabla r\right)}^2 dx\\ {}+{\int}_{\varOmega }{\sigma}_3^k\left(q-\nabla \overrightarrow{n}\right) dx+\frac{\mu_3}{2}{\int}_{\varOmega }{\left(q-\nabla \overrightarrow{n}\right)}^2 dx+{\int}_{\varOmega }{\overrightarrow{\sigma}}_4^k\left(\overrightarrow{n}-\overrightarrow{m}\right) dx\\ {}+\frac{\mu_4}{2}{\int}_{\varOmega }{\left(\overrightarrow{n}-\overrightarrow{m}\right)}^2 dx+\beta {\int}_{\varOmega}\mid \overrightarrow{w}\mid dx+{\int}_{\varOmega }{\overrightarrow{\sigma}}_5^k\left(\overrightarrow{w}-\nabla s\right) dx\\ {}+\frac{\mu_5}{2}{\int}_{\varOmega }{\left(\overrightarrow{w}-\nabla s\right)}^2 dx+\frac{\lambda }{2}{\int}_{\varOmega }{\left(r-{r}_0\right)}^2 dx\end{array}\right\} $$
(11)

where σ1,\( {\overrightarrow{\sigma}}_2 \),\( {\sigma}_3,{\overrightarrow{\sigma}}_4 \),\( {\overrightarrow{\sigma}}_5 \) are the Lagrange multipliers, and μ1, μ2, μ3, μ4, μ5 are positive parameters to compensate for the five Lagrange items. k represents the step number of iterations.

Then the minimization problem (11) can be decomposed into eight subproblems and each sub-problem of minimization is carried out with respect to one variable while keeping other variables constant temporarily. Their detailed solving procedures are given step-by-step as follows.

Step 1: fix \( s,q,\overrightarrow{p},\overrightarrow{n},\overrightarrow{m},\overrightarrow{w} \) to solve r

$$ {\varepsilon}_1(r)=\min \left\{\begin{array}{l}E(r)=\frac{1}{2}{\int}_{\varOmega }{\left(s-l-r\right)}^2 dx+{\int}_{\varOmega }{\overrightarrow{\sigma}}_2^k\left(\overrightarrow{p}-\nabla r\right) dx\\ {}+\frac{\mu_2}{2}{\int}_{\varOmega }{\left(\overrightarrow{p}-\nabla r\right)}^2 dx+\frac{\lambda }{2}{\int}_{\varOmega }{\left|r-{r}_0\right|}^2 dx\end{array}\right\} $$
(12)

The Euler-Lagrange (E-L) equation of Eq. (12) is

$$ r+l-s+\nabla {\overrightarrow{\sigma}}_2+{\mu}_2\nabla \left(p-\nabla r\right)+\lambda \left(r-{r}_0\right)=0 $$
(13)

The final r can be obtained using Jacobi iteration method:

$$ {r}_{i,j}^{k+1}=\frac{s^k-{l}^k-\nabla {\overrightarrow{\sigma}}_2^k-{\mu}_2\nabla {\overrightarrow{p}}^k+{\mu}_2{R}^k+\lambda {r}_0}{1+4{\mu}_2+\lambda } $$
(14)

where \( R={r}_{i+1,j}^k+{r}_{i-1,j}^k+{r}_{i,j+1}^k+{r}_{i,j-1}^k-4{r}_{i,j}^k \)

Step 2: fix \( r,\overrightarrow{p},q,\overrightarrow{n},\overrightarrow{m},\overrightarrow{w} \)to solve s:

$$ {\varepsilon}_2(s)=\min \left\{E(s)=\frac{1}{2}{\int}_{\varOmega }{\left(s-l-r\right)}^2 dx+{\int}_{\varOmega }{\overrightarrow{\sigma}}_5^k\left(\overrightarrow{w}-\nabla s\right) dx+\frac{\mu_5}{2}{\int}_{\varOmega }{\left(\overrightarrow{w}-\nabla s\right)}^2 dx\right\} $$
(15)

Its Euler-Lagrange (E-L) equation is

$$ s-l-r+\nabla {\overrightarrow{\sigma}}_5+{\mu}_5\nabla \overrightarrow{w}-{\mu}_5\varDelta s=0 $$
(16)

Then

$$ {s}^{k+1}=\frac{\mu_5S+r+l-\nabla {\overrightarrow{\sigma}}_5^k-{\mu}_5\nabla \overrightarrow{w}}{1+4{\mu}_5} $$
(17)

where \( S={s}_{i+1,j}^k+{s}_{i-1,j}^k+{s}_{i,j+1}^k+{s}_{i,j-1}^k-4{s}_{i,j}^k \)

Step 3: fix \( r,\overrightarrow{p},s,\overrightarrow{n},\overrightarrow{m},\overrightarrow{w} \) to solve q

$$ {\varepsilon}_3(q)=\min \left\{E(q)=\alpha {\int}_{\varOmega }|q| dx+{\int}_{\varOmega }{\sigma}_3^k\left(q-\nabla \overrightarrow{n}\right) dx+\frac{\mu_3}{2}{\int}_{\varOmega }{\left(q-\nabla \overrightarrow{n}\right)}^2 dx\right\} $$
(18)

Based on the generalized soft threshold formulation (GSTF), Eq. (18) can be solved as

$$ {q}^{k+1}=\max \left(|\nabla {\overrightarrow{n}}^k-\frac{\sigma_3^k}{\mu_3}|-\frac{\alpha }{\mu_3},0\right)\frac{\nabla {\overrightarrow{n}}^k-\frac{\sigma_3^k}{\mu_3}}{\mid \nabla {\overrightarrow{n}}^k-\frac{\sigma_3^k}{\mu_3}\mid },0\frac{\overrightarrow{0}}{\mid \overrightarrow{0}\mid }=0 $$
(19)

Step 4: fix \( r,s,q,\overrightarrow{n},\overrightarrow{m},\overrightarrow{w} \) to solve \( \overrightarrow{p} \)

$$ {\varepsilon}_4\left(\overrightarrow{p}\right)=\min \left\{\begin{array}{l}E\left(\overrightarrow{p}\right)={\int}_{\varOmega }{\sigma}_1^k\left(|\overrightarrow{p}|-\overrightarrow{p}\cdot \overrightarrow{m}\right) dx+\frac{\mu_1}{2}{\int}_{\varOmega}\left(|\overrightarrow{p}|-\overrightarrow{p}\cdot \overrightarrow{m}\right) dx\\ {}+{\int}_{\varOmega }{\overrightarrow{\sigma}}_2^k\left(\overrightarrow{p}-\nabla r\right) dx+\frac{\mu_2}{2}{\int}_{\varOmega }{\left(\overrightarrow{p}-\nabla r\right)}^2 dx\end{array}\right\} $$
(20)

Its GSTF is

$$ {{\overrightarrow{p}}_i}^{k+1}=\mathit{\max}\left(|\nabla r-\frac{{\overrightarrow{\sigma}}_2^k}{\mu_2}+\frac{\left({\sigma}_1^k+{\mu}_1\right){\overrightarrow{m}}^k}{\mu_2}|-\frac{\sigma_1^k+{\mu}_1}{\mu_2},0\right)\frac{\nabla r-\frac{{\overrightarrow{\sigma}}_2^k}{\mu_2}+\frac{\left({\sigma}_1^k+{\mu}_1\right){\overrightarrow{m}}^k}{\mu_2}}{\mid \nabla r-\frac{{\overrightarrow{\sigma}}_2^k}{\mu_2}+\frac{\left({\sigma}_1^k+{\mu}_1\right){\overrightarrow{m}}^k}{\mu_2}\mid },0\frac{\overrightarrow{0}}{\mid \overrightarrow{0}\mid }=0 $$
(21)

Step 5: fix \( r,s,q,\overrightarrow{p},\overrightarrow{m},\overrightarrow{w} \) to solve \( \overrightarrow{n} \)

$$ {\varepsilon}_5\left(\overrightarrow{n}\right)=\min \left\{\begin{array}{l}E\left(\overrightarrow{n}\right)={\int}_{\varOmega }{\sigma}_3^k\left(q-\nabla \overrightarrow{n}\right) dx+\frac{\mu_3}{2}{\int}_{\varOmega }{\left(q-\nabla \overrightarrow{n}\right)}^2 dx\\ {}+{\int}_{\varOmega }{\overrightarrow{\sigma}}_4^k\left(\overrightarrow{n}-\overrightarrow{m}\right) dx+\frac{\mu_4}{2}{\int}_{\varOmega }{\left(\overrightarrow{n}-\overrightarrow{m}\right)}^2 dx\end{array}\right\} $$
(22)

Its E-L equation is

$$ \nabla {\sigma}_3+{\mu}_3\nabla \left(q-\nabla \overrightarrow{n}\right)+{\overrightarrow{\sigma}}_4+{\mu}_4\left(\overrightarrow{n}-\overrightarrow{m}\right)=0 $$
(23)

Given that

$$ \Big\{{\displaystyle \begin{array}{l}N1={\mu}_3\left(n{1}_{i+1,j}^k+n{1}_{i-1,j}^k+\frac{1}{4}\left(n{2}_{i+1,j+1}^k+n{2}_{i-1,j-1}^k-n{2}_{i+1,j-1}^k-n{2}_{i-1,j+1}^k\right)\right)\\ {}N2={\mu}_3\left(n{2}_{i,j+1}^k+n{2}_{i,j-1}^k+\frac{1}{4}\left(n{1}_{i+1,j+1}^k+n{1}_{i-1,j-1}^k-n{1}_{i+1,j-1}^k-n{1}_{i-1,j+1}^k\right)\right)\end{array}} $$
(24)

The solution of \( \overrightarrow{n} \) is

$$ \Big\{{\displaystyle \begin{array}{l}\overrightarrow{n}{1}^{k+1}=\frac{\left(N1+{\mu}_4\left({\overrightarrow{m}}_1\right)-{\overrightarrow{\sigma}}_{41}-\frac{\partial {\sigma}_3}{\partial x}-{\mu}_3\left(\frac{\partial q}{\partial x}\right)\right)}{\left(2{\mu}_3+{\mu}_4\right)}\\ {}\overrightarrow{n}{2}^{k+1}=\frac{\left(N2+{\mu}_4\left({\overrightarrow{m}}_2\right)-{\overrightarrow{\sigma}}_{42}-\frac{\partial {\sigma}_3}{\partial y}-{\mu}_3\left(\frac{\partial q}{\partial y}\right)\right)}{\left(2{\mu}_3+{\mu}_4\right)}\end{array}} $$
(25)

Step 6: fix \( r,s,q,\overrightarrow{p},\overrightarrow{n},\overrightarrow{w} \) to solve \( \overrightarrow{m} \)

$$ {\varepsilon}_6\left(\overrightarrow{m}\right)=\min \left\{\begin{array}{l}E\left(\overrightarrow{m}\right)={\int}_{\varOmega }{\sigma}_1^k\left(|\overrightarrow{p}|-\overrightarrow{p}\cdot \overrightarrow{m}\right) dx+\frac{\mu_1}{2}{\int}_{\varOmega}\left(|\overrightarrow{p}|-\overrightarrow{p}\cdot \overrightarrow{m}\right) dx\\ {}+{\int}_{\varOmega }{\overrightarrow{\sigma}}_4^k\left(\overrightarrow{n}-\overrightarrow{m}\right) dx+\frac{\mu_4}{2}{\int}_{\varOmega }{\left(\overrightarrow{n}-\overrightarrow{m}\right)}^2 dx\end{array}\right\} $$
(26)

Its E-L equation is

$$ {\overrightarrow{m}}^{k+1}=\frac{{\overrightarrow{\sigma}}_1^k{\overrightarrow{p}}^k+\frac{1}{2}\left({\mu}_1{\overrightarrow{p}}^k\right)}{\mu_4}+{\overrightarrow{n}}^k $$
(27)

In order to meet the constraint\( \mid \overrightarrow{m}\mid \le 1 \), the following projection formation is imposed on\( \overrightarrow{m} \):

$$ {\overrightarrow{m}}^{k+1}=\frac{{\overrightarrow{m}}^k}{\max \left(|{\overrightarrow{m}}^k|,1\right)} $$
(28)

Step 7: fix \( r,s,q,\overrightarrow{p},\overrightarrow{m},\overrightarrow{n} \)and solve\( \overrightarrow{w} \):

$$ {\varepsilon}_7\left(\overrightarrow{w}\right)=\min \left\{E\left(\overrightarrow{w}\right)=\beta {\int}_{\varOmega }|\overrightarrow{w}| dx+{\int}_{\varOmega }{\overrightarrow{\sigma}}_5^k\left(\overrightarrow{w}-\nabla s\right) dx+\frac{\mu_5}{2}{\int}_{\varOmega }{\left(\overrightarrow{w}-\nabla s\right)}^2 dx\right\} $$
(29)

Its GSTF is

$$ {\overrightarrow{w}}^{k+1}=\max \left(|\nabla {s}^k-\frac{{\overrightarrow{\sigma}}_5^k}{\mu_5}|-\frac{\beta }{\mu_5},0\right)\frac{\nabla {s}^k-\frac{{\overrightarrow{\sigma}}_5^k}{\mu_5}}{\mid \nabla {s}^k-\frac{{\overrightarrow{\sigma}}_5^k}{\mu_5}\mid },0\frac{\overrightarrow{0}}{\mid \overrightarrow{0}\mid }=0 $$
(30)

Step 8: update the Lagrange multipliers

$$ \Big\{{\displaystyle \begin{array}{l}{\sigma}_1^{k+1}={\sigma}_1^k+{\mu}_1\left(|{\overrightarrow{p}}^k-{\overrightarrow{p}}^k\cdot {\overrightarrow{m}}^k|\right)\\ {}{\overrightarrow{\sigma}}_2^{k+1}={\overrightarrow{\sigma}}_2^k+{\mu}_2\left({\overrightarrow{p}}^k-\nabla {r}^k\right)\\ {}{\sigma}_3^{k+1}={\sigma}_3^k+{\mu}_3\left({q}^k-\nabla {\overrightarrow{n}}^k\right)\\ {}{\overrightarrow{\sigma}}_4^{k+1}={\overrightarrow{\sigma}}_4^k+{\mu}_4\left({\overrightarrow{n}}^k-{\overrightarrow{m}}^k\right)\\ {}{\overrightarrow{\sigma}}_5^{k+1}={\overrightarrow{\sigma}}_5^k+{\mu}_5\left({\overrightarrow{w}}^k-\nabla {s}^k\right)\end{array}} $$
(31)

For clarity, the overall algorithm framework for solving the proposed variational model is summarized as follows.

Algorithm 1

The computational procedure of the proposed method.

figure f

4 Experimental results and discussion

In this section, comparison experiments conducted on real underwater images and the analysis of the influence of experimental parameters are carried out, respectively. In the qualitative and quantitative comparisons, the state-of-the-art methods used for comparison include contrast limited adaptive histogram equalization (CLAHE), multi-scale Retinex with color restoration (MSRCR), dark channel prior (DCP) [16], wavelength compensation and image dehazing (WCID) [4], and automatic red-channel (ARC) [11], multi-scale correlated wavelet (MSCW) [27], Kimmel’s variational framework for Retinex [23]. All the experiments are implemented in MATALAB R2016b on an Intel 2.8GHz PC with 4GB RAM.

4.1 Qualitative assessment

First, we present the recovered results compared with the original ones to demonstrate the performance of the proposed method, as shown in Fig. 1. Figure 1 (a) shows three underwater degraded images suffer from low contrast, haze and noise. Figure 1 (b) and (c) present the result of proposed method and the corresponding energy curve verse number of iterations. It can be seen that our proposed algorithm achieves a good visual outcome in removing haze, suppress noise and improving contrast. Moreover, its high-efficiency in convergence with fewer iterations is illustrated in Fig. 1 (c). In addition, to evaluate the computational efficient of the proposed method, we set the times of iterations to 5, 10, 20 to restore different size of Fig. 1(a)-1(c). Their calculated runtimes with different iterations are shown in Table 1. It can be concluded that the cost of one iteration of the proposed method is around 0.1 s to 0.3 s, which achieves acceptable outcomes of computational efficiency using the fast ADMM algorithm.

Fig. 1
figure 1

Real underwater image tests. (a) raw underwater image, (b) the corresponding recovered results, (c) the energy convergence curve

Full size image
Table 1 Runtime of proposed method
Full size table

As shown in Eq. (10), the proposed energy functional contains three parameters including α, β, and λ. As mentioned before, all of them are free positive parameters to weight the three corresponding terms. In this section, we will analyze the influence of these three parameters. Figure 2 shows the results with different values of parameters α, β, and λ after employing our dehazing and denoising method. In our experiment, we vary one by fixing another two parameters.

Fig. 2
figure 2

The results with different α, β, λ in (a), (b), (c), respectively

Full size image

First, we set the constant values of β = 7 and λ = 1, and change α from10−4 to 104. From Fig. 2 (a), we can see that some valuable information will be missing when α is too small or too large, especially in the shaded areas. Figure 2 (b) displays the recovered results when we vary the values of β. Here, we fix α = 7, λ = 1 and change β from 10−4 to 104. Visually, the results are basically similar no matter when β increases and β decreases. Thus, our algorithm is minimally affected by varying the parameter β. Next, we study the effect of λ. Figure 2 (c) presents the result with different λ by fixing α = 7 and β = 7. Obviously, the parameter λ determines the dehazing level. We can observe that the results have less haze and become darker when λ increases. It should be noted that a larger λ will produce a poorer outcome. In our test, we find that λ ∈ [0.8 − 1.5] could be more suitable.

For comparison purposes, we choose six typically underwater images captured in different challenging conditions, such as hazy, fuzzy, color faded, nonuniform illumination, and noise, as shown in Fig. 3(a). Figure 3(b)-(h) lists the corresponding restored results after employ CLAHE, MSRCR, DCP, WCID, ARC, MSCW, Kimmel’s Variational Retinex and the proposed method, respectively. As we can observe from Fig. 3(b), CLAHE method achieves a good improvement in color correction and contrast enhancement. However, CLAHE algorithm also has a problem of over-amplifying the noise. As shown in Fig. 3(c), the MSRCR method can generate a high color saturation recovered image, but it may generate some overexposed regions in the foreground. Although the contrast and color saturation are enhanced by CLAHE and MSRCR methods, there is still room for improvement in details and visibility due to these methods without considering the underwater optical image formation model (i.e. the details of the men’s hand and head in the results obtained from Fig. 3(a-6) are blurred). From Fig. 3(d) and 3(e), it can be seen that DCP and WCID methods can effectively remove haze because both the algorithms are designed based on image formation model. Nevertheless, the backgrounds of some restored images are too dark owing to inaccurate estimation of background light and they also almost have no effect on improving contrast and reducing noise. Besides, in some cases, WCID method may bring color distortion. Similarly, the recovered results in Fig. 3(f) shows that the ARC method has a poor performance on removing haze. The MSCW method successes in dehazing and enhancing contrast, but the exposure operation may lead to a non-uniform illumination, as show in Fig. 3(g). As shown in Fig. 3(h), in most cases, Kimmel’s method produces unsatisfied results and brings considerable artifacts and noise. Although the variational Retinex method has a good performance on low-light scene. However, they often saturate pixels without taking into account the underwater image information model, leading to no estimation of the underwater background light B and underwater medium transmission map t. Hence, it is not suitable for directly application to underwater image restoration. Fortunately, in Fig. 3(i), we can observe that the proposed method achieves a better visual result, especially in dehazing and denoising.

Fig. 3
figure 3

Qualitative comparison, (a) Original underwater image under different challenging scenes, the corresponding recovered results of (b) CLAHE method, (c) MSRCR method, (d) DCP method, (e) WCID method, (f) ARC method, (g) MSCW method, (h) Kimmel’s, (i) the proposed method

Full size image

To further see the detailed superiority of the proposed method, we crop a rectangle area from the image of Fig. 3(a-4). The corresponding zoomed small sub-regions are displayed in Fig. 4. From Fig. 4(b)-(h), the seven methods used for comparison have similar problems in preserving texture features and eliminating non-uniform illumination. In contrast, it can be observed that the edge and valuable information in box are well preserved after using our proposed method from Fig. 4(i).

Fig. 4
figure 4

Zoomed small sub-regions for detail comparison, (a) the original sub-region, (b)-(i) the recovered results of the seven compared methods

Full size image

4.2 Quantitative assessment

To objectively evaluate the superiority of our proposed method, four non-reference metrics namely fog aware density evaluator (FADE) [5], colour quality enhancement (CQE) [31], natural scene statistics (NSS) [9] and the underwater color image quality evaluation metric (UCIQE) [46] are used for quantitative comparison. The FADE metric is used frequently to predict perceptual fog density, which correlates well with the viewpoint of subjective human perception. The indicator CQE is a simple combination of colorfulness, sharpness and contrast in RGB color space. The UCIQE metric is similar to the CQE metric but differs in the combination of chroma, saturation, and contrast in CIELab color space. The NSS metric is often used for assessment of contrast-distorted images.

The calculated results of these metrics on the recovered image in Fig. 3 are illustrated in Tables 2, 3, 4, and 5, respectively. The best outcomes are highlighted in bold format in these tables. From Tables 2, 3, 4, and 5, it can be seen that our dehazing and denoising method outperforms the other seven methods in terms of FADE CQE, NSS, and UCIQE values. The highest FADE values of the proposed methods verify its superiority in defogging, as shown in Table 2. For CQE in Table 3, we can observe that our proposed method achieves highest average value with 1.0383, followed by MSCW method. The average values obtained from MSRCR, DCP, ARC, WCID and Kimmel’s are all less than 1.0. Similarly, as shown in Table 4, the higher NSS score of the proposed method also demonstrate that our method can effectively enhance contrast and reveal more details. In Table 5, we can see that the average UCIQE scores of the seven compared methods are all below 0.6. Badly, Kimmel’s method achieves the lowest value with 0.4202. In contrast, in most cases, the UCIQE values of ours’ are more than 0.6. The best UCIQE value indicates that the recovered underwater images of our proposed method have a good balance in chroma, saturation, and contrast.

Table 2 FADE value of different compared methods
Full size table
Table 3 CQE value of different compared methods
Full size table
Table 4 NSS value of different compared methods
Full size table
Table 5 UCIQE value of different compared methods
Full size table

For further assessment of the denoising effect, variations of noise (i.e. salt & pepper noise with 0.2, Gaussian noise with 0.05, and Poisson noise) are further added to six real underwater images, as shown in Fig. 5. The ratio indicator PSNR is employed for the performance evaluation, the comparisons of PSNR values using different models are presented in Table 6. It should be noted that higher PSNR values mean better results. As shown in the first two columns of Table 6, for salt & pepper noise, all the obtained PSNR values after employing these compared algorithms are lower than their original values. However, our denoising method still get the highest score among them. For Gaussian noise, the lower PSNR values of CLAHE and MSRCR indicate that noise is amplified. For Poisson noise, only DCP and our method can effectively reduce it. Overall, we can conclude that our proposed method stands out among these compared methods in suppressing noise, especially for Gaussian and Poisson noise.

Fig. 5
figure 5

Six real underwater images added with various noise, (a-b) salt & pepper noise added, (c-d) Gaussian noise added, (e-f) Poisson noise added

Full size image
Table 6 PSNR value of different compared methods
Full size table

5 Conclusion

A novel variational approach incorporating curvature regularization and underwater image formation is proposed in this paper, which aims to tackle problems of underwater image dehazing and denoising. The three prominent aspects of our variational framework lie in: (1) the dark channel prior and underwater red channel prior are used as a guidance to provide information of the underwater global background light and the underwater transmission map; (2) Integration with variational model using high-order curvature term and total vatiation term as regularizers, a significant accuracy increase is achieved reflected in the evaluation section; (3) The ADMM algorithm is designed into the calculation procedure for the non-smooth optimization problem, which can also guarantee the convergence and efficiency. Extensive experiments demonstrate that the proposed method not only achieves satisfactory haze-free results, but also possesses capabilities of color balancing and contrast enhancing, especially produces excellent details and edge preservation results. It can be easily observed that all these advantages stand out more through the qualitative and quantitative comparisons with the other seven algorithms.