1 Introduction

Images captured in poor weather conditions may lose their potential information, due to a dirty medium such as particles and water droplets in an atmospheric veil. Thus, these hazy images do not provide enough significant details for future vision applications [57]. Poor illumination decreases the visibility of images. Thus, these images are not suitable for future vision applications such as weather forecasting, radar tracking system, lane detection system, etc. Therefore, these kinds of applications demand haze removal technique as a pre-processing tool to improve the performance of vision applications under poor environmental conditions [24].

The number of particles available in the atmosphere are fluctuate according to the weather condition. An enormous attempt has been made to quantify the size of these particles as shown in Table 1. Depending upon the category of the visual belongings, poor weather circumstances are categorized into two types: Steady and dynamic [24]. In steady poor weather, ingredient droplets are minimum (1–10 μ m) and steadily float in the atmosphere. Haze, mist, and fog are the examples of steady weather conditions. The illumination effect at a given pixel is because of the collective consequence of the high degree of droplets within the pixel’s solid angle. In dynamic poor weather circumstances, ingredient droplets are 1000 times more (0.1–10 μ m) than steady weather [33]. Snow and rain are the examples of dynamic weather circumstances.

Table 1 Weather conditions and associated particle types and sizes
Full size table

The majority of vision applications provide poor results in case of weather degraded images [3]. Thus, haze removal algorithms become significant for several vision applications like aerial imagery, object recognition, image retrieval and object analysis [42].

In hazy days, illumination observed from a scene is sprinkled and immersed because of the considerable occurrence of molecules and aerosols hanging in the environment [4]. Because of the effect of intense haze, the perceptibility of environment and visibility become poorer considerably, that result in significant disturbance to different vision applications [22]. In haze environment, objects have poor visibility [54] and such images are often recognized as of low contrast and poor intensity [53]. Poor brightness considerably influences the consistency of image processing applications [41].

1.1 Imaging under different weather environments

Figures 1 and 2 demonstrate the imaging procedure. In the haze-free circumstances, the object imitates energy from illumination source such as direct sunlight, diffuse skylight and illuminate reflected by the source. The energy of the input scene is reduced when it arrives at the vision system. Vision system integrates the received energy and centers it onto the image plane. Excluding the haze, images have brighter colors as shown in Fig. 1. In hazy circumstances, the situation turns out to be more difficult (as in Fig. 2) [57].

Fig. 1
figure 1

Procedure of imaging under sunny weather [57]

Full size image
Fig. 2
figure 2

Procedure of imaging under hazy weather [57]

Full size image

This review paper has the following structure: Section 2 describes the general framework and mathematical model of the haze removal algorithms. Section 3 describes the comprehensive review on existing well-known haze removal techniques along with their categorization. Comparative analysis of haze removal techniques is given in tabular form in Section 4. In Section 5, the challenges of different haze removal approaches and directions for future research are given. Section 6 describes different quality metrics which can be used to evaluate the performance of haze removal techniques. In Section 7, various applications of haze removal techniques are discussed. The concluding remarks and future scope are presented in Section 8.

2 General framework and mathematical formulation

The existing image enhancement and restoration techniques are not so useful to reduce the effect of haze from hazy images. As known a prior, haze reduces the optical information and thus decreases the accuracy of data analysis. Remotely sensed, underwater and road side images are primarily susceptible to weather effects [22]. The effect of haze increases with the distance, which makes image dehazing a challenging problem. Under poor weather circumstances, the illumination arriving at the visual sensor is harshly sprinkled by the atmosphere [54]. The resulting decay in brightness differs across the object and is exponential in the depth of object points. Thus, traditional space invariant vision processing algorithms are not so effective to reduce haze effects from digital images. Therefore, haze removal technique is required to remove the haze from digital images [41].

Several researchers have shown that the effect of haze in digital images increases when the distance from camera and scene increases [41, 54] and [37]. If an input is just a particular hazy image, then an assessment of the depth knowledge is under certain assumptions. Usually, evaluation of depth map demands two images. Thus, numerous techniques have been developed that may utilize several images [52].

However, these techniques are unable to apply on standard vision framework. There are various techniques that can reduce haze by a single image. To improve the estimation of the depth map, these methods utilize certain constrains [34]. A general structure of the haze removal technique is depicted in Fig. 3.

Fig. 3
figure 3

Generic framework of haze removal techniques

Full size image

The step by step detail of Fig. 3 is demonstrated by using mathematical models of each step. Table 2 represents various symbols with their respective meanings which are utilized to mathematically describe the steps of the generic framework of haze removal techniques.

Table 2 Nomenclature used
Full size table

2.1 Depth map estimation

Figure 4 shows the effect of light on outdoor images. This figure clearly represents that the effect of haze on outdoor images increases as the distance of scene becomes larger. A hazy image formed as shown in Fig. 4 can be mathematically modeled as follows.

Fig. 4
figure 4

Haze imaging model

Full size image

The haze removal techniques demand the estimation of a depth map. The evaluated depth map can be used to evaluate the airlight and transmission map. Many developed methods have predicted the depth map by using the scene characteristics. These features can be shading technique, human visual function, or illuminate based function. Following are some well-known methods which have been used so far by researchers to estimate the depth.

2.1.1 Optical model

The classical hazy image formation model i.e., optical model is proposed by Cozman and Krotkov [8] and given as in (1):-

$$ I_{mg}(k)=J_{sr}(k)M_{tx}(k)+G_{al}(1-M_{tx}(k)) $$
(1)

where I m g is the observed image intensity, J s r is the scene radiance, G a l is the global atmospheric light and k is the pixel position. The medium transmission, M t x , is an exponential function of distance between object and camera, describing the portion of light i.e., not scattered but directly reaches the camera [14]. This model is widely applied in computer vision and the objective of image haze removal is to recover scene radiance J s r from observed image intensity I m g , estimated atmospheric light G a l and transmission M t x .

2.1.2 Refined optical model

Fattal [43] proposed a refined image formation model by replacing the unknown image J s r with a product, S a c × S f , where S a c is the surface albedo coefficient and S f is the shading factor. In this model, both the surface shading and transmission functions are taken into consideration. The ambiguities faced by traditional single image de-hazing methods, were solved by searching for a solution, in which the resultant shading and transmission functions are not similar with each other. Using the same principle, the atmospheric light can be estimated.

2.1.3 Visibility restoration

Instead of calculating image transmission M t x , the atmospheric veil A v e i l (k) = G a l (1 − M t x (k)) was introduced to avoid the separation between the medium extinction coefficient and the scene distance depth [44]. These two factors, which are not always possible to calculate, influence the transmission. Therefore, the image formation model can be rewritten as follows:

$$ I_{mg}(k)={J_{sr}(k)\left( 1-\displaystyle\frac{A_{veil}(k)}{G_{al}}\right)}+ A_{veil}(k) $$
(2)

White balance was initially adopted to set airlight M t x to [111]T and the observed image I m g was normalized. Therefore, the atmospheric veil A v e i l (k), rather than the transmission, is required to be calculated for image restoration.

2.1.4 Dark channel prior

The Dark Channel Prior assumption proposed by He [22] is based on observations about haze-free images, in which there is at least one color channel with some pixels whose intensities are very low or close to zero. In dark channel prior based approaches, the image formation model used is the traditional one (1) and the dark channel J d c is calculated as follows:

$$ J_{dc}=\min_{z \in \mathcal{O}(k)} \left( \min_{cc \in {RGB}} \left( J^{cc}_{sr}\right)\right) $$
(3)

where cc is a color channel of J s r and \(\mathcal {O}(k)\) is a local patch centered at k. The two minimum operators are commutative. Both transmission M t x and atmospheric light G a l can be obtained through the dark channel prior based method.

2.1.5 Learning based color attenuation prior

Instead of searching for the transmission M t x (k), Zhu [57] introduced a novel color attenuation prior to obtaining the scene depth I d e p t h (k). A linear model was employed to relate I d e p t h (k) with scene brightness S b t(k) and saturation I s t (k), which can be mathematically written as follows:

$$ I_{depth}(k)=LC_{0}+ LC_{1} S_{b}(x)+ LC_{2} I_{st}(k)+R_{r}(k) $$
(4)

where L C 0, L C 1 and L C 2 are three unknown linear coefficients, to be obtained through the supervised learning method, R r is the random error of this model and R r can be regarded as a random image. This random error is assigned with a Gaussian density, which gives R r (k) ∈ N(0, σ 2). Furthermore, 500 haze-free images were used in [57] as the training samples, and the Maximum Likelihood Estimation method was adopted to achieve the best learning results, which is L C 0 = 0.12, L C 1 = 0.95, L C 2 = 0.78 and μ = 0.04.

2.2 Depth map refinement

Depth map refinement is a technique which reduces the errors of depth map form haze, noise, poor defined edges or other undesired artifacts. Filtering is a well-known method to refine the depth map. The subsequent section demonstrates the mathematical model of Joint Trilateral Filter (JTF), [37]. As in [37, 44] have set \(A_{viel}^{cc}(k) = G_{al}(1-M_{tx}(k))\) as the transmission veil, M c c (k) = minc c(I c c (k)) is the min color components of I m g (k). As known a prior, 0 ≤ A v e i l (k) ≤ M c c (k), thus for gray scale image, M c c = I m g . JTF [37] have computed the \(T_{xf}(k) = median(k)-J_{\mathcal {O}}^{tf}\left (|M_{cc}- median(k)| \right )\). And then, [37] have acquired it by A v e i l (k) = max ((min (α T x f (k), M c c (k))), 0). Here, α is the parameter in (0,1). Finally, the transmission of each patch can be written as follows:

$$ \overline{\text{M}_{\text{tx}}}= 1-\frac{A_{veil}}{G_{al}^{cc}} $$
(5)

The background \(G_{al}^{cc}\) is usually assumed to be the pixel intensity with the highest brightness value in an image. However, in practice, this simple assumption often renders erroneous results due to the presence of self-luminous organisms. Consequently, the authors [37] computed brightest pixel value among all local min corresponding to the background light \( G_{al}^{cc} \) as follows:

$$ G_{al}=\max_{z \in I_{mg}} \left( \min_{z \in \mathcal{O}(k)} \left( I^{cc}_{z}\right)\right) $$
(6)

where \(I_{mg}^{cc}(z)\) is the local color components of I m g (k) in each patch.

JTF can overcome the gradient reversal artifacts occurring. The filtering process of JTF is first performed under the guidance of the image (G d ). In JTF, input image (I m g ) itself is taken as G d . Let I v a l and G v a l be the intensity values at pixel q of the minimum channel image and guided input image respectively. W r be the kernel window centered at pixel k. JTF is then formulated by follows:

$$ J_{tf} (I_{mg})=\frac{1}{{\sum}_{q \in K_{r}} M_{cc_{pq}}(G_{d})} \left( \sum\limits_{q \in k_{r}} M_{cc_{pq}}(G_{d}) I_{q} \right) $$
(7)

Here, the kernel weights function \( M_{cc_{pq}} (G_{d})\) can be written as follows:

$$ M_{cc_{pq}}(G_{d}) = \frac{1}{{|n|}^{2}} \sum\limits_{n:(p, q) \in k_{r} } \left( 1+ \frac{({G_{d}}_{p}-\mu_{n})({G_{d}}_{q}-\mu_{n})}{{\sigma_{n}^{2}}+\epsilon} \right) $$
(8)

where m u n and \({\sigma ^{2}_{n}}\) are the mean and variance of G d in local window k r , respectively. |n| is the number of pixels in this window. When both G d p and G d q are on the same side of an edge, the weight assigned to pixel q is large. When G d p and G d q are on different sides, a small weight will be assigned to pixel q.

2.3 Restoring the haze-free image

After refining the depth map, it is just required to restore the hazy image using haze removal restoration function. After refining the transmission map, the scene radiance can be mathematically recovered as follows:

$$ J_{sr}(k)=\frac{{I_{mg}(k)-G_{al}}} {\max(M_{tx}(k), L_{b})}+G_{al} $$
(9)

where L b , is a lower bound whose typical value is 0.1 that is introduced to make this algorithm more robust to noise.

3 Haze removal techniques

This section contains comprehensive review on existing well-known haze removal techniques. The categorization of these techniques is also done. The categories of image dehazing techniques are shown in Fig. 5. Haze removal methods are divided into seven broad categories i.e. (1) Depth estimation based haze removal, (2) Wavelet based haze removal, (3) Enhancement based haze removal (4) Filtering based haze removal, (5) Supervised learning based haze removal, (6) Fusion based haze removal and (7) Meta-heuristic techniques based haze removal. The subsequent section contains the details of various haze removal methods along with their strengths and weaknesses.

Fig. 5
figure 5

The categories of image dehazing techniques

Full size image

3.1 Depth estimation based haze removal

The multi-scale tone strategy is utilized to evaluate the atmospheric veil in an optimistic way. Thus, it can manipulate the quality and illuminate of an input image at multiple scales [56]. But, this technique experiences the same issue of the majority of haze-free techniques i.e., it does not attain consistent results, especially for heavy haze images. It performs poorly, whenever it fails to recognize local maxima and minima precisely.

Due to a maximum intensity of airlight, existing methods select pixels with the greatest intensity to evaluate the airlight map. However, some pixels with heavy illuminate are also produced by some other light foundations, such as train headlights. The Gaussian distribution based haze removal technique selects the airlight contenders from the brightest segment of haze image. The color similarity assessment is also utilized to hierarchically filter the airlight contenders. In the end, mean color from filtered airlight contenders is utilized for airlight estimation [6]. In [19], fast haze removal technique is introduced. This model can evaluate the atmospheric light by utilizing an infinite sky area and close white area. However, it still suffers from the edge preservation issue, because the potential edges may degrade during the haze free process.

Change of detail prior is utilized in [29] which can remove the haze from an image by utilizing multiple scattering occurrences in the dissemination of illumination. By using this technique, a thickness of haze can be evaluated successfully to restore a haze-free image. The change of detail prior is stable to local areas of haze image which contain objects in dissimilar depths [29]. However, it cannot preserve the edges of the haze free images.

A superpixel technique is designed for evaluating the transmission on sky and as well as non-sky areas, to diminish the effect of halo artifacts around edges and the decreases the color distortion in the sky area. Thus developed technique can overcome the halo artifacts issues with most of the existing haze removal techniques. [50]. However, this technique can be improved further by efficiently estimating the atmospheric veil, to restore the image in more consistent manner.

3.2 Wavelet based haze removal

The improved wavelet transform technique for proficient image haze removal is proposed by [36]. This technique initially utilizes wavelet transform for removing the haze from image, and then retinex technique is utilized to improve the color performance and to enhance the color effect after implementing the wavelet transform for image haze removal.

The improvement in the dark channel prior is done by utilizing the wavelet transform in [55]. This technique applies wavelet transform and guided filter to evaluate and improve the depth information of hazy images. Also contrast enhancement techniques are utilized as pre-processing techniques to recover the illuminate of hazy image.

Fast wavelet transform technique is proposed for improving the speed of haze removal technique without considering any prior knowledge. This method concurrently removes haze from the image and improves sharpness of the image. In the haze removing stage, two coarse transmission maps using dark channel prior are fused. One is obtained based on single-point pixel and the other is obtained by patch. For the sake of dehazing and enhancing sharpness simultaneously, a modified fast wavelet transform based unsharp masking framework is applied to control the effectiveness of sharpness by constructing a sigmoid function adaptively [48].

3.3 Enhancement based haze removal

In this section, various haze removal techniques are discussed which are based upon several image enhancment techniques. The dark channel prior utilizes soft matting technique that requires more memory and time. Thus, dark channel prior is efficient for small size images only. To overcome this issue, soft matting is replaced by adaptively subdivided quadtrees built in image space. The quadtree improves the speed by converting the problem of solving a N-variable linear model of soft matting, to a much lesser m-variable linear model, where N is number of pixels and m is number of corners in quadtree. Therefore, quadtrees considerably decrease both space and time cost while still preserve visual reliability [9].

3.4 Filtering based haze removal

The gamma correction and median filtering by utilizing look up table that can determine the haze free images in proficient way. This method has minimum computation time than existing techniques without losing the brightness of the haze free image [26]. Because this method cannot preserve the edges of haze free images.

L2-norm based haze removal method can evaluate the depth by calculating average vector L2-norm of sample window. After that it filters the evaluated transmission map by utilizing a guided filter. Thus, it uses the guided filter to preserve edges of haze free image [10]. However, it still suffers from the halo artifacts issue, which may introduce during fusion process. The weighted guided image filter utilizes an edge aware weighting to improve the guided image filter further. This method has overcome the problem of halo artifacts with the help of guided image filter. Weighted guided image filter has minimum computation time than existing techniques without losing the brightness of the haze free image [30]. The bilateral filter is utilized in order to attain local smoothness and as well as edge preservation of haze free image. This technique reduces the adverse effects due to difference in evaluating the global atmospheric illuminate [42]. The bilateral filter suffers from the halo artifacts issue, which may introduce during the fusion process. The weighted guided image filter and Koschmiedars law [28] without using any prior is utilized to simplify the dark channel of haze image into a base and detail layer. The transmission map is evaluated using the base layer, and it is utilised to recover haze free image. But, this method has poor computation time than majority of existing techniques.

3.5 Supervised learning based haze removal

By developing a linear model with supervised learning technique, depth of haze image can be evaluated in more consistent way than most of existing techniques. By using this depth information one can easily evaluate the transmission map and therefore recover the scene brightness by utilizing the atmospheric scattering method. [57]. But, for supervised filtering a lot of hazy and haze free images of the dissimilar scenes and environments are required which make it difficult for real time implementation.

Supervised learning based techniques typically require well-designed models and also require dummy hazy images for estimating the haze depth in efficient way. But, it cannot always contain significance depth knowledge of the natural images in practice. The two layer Gaussian regression [12] is proposed to overcome this issue. By utilizing training hazy and haze free image, the two layer Gaussian regression found an direct association among haze image and its depth knowledge.

A trainable end-to-end system called DehazeNet [2] is proposed for efficient monitoring of the medium transmission. DehazeNet can significantly estimate the transmission map by using the atmospheric scattering method. DehazeNet implements standard artificial neural network to estimate the transmission map from the hazy image. A nonlinear activation function is also considered in DehazeNet, called bilateral rectified linear unit, which has an ability to enhance the quality of restored haze free image.

The transmission estimated by dark channel prior is not smooth and possesses no local neighbor information which leads to the block effects. An improved haze removal method is proposed using Kernel Regression Model on local neighbor data. In this approach firstly, the initial transmission in atmospheric light model is estimated by dark channel prior. Secondly, the transmission is refined according to Kernel Regression Model. Then restoration model comes in action to remove the effect of haze from the image [52].

3.6 Fusion based haze removal

A multiscale depth fusion technique is described for removing the haze from single image [47]. The results of multiscale filtering are probabilistically combined into a fused depth map depending upon the model. The fusion is devised as an energy minimization issue that integrate spatial Markov dependence. The multiscale depth fusion technique can estimate the depth map in more consistent way and also has the ability to preserve the edges of haze free image with sharp details.

An efficient technique for transmission map estimation by using the guided fusion is presented in [35]. By utilizing the reliability guided fusion of block-level and pixel-level dark channels, a high-quality refined transmission map is evaluated. This technique successfully reduces the dark channel prior failure probability and haloes by growing the mask size in an edge-preserving manner. Dark channel prior failure in the sky (bright) regions is handled by limiting the contrast boost of sky-like surfaces. Thus it produces a more natural recovery of the sky regions.

A fusion strategy based haze removal technique is proposed in [32], which fuses the outcomes of the linear transform with the guided image filtering. Main steps of the algorithm are as follows. First, the first input image of the fusion process is obtained via a simple linear transformation. Second, an improved high-boost filtering algorithm based on guided image filtering is proposed to obtain the second input image of the fusion process. Third, a simple fusion method is used to fuse the above two input images. The final dehazing result is obtained by a simple white balance process. This algorithm not only greatly enhances the visibility of outdoor image, but also has high computational efficiency.

3.7 Meta-heuristic techniques based haze removal

Most of existing haze removal technique are unable to select best parameters for better haze removal. Therefore, does not provide optimistic results. The use of genetic algorithm for haze removal is that the parameter selection and function maximization can be strongly related issues. The genetic algorithm can attain the optimistic haze removal parameters by using contrast gain as fitness function [20]. However, genetic algorithm does not guarantee the global optimal solution, therefore requires its hybridization with others.

Haze Removal from the Noise Filtering Perspective is proposed by [31]. Images contaminated by haze in the form of noise possess two main characteristics: high intensity and low saturation. Therefore, a weighted sum of input image intensity and saturation is used to describe the haze severity. Atmospheric light can be estimated by the same principle, while a small correction is needed when images contain over-bright objects. After the two weighted maps are constructed, local statistics of the severity map are applied in image noise filtering. Four parameters involved are optimized via particle swarm optimization. The objective function, in this work, is to maximize the saturation of output image. Furthermore, a penalty function to control the hue change is introduced while calculating the overall fitness.

3.8 Variational image dehazing

Existing dehazing approaches estimate depth map to remove the haze from images. Thus, these techniques are vulnerable to failure whenever the physical assumptions are violated. Image enhancement techniques do not evaluate the depth map. Therefore, these techniques do not suffer from this issue. However, these suffer from the over-enhancement issue. Fortunately, variational image dehazing technique can overcome the physical assumptions failure issue and over-enhancement problem. Succeeding section describes some well-known variational image dehazing techniques [17].

Fang et al. [13] have designed a unified variational technique to restore hazy images and to remove noise from a single image. Total variation regularization is utilized as energy model for dehazing. Negative gradient descent technique is implemented to handle the corresponding Euler-Lagrange equations. To evaluation efficient initial attributes, the depth map is also improved with windows adaptive technique based on DCP which can remove the block artifacts. Galdran et al. [17] utilized a perceptually inspired variational based dehazing technique to develop an energy minimization model. The energy model is dependent upon a hazy image under a gray-world assumption. This assumption is further improved by estimating a average value for a dehazing image, and a local contrast evaluation is involved in the designed model. This technique outperforms in terms of visible edges in local areas. However, dark masks as a kind of unwanted artifacts, may be found in close-range areas.

Chen et al. [5] designed a dehazing technique for reliable suppression of several artifacts in images. Initially, the depth-edge-aware smoothing technique is implemented to improve the initial atmosphere veil estimated using local priors. In the image restoration step, Gradient Residual Minimization is implemented for jointly remove the haze from image while explicitly decreasing the various artifacts. Galdran et al. [18] designed fusion-based variational image-dehazing method. Fusion based variation dehazing does not rely on a physical model from which to estimate a depth map, nor does it require a training stage on a database of human-labeled examples.

4 Comparative analysis of haze removal techniques

In this section, comparisons have been shown among existing techniques by considering the various attributes.

Table 3 contains the comparison of existing techniques based upon certain features and artifacts. It clearly shows that each technique has its own features and limitations. No technique is effective for every case of haze removal. Thus it shows haze removal is still an open area of research.

Table 3 Comparative analysis of existing haze removal techniques
Full size table

Ranking a hazy algorithm is difficult task, because various kinds of vision applications may focus on different issues. Such as real time applications may demand dehazing techniques with good speed, Remote sensing image processing systems demand dehazing technique with lesser artifacts and large haze gradient, and some other outdoor applications may demand the removal of haze from hazy images with large haze gradient. However, we have given the highest rank to dehazing technique that cover almost all the issues at a same time.

Table 4 contains the comparison of existing techniques based upon their respective ranks. Among the existing dehazing techniques, High boost filtering based fusion has the highest rank among other techniques.

Table 4 Ranking of existing haze removal techniques
Full size table

Table 5 summarizes the pros and cons of several typical image dehazing techniques.

Table 5 The comparison of several typical image dehazing techniques
Full size table

5 Challenges and future directions

In hazy weather, water droplets float in the atmosphere. These droplets are extremely tiny in size. Thus, image illuminate developed at a pixel is the integrated effect of the maximum numbers of water droplets inside the pixels solid angle. The energy reflected by the object’s surface is not only attenuated by the overhanging water droplets but also merges with airlight when it perceived by the viewer. Thus, qualtiy of the captured image is not so significant as in haze free image. The primary objective of haze removal techniques is to restore color and significant details of the image. Attenuation and airlight are primary functions of distance of scene from the camera. Therefore, haze removal techniques require depth information of the hazy image. In real time applications, it is required to estimate the depth information. But, estimating the depth map is challenging issue, because the airlight attenuation ambiguity holds for every pixel and cannot be determined autonomously. Therefore, to handle ambiguity issue, an assumption or prior information is required [45]. The subsequent section contains various challenges associated with the haze removal techniques.

5.1 Atmospheric light monitoring

The atmospheric light is reliably monitored by using the dark channel prior, particularly when the dark channel is evaluated by utilizing a large local mask. Thus, if the local mask size utilised in dark channel evaluation is not sufficiently large, it is suggested to employ an supplementary dark channel with a larger local mask size only for atmospheric light monitoring. The utilization of local entropy is also found to be useful in improving the monitoring accuracy because atmospheric light monitoring from intense objects can be prohibited [27].

5.2 Over enhancement

Enhancement of the hazy image is found to be critical task because of the complexity in restoring the illuminate and color while holding the color reliability. During enhancement of hazy images, over enhancement leads to saturation of pixel value. Thus, enhancement should be restricted by several assumptions to avoid saturation of image and maintain suitable color reliability.

5.3 Large haze gradients

The primary drawback of majority of existing methods is that they may lose significant details of restored images with large haze gradients.

5.4 Adaptive parameters selection

Since most of dehazing algorithms may produce oversaturated or undersaturated intensity values due to manual parameter selection. Generally these parameters are patch size, restoration value, lower bound and white balance factor. The majority of existing haze removal techniques has taken these values manually, which depends upon the given set of images. This limits the performance of haze removal as restoration value needs to be adaptive as the effect of haze on given image varies scene to scene and atmospheric veil.

5.5 Meta-heuristic algorithms

The use of meta-heuristic algorithms to adaptively find the haze restoration parameters is ignored by majority of existing researchers. [20] have utilized Genetic algorithm to optimistically find the haze restoration parameters. But, the Genetic algorithm suffers from local optima issue and premature convergence issue. Also particle swarm optimization [31] based haze removal technique suffers from premature convergence and also initial amount of particles limits the performance of the particle swarm optimization. Thus, it is required to explore and apply other meta-heuristic techniques to restore the hazy images.

6 Performance metrics

Performance metrics are used to analyze the quality of an haze removal algorithms. In haze removal techniques, quality metrics are divided into two parts i.e., when ground truth image is given and when ground truth image is not given.

6.1 When ground truth image is given

In this case a ground truth image also called reference image is given in advance. It is an actual haze free image of the same hazy image. However, actual haze free images are only given when someone want to validate its haze removal algorithm on standard hazy images data sets. For objective evaluation of haze removal techniques when reference image is given, several quality metrics can be considered like Mean Squared Error (MSE), Peak Signal to Noise Ratio (PSNR), and Structural Similarity Index Metric (SSIM).

6.1.1 Mean Square Error

The Mean Square Error (MSE) is an error measure, which is utilised to evaluate the difference between the Ground Truth (GT) image and the Haze free image (OP) produced by given algorithm. It is basically a positive integer which ranges from 0 to ∞. Close to 0 is required. MSE can be calculated as follows [1, 40]:

$$ MSE = \displaystyle\frac{1}{K \times L}\sum\limits_{q = 1}^{K}\sum\limits_{p = 1}^{L}\left[GT(p, q)-OP(p, q)\right]^{2} $$
(10)

G T(p, q) represents pixel intensities of ground truth image whereas O P(p, q) depicts the pixel value of haze free image. Also p and q represents the pixel’s coordinate values

6.1.2 Peak signal to noise ratio

With respect to haze free image, Peak Signal to Noise Ratio (PSNR) evaluates mean squared error after applying haze free technique. Maximum PSNR value represnts that haze is removed proficiently. Similarly, lesser PSNR value represents poor capability of haze free technique. PSNR can be evaluated as follows [1, 40]:

$$ PSNR = \displaystyle10\log_{10}\left( \frac{255^{2}}{MSE}\right) $$
(11)

6.1.3 Structural similarity index metric

Structural Similarity Index Metric(SSIM) evaluates degree of relationship among hazy and haze free image. SSIM was designed to have a quality reconstruction measure which also considers the relationship of edges (high frequency content) that was not there in case of PSNR. SSIM always lies between 0 and 1. Closer to 1 means higher structural quality of haze free image. It is used to evaluate the structural similarity of edges among GT and OP image. SSIM can be calculated as follows [40]:

$$ SSIM(p,q) = \displaystyle\left( \frac{2\mu_{p}\mu_{q}+c_{1}}{{\mu_{p}^{2}}+{\mu_{q}^{2}}+c1}\right)\left( \frac{2\mu_{pq}+c_{2}}{{\sigma_{p}^{2}}+{\sigma_{q}^{2}}+c2}\right) $$
(12)

In (12) p and q represents the pixel coordinates. Also μ p and μ q are sample means of p and q respectively. \({\sigma _{p}^{2}}\) and \({\sigma _{q}^{2}}\) are the sample variances of p and q, and σ p q is the sample cross-covariance between p and q. The default values for c 1 and c 2 are 0.01 and 0.03, respectively.

6.2 When ground truth image is not given

In real time applications, ground truth images are not given. Then, it becomes difficult to measure the effectiveness of the given algorithm. In case of haze removal techniques, a haze free image has more contrast compared to hazy image. Therefore, contrast gain (Ω) and Percentage of saturated pixels (PSP) can be effective parameters for evaluating the best haze removal technique.

6.2.1 Contrast gain

Contrast gain (Ω) is defined as the average contrast difference between hazy and haze-free image [45]. Higher the value of Ω indicates that the given dehazing technique is more efficient than others. Assume A C h f i and A C h i are average contrast values of haze free and hazy image respectively, then Ω can be computed as follows [38, 39]:

$$ {\Omega} = AC_{hfi} - AC_{hi} $$
(13)

Assume an image of size K × L can be represented by I κ (K, L). Then, average contrast AC can be computed as follows:

$$ AC = \displaystyle\frac{1}{K \times L}\sum\limits_{p = 1}^{K}\sum\limits_{q = 1}^{L} I_{\kappa}(p, q) $$
(14)

6.2.2 Percentage of saturated pixels

The Contrast gain (Ω) should not be so high that the pixels of haze free image become saturated. Therefore, the Percentage of saturated pixels (τ) is needed to be computed [45]. τ can be mathematically represented as follows [38, 39]:

$$ \tau = \displaystyle\frac{S_{p}}{K \times L} $$
(15)

Here, S p represents the number of pixels that are saturated either completely black or white, after the haze removal technique, which were not present in the hazy image. The lower value of τ indicates the given dehazing technique is better than others.

6.2.3 Visible edges ratio

The ratio of new visible edges (e) and ratio of average gradient \((\bar {r})\) are also utilized to monitor the performances of the proposed approach. The e represents the improved rate of visible edges of haze free images, and is calculated as follows [21]:

$$ e= \frac{n_{k}-n_{l}}{n_{l}} $$
(16)

where n k and n l represents the cardinal number of the visible edges in the hazy image I κ and the haze free image O p , respectively.

The maximum e states that the edges of haze free image are stronger. The \(\bar {r}\) utilizes the gradients of visible edges in the haze free image, to depict the restoration degree of the image edge and texture information. \(\bar {r}\) is described as follows:

$$ \bar{r} = e^{{\left[\frac{1}{n_{k}} \sum\limits_{i\in\phi_{k}}logr_{i}\right]}} $$
(17)

where r i = \(\frac {\Delta k}{\Delta l}\), k and l are the gradients of Δk and Δl, respectively, r i denotes the set of visible edges of O p . A maximum \(\bar r\) state that the corresponding dehazing technique has improved capacity of edge preservation than others.

6.2.4 Perceptual haze density

An effective technique for haze density prediction is discussed in [7] in which the input image is divided into N × N sections and aggregate average values are computed. All N × N sections are utilized to evaluate various haze aware features such as variance, sharpness, contrast energy, image entropy, dark channel prior, color saturation, colorfulness etc. Mahalanobis-like measure [46] is applied on these features to evaluate the Multivariate Gaussian (MVG) fit of n dimensions can be mathematically calculated as follows:

$$ P(s) =\frac{1}{\sqrt{(2\pi)^{n}}|D|}exp\left( -0.5*(s-\mu)^{t} C^{-1}(s-\mu)\right) $$
(18)

Here, s denotes the haze aware statistical features, μ represents mean and n × n demonstrates the covariance matrix of different hazy features. Also, D represents determinant and C − 1 depicts the covariance matrix inverse for MVG. D and C − 1 can be derived using maximum likelihood (ML) estimation [11]. Next, Mahalanobis-like distance can be calculated as follows:

$$ D = \sqrt{(m_{1}-m_{2})^{t}\left( \frac{C_{1}+C_{2}}{2}\right)^{-1}(v_{1}-v_{2})} $$
(19)

where, m 1 and m 2 are mean vectors and C 1 and C 2 are covariance matrices for MVG model of the haze free corpus and MVG fit of the test image.

Another metric L f which has haze free level of the test haze image is calculated which is the distance norm of MVG versus haze aware statistical features. This information is extracted from a haze test image and normal MVG model from a group of 500 hazy images [7]. Afterwards haze density D can be calculated calculated as follows:

$$ D_{h} = \frac{D}{1+L_{f}} $$
(20)

Values of D h are proportional to the corresponding haze density.

7 Significance and benefits to society

Haze removal techniques play an important role in vision processing applications. Subsequent section briefly explains some of the most significant applications in which haze removal techniques are utilized.

7.1 Airplanes

Generally, takeoff and landing of airplanes become challenging task in hazy environment. Many flights get delayed or some times are canceled due to hazy environment. To handle this issue, one can use haze removal algorithms to make the perceived scene as haze free.

7.2 Underwater image processing

For researchers and divers, it is hard to attain maximum information from underwater images. Like during shark attack, underwater scene analysis, etc. [37].

7.3 Remote sensing

Remotely sensed images play an important role in vision processing application. Due to high difference in camera and scene, haze will be introduced in the captured scene. Such as weather forecasting, detection of some particular objects demands haze free images[34].

7.4 Intelligent transportation vision system

In hazy days, due to poor visibility of roads, many accidents occurs on highways, especially in hilly areas. So, in order to prevent accidents on highways and hilly areas a haze removal technique is required to provide haze free image to driver using some visual equipment etc. However, due to high speed of vehicles, it needs a haze removal technique with constant time complexity [23].

7.5 Intelligent railway

Many trains get delay or some times even canceled due to hazy environment. So, in order to handle this issue, one can use haze removal algorithms to make the perceived scene as haze for train drivers.

8 Conclusion and future work

In this paper, evolution of techniques for removal of haze from hazy images has been studied. Framework and challenges for the haze removal techniques have been discussed. Here, haze models have been studied which discovered the cause of poor visibility of the hazy image due to haze. Several features of the existing haze removal techniques are explored to encourage further research. Removal of the haze from single image is an difficult task because depth map is required to be estimated. Therefore, haze removal techniques demand certain constraints or prior knowledge. It is essential that during recovery of hazy image, both the illuminate and color characteristics should be restored in efficient way to preserve the color fidelity and appearance. Hence, future research will center on optimistic estimation of depth map and restoration parameters with better visual quality by using meta-heuristic techniques. A fast and optimistic monitoring of the depth information improves the speed and perceptual image quality.