Hybrid Domain Analysis of Noise-Aided Contrast Enhancement Using Stochastic Resonance

Abstract

This paper aims to present an analysis of a noise-aided contrast enhancement algorithm in hybrid transform domains. The performance of our earlier noise-enhanced iterative algorithm, formulated from the motion dynamics of a double-well system exhibiting dynamic stochastic resonance, has been investigated here on hybrid coefficients, viz. singular values (SVs) of wavelet coefficients, SVs of discrete cosine transform (DCT) coefficients, and DCT of wavelet coefficients, of a dark image. The performance of the algorithm is gauged using metrics indicating relative contrast enhancement and perceptual quality. Colorfulness, subjective visual scores and logarithmic contrast metrics for outputs are also observed. Experimental results display noteworthy enhancement of contrast on both natural and synthetically-darkened images. It can be inferred from comparative analysis with respect to other conventional methods that while the algorithm is observed to work well in all three hybrid domains, the SV-DCT domain performs better in terms of iteration count, while DCT-DWT is found to outperform others in terms of perceptual quality.

1 Introduction

Research in noise-enhanced image processing has gained widespread momentum in the past two decades. Particularly, the use of stochastic resonance (SR) – a counterintuitive phenomenon, where addition of optimal amount of noise leads to enhanced sensitivity of a non-linear system towards a weak subthreshold signal - has been widely used in signal and image processing applications. In the past decade, several works on application of stochastic resonance for grayscale image or edge enhancement that have been reported in literature [1–14]. The broad framework of each of these works is to add controlled amounts of noise to the image (values or coefficients) so as to increase its contrast or comprehension.

A common problem when addressing dark images is the implicit presence of noise in the enhanced image. Most of the algorithms available in literature produce remarkable increase in contrast of the images, but the use of an iterative approach utilizing internal degradation and focus on perceptual quality of enhanced images had been little explored. Since the noise-aided iterative algorithm of [9] was found to perform well in individual domains [9, 10, 12, 13], we found it interesting to study its performance in hybrid domains, where the advantages of individual domains may be combined to produce enhancement with fewer iteration. The unique feature of the SR-based approach is the use of internal noise, instead of externally added noise to induce resonance, and the selection of parameters by Signal-to-Noise Ratio (SNR) maximization, as described in details in [9, 13]. Apart from noise-aided algorithms, literature is abundant in various remarkable algorithms of image enhancement in a wide range of processing domains.

Spatial domain processing is usually popular amongst researchers working on dark images due to its low computation requirements. Jobson et al. [15] have reported the retinex theory that requires filtering with multiscale Gaussian kernels and post-processing stages for adjusting colors. They have also reported an extension of the previously designed single-scale center/surround retinex to a multiscale version that achieves simultaneous dynamic range compression, color consistency and lightness rendition [16]. A modified high-pass filtering has been described in [17], where some specific spatial frequencies are selectively magnified by exaggerating the local visibility of an image, followed by high-pass filter to adjust those critical frequencies. Since singular values of an image hold luminance information on an image, nonlinear scaling of these values also leads to increase in overall luminance of the image [9]. Older algorithms like histogram equalization and gamma correction are already well-known to the image processing community.

The above mentioned enhancement techniques are based on spatial-domain operations. However, DFT and DCT domains provide spectral separation, and due to this property it is possible to enhance features by treating different frequency components differently. A popular technique called alpha-rooting [18] produces an increase in contrast of the image when the magnitude of each transform coefficient is raised to a power α, 0<α<1, and the sign or the phase of the coefficient is unchanged. Many algorithms reported in literature have been designed for both colored and grayscale images in block DCT domain [10, 19, 20]. However, problems of blocking artifacts occur when operating in block neighborhood. Wavelet domain is another promising platform for image enhancement due to its multiresolutional characteristics. As the approximation band consists of illumination content, while the other subbands contain edge information, processing of this approximation band may, therefore, protect the edges and details from degradation. Hybrid domains like singular values of wavelet transform have been explored in [21, 22] where the broad mechanism adopted is the equalization of singular values (of image or approximation band) by a coefficient found by ratio of largest singular values of the image and an equalized image. Similar equalization of singular values of DCT coefficients was reported in [23] for enhancement of low-contrast satellite images. Note that in the proposed work, we aim to explore three hybrid domains to study the performance of an existing SR-based enhancement model.

The organization of the rest of the paper is as follows: Section 3 describes the theory and mathematical formulation of the approach. Section 4 explains the choice of chosen hybrid domains for this study, and Section 5 describes the mechanism of how the SR-based processing modifies the coefficients or values of the respective hybrid domains to produce contrast enhancement. Section 6 lists and briefly explains the quantitative and qualitative metrics used for characterizing the performance of the algorithm. Section 7 presents the general SR-based enhancement algorithm and its description when the input dataset are the hybrid coefficients/values. Section 8 presents and discusses in details, various aspects of the performance of the SR-based algorithm in hybrid domains. Inferences and concluding remarks are summarized in Section 9.

2 Key Contribution

As stated before, this paper explores the combined advantage of various transform domains for contrast enhancement of dark images using dynamic stochastic resonance. An established input statistics-dependent dynamic SR (DSR) model for contrast enhancement [13], is investigated here in various hybrid domains, such as singular values of wavelet transform (SV-DWT), singular values of discrete cosine transform (SV-DCT), and discrete cosine transform of wavelet transform (DCT-DWT). The choice of these hybrid domains and the mechanism of enhancement through iterative processing on each of these domains is analyzed. The use of internal degradation (due to lack of illumination) to induce stochastic resonance and use of perceptual quality to participate in the termination of the iterative algorithm are some unique features of employed approach. The performance is characterized using quantitative metrics of contrast, color, perceptual quality, as well as qualitatively through subjective evaluations. Various aspects of the performance, such as quality, artifacts, computation, parameter selection are also discussed in details. After careful study of the algorithm on several test images and comparisons with several enhancement algorithms, the noteworthy performance of the hybrid-SR domain is validated. More importantly, the superior performance of the SR-based enhancement model in hybrid domains is established through rigorous experimentations and analyses.

3 Mathematical Formulation of SR-Based Enhancement Algorithm

Stochastic resonance is a phenomenon in which a system (embedded in a noisy environment) acquires an enhanced sensitivity towards a small external periodic forcing, when the noise intensity reaches some finite level [24]. The underlying mechanism of stochastic resonance may be understood using the double-well model proposed by Benzi et al. [25, 26] as follows: If a particle (image) is placed in a double-well potential system having two stable states (or wells), and a weak force is exerted on the system, the double-well would oscillate asymmetrically. The particle may also oscillate, but may not have sufficient force to transit into the other well, even after application of a random noise fluctuation on the particle. However, at some optimum amount of noise intensity, the particle may make the transition to the other well due to constructive cooperation between the weak force and the noise. This periodic inter-well transition occurs when the noise is tuned to the signal using the Kramer’s equation [27]. If the position of the particle is considered as output at any instant, the periodic nature of the weak subthreshold signal is amplified and exhibited at the output. The above analogy in the context of images and following mathematical formulation has been extensively explained in our earlier works in SVD [9], and intensity [13] domains respectively. It is important to note that the present paper presents an extension of the application of this SR model into hybrid domains. As the focus of the paper is the application of the model, its mathematical formulation has been briefly included in the interest of reading continuity.

The dynamics of the motion of this particle is governed by Langevin equation of motion [27] as follows:

$$ \frac{dx(t)}{dt} = -\frac{dU(x)}{dx}+ B\sin(\omega{t}) + \sqrt{D}\xi(t) $$

(1)

where B and ω are respectively, the amplitude and frequency of the weak periodic signal. Here, ξ(t) is the additional stochastic force (noise) of intensity D.

Here, U(x) is a bistable quartic potential given by:

$$ U(x)=-a\frac{x^{2}}{2}+b\frac{x^{4}}{4} $$

(2)

Here, a and b are positive bistable double-well parameters. The double-well system is stable at $x_{m}=\pm {\sqrt {{a}/{b}}}$ separated by a barrier of height ΔU = a ²/4b when the ξ(t) is zero.

In the context of images, the double-well represents image quality, while the dark input image serves as the weak input signal due to its low excursion about mean intensity. Similarly, some transform coefficients, or hybrid transformed coefficients of a dark image may also qualify as a weak signal input to the SR system. Therefore, if stochastic resonance is induced in the hybrid coefficients or values (to be called as SV-DWT or SV-DCT or DCT-DWT in this paper), each of the hybrid values may be non-linearly scaled up. The position of the particle along x-axis represents the state of image coefficients at any instant (iteration). If one stable well represents the low-contrast state, and the other the final enhanced state, we consider iteration until single hopping of the image from the low-contrast state to high-contrast state.

Equation 1, when discretized for this bistable double-well (as described in [9], [13]), produces the following iterative equation:

$$ {x(n+1)} = {x(n)} + {\Delta}{t}\left[ax(n)-bx^{3}(n) + Input\right] $$

(3)

Note that $Input=B\sin (\omega {t}) + \sqrt {D}\xi (t)$ denotes the sequence of input signal + noise. We assume that the noise is due to degradation arising out of insufficient illumination, and the signal is the image information. Therefore, both signal and noise are inseparable and inherently present in the image, or the image’s transformed coefficients. Hence, here the term Input assumes the value of the transformed coefficients.

Δt is the sampling step size for discretization, and a and b are the double-well parameters as described before. x(0) is a zero matrix for mathematical convenience. x(n+1) are the tuned (iterated) transform coefficient matrices.

By differentiation of the SNR equation for dynamic stochastic resonance w.r.t a, the optimum value of a for maximum SNR is found to be a=2 σ ₀ ² (as derived in [9]). Another condition is needed to ensure that the maximum allowable force on the bistable well maintains its stability, i.e. the periodic input signal is less than or equal to maximum restoring force or gradient of potential function. This is ensured by the condition b<4a ³/27.

4 Choice of Hybrid Domains

Our earlier studies on SR-based processing on singular values [9], DWT coefficients [12], and DCT coefficients [10] indicated the advantage of operating in these domains. In context of a dark or low-contrast image, the intensity values constitute a weak signal due to their low excursion about the mean. Similarly, the wavelet approximation band (which is a coarse representation of such an image) also qualifies as a weak signal. Note that using singular value decomposition an image can be represented as a weighted summation of image layers such that the weight of each layer is represented by the corresponding singular value that denotes the luminance of that layer. Since SR processing on SVs and wavelet coefficients individually produced contrast enhancement, we were interested to explore their hybrid combination. As the approximation subband of the DWT decomposition of an image simply represents a coarse (low-pass) version of the image itself, the singular values of such a subband can also indicate luminance values. However, due to the lower resolution of the approximation band, its singular values would denote coarse and more compact luminance information.

Similarly, as derived in [20], if the normalized DC and AC coefficients of an image are scaled by certain value, the contrast of the processed image also gets similarly scaled. Since SR processing on DCT coefficients of a dark image produced a non-linear scaling and multifold enhancement in image contrast [10], and singular value decomposition of DCT coefficients denotes a weighted representation of the same, we chose to study the effect of SR processing in this hybrid SV-DCT domain. Also, since wavelet approximation is a coarse representation of the original image, we also studied the effect of SR processing on DCT of these wavelet coefficients, hence, the choice of DCT-DWT hybrid domain.

5 Mechanism of Contrast Stretching in Hybrid Domains by Noise-Enhanced Iterations

The effect of noise-enhanced iterations on hybrid domain values or coefficients has been discussed in the following section.

5.1 Singular Values of Wavelet Coefficients (SV-DWT)

From an investigative point of view, the effect of dynamic SR (DSR) iterations has been observed on singular values (SV) of all four wavelet subbands. The effect of iterative processing on distribution of singular values of DWT (SV-DWT) has been shown in Fig. 1. Note that the range of input values is significantly different from the tuned values, and thus the axes of the input distribution are scaled differently to ensure that characteristics of the input are clearly visible. It can be observed that each of the distributions shifts and gets skewed towards the larger end. This would mean that the count of smaller SVs, that was higher before DSR, has now decreased, while count of larger SVs which were very few has now increased. Also, the rate of distribution flattening is much more for the approximation coefficients. Since singular values denote weights of image luminance, as the number of larger SVs increase, this can be attributed to increase the overall luminance of the image in question. The variance of the SV-DWT distribution also displays a resonant nature, for instance, giving a peak at around iteration count, n=60 (for the Lena test image in question) as shown in Fig. 2 that also corresponds to the best contrast image as shown in Fig. 1. Processing of SVs of all subbands produced more blocking effects than processing of only approximation subband [28]. Since only approximation band contains luminance information, we choose to process only SV-Approximation subband.

Figure 1

a shows distribution of SV-DWT values of a dark input image. b-e show the probability density function (pdf) of SV-DWT values (approximation, horizontal, vertical, diagonal details) of a dark low contrast image after 20, 40, 60 and 80 iterations respectively. f shows the input dark low-contrast image. g-j show output image after 20, 40, 60 and 80 iterations respectively (for Δt=0.01).

Figure 2

Characteristics of variance of the SV-DWT distribution with respect to iteration.

5.2 Singular Values of DCT Coefficients (SV-DCT)

As discussed in Section 5.1, the singular values of an image represent luminance values of individual image layers. Also, DCT of an image signifies distribution of frequencies over the entire image. The DCT magnitude spectrum (squared) denotes the power (or energy) of the image. Singular value decomposition of these coefficients implies that the DCT coefficients distribution is now represented as a sum of layers with different weights (now denoted by the singular values of the DCT coefficients).

Similar to our earlier observation on SV-DWT, it can be observed here (in Fig. 3) that the distribution of SV-DCT, is exponential in nature, and shifts and gets skewed towards the larger end with iteration. This change in skewness and increase in variance is validated by Fig. 4a, b. This would again mean that the count of smaller SVs, that was higher before DSR, has now decreased, while count of larger SVs, which was very less, has now increased. It should be noted here that these are the SVs of DCT coefficients, and denote the weights if the DCT matrix is represented as a summation of several matrices. Since the number of high-valued SVs increases with DSR, this can be related to increase of larger weights of DCT summation matrix. The larger weights of DCT summation matrix in turn denote the contribution of larger DCT values which surround the DC coefficient (denoting average brightness). In this way, an increase in high-valued SV of DCT coefficients can be attributed to increase the overall brightness of a dark image, and the increase in energy (magnitude squared) of DCT coefficients leads to an increase in contrast with iteration. Another observation from Fig. 3 is that too much iteration after optimal count leads to a textured and over-bright image due to increased weights (SVs) of lower ordered layers.

Figure 3

a shows distribution of SV-DCT values of a dark input image. b- e show the probability density function (pdf) of singular values of DCT coefficients of a dark low-contrast image after 18, 22, 26 and 30 iterations respectively. f shows the input dark low-contrast image. g-j show output image after 18, 22, 26 and 30 iterations respectively (for Δt=0.005).

Figure 4

a shows the characteristics of skewness and b variance of the SV-DCT distribution with respect to iteration.

5.3 DCT of DWT Coefficients (DCT-DWT)

While there are several representation for the nature of DCT coefficient distribution, in totality, it may be considered to follow general gaussian normal distribution [29]. When both DC and AC coefficients are tuned using iterative dynamic stochastic resonance equation, the variance of the DCT coefficient distribution is found to increase with iterations. It is known that the DC coefficient represents the average brightness of an image, while the sum of squares of the normalized AC coefficients gives the variance of an image [20]. Thus, modification of DC coefficient of the DCT matrix would increase the overall image brightness (and this would be very useful for enhancement of dark images) [10]. As already stated, since the approximation coefficients emulate the behavior of image intensity values [12], the DCT of approximation coefficients may also be modified for producing contrast enhancement.

In Fig. 5, the distribution of DCT of wavelet subband coefficients is also observed to be close to gaussian distribution (by curve fitting). For a dark input image, the transformed (DCT) coefficient distribution (as observed in [10]) as well as is DCT-DWT coefficient distribution is observed to be of low spread. Since squared magnitude of the coefficients implies energy, a low-variance distribution implies that the energy distribution is concentrated in only certain areas, confirming that the image in question is of low-contrast. Iterative processing is observed (in Fig. 5) to increase the variance of all DCT-DWT coefficient sets, and therefore, can be attributed to increase the contrast and brightness of the image in question. This increase in variance is also reflected in the characteristics shown in Fig. 6, that clearly shows that the increase in variance of approximation is multifold times that of other details subbands.

Figure 5

a shows distribution of DCT of DWT coefficients of a dark input image. b-e show the probability density function (pdf) of DCT-DWT coefficients (i.e. DCT of approximation, horizontal, vertical, diagonal details) of a dark low-contrast image after 100, 200, 400 and 500 iterations respectively. f shows the input dark low-contrast image. g-j show output image after 100, 200, 400 and 500 iterations respectively (for Δt=0.001).

Figure 6

Characteristics of variance of the DCT of DWT (approximation, vertical, horizontal and diagonal details) coefficients distribution with respect to iteration.

During experiments, no visible difference was observed in outputs obtained from SR-processing on DCT of all subbands and DCT of approximation subband. Therefore, to omit unnecessary computation, we have continued our study only on DCT of approximation band.

6 Performance Characterization

The performance of the algorithm is gauged in terms of metrics of contrast enhancement and visual quality. The following metrics of performance are used in the iterative algorithm to serve as termination criteria.

Contrast Enhancement Factor (F)

Metric of contrast enhancement (F) is based on global variance and mean of original and enhanced images [6]. Therefore, a descriptor called contrast quality index, Q, has been used such that Q = σ ²/μ where σ and μ are, respectively, the standard deviation and mean of the image intensity values. Relative contrast enhancement factor, F, is computed as the ratio of values of contrast quality indices post-enhancement, (Q _B ), and pre-enhancement, (Q _A ).

Perceptual Quality Measure (PQM)

There is no universal measure that can specify both objective and subjective validation for any enhancement technique. Therefore, for quantitative evaluation of perceptual quality, a no-reference metric of image quality, PQM, is used that takes into account visible blurring or blocking artifacts (if any) present in the enhanced image [30]. According to [20], PQM should be close to 10 for good perceptual quality (digression away from 10 on either direction is an indication of decrease in perceptual quality). The code available at [31] has been used to compute PQM.

Some other metrics of enhancement are also observed to characterize the quality of the enhanced images.

Measure of image enhancement (EME) and measure of enhancement based on entropy (EME Entropy)

[32, 33]: Agaian et al. [32, 33] have proposed logarithmic metrics of image enhancement related to the concepts of the Weber’s Law of the human visual system (HVS). Though the metrics are particular useful for selection of optimal parameter and transform in frequency-based enhancement algorithms, here a simpler version of the metrics, EME and EMEE, has been used to obtain and observe an empirical HVS-based performance value. The basis of these metrics is the logarithmic relationship between stimulus and perception.

Let an image, I, be split into k ₁×k ₂ blocks B(k,l) with center (k,l) of size M ₁×M ₂,

$$EME_{k_{1}, k_{2}} = \frac{1}{{k_{1}}{k_{2}}}\sum\limits_{l=1}^{k_{1}} \sum\limits_{k=1}^{k_{2}} 20 \ln \frac{I^{w}_{max;k,l}}{I^{w}_{min;k,l} + c} $$

where $I^{w}_{max;k,l}$ and $I^{w}_{min;k,l}$ are the maximum and minimum in a given block, B(k,l); c is a small constant (experimentally) set to 0.0001 to avoid dividing by 0.Measure of enhancement based on entropy (EMEE) [33],

$${EMEE}_{\alpha, k_{1},k_{2}}=\frac{1}{{k_{1}}{k_{2}}}\sum\limits_{l=1}^{k_{1}} \sum\limits_{k=1}^{k_{2}}\alpha{\left( \frac{I^{w}_{max;k,l}}{I^{w}_{min;k,l}+c}\right)}^{\alpha}\ln\frac{I^{w}_{max;k,l}}{I^{w}_{min;k,l}+c} $$

where α is an enhancement parameter, here set to 1.

Color Enhancement Factor (CEF)

Though our processing leaves the chromatic components undisturbed, the reverse mapping of the enhanced value vector (of HSV color space) into RGB space may cause some change in overall colorfulness. To observe the colorfulness of the enhanced image, we have considered a colorfulness metric, (CM) [34]. The color enhancement factor (CEF) has been defined as ratio of colorfulness of output to input image. For good color and contrast enhancement, respective values CEF and F should be greater than 1.

Mean Opinion Score (MOS)

of subjective visual quality of enhanced images is also observed for a representative size of 20 subjects. The subjects were asked to score the test images following the code: 0 - Very poor, 1 - Poor, 2 - Average, 3 - Good, 4 - Very good, 5 - Excellent. Eight images were presented to twenty subjects. Thirteen of the subjects were image processing graduate students, while seven were naive, all belonging to age group of 22 - 40 years, and with normal vision. Scores were obtained by showing (blind) enhanced images on a LED screen (resolution 1366×768, Distance from subject: approx. 1.5 feet, Ambience - 33 by 16 feet room illuminated with eight fluorescent lights) with paired comparisons w.r.t. input.

7 General SR-Based Enhancement Algorithm

The basic steps of our noise-enhanced contrast stretching algorithm are as follows:

Step 1 :

- Color model conversion, followed by transformations The input image is projected into H-S-V color space to process only luminance vector (so as to leave chromatic components undisturbed). DCT transformation, here, implies global transformation, unless mentioned otherwise. Depending on the domain of transformation, the value vector is then

• decomposed into approximation (LL) and detail (HL, LH, HH) coefficients using 1-level discrete wavelet transform (here, the Haar or Biorthogonal CDF wavelet). Singular value decomposition of the approximation (LL) band is done using

  $$ LL = U~{SV_{LL}}~{V^{T}} $$

  (4)

  where U and V are left and right singular matrices. Here, the data to be processed are the SV-DWT values, S V _{L L} .or

• DCT transformation of the value component is computed, and singular values of the DCT coefficients is found, thereby producing the data set to be processed as SV-DCT coefficients, S V _{D C T} .or

• Discrete cosine transformation of the (LL) band is computed, producing data to be processed as D C T _{L L} .

Let us call the general set of hybrid domain coefficient or values as a vector, H, such that H= {S V _{L L} ,S V _{D C T} ,D C T _{L L} }.

Step 2 :

- Computing SR parameters Assume a small value of Δt, say 0.01 or 0.005, a = k×2σ ₀ ², b = m×(4a ³)/27, where σ ₀ is the standard deviation of hybrid coefficient or value set, H.

Here, k is a factor denoting image dullness (given by inverse of (variance × dynamic range)) and m is a factor much less than 1 (so that b<4a ³/27).

Step 3 :

- SR processing of hybrid coefficients or values, H, viz. S V _{L L} or S V _{D C T} or D C T _{L L} Initialize a zero vector, x, such that x(0)=0. Using the bistable dynamic SR parameters tune the hybrid coefficient sets (or values) according to Eq. 3 as follows:

$$\begin{array}{@{}rcl@{}} x(n+1) = x(n) + {\Delta}{t}\left[~a x(n)-b {x}^{3}(n) + H~\right] \end{array} $$

(5)

Figure 7

Results on the proposed SR-based algorithm in hybrid domains for various test images.

Inverse transformation and back-projection (to R-G-B) is performed on each SR-processed data set, and performance metrics, F, and PQM are computed for each back-projected image after every iteration. Iteration is terminated at count, n ₀, such that the sum of F(n) becomes maximum in the nearest possible vicinity of PQM = 10, (say P Q M±2.5) for this n= n ₀. This termination criterion is chosen so as to give primary importance to contrast as well as perceptual qualities. This would ensure that the optimal output has high contrast quality as well as acceptable perceptual quality.

Note

Experiments have shown that human visual preference of good quality may not always be in identical accordance with the empirical value of PQM. For instance, contrast enhancement factor may be to too low at PQM = 10, but may increase till PQM = 9 or 8. This is why, a flexible window of PQM observation is considered. If this window is too constrained, the enhanced image may quantitatively display high values, but may lack visually and contrast-wise. Therefore, an allowance of ± 2.5 has been allowed in this paper so that higher contrast qualities may be reached with acceptable visual quality.

8 Results and Discussion

Iterative algorithms offer the advantage of allowing observation of incremental refinement with each iteration, and as a result allow us to monitor multiple performance measure so that an optimal combination of the two can be achieved. Note that an inherent challenge while trying to enhance dark images is the implicit graininess (noisy) present in the very dark regions of enhanced image. This is because non-linear processing of values tending to zero (dark intensities) leads to spurious artifacts. This is a common observation when any enhancement algorithm is used to enhance a very dark image. For example, the output of histogram equalization of a dark input image would give a high contrast quality, but poor perceptual quality measure. An iterative algorithm, on the other hand, allows us to monitor the perceptual quality with each iteration so that a constraint may be employed to limit perceptual quality within an acceptable range (not allowing it to deteriorate in order to increase contrast). This helps ensure that the optimal output has meaningful enhancement and smooth natural appearance.

The iterative processing using domain-specific double-well parameters was performed on several test images, and some of those results are displayed and discussed in this paper. Figures 7 and 8 show the SR-enhanced outputs for SV-DWT, SV-DCT and DCT-DCT domains, along with their optimal iteration counts and performance metrics. Test images, Flowers and Grass are made low-contrast by random manipulation of original good-contrast colored images. Test images, Pencil, Brushes, Watch, and Chair are naturally dark colored images, captured indoor under very poor illumination. Test images, Cameraman and Pepper have been made dark by random (nonlinear) manipulation on original standard grayscale images. Therefore, the test image dataset contains equal distribution of phantom and naturally dark images.

Figure 8

Results on the proposed SR-based algorithm in hybrid domains for various test images.

Figure 9 display characteristic of performance metrics, F and PQM, with respect to iteration in all three hybrid domains, for a test image Chair. The termination criterion has also been illustrated to clarify how the output is obtained. The iteration count, n ₀, as marked in the graphs is the terminating iteration that is found when F is maximum in the vicinity of P Q M∼10 (here, 10 ± 2.5). This relaxation in value of PQM may be made taut for better empirical visual quality, but here it has been kept as a wide window of 5 units about 10 so that contrast enhancement factor (F) may be allowed to reach higher values.

Figure 9

Characteristic of performance metrics, F and PQM with respect to iteration count, showing the point of termination for test image, Chair.

The proposed noise-enhanced iterative algorithm is observed to give noteworthy enhancement in contrast, perceptual quality and colorfulness. The important aspects of the performance shall be discussed in this section.

8.1 Observations and Analysis

The application of SR-based iterative processing is observed to give remarkable increase in contrast quality of the image while ascertaining good perceptual quality. The empirical performance metrics for three test images have been tabulated in Table 1, showing comparison of SR in various hybrid and other domains, along with some non-SR based enhancement algorithms. For the purpose of comparison among enhancement algorithms, the results of DSR on the three tested hybrid domains shall be referred to as SV-DWT-DSR, SV-DCT-DSR, and DCT-DWT-DSR, corresponding to SV-DWT, SV-DCT and DCT-DWT domains, respectively. The following may be observed from the visual outputs and quantitative metrics.

• The perceptual quality metrics (PQM) of some of the input test images is less than or greater than 10, indicating its (quantitatively) poor visual quality. However, since the input image is poor only in terms of contrast, brightness and illumination but not with respect to blurring, blocking artifacts, grains and noise, the PQM value is similar to (and in some cases as high as) that for enhanced images. The main aspect to study PQM is in comparison with enhanced images, because enhancement of very dark images produces implicit noise in the enhanced images.

• Contrast and visual quality: In each of the three hybrid domains, the contrast enhancement factor of the enhanced image is observed to be nearly the same. In other words, the contrast enhancement observed in each of the hybrid domains is comparable. However, the perceptual quality obtained for similar contrast quality is higher (closer to 10) for DCT-DWT domain than other two domains (due to less artifacts).

• Blocking Artifacts: The zoomed-in portions of images enhanced in SV-DWT, SV-DCT and DCT-DWT domains for two test images are shown in Figs. 10 and 11. A stark observation is the presence of blocking artifacts in images enhanced in SV-DWT domains. Note that it is the collective contribution of both SVD and approximation coefficients that cause the blocking artifacts in SV-DWT processing. Singular values, representing descending order weights of low resolution coarse (emulating the nature of intensity values), are scaled up by non-linear iterative processing. As a result, after back-projection (IDWT) and upsampling, the boundaries and details appear pixelated. In other words, singular values inherently represent coarse information about an image. Therefore, SV of the LL subband implies incorporating double coarseness, once due to the approximation subband, and then by taking SVs of that band.

  Table 1 Comparative performance of the proposed technique with various existing techniques using performance metrics F [6], PQM [35], CEF [20], and Mean Opinion Score ^∗(MOS) for three test images.

  Figure 10

  

  Zoomed-in regions of DSR-enhanced Flower image for each hybrid domain, highlighting the extent of blocking effects at discontinuities for SV-DWT.

  Figure 11

  

  Zoomed-in regions of DSR-enhanced Grass image for each hybrid domain, highlighting the extent of blocking effects at discontinuities for SV-DWT.

  SV-DCT, sometimes produces over-brightening of details, but preserves the smoothness of boundaries to a great extent. It is important to note that SV-DCT does not display significant blocking as we do not operate on block neighborhood, but on entire image. Though, we are still processing the SVs, but here they represent weights of frequency coefficients of the original image (and not a low resolution intensity image). Thus, more information is being processed, leading to lesser possibilities of blocking/pixelating effects.

  DCT-DWT, on the other hand, remarkably preserves the quality of output with negligible artifacts, due to non-linear scaling of normalized DCT coefficients instead of layer weights. Note that here, the frequency components (and not layer weights) of low resolution subband are processed. All the frequency components are processed unlike coarse weights in case of SV-DWT. For example, for a 512×512 image with an approximation subband of 256×256, the DCT-DWT processing iterates on (256×256 matrix of DCT of the approximation band), unlike coarse weight array (1×256 array of SV of the approximation band).

• Iteration: According to Table 1, SV-DCT-DSR requires least iteration count among the three hybrid domains in question, while DCT-DWT-DSR requires maximum iteration to reach target output. Despite having similar computation complexity (discussed later in Section 8.4), SV-DCT gives faster performance than SV-DWT because the SR-enhanced iterative processing on singular values of DCT coefficients causes the image energy (or variance of distribution) to increase more rapidly (as shown in Fig. 4) than that of SV-DWT (as shown in Fig. 2).

• Choice of Δt in relation with iteration count: Since the algorithm is iterative in nature, and the term, Δt, or the sampling time acts as a multiplicative step-size in the iterative equation, larger values of Δt would naturally lead to similar performance metrics in lesser iteration. However, large values of Δt would not allow incremental observation of performance, and might sometimes cause situations where the perceptual quality might overshoot the desired window centered about value 10. Moreover, a large value would also significantly modify the dynamics – it would no longer approximate the dynamics of the continuous time system and would become something quite different i.e. a nonlinear mapping/filter with distinct properties that depend directly on the choice of time step. Hence, Δt should be chosen such that its value is small enough to allow incremental observation of refinement but does not require excessively large number of iterations.

  In the current case of hybrid domains, initial experiments and performance was observed for 50 iterations. Values of Δt chosen for SV-DWT was around 0.01, for SV-DCT was 0.001 to 0.005, and for DCT-DWT was 0.01 to 0.05. The choice of these values was a result of initial experimental observations. It can be noted that despite having higher operating values of Δt for DCT-DWT, the iteration count for still the highest for this hybrid domain. Our earlier work on DCT of images showed an nominal iteration count of 15 - 50 iterations. Since, here processing is being done on DCT of approximation band, the iteration count is comparable to that of DCT alone.

  Note that the algorithm iteratively processes each input image using the parameters that are computed from the global statistics of the coefficients or values of the input itself. Therefore, computation of optimal iteration count, n ₀ cannot be generalized because different input images would give rise to different parameter values that, in turn, would lead to different optimal iteration counts for each image. However, parameters a, b and Δt could be adaptively tuned and further studied to ensure that algorithm converges within a predefined number of iterations.

8.2 Comparative Analysis

Comparison of the proposed work in hybrid domains has been performed with our earlier work on stochastic resonance-based enhancement, along with some other non-SR-based techniques. Though there are several enhancement algorithms available in literature, we have presented a comparative analysis for only those that can be used for contrast enhancement of dark images. As stated before, for the purpose of comparison among enhancement algorithms, the results of DSR on the three tested hybrid domains shall be respectively referred to as SV-DWT-DSR, SV-DCT-DSR, and DCT-DWT-DSR, corresponding to SV-DWT, SV-DCT and DCT-DWT domains. Table 1 and Figs. 12, 13 and 14 display the quantitative and qualitative results for three test images in comparison with the following:

• Dynamic SR-based contrast enhancement in individual SVD (SVD-DSR) [9], DWT (DWT-DSR) [12], DCT (DCT-DSR) [10], intensity values (Intensity-DSR) [13]

• Non-dynamic Suprathreshold SR-based contrast enhancement (SSR) [8]

• Spatial domain algorithms - adaptive histogram equalization (AHE) [36] and Gamma correction (for γ=1.5). Note: Since histogram equalization is known to be unsuitable for very dark images [37], adaptive histogram equalization has been used for comparison here.

• ‘Auto Contrast’ control of Adobe Photoshop (Automatic)

• Spatial domain algorithms, Modified High-Pass Filtering (MHPF) [17], Single-scale Retinex (Retinex) [15], and Multiscale Retinex (MSR) [16]

• Frequency and hybrid domain enhancement techniques: Alpha rooting (AR) [18], Equalization of singular value of DWT (E-SVD-DWT) [22] coefficient and DCT coefficients (E-SVD-DWT) [23].

Figure 12

Proposed hybrid domain enhancement results on a low-contrast input image in comparison with other existing enhancement techniques.

Figure 13

Proposed hybrid domain enhancement results on a low-contrast input image in comparison with other existing enhancement techniques.

Figure 14

Proposed hybrid domain enhancement results on a low-contrast input image in comparison with other existing enhancement techniques.

In comparison with SR-based algorithms in other domains, the contrast and visual qualities obtained in hybrid domains are comparable (except SV-DWT-DSR where F is high, but PQM lower than other SR-based algorithms). Iteration-wise, SV-DCT-DSR appears to give minimum iteration count amongst the SR-based algorithms. For Flower image, the performance in comparable with some algorithms, but better than Gamma correction, MHPF, Retinex, MSR and Alpha rooting. For Grass and Chair, the outputs are inferior to Photoshop and MSR, but better than most of the other algorithms. Overall, the DCT-DWT-DSR domain appears to perform better than most of the compared algorithms in terms of F. SV-DWT-DSR performs well individually, but appears unsuitable for further application on comparison with other domains. Over all, for comparable contrast metrics, the perceptual quality of SR-based algorithms is next to Photoshop. Non-SR-based hybrid domain algorithms reach good contrast qualities, but have very low PQM for most of the test images.

In other words, DCT-DWT-DSR is observed to give noteworthy performance when compared with other existing non-SR-based enhancement techniques in terms of perceptual and contrast qualities. Among the SR-based algorithms, the DCT-DWT-DSR displays comparable perceptual quality but requires significantly fewer number of iterations than individual DWT-DSR. On the other hand, SV-DCT-DSR requires the least number of iteration counts and computation time among all SR-based algorithms.

Also, if the relaxation on PQM is loosened a little, higher contrast metrics may be obtained for all SR-based algorithms.

8.3 Observations on Colorfulness, HVS-Based Logarithmic Metrics and Subjective Scores

We have also attempted to present an initial analysis of other quality metrics for the enhanced images, in particular, some human visual system (HVS)-based metrics, colorfulness, and subjective visual quality.

8.3.1 Enhancement of Colorfulness

Though our entire processing is only on luminance vectors so as to leave the chromatic components undisturbed, the back-projection of H-S-V color space to R-G-B produces noticeable enhancement in colorfulness of the image. This primary reason behind this enhancement is the increased value of component V, and the resultant corresponding mapping into R-G-B. An interesting observation (as shown in Fig. 15 for seven test images) is that the color enhancement factor (CEF) is also observed to increase with SR iterations, and displays a resonant nature (giving peak at certain iteration count). In many cases this peak either corresponds to the peak in contrast enhancement factor, F, or is located nearby.

Figure 15

Characteristic of colorfulness performance metrics, CEF, with respect to iteration count for seven test images. Note the semi-resonant nature of the curves.

8.3.2 Logarithmic Contrast Metrics

Table 2 displays logarithmic metrics, EME and EMEE, for four test images and their hybrid-domain enhanced outputs. As stated earlier in Section 6, the logarithmic metrics have been observed just to present an initial reading of HVS aspects of the SR-based algorithm in hybrid domains. With respect to the EME and EMEE values of the low-contrast input images, the EME for DCT-DWT seems to be comparable and in some cases better than input. This is due to the absence of blocking artifacts in the DCT-DWT results. SV-DCT also appears to give sometimes better and some times lesser EME. On the other hand, EME for SV-DWT, however, is always less than that of input. This is due to the blocking artifacts that is clearly discerned by the human visual system in terms of visual quality.

Table 2 Logarithmic contrast metrics: E M E and E M E E (EME based on entropy), and subjective visual quality: Mean Opinion Score, (M O S) along with their standard deviation (S D).

EMEE (EME based on Entropy), is a entropy-based version of EME. Since entropy denotes average information, and maximum entropy denotes most uniform distribution of intensities, EMEE values appear to more closely be associated with the contrast-specific aspect of the human vision. This is why the EMEE metrics for both SV-DWT and DCT-DWT are (on an average) much higher for most of the test images. SV-DWT is observed to give lower EMEE than input.

8.3.3 Mean Opinion Score (MOS)

Tables 1 and 2 also display the Mean Opinion Score (MOS) of subjective visual quality provided by twenty subjects for some of the test images. Scores on a scale of 0 to 5 were provided to each test image and their enhanced results (code explained in Table 1). As expected, the dark and low-contrast inputs scored a minimum (between 0 and 1) denoting poor visual presence. The confidence of subjective scores (denoted by their standard deviations) are also within satisfactory range. The dynamic range of this scale is 5, and nearly all SD values (with very few exceptions) lie within a margin of 25 % (≤1.25).

On an average all dynamic SR-based algorithms, Photoshop and plain SVD-DCT scored the highest (between 2 to 3, signifying Average to Good visual quality), with comparable performance of hybrid domain SR algorithms with respect to the results of Photoshop.

It can be noted from both tables that overall, dynamic SR-based algorithms (and Photoshop) results have been rated best subjective visual quality (especially for naturally dark images, like Chair and Brushes). Also, at an average, the SR-based SV-DWT scored lower than other SR-based hybrid domain (and individual domain) results due to reasons discussed before.

8.4 Computational Complexity

The computation time of the algorithm is guided by two factors: the iteration count and complexity of transformation. Let the size of an image be M×N, and optimal iteration count by n ₀. Let c(T) denote the cost of transformation, and c(T ⁻¹) denote the cost of inverse transformation. Therefore, the cost of computation of an SR-based iterative algorithm in transformed domains is:

$$c(T) + n_{0}\cdot\left[\text{\textit{Cost of an iterative (SR) step}}+ c(T^{-1})\right] $$

The SR-iterative step requires a cost of O(N) for hybrid domains SV-DWT and SV-DCT (since only singular values of approximation of DCT coefficients are being processed). For domains DCT-DWT, the cost of a single SR step is O(M N). The individual and total costs of computation for each of the hybrid domains (excluding calculation of performance metrics) is tabulated in Table 3. Experiments were performed using on MATLAB ^TM (v.7.10) on an Intel (R) Core (TM) i5-3210 CPU working at 2.5 GHz and 4GB RAM.

Table 3 Cost of computation for SR-based Enhancement algorithm in each of the domains. Here, n ₀ is the optimal iteration count, and M and N are the image dimensions, respectively.

To serve as a platform-independent comparison in computation speeds, Table 3 displays computation costs of various DSR-based algorithms in big-O notation for a standard 512×512 grayscale image. It is important to note that there is no deterministic guarantee that the same F value shall be reached for each individual DSR-based method, because the optimal output of each algorithm is bound by constraints of both perceptual quality and enhancement factors. Therefore, gauging speed via only one constraint will not adequately represent the speed of the algorithm (which is also image-dependent). We call it ‘image-dependent’ because the selection of parameters, a and b are derived from input image statistics.

The time to reach optimal output in seconds for a sample 512×512 grayscale (Pepper) image is also displayed. Due to different underlying transformations, an iteration of one method may take longer/shorter than one iteration of another method. For example, while SVD-DSR gives an optimal iteration count of 5, it takes a massive 12 seconds to reach it. As apparent, the intensity based and SV-DCT domain processing are the top scorers in terms of processing time.

9 Conclusions

A study of noise-enhanced stochastic resonance-based iterative algorithm in hybrid domains was presented in this paper. An iterative equation, derived from the motion dynamics of a particle moving under weak force and noise in a bistable system, has been used here to induce resonance in the hybrid SV-DWT, SV-DCT and DCT-DWT domains. The main feature of the concept is the utilization of internal noise due to lack of illumination in a dark image. As a result of the SR-based iterative processing, the hybrid coefficients or values are non-linearly scaled, thereby producing noteworthy contrast enhancement of the image.

By keeping the termination criteria of optimal contrast while ascertaining good perceptual quality, a trade-off between perceptual quality measure and contrast metric is observed. Color enhancement, logarithmic contrast metrics, and subjective scores have also been observed to present an estimate of actual visual quality. It was found that noise-enhanced processing on SVs of DWT approximation band produces maximum blocking artifacts in the enhanced image. While SV-DCT performed best amongst the SR-based algorithms in terms of iteration count to reach target output, processing on DCT-DWT produced high contrast and visual quality with somewhat larger iteration count. The hybrid domain analysis proves that noise-enhanced iterative processing can be used in hybrid domains to produce noteworthy enhancement of dark images, and specific to application requirements, the SV-DCT and DCT-DWT may be further explored and modified. While currently the reported iterative processing is able to enhance only overall dark images, and not dull or bright images, suitable modifications can be made to address images of different types. Also, the SR mathematical model may be subjected to revision so as to develop an analytical model specific to images, and to address a wide variety of images. Moreover, depending on the input given to an SR system, global features such as edges, or local features such as textures may be enhanced by modeling the SR phenomenon in these contexts.