Keywords

1 Introduction

Image fusion [1, 2] is a technique that integrates complementary information, scattered in different images into a single composite image. The fused image contains more information than any of its source images with reduced noise and artifacts. Hence, the results of the fusion are more useful for human perception and further image processing tasks. The idea behind this technique is the sensors of limited capacity that cannot capture the complete information of a scene. For example, in medical imaging [1], the computed tomography (CT) image is suitable for bone injuries, lungs, and chest problems, whereas the magnetic resonance imaging (MRI) is used for examining soft tissues like spinal cord injuries, brain tumors, etc. Various fusion methods are developed by the researchers. These methods are classified into pixel-level [3], feature-level [4], and decision-level fusion [5]. Pixel-level fusion is again divided into spatial- and transform-domain fusion. Spatial-domain fusion methods (averaging, weighted averaging, principal component analysis) are simple but sometimes give poor fusion results. Transform-domain fusion methods avoid the problems of spatial-domain methods and give better results. Different transforms are developed by the researchers such as Laplacian pyramid, gradient pyramid, wavelet transform [6], and contourlet transform [7]. Wavelet transform is very popular among the researchers, but it has limited number of directions, due to which it is not efficient in handling two-dimensional singularities. Contourlet transform, proposed by Do and Vetterli [7], provides flexible number of directions but is shift variant. Shift variance causes many problems like pseudo-Gibbs phenomenon near singularity. To tackle this problem, Chunha et al. [8] gave a multiscale, multidirectional, and shift-invariant transform known as nonsubsampled contourlet transform (NSCT). It has better frequency selectivity and improved temporal stability, and it removes the problem of pseudo-Gibbs phenomenon that causes smoothness near singularities. All these properties make it a suitable transform for image fusion.

In this work, we have proposed a local energy-based image fusion in NSCT domain. The source images are decomposed in transform domain by applying NSCT. Then, we have calculated local energy of each coefficient and then compared the local energy of corresponding coefficients of source images. The coefficient with highest local energy is selected. After obtaining the coefficient set for the fused image, the inverse NSCT is applied to get the final fused image.

The rest of the paper is organized as follows: Sect. 2 gives the basic concept of NSCT. Section 3 describes the proposed method in detail. Experimental Results and discussion are discussed in Sect. 4. Finally, the paper is concluded in Sect. 5.

2 Nonsubsampled Contourlet Transform (NSCT)

NSCT [9, 10] is a multiscale, multidirectional, and geometrical transform. Unlike contourlet transform, it is shift invariant. In contourlet, the shift variance is caused by the downsampling in Laplacian pyramid filter banks (LPFB) as well as in directional filter banks (DFB) [11]. NSCT achieves shift invariance by removing the downsampling and upsampling from the LPFB and DFB. The construction of NSCT is the combination of nonsubsampled pyramidal filter banks (NSPFB) and nonsubsampled directional filter banks (NSDFB). The NSPFB provides the multiscale property, whereas the NSDFB gives multidirectional property to NSCT. Its implementation is similar to that of nonsubsampled wavelet transform obtained by a`trous algorithm. Here, the filter of the next decomposition level is obtained by upsampling the filter of previous stage. Hence, multiscale property is achieved easily without extra filter design algorithm.

The DFB is proposed by Bamberger and Smith [11]. It is obtained by incorporating the critically sampled two-channel fan filter banks and resampling operator. The DFB has tree-like structure which splits the frequency plane in directional wedges. But it is not shift invariant. To make it shift invariant, the downsampler and upsampler are removed. This results in tree-shaped two-channel nonsubsampled filter banks.

The NSCT is constructed by combining the NSPFB and NSDFB. It is invertible and satisfies the anisotropic scaling law. Also, the removal of downsampling makes the filter designing simpler.

3 The Proposed Method

In this paper, we have proposed local energy-based image fusion in NSCT domain. In local energy-based fusion, the decision of coefficient selection is based not only on the coefficient itself but also on its neighboring coefficients. Hence, local energy-based fusion gives more efficient results than single coefficient-based fusion rules such as maximum selection or average fusion rules. The local energy is calculated by taking absolute sum of the neighboring coefficients of the current coefficient. Then, we compare the local energy of the coefficients of the source images and select the coefficient that has higher value of local energy.

In the proposed method, we have performed fusion in NSCT domain. It has better directional selectivity and is shift invariant. Also, it avoids the pseudo-Gibbs phenomenon, which is a big problem in wavelet transform- and contourlet transform-based fusion methods. The proposed algorithm can be summarized as below:

  1. Step 1

    The source images are decomposed into coefficient sets using the NSCT.

    $$ Im_{1} (i,j)\mathop \to \limits^{\text{NSCT}} Cf_{1} (i,j)\quad Im_{2} (i,j)\mathop \to \limits^{\text{NSCT}} Cf_{2} (i,j) $$
  2. Step 2

    Local energy of each coefficient is calculated in 3 × 3 window using the following formula:

    $$ e_{1} (i,j) = \sum\limits_{i - 1}^{i + 1} {} \sum\limits_{j - 1}^{j + 1} {Cf_{1} } (i,j)\quad e_{2} (i,j) = \sum\limits_{i - 1}^{i + 1} {} \sum\limits_{j - 1}^{j + 1} {Cf_{2} } (i,j) $$
  3. Step 3

    The coefficient with higher local energy is selected from the two.

    $$ Cf(i,j) = \left\{ {\begin{array}{*{20}l} {Cf_{1} (i,j)} \hfill & {{\text{if}}\;|e_{1} (i,j)| \ge |e_{2} (i,j)|} \hfill \\ {Cf_{2} (i,j)} \hfill & {\text{otherwise}} \hfill \\ \end{array} } \right. $$
  4. Step 4

    Reconstruct the final fused image by applying inverse NSCT on the above coefficients.

    $$ Cf(i,j) \, \mathop \to \limits^{{{\text{Inverse}}\;{\text{NSCT}}}} \, F(i,j) $$

4 Experimental Results and Discussion

In this section, we have shown the results of the proposed method and its comparison with other methods. For experiment, we have taken two different multimodal medical images (Figs. 1a, b and 2a, b) of size 256 × 256. Each set contains one CT image and one MRI. To show the effectiveness of the proposed method, it is compared with curvelet transform [12], contourlet transform, and dual-tree complex wavelet transform [13] and Duabechies complex wavelet transform-based [14] methods with three different fusion rules (absolute maximum, local energy, and edge preserving) [10, 15, 16]. The visual results of the proposed and other methods are shown in Figs. 1 and 2. From Figs. 1 and 2, we see that the proposed method has better results than the other methods. Since human eye has limited vision capacity, it cannot observe the minute difference in the images. Hence, we have also evaluated the results quantitatively. Mutual information, edge strength (\( Q_{AB}^{F} \)), and fusion factor quality metrics [17] are used for comparing the results quantitatively. Results are presented in Table 1 for both set of images. From the combined observation of qualitative and quantitative results, we can say that the proposed method has better performance than the other methods.

Fig. 1
figure 1

Fusion results for first set of multifocus images. a Image 1. b Image 2. c The Proposed Method. d Cvt (Max). e Cvt (Energy). f Cvt (Edge). g CT (Max). h CT (Energy). i CT (Edge). j DTCWT (Max). k DTCWT (Energy). l DTCWT (Edge). m DCxWT (Max). n DCxWT (Energy). o DCxWT (Edge)

Full size image
Fig. 2
figure 2

Fusion results for second set of multifocus images. a Image 1. b Image 2. c The Proposed Method. d Cvt (Max). e Cvt (Energy). f Cvt (Edge). g CT (Max). h CT (Energy). i CT (Edge). j DTCWT (Max). k DTCWT (Energy). l DTCWT (Edge). m DCxWT (Max). n DCxWT (Energy). o DCxWT (Edge)

Full size image
Table 1 Fusion results for both set of medical images
Full size table

5 Conclusions

This paper presents a local energy-based image fusion method using NSCT. NSCT provides better fusion results because of its shift-invariant property. It has better directional selectivity and avoids the pseudo-Gibbs phenomenon. These features of NSCT improve the results of the proposed method. The proposed fusion method is based on local energy. Local energy contains information of current coefficient and its neighboring coefficients. It carries more information than a single pixel has. Hence, it is more reliable and efficient than single pixel-based fusion rules. To prove the effectiveness of the proposed method, it is compared with curvelet transform, contourlet transform, dual-tree complex wavelet transform, and Daubechies complex wavelet transform-based fusion methods. Three standard quantitative measurements—mutual information, edge strength, and fusion factor—are used for evaluating the results quantitatively. Both the visual and quantitative results show that the proposed method has better performance than the curvelet, contourlet, dual-tree complex, and Daubechies transform-based fusion methods.