Visible and Infrared Image Fusion Using Low-Frequency Coefficients Mapping Fusion Rule in the LWT

In order to improve the effect of detect object in visible and infrared fusion image, a Low-Frequency Coefficients Mapping fusion rule is proposed. The overall fusion scheme based on lifting wavelet transforms. Firstly, the source images of same scene are decomposed using lifting wavelet transform (LWT). Secondly, a Low-Frequency Coefficients Mapping fusion rule is used to select low frequency lifting wavelet coefficients of the visible and infrared images. The fusion rule of local square maximum is used to combine corresponding high frequency coefficients. After fusing low and high frequency coefficients of the source images, the final fused image is obtained using the inverse LWT. The experiments show that the proposed Low Frequency Coefficients Mapping Fusion Rule in the LWT obtains a good fusion results as compared to previous image fusion rule in the LWT such as local energy weighted average and average gradient maxima.


Introduction
Visible spectral and infrared spectral sensors are two kinds of the most commonly used sensors. The infrared imaging sensor is only sensitive to radiation of the target scene, which is mainly determined by its emissivity and temperature difference. And visible spectral imaging sensor is only sensitive to the reflection of target scene instead of thermal contrast. Through image fusion, images from visible spectral and infrared spectral sensors are processed in different aspects and levels to achieve more reliable information of the target. Image fusion is widely used in remote sensing, automatic target recognition, machine vision and other fields [1,2] .
Many fusion techniques and algorithms for pixel-level fusion have been proposed, including the simplest weighted averaging, advanced pyramidal decomposition and wavelet transform methods. In this paper, a new image fusion rule based on LWT is put forward. The paper is organized as follows. In Section 2, the proposed LWT-based low frequency coefficients mapping image fusion rule for visible and infrared images is described. The experiment results are given in Section 3. Finally, Section 4 provides the conclusion.

The Proposed Fusion Approach
The proposed method of visible and infrared image fusion uses the lifting wavelet transform for decomposition and reconstruction of the source images. The overall fusion scheme based on LWT is shown in Figure 1. Firstly we decompose source images of same scene using LWT and then low frequency coefficients are obtained using low frequency coefficients mapping fusion rule. The fusion rule of local square maximum is used to combine corresponding high frequency coefficients. After fusing low and high frequency coefficients of the source images, the final fused image is obtained using the inverse LWT. We strongly encourage authors to use this document for the preparation of the camera-ready. Please follow the instructions closely in order to make the volume look as uniform as possible.

The basic principle of lifting wavelet
Lifting scheme [3,4,5] , put forward by Sweldens and Daubechies in the 1990s, is a kind of wavelet construction method does not rely on the Fourier transform. The wavelet transform based on lifting method is also called second generation wavelet transform. It can not only maintain the timefrequency localization and other characteristics of traditional wavelet, but also overcome limitations of its long time execution and the need of requiring a large memory. Lifting wavelet algorithm realization is divided into three steps: division, prediction, update and reconstruct.

1) Split
The original signal X n (n is behalf of the resolution) is divided into a low-resolution x n (even-numbered sequence 2n ) andx n (odd-numbered sequence X 2n 1 ) two parts in this step: x n x 2n x n x 2n 1 The signal is simply divided into two parts in this step, and does not change the description of the signal. The next lifting step is adopted to regroup both sequences to reduce the correlation between them.

2) Predict Predictx
n and define prediction P by x n . The prediction signal is as follows: The value of the error between and the predicted value represents the signal details . c) Update In order to maintain certain properties of the original signal X(n), update is defined and update with the subset of the data , as follows: This step is known as the primal lifting in lifting method. If the decomposed approximate signal performs the above three decomposition steps and after a certain number of iterations, we can get a multi-level decomposition of the original signal. As can be seen from above, using lifting method to implement wavelet decomposition has the advantage that it can decompose wavelet transformation into a few very simple steps.

4) Reconstruction
Reconstruction is the inverse step of the decomposition process. It has three steps: anti-Update, anti-predict and merge. The reconstruction equation is as follows: x n Merge x n , x n Image fusion algorithm based on lifting wavelet has following steps. a)At First, take one-dimensional lifting transform by row direction in image matrixes of fused image A and B, get the approximate coefficient matrix and the details of the coefficient matrix respectively; the predict and update equation is as follows: d n x n x n c n x n b) Then take one-dimensional lifting transform by column-direction in the approximate coefficient matrix and the detail coefficients matrix that get from decomposition of A and B, get a low frequency coefficient matrix and three high-frequency coefficient matrix of the image, which is the completion of a layer wavelet decomposition of the image.
Repeat decomposition process to coefficient matrix of the image, the image can be wavelet decomposed at any scale as traditional.
c) Use certain fusion rules for low frequency coefficients and high frequency coefficients from the decomposition of image A, B, and then get fused image through inverse transform.

Fusion Rule
The low-frequency part reflects the overview of the source image at this resolution, including key details of the source image information. The high-frequency part contains a large number of the source edge details. Considering the characteristics of decomposition sub-bands, we adopt different fusion rules to low-frequency coefficient and high-frequency coefficient in this section.

Low-Frequency coefficient mapping Fusion Rule
To improve target's thermal feature, a low-frequency coefficients mapping fusion rule is proposed, in which the weights are obtained using the low frequency lifting wavelet coefficients mapping of the infrared image: L x, y α x, y L x, y α x, y L x, y Where L x, y , L x, y expresses visible and infrared image's low-Frequency coefficient matrix at the N-th lifting wavelet decompose level. L x, y denotes the fused image's low-frequency coefficient matrix. The weightsα x, y , α x, y are estimated as: Where E denoted low frequency coefficient mean of infrared image B. When the weight is zero, this means the substitution of an image by another. 1 k , 2 k are estimated as:

Fusion Rule of High-Frequency coefficient
We use the fusion rule of local square maximum to combine corresponding high sub-band coefficients. We first calculate the local square features V x, y and V x, y of the N-th highfrequency coefficients of the visible and infrared images. V x, y , x, y are calculated over a 3-by-3 or 5-by-5 window using the formula: Where H matrix at t coefficient coefficient   From Table 2, it can be seen the fusion image based on the proposed method is the highest on Entropy, Average Gradient, Standard deviation, spatial frequency and Speed. It indicates that the featured information and average information volume are comparatively highest. Meanwhile, it proves that the image is the clearest.

Conclusions
In this paper, by using LWT, we have decomposed source visible and infrared image into couples of sub-band images which are separately approximate images or detail images. The low frequency components give an overview of the target object or scene and the high frequency components highlight the detail information. By using low-frequency coefficients mapping and high-frequency coefficients local square maximum fusion rules to sub-band images, we can make full use of the temperature and visible features on the target object or scene. Experimental results show that the proposed image fusion rule obtained better results in LWT domain when tested with performance measures, Entropy, Average Gradient, Standard deviation, Spatial frequency and Speed.