Research on satellite remote sensing image fusion algorithm based on compression perception theory

Abstract

With the development of space technology in recent years, various spacecraft with different sensors have been launched one after another, and there are more and more satellite remote sensing images in different situations. How to obtain better quality images has become the main research direction in the field of image fusion. Image fusion is an important branch and main research object of information fusion. Generally speaking, with the rapid development of the information society, people have higher and higher quality requirements for a variety of images, so There are serious challenges in storing, transmission and signal sampling. Nowadays, with the development of compressed sensing (CS) theory, a new sampling method has been extensively studied by many scholars. Compared with traditional Nyquist sampling theorem, Compression sensing can respond to the original signal with fewer sampling points. This paper is introduced the fusion algorithm of satellite remote sensing image based on compressed sensing. Firstly, the basic theory of compressed perceptual discretization signal is introduced. Secondly, a sparse CS remote sensing image fusion algorithm based on wavelet transform is proposed. Finally, through simulation verification, comparing the widely used IHS fusion and PCA fusion image methods, the method in this paper can get higher correlation coefficient and lower interaction entropy and spectral distortion after fusion. Compared with other methods in this paper, the fusion image can carry more space information and the original image is more similar.

Keywords

Compressed sensing image fusion remote sensing wavelet transform

1. Introduction

Information fusion technology is formed with the development of radar information processing and command automation system. It is about how to use multi-source information collaboratively in order to obtain a more objective and essential understanding of the same thing and the goal is comprehensive information processing technology. With the development of computer hardware, the amount of information obtained by a single sensor is far from satisfying the requirement of information acquisition. More and more attention has been paid to the theoretical research of multi-sensors. The appearance of image fusion also develops with the diversity of signal processing. At the same time, it is an important research content in the development of information fusion, which can be said to be the development of a small subject in a wide range. With the frequent appearance of the range of image processing in our daily work, we need to process the image data with different image features and to fuse the image, so that the final fusion image has more accurate signal interpretation.

Multi-sensors can get multiple images and a single sensor can get the image information at multiple time points. After a process of feature extraction and feature fusion of source image, the image information can be obtained more accurately, the image information can be obtained more accurately in the process of feature extraction, feature fusion and so on. The image quality is more perfect and the computer processing is more convenient to fuse the image to generate a new image signal with rich feature information. Remote sensing technology [4, 5] is based on different space, time, spectrum, radiation resolution to provide different bands of electromagnetic spectrum data. The remote sensing data obtained by a single remote sensor is limited and cannot provide comprehensive and effective feature information. With the development of multi-spectral remote sensing in the field of environmental detection, the research on image fusion is of great significance to human life. Image fusion [6] refers to the process of using two or more images to fuse useful information into a new image. Image fusion can be applied to a variety of image data, from different ways, different times, on the same sensor to obtain the image; There are also images from different sensors, and then according to the needs of the actual situation, these source images containing all the feature information are fused into a more accurate, more complete and more visible image information, which is convenient for people to study. Image fusion can fully complement each other and eliminate redundancy in time and space. The analysis and extraction ability of image information is also improved. Therefore, the research of remote sensing image fusion is an inevitable trend.

Satellite remote sensing [7, 8] mainly refers to the use of visible light, microwave and other detection instruments through photography or scanning, information sensing, transmission and processing from platforms in outer space. Which can identify the nature and state of motion of ground matter. With the rapid development of internet and communication technology, the technology of remote sensing image processing has been continuously updated and optimized. At present, the application field of remote sensing image is increasing day by day, which brings great convenience to the survival and development of human society. The industries of military affairs, spaceflight, communication, urban planning, environmental monitoring and so on are inseparable with remote sensing technology at the same time. Because of the variety of remote sensing equipment and the different shooting time, location and angle, it is not advisable to obtain the information from a single equipment in order to maintain the integrity of the information. Remote sensing image data fusion is a new technology in recent years. It refers to the processing process of multi-sensor image data and other information. It focuses on processing multi-source data according to certain rules (or algorithms). Obtain more accurate and abundant information than any single data, and generate a composite image with new spatial, spectral and temporal characteristics [12].

Compression sensing [13, 14, 15, 16], also known as compressed sampling or sparse sampling, which is a technique for finding sparse solutions of underdetermined linear systems. It is widely used in electronic engineering, especially in signal processing to obtain and reconstruct sparse or compressible signals. Compared with Nyquist’s theory [17, 18], this method is able to restore the whole signal from a few measurements by using the characteristic of sparse signal. Compression sensing is based on the compressibility of the signal. It realizes the perception of high-dimensional signal by non-correlation observation of low-dimensional space, low-resolution and under-Nyquist sampled data, which enriches the optimization strategy of signal recovery. It greatly promotes the combination of mathematical theory and engineering application. Although compression sensing was proposed by scientists such as [19], as early as the last century, there were considerable theoretical and applied groundwork in related fields, including image processing, medical imaging, geophysics, computer science, applied mathematics, etc. The appearance of compressed sensing provides a rich content for information processing, and shows a new vitality in application. In 2008, Wan used compressed sensing technology to carry out image fusion experiments, and verified the feasibility of this algorithm [20]. At present, many fusion algorithms have been widely used, such as IHS (intensity-hue-saturation) transform and Curvelet transform [21], NSCT (nonsubsampled contourlet transform) transform [22] and principal component analysis (PCA) image fusion method [23], wavelet transform fusion method for enhanced flight vision system, Brovey transform method for improved low-pass filtering, SVR (synthetic variable ratio) transform method for spectral and spatial local correlation [24], Nearest neighbor diffusion-based Pan-sharp [25] fusion method and Gram Schmidt (GS) transform [26], etc.

Compressed sensing theory has only been established for several years, but its influence has swept over most applied sciences, which has attracted wide attention in academia and industry. Although compression sensing has a wide range of applications, few innovations and breakthroughs have been made in the field of remote sensing images. Although some scholars have applied over-compression sensing technology in the process of remote sensing image fusion, and proposed a remote sensing image fusion algorithm based on FFT sparse compression sensing domain, but this algorithm is not perfect [27]. The spatial resolution and spectral information of the image still need to be improved.

In this paper, a remote sensing image fusion algorithm based on wavelet sparse basis in compressed sensing domain is proposed. Firstly, the article introduces the basic theory of compressive perception. Secondly, a sparse CS remote sensing image fusion method based on wavelet transform is proposed based on the combination of compression perception and image fusion, in which the traditional basic methods and evaluation criteria of image fusion algorithm are mainly introduced. At the same time, the whole evaluation of the fused image is carried out, and the flowchart of the method is shown to facilitate the follow-up research. Finally, the simulation results show that the proposed method has a better effect than other methods, and provides a reliable guarantee for image fusion. In general, this paper combines compression sensing technology with image fusion, which provides a medium for the current large amount of data, storage speed and so on, and lays a foundation for future research.

2. Proposed method

2.1 Compression perception foundation

In 1928, Nyquist, an American engineer, first proposed that the sampling frequency should be more than or equal to twice the highest frequency in the spectrum of analog signals. This theorem occupies a dominant position in the field of signal processing and other fields for a long time. With the development of information society, the volume of information processing is increasing, mass data is facing great pressure in the aspects of transmission, processing and storage, and the data information obtained according to traditional Nyquist sampling theorem is very huge. The limitations are increasingly apparent. The appearance of compression sensing abandons the redundant information in the current signal sampling. It gets the compressed sample directly from the continuous time signal transformation, and then uses the optimized method to process the compressed sample in the digital signal processing. It can recover the original image information perfectly and accurately under the condition that the sample rate is much less than Nyquist sampling rate, which shows great superiority.

Signal sampling is a bridge between the analog world and the digital world. With the rapid development of information technology, the demand for information increases dramatically, which results in great pressure on signal sampling, transmission and storage. The traditional signal sampling criterion Shannon/Nyquist sampling theorem has been difficult to keep pace with the rapid development of information technology. Some literatures [28, 29, 30, 31, 32, 33, 34] also introduce some compression-aware image fusion methods, which have their own advantages. It is pointed out that when the signal is sparse or compressible, the original signal can be reconstructed accurately or approximately precisely with far less than the number of samples required by Nyquist’s sampling theorem. The general framework of CS theory can be divided into the following three steps: first, select an orthogonal basis and sparse the signal on the orthogonal basis. Second, select the measurement matrix which is not related to the orthogonal basis, and measure the sampling. Third: reconstruct the signal from the measured sample value.

2.1.1 Sparse representation of signal

The signal sparse representation is first used before the compression sensing. The basic design method of defining the signal sparse representation is as follows: Signal $X$ has $B=\psi^{T}X$ under orthogonal basis $\psi$ . For M and N, the coefficients satisfy the following formulaï¼š

$\displaystyle\left\|B\right\|_{p}\equiv\left({\sum{\left|{B_{i}^{p}}\right|}}% \right)^{1/p}\leqslant R$ (1)

It is shown that signal $X$ can be sparse under $\psi$ orthogonal basis. If the potential of the support region $\left\{{i:\theta_{i}\neq 0}\right\}$ of the transformation coefficient $\theta_{i}\leqslant x,\psi_{i}>$ is not greater than K, it is considered that the signal $X$ is K-sparse. If the signal is sparse or can be expressed sparsely on an orthogonal basis, the original high-dimensional signal can be projected onto a low-dimensional space by designing a measurement matrix independent of the transform basis. Then, by solving an optimization problem, the original signal can be reconstructed with a high probability by using these small quantities of measurements. It can be seen that the sparse characteristics of the signal is the premise and basis for the application of compression sensing theory, and it is directly related to the reconstruction accuracy of compression sensing. The accuracy of the signal reconstruction process can be guaranteed only by selecting more efficient representation signals to make the signals sparse. In practice, however, our signals are not sparse or ideal sparse. So in this case, according to the research in Candes and Tao [28], when the sparse of the original signal in the transform domain presents an exponential decrease, compression sensing can also reconstruct the sparse signal. The error generated in the reconstruction is:

$\displaystyle E=\left\|{\hat{x}-x}\right\|_{2}\leqslant A_{p}\cdot R\cdot\left% ({\frac{K}{\log N}}\right)^{-r}$ (2)

$A_{p}$ is constant, affected by $r$ , $r=1/p-1/2,0<\rho<2,R>0$ .

2.1.2 Signal observation sampling

In the compressed sensing measurement model, it is not the direct measurement signal itself, but the measurement value obtained by projecting the signal $x$ onto a set of measurement vectors $\Phi=\left(\varphi_{1},\varphi_{2},\ldots,\varphi_{M}\right)$ . Written in matrix form as follows:

$\displaystyle y=\Phi x$ (3)

In the formula $x$ is the $N\times 1$ matrix, $y$ is the $M\times 1$ matrix and $\Phi$ is the measurement matrix of $M\times N$ .

$\displaystyle y=\Phi x=\Phi\Psi s=\Theta s$ (4)

The measurement dimension $M$ is much smaller than the signal dimension $N$ , and the inverse problem of the solution Eq. (3) is a ill-conditioned problem, so the signal $x$ cannot be solved directly from the $M$ measurements. Since s in Eq. (2) is K-sparse, that is, there are only K non-zero coefficients, and $K<M<N$ , then by using the sparse decomposition algorithm in signal sparse decomposition theory, the sparse coefficient s can be obtained by solving the inverse problem of Eq. (3). Then return Eq. (1) to further obtain the signal. In order to ensure the convergence of the algorithm, so that $K$ coefficients can be accurately recovered from $M$ measurements, the matrix in Eq. (3) must satisfy the restricted equidistant characteristic (RIP) criterion, that is, if there is a strict $K$ sparse (compressible case), the K coefficients in the matrix must satisfy the restricted equidistant characteristic (RIP) criterion. The vector $p$ , matrix $\Theta$ , which requires 3K), guarantees the following inequalities:

$\displaystyle 1-\varepsilon\leqslant\left\|{\Theta v}\right\|_{2}/\left\|v% \right\|_{2}\leqslant 1+\varepsilon$ (5)

$\varepsilon>$ 0. RIP criterion in the formula is that the measurement matrix $\Phi$ and sparse matrix $\Psi$ satisfy the requirement of irrelativity. The signal can be reconstructed better when the measurement number $M$ satisfies $M\geqslant K\log(N/K)$ .

2.1.3 Signal reconstruction

Define a vector $x=[x(1),x(2),\ldots x(L)]^{T}$ $l$ -norm:

$\displaystyle\left\|x\right\|=\left(\sum_{l=1}^{L}\left|{x_{i}}\right|\right)^% {1/l}$ (6)

When $l=$ 0, 0-norm is obtained, which represents the number of non-zero terms in vector $x$ . The most direct method of signal reconstruction is to write the solution in $l_{0}$ norm.

$\displaystyle\min\left\|x\right\|_{l_{0}}\ \textit{s.t.}y=\phi\psi_{s}$ (7)

However, the optimization problem based on $l_{0}$ -norm is a NP problem and cannot be solved at all. From a mathematician’s point of view, this is very similar to the sparse decomposition problem. All the existing sparse decomposition algorithms can be directly applied to signal reconstruction. In general, it is transformed into a solution in $l_{1}$ norm, which transforms a non-convex optimization problem into a strict convex optimization problem. All of the above considerations are equality constraints, but in practice, the measurement process may over-introduce noise, so $y=\phi\psi_{s}$ becomes $y=\phi\psi_{s}+n$ where $n$ is Gaussian white noise. Therefore, the above-mentioned problems become the following inequality constraints.

$\displaystyle\min\left\|x\right\|_{l_{0}}\textit{s.t.}\left\|{y-\phi\psi_{s}}% \right\|_{2}<\varepsilon$ (8)

At present, the most commonly used reconstruction algorithms are constructed based on the optimization method and the matching and tracking method, and the course is roughly classified into the following three categories: a series of greedy algorithms for the minimum $l_{0}$ -norm, the linear programming optimization algorithm proposed for the minimum $l_{1}$ -norm, And a statistical optimization reconstruction algorithm.

2.2 Sparse CS remote sensing image fusion based on wavelet transform

2.2.1 Remote sensing image and fusion algorithm

Image fusion refers to the combination of two or more images into a new image by a specific algorithm. It is a branch of data fusion, proposed in the late 1970s. Remote sensing image refers to using remote sensing technology, sensors to detect and receive information from the target object to form an image. In the middle and late 1990s, with the launch of several remote sensing radar satellites, the number of remote sensing images with different spatial, spectral and temporal resolutions available in the same area became extremely abundant. Satellite remote sensing image fusion technology can be divided into three levels: pixel level, feature level and decision level. The real-time data of satellite remote sensing image reconnaissance is strong, the data source is large, and the amount of data is huge. It is necessary for the image to extract useful information and form valuable information through the process of de-coarding and refining, identifying fake things and keeping the real things in order to extract useful information from them. The traditional satellite remote sensing image processing methods cannot effectively utilize the massive satellite remote sensing image data, and cannot fully meet the requirements of accurate geographic information extraction. In this context, satellite remote sensing image fusion technology, especially multi-spectral image and panchromatic image fusion, to improve the spatial resolution and spectral information.

The remote sensing image fusion technology is to combine the sensor images with different properties with each other, and the satellite remote sensing technology is developing towards the high spatial resolution and the high spectral resolution direction, and the accuracy of the interpretation image is improved. Although the fused images are a wide variety of images, it is generally divided into three fusion principles:

1.
The fused image must ensure the clarity and texture of the image to the maximum extent. Preserves the basic salient features of the original image.
2.
Basic use of raw data for fusion, avoid unnecessary pigment addition.
3.
Improve the signal-to-noise ratio of the fused image.

2.2.2 Traditional methods of remote sensing image fusion

In this paper, we only study the pixel-level fusion level. The fusion methods widely used in pixel-level fusion include weighting, IHS transform, PCA and wavelet transform, etc.

(1)
OMP algorithm

OMP algorithm is based on the core idea of matching pursuit algorithm (MP), and it is optimized. Matching pursuit algorithm is a greedy algorithm, which is to find the atoms (also known as the best atoms) that match the original signal best from the super complete dictionary through iterative calculation. The purpose is to combine the best atoms linearly to get the original signal.
(2)
Weighted fusion

Weighted fusion is the process of multiplying pixel points at corresponding positions in different images and determining weighted coefficients, and then simply fusing the weighted images. Although this method is a simple method to realize in image fusion, it is very difficult to determine the weighted coefficient because different weights will affect the quality of the fused image to a great extent. The image A1 and A2 to be fused, both of which are of $M\times N$ and the fused image is A, the mathematical expression of the fusion process is as followsï¼š $A\left({n_{1},n_{2}}\right)=\alpha A_{1}\left({n_{1},n_{2}}\right)+\beta A_{2}% \left({n_{1},n_{2}}\right)$ . $\alpha$ and $\beta$ are weighted coefficients. Generally, there is $\alpha+\beta=$ 1, and the average weighted fusion is to weight the pixels of each image by 0.5.
(3)
IHS transform

Color images are usually represented by dividing the images into three primary colors, but the tricolor images are not suitable for people’s intuitive feelings, but the brightness of the image is I (Intensity), However, tonal H (Hue) and saturation S (Saturation) are the main factors to evaluate the basic feelings of an image. The principal wavelength in the image spectrum determines the size of H, the saturation is the ratio of the spectral intensity of the principal wavelength to all the spectra, and I is still the luminance of the spectrum.
(4)
Principal component analysis

Principal component analysis (PCA) is a mathematical dimensionality reduction method to find several synthetic variables to replace the original many variables, so that these comprehensive variables can represent the information of the original variables as much as possible, and they are not related to each other. On the basis of the new synthesis variables, we can further statistical analysis. PCA is essentially a multi-dimensional orthogonal linear transform based on statistical characteristics. PCA does not limit the number of fusion bands in fusion. The principal component analysis (PCA) algorithm is similar to the HIS algorithm, but its advantage is that it can handle arbitrary multi-channel multispectral images. Its algorithm idea is: it first carries on the PCA transform to the multispectral image. In general, the first principal component contains most of the gray-scale information of multispectral images, while the other principal components contain the spectral information of multispectral images. Then, the panchromatic image is matched with the first principal component by histogram matching, and the panchromatic image with histogram matching is replaced by the first principal component with other principal components for anti-PCA transform to get the fusion image. The convergence rates of the four fusion algorithms are compared as follows.

Table 1
Comparison of convergence of algorithm

Fusion algorithm Degree of convergence

Weighted fusion Low

IHS transform Low

PCA Middle

OMP High

2.2.3 Algorithmic framework and basic steps

Fusion algorithm	Degree of convergence
Weighted fusion	Low
IHS transform	Low
PCA	Middle
OMP	High

The approximate block diagram of remote sensing image fusion algorithm based on CS theory is shown as follows.

Figure 1.

Flow chart of remote sensing image fusion algorithm based on CS theory.

In the current environment, CS is still in the initial stage, and the development of CS hardware level is completely blank, so CS can’t be processed in practice. In general, CS theory is applied to remote sensing image fusion at this stage. Only the images that have been sent back from space to the ground are measured by CS, sampled, then restored by a certain fusion algorithm. Finally, the fused remote sensing images are reconstructed by solving the optimal problem.

In response to the above statement, the key steps of the algorithm are as follows:

(1)

Sparse treatment of wavelet sparse base

Most of the signals in nature are not sparse, so we need to turn them into sparse representations in some ways. For example, $x=\sin({2\pi*697*t})+\sin({2\pi*1633*t})$ is a continuous multi-valued signal in time domain, but a discrete two-valued signal in frequency domain. At present, fast Fourier transform (FFT), discrete cosine transforms (DCT) and discrete wavelet transform (DWT).) are commonly used in sparse transform. But in the actual image processing process, the sparse effect of non-redundant orthogonal transform, such as wavelet transform and discrete cosine transform, is the best. In this paper, wavelet transform is used to sparse remote sensing images.

(2)

Reconstruction algorithm using OMP algorithm

The OMP algorithm is used in image reconstruction. The OMP algorithm is improved from the early MP algorithm. The improved OMP algorithm converges faster than the MP greedy algorithm. The input elements of the algorithm are measurement matrix $\Phi$ , sampling data $y$ and sparsity $K$ . The main flow of the algorithm is as follows:

Algorithm flow

1. Given

r_{0}

, index set

\Lambda_{0}

and iterations

t

initialed value:

r_{0}=y

\Lambda_{0}=\emptyset

t=

2. Find the solution of satisfying

\lambda_{t}=\arg\max_{j=1,\ldots,N}\left|{\left\langle{r_{t-1},\varphi_{j}}% \right\rangle}\right|

;

3. Update index set

\Lambda_{t}=\Lambda_{t-1}\cup\left\{{\lambda_{t}}\right\}

, also record the set of reconstructed atoms found in the measurement matrix

\Phi_{t}=\left[{\Phi_{t-1},\Phi_{\lambda_{t}}}\right]

;

4. According to the least square method

\hat{x}_{t}=\arg\min\left\|{y-\Phi_{t}\hat{x}}\right\|_{2}

t=t+1

;

5. Update residuals

r_{t}=y-\Phi_{t}\hat{x}_{t}

t=t+1

;

6. Judge

t

and

K

, if

t>K

break; else return 2;

(3)

Determine fusion factor

The fusion rule in this paper only uses simple weighted fusion, passing coefficient and weighting coefficient as panchromatic remote sensing image and multi-spectral image fusion. Where $\alpha+\beta=$ 1. Nine experiments have been carried out in this paper to select the value of fusion coefficient, and the implementation process is introduced in the following paper.

3. Experiments

In order to verify the feasibility and efficiency of the CS-based remote sensing image fusion method discussed in this chapter, the following two aspects are compared. On the one hand, it is compared with the method in reference [29], on the other hand, it is compared with the widely used IHS fusion and PCA fusion. The fusion flow of this algorithm is as follows: First, The $R, G, B$ values of the source image A and B are read, the three primary color $R_{a},G_{a},B_{a},R_{b},G_{b},B_{b}$ values of the two images are divided, and then the six color values are sparsely processed using wavelet basis respectively. then the six sparse matrices are measured and sampled through the random gaussian matrix. the random gaussian matrix is a random matrix with an obedience to the gaussian distribution, the mean value is zero, and the variance is 1. then the sample value is weighted fusion, and finally the fusion image is reconstructed by the OMP algorithm.

The main flow chart of the algorithm is as follows.

Figure 2.

Sparse CS remote sensing image fusion based on wavelet transform.

The specific algorithm flow is described as:

Step 1:

Two images were extractedï¼Œimage A is $\left\{{R_{a},G_{a},B_{a}}\right\}$ , image $B$ is $\left\{{R_{b},G_{b},B_{b}}\right\}$ .

Step 2:

Sparse Processing. Wavelet sparse processing of six values of $R_{a},G_{a},B_{a},R_{b},G_{b},B_{b}$ , we get $r_{a},g_{a},b_{a},r_{b},g_{b},b_{b}$ . Sparse processing mainly through wavelet sparse basis db2 for sparse transformation.

Step 3:

Building a stacked Gaussian matrix $\Phi$ .

Step 4:

Measurement sampling. Through the Gaussian matrix pair Sampling the measurements $r_{a},g_{a},b_{a},r_{b},g_{b},b_{b}$ to:

$\displaystyle Y_{R_{a}}=\Phi\times r_{a},Y_{R_{b}}=\Phi\times r_{b}$ $\displaystyle Y_{G_{a}}=\Phi\times g_{a},Y_{G_{b}}=\Phi\times g_{b}$ (9) $\displaystyle Y_{B_{a}}=\Phi\times b_{a},Y_{B_{b}}=\Phi\times b_{b}$

Step 5:

Weighted fusion by $\alpha,\beta$ .

$\displaystyle r=\alpha Y_{R_{a}}+\beta Y_{R_{b}}$ $\displaystyle g=\alpha Y_{G_{a}}+\beta Y_{G_{b}}$ (10) $\displaystyle b=\alpha Y_{B_{a}}+\beta Y_{B_{b}}$

Step 6:

Reconstruct the image through the OMP algorithm.

$\displaystyle\textit{RGB}\left({:,:,1}\right)=\textit{OMP}\left(r\right)$ $\displaystyle\textit{RGB}\left({:,:,2}\right)=\textit{OMP}\left(g\right)$ (11) $\displaystyle\textit{RGB}\left({:,:,3}\right)=\textit{OMP}\left(b\right)$

4. Discussion

4.1 Evaluation criterion

In this paper, the following objective evaluation indexes are used to evaluate the fused images: (1) Average, (2) information entropy, (3) cross entropy, (4) spectral distortion, (5) correlation coefficient.

(1)
Average means the average of gray in the image, and the basic feeling of the naked eye in the image is a perceptive process to the average brightness of the image. If the brightness is moderate, the naked eye will be more comfortable. If the pixel in an image is $M\times N$ , so the average is $\overline{F}=1/M\times N\sum_{i=1}^{M}{\sum_{j=1}^{N}{F\left({i,j}\right)}}$ .
(2)
The information entropy is used to measure the physical quantity of the information contained in the image. The larger the information entropy, the more detailed information in the image. For remote sensing image, the larger the information entropy after fusion, the more spatial information the image gets, calculated by $H=-\sum_{j=0}^{L-1}{P(i)\log P(i)}$ , where $P=\{P_{1},P_{2},P_{3},\ldots P_{i},\ldots,P_{n}\}$ represents gray value distribution and $P_{i}$ shows the probability of pixel gray value $i$ in the fused image.
(3)
Cross entropy is also called relative entropy. It represents statistically the difference in probability distribution between pre-and post-fusion images, and a relative evaluation index for the probability of information contained in images before and after fusion. The smaller the cross-entropy, the smaller the difference between the images before and after fusion, calculated by $\textit{CF}_{({A,F})}=\sum_{l=0}^{L}{P_{A}\left(l\right)}\ln\left({{P_{A}\left% (l\right)}\mathord{\left/{\vphantom{{P_{A}\left(l\right)}{P_{F}\left(l\right)}% }}\right.\kern-1.2pt}{P_{F}\left(l\right)}}\right)$ .

In general, the average value MCE can be used to describe the synthetic difference between the fusion image $A_{1}$ and image $A_{2}$ and the fused image $F$ . The Cross Entropy of Image $A_{1}$ and Image $F$ is $\textit{CE}_{A_{1},F)}$ , The Cross Entropy of Image $A_{2}$ and Image $F$ is $\textit{CE}_{A_{2},F)}$ , The cross entropy before and after fusion can be expressed in MCE. The formula is:

$\displaystyle\textit{MCE}=(\textit{CE}_{(A_{1},F)}+\textit{CE}_{(A_{2},F)})/2$ (12)
(4)
The spectral distortion reflects the spectral distortion of the image after image fusion. When the fusion distortion increases, the value of spectral distortion also increases. Ideally, the picture is undistorted, and the spectral distortion of the undistorted picture is 0. The formula of spectral distortion D is as follows:

$\displaystyle D=\frac{1}{\textit{MN}}\sum_{i=1}^{M}{\sum_{j=1}^{N}{|F(i,j)-A(i% ,j)|}}$ (13)
(5)
The correlation coefficient indicates that the spectral features of the images before and after fusion are correlated with each other. Ideally, the correlation coefficient is 1, which means that there is no difference between the images before and after fusion. When infinitely close to 1, the spectral information of the fused image and the original image is the most acacia. The correlation coefficient is calculated as follows:

$\displaystyle C(F,A)=\frac{\sum\limits_{i=1}^{M}\sum\limits_{j=1}^{N}[F(i,j)-% \bar{I}_{F}][A(i,j)-\bar{I}_{A}]}{\sqrt{\sum\limits_{i=1}^{M}\sum\limits_{j=1}% ^{N}[F(i,j)-\bar{I}_{F}]^{2}[A(i,j)-\bar{I}_{A}]^{2}}}$ (14)

Table 2
Cross entropy and spectral distortion under different fusion coefficients

Fusion coefficient Mutual Light spectrum

$\alpha$ $\beta$ entropy torsion resistance

0.1 0.9 0.1152 4.6752

0.2 0.8 0.1075 6.9335

0.3 0.7 0.1032 9.1852

0.4 0.6 0.0952 11.9802

0.5 0.5 0.0758 14.4562

0.6 0.4 0.0651 17.2584

0.7 0.3 0.0456 19.8562

0.8 0.2 0.0325 22.4382

0.9 0.1 0.0184 25.3215

Table 3
The value of evaluation index

Entropy of information Mean value Correlation coefficent Cross entropy Light spectrum torsion resistance

This paper 7.4072 90.7495 0.9235 0.0775 14.4932

Ref. [29] 7.4060 90.9665 0.9010 0.1250 16.8780

HIS method 7.4028 85.5968 0.7080 0.0523 26.72

PCA method 7.4107 85.5806 0.7097 0.0653 27.7564

Figure 3.
Weight ( $\alpha=$ 0.4, $\beta=$ 0.6).

In addition to the above indicators, there are also objective evaluation indexes such as spatial frequency, root mean square error, peak signal to noise ratio, contrast change, and so on. It is difficult to obtain standard reference images in practical remote sensing image fusion applications. Therefore, these objective evaluation methods may be limited. However, any objective evaluation method cannot be separated from subjective evaluation, so a good evaluation index must be a combination of subjective visual factors and objective quantitative evaluation of human eyes.

Table 4
The value of evaluation index

Entropy of information Mean value Correlation coefficient Cross entropy Light spectrum Torsion resistance

This paper 7.452 91.3245 0.9348 0.0645 13.4512

Ref. [29] 7.356 90.5063 0.9176 0.1560 17.8591

HIS method 7.388 84.6678 0.7589 0.0688 25.3329

PCA method 7.390 83.3567 0.7924 0.0691 29.9816

Figure 4.
Weight ( $\alpha=$ 0.5, $\beta=$ 0.5).

Figure 5.
Weight ( $\alpha=$ 0.6, $\beta=$ 0.4).

4.2 Determination of fusion coefficient

Fusion coefficient	Mutual	Light spectrum
0.1	0.9	0.1152	4.6752
0.2	0.8	0.1075	6.9335
0.3	0.7	0.1032	9.1852
0.4	0.6	0.0952	11.9802
0.5	0.5	0.0758	14.4562
0.6	0.4	0.0651	17.2584
0.7	0.3	0.0456	19.8562
0.8	0.2	0.0325	22.4382
0.9	0.1	0.0184	25.3215

	Entropy of information	Mean value	Correlation coefficent	Cross entropy	Light spectrum torsion resistance
This paper	7.4072	90.7495	0.9235	0.0775	14.4932
Ref. [29]	7.4060	90.9665	0.9010	0.1250	16.8780
HIS method	7.4028	85.5968	0.7080	0.0523	26.72
PCA method	7.4107	85.5806	0.7097	0.0653	27.7564

	Entropy of information	Mean value	Correlation coefficient	Cross entropy	Light spectrum Torsion resistance
This paper	7.452	91.3245	0.9348	0.0645	13.4512
Ref. [29]	7.356	90.5063	0.9176	0.1560	17.8591
HIS method	7.388	84.6678	0.7589	0.0688	25.3329
PCA method	7.390	83.3567	0.7924	0.0691	29.9816

Based on the analysis of the third chapter, the fusion coefficients used in the fusion algorithm are determined by several sets of different values of fusion coefficients in this section. The main values are as follows:

$\displaystyle\alpha=0.1,0.2,\ldots,0.9;\beta=0.9,0.8,\ldots,0.1$ (15)

The fused images are evaluated subjectively. When the fusion coefficient is $({\alpha=0.4,\beta=0.6})$ $({\alpha=0.5,\beta=0.5})({\alpha=0.6,\beta=0.4})$ , the fusion of the three sets of images can guarantee high resolution of spectral information. The fusion images under the above three fusion coefficients are shown in the following figure, and the values of cross entropy and spectral distortion for the above different fusion coefficients are listed in the following table

Figure 6.

The method based on the fusion method of CS.

Figure 7.

The method of [35].

Figure 8.

IHS fusion method.

Through the simulation of the fusion coefficients and the data results, it can be seen that with the increase of the values, the value of the interactive entropy gradually decreases, and the spatial resolution of the fused image becomes better and better. However, the spectral distortion of the image increases, which destroys the spectral imaging of the image. In the light of the above-mentioned evaluation, The selection of fusion coefficients in this paper is ( $\alpha=$ 0.5, $\beta=$ 0.5). In order to verify the correctness of the algorithm and image congruence, the experiment of remote sensing image fusion from multiple regions and different satellites is carried out in this paper.

In order to further reflect the performance of the fusion algorithm, the following table gives the experimental data after different methods of image fusion.

Some of the parameters in the above table are described below. The larger the information entropy, the more spatial information the fused image carries, so the higher the ratio is better. The average value is the average value of the pixel gray value of the image, through the naked eye sensory flexibility to distinguish, from the table can be seen that the four methods are almost equal, the naked eye cannot distinguish the quality of the image after fusion; The value of correlation coefficient is closer to 1, which indicates that the fused image is more similar to the original image, and the smaller the cross entropy is, the better the ideal value is 0, and the larger the value of spectral distortion, the more serious the spectral distortion of the fused image is. Information entropy indicates that the larger the spatial information carried by the fused image, the better for fused image. In order to reflect the fusion effect of the algorithm in this paper, the average effect of the algorithm and other algorithms is shown in the following table.

Figure 9.

PCA fusion method.

From the above four charts and the data in the table, the fusion algorithm introduced in this paper has very good fusion results, the spatial resolution and spectral information of remote sensing images are better. The superiority of the method in information entropy, correlation coefficient, cross entropy, spectral distortion and so on can also be seen from the data in the table, which is superior to the method in reference [29].

5. Conclusions

The compression sensing theory has influenced many fields of application since it was put forward. Both from the theoretical basis of research and related field research has been rapid development. The innovation of compressed sensing theory is that it preserves useful information in the original signal sampling and removes redundant information at the same time. The original Nyquist sampling law is updated. Compression sensing solves the traditional data sampling mode. Firstly, the redundant information before collecting data is compressed, so that the compressed samples are reduced, and then the appropriate optimization method is selected to deal with the compressed samples. The optimization method required here is often an underdetermined linear inverse problem. In order to change the status quo of the massive data caused by the rapid development of information technology, it is very meaningful to find a solution to compress the data processing algorithm in the perceptual domain. Remote sensing image fusion is a very important part of remote sensing image processing technology. For the limitation and difference of single sensing image, the fusion algorithm can make up effectively. The fused image has high resolution and clear image, so the remote sensing image is used in more and more fields. The traditional image fusion method has led to a large amount of data, the strong pressure of storage and transmission, the emergence and development of compression perception theory, alleviate this kind of pressure, and at the same time provides an effective way to solve the problem. The unique advantages of compressed perceptual domain are more and more prominent in image fusion processing. In this paper, a sparse CS remote sensing image fusion algorithm based on wavelet transform is proposed. The simulation results show that this method can realize the image fusion well in the actual image fusion process, and improve the operation speed on the basis of compression perception. It is significance for realization of big data complex environment under the situation of image fusion. By comparing the algorithms in this paper with other algorithms, the method can get higher correlation coefficient and lower interaction entropy and spectral distortion. Compared with other methods in this paper, the fusion of the image can carry more space information and the original image more similar.

Footnotes

Acknowledgments

This work was supported by Key Scientific Research Projects of Higher Education Institutions in Henan Province (No. 18B170008).

References

Yong

and Chang

, Infrared and visible image fusion methods and applications: A survey, Information Fusion 45 (2019), 153–178.

Hausler

Jacobson

and Milford

M.J.

, Multi-Process Fusion: Visual Place Recognition Using Multiple Image Processing Methods, IEEE Robotics and Automation Letters, 2019: 1.

Núñez

Otazu

Fors

et al., Multiresolution-based image fusion with additive wavelet decomposition, IEEE Transactions on Geoscience & Remote Sensing 37(3) (1999), 1204–1211.

Ingole

N.A.

Nain

A.S.

Kumar

et al., Monitoring and Mapping Invasive Aquatic Weed Salvinia molesta using Multispectral Remote Sensing Technique in Tumaria Wetland of Uttarakhand, India, Journal of the Indian Society of Remote Sensing 46(5) (2018), 863–871.

Singh

Sinha

L.K.

and Bhavani

A.G.

, Application of GIS and Remote Sensing Technique in Drainage Network Analysis: A Case Study of Naina-Gorma Basin of Rewa District, M.P. India, 2019, pp. 403–411.

Rui

Vorobyov

and Hong

, Image fusion with cosparse analysis operator, IEEE Signal Processing Letters, 2017, PP(99): 1-1.

Marra

Morin

Peleg

et al., Intensity-duration-frequency curves from remote sensing rainfall estimates: comparing satellite and weather radar over the eastern Mediterranean, Hydrology & Earth System Sciences 21(5) (2017), 2389–2404.

Yang

Wang

and Ren

, The first quantitative remote sensing of ocean internal waves by Chinese GF-3 SAR satellite, Acta Oceanologica Sinica 36(1) (2017), 118–118.

Gong

Yang

Yao

et al., When Deep Learning Meets Metric Learning: Remote Sensing Image Scene Classification via Learning Discriminative CNNs, IEEE Transactions on Geoscience & Remote Sensing 56(5) (2018), 2811–2821.

10.

Qiang

Yuan

Chao

et al., Missing data reconstruction in remote sensing image with a unified spatial-temporal-spectral deep convolutional neural network, IEEE Transactions on Geoscience & Remote Sensing 2018, PP(99): 4274–4288.

11.

Chen

Sun

Yang

et al., An improved optimum-path forest clustering algorithm for remote sensing image segmentation, Computers & Geosciences 112 (2018), 38–46.

12.

Ming

Duan

Wei

et al., A novel infrared and visible image fusion algorithm based on shift-invariant dual-tree complex shearlet transform and sparse representation, Neurocomputing 226(C) (2017), 182–191.

13.

Meng

Wang

et al., A holistic approach to reconstruct data in ocean sensor network using compression sensing, IEEE Access 6(99) (2018), 280–286.

14.

Oleksyuk

Rajan

Saleheen

et al., Risk score based pre-screening of breast tumor using compression induced sensing system, IEEE Sensors Journal, 2018, PP(99): 1-1.

15.

Zhao

Sun

et al., On-chip neural data compression based on compressed sensing with sparse sensing matrices, IEEE Trans Biomed Circuits Syst 2018, PP(99): 242–254.

16.

Ponuma

and Amutha

, Compressive sensing based image compression-encryption using Novel 1D-Chaotic map, Multimedia Tools & Applications (4) (2017), 1–26.

17.

Gomes

, Jr. Guimarães

C.H.C.

Martins

et al., Damped Nyquist Plot for a pole placement design of power system stabilizers, Electric Power Systems Research 158 (2018), 158–169.

18.

Chang

Omomukuyo

Xiang

et al., Robust Faster-Than-Nyquist PDM-mQAM systems with Tomlinson-Harashima precoding, IEEE Photonics Technology Letters 28(19) (2018), 2106–2109.

19.

Candès

E.J.

and Wakin

M.B.

, An introduction to compressive sampling, IEEE Signal Processing Magazine 21 (March 2008).

20.

Wan

H.S.

, New approach for dynamic range compression of remote sensing image using wavelet fusion, Journal of Applied Remote Sensing 4(1) (2010), 2017–2039.

21.

Vollnhals

Audinot

J.N.

Wirtz

et al., Correlative Microscopy Combining Secondary Ion Mass Spectrometry and Electron Microscopy: Comparison of Intensity-Hue-Saturation and Laplacian Pyramid Methods for Image Fusion, Analytical Chemistry, 2017:acsanalchem.7b01256.

22.

Wang

and Gao

, Image denoising method based on nonsubsampled contourlet transform and bandelet transform, Iet Image Processing 12(5) (2018), 778–784.

23.

Ĺukasz

Kokoszka

and Jouzdani

N.M.

, Principal component analysis of periodically correlated functional time series, Journal of Time (2018), 39.

24.

Bell

M.C.

and Baum

W.M.

, Concurrent variable-interval variable-ratio schedules in a dynamic choice environment, Journal of the Experimental Analysis of Behavior 108(3) (2017), 75–89.

25.

Kuang

Yan

Zhu

et al., Predicting duration of traffic accidents based on cost-sensitive Bayesian network and weighted K-nearest neighbor, Journal of Intelligent Transportation Systems (2019)(4), 1–14.

26.

Lei

Yong

Liu

et al., Accurate phase extraction algorithm based on Gram-Schmidt orthonormalization and least square ellipse fitting method, Journal of Modern Optics (2018)(3), 1–11.

27.

Hsieh

S.H.

C.S.

and Pei

S.C.

, Compressive Sensing Matrix Design for Fast Encoding and Decoding via Sparse FFT, IEEE Signal Processing Letters (2018), PP(99):1-1.

28.

Zhu

and Bamler

, A sparse image fusion algorithm with application to pan-sharpening, IEEE Trans Geosci Remote Sens 51(5) (2013), 2827–2836.

29.

Cheng

Wang

and Li

, Sparse representation based pansharpening using trained dictionary, IEEE Geosci. Remote Sens. Lett. 11(1) (2014), 293–297.

30.

Ghahremani

and Ghassemian

, Remote sensing image fusion using ripplet transform and compressed sensing, IEEE Geosci. Remote Sens. Lett. 12(3) (2015), 502–506.

31.

Vicinanza

M.R.

Restaino

Vivone

Mura

M.D.

and Chanussot

, Pan-sharpening method based on the sparse representation of injected details, IEEE Geosci. Remote Sens. Lett. 12(1) (2015), 180–184.

32.

Ghahremani

and Ghassemian

, A compressed-sensing-based pan-sharpening method for spectral distortion reduction, IEEE Transactions on Geoscience and remote Sensing 54(4) (2015), 2194–2206.

33.

Zhao

Yang

and Chan

J.C.-W.

, Hyperspectral imagery super-resolution by spatial-spectral joint nonlocal similarity, IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens. 7(6) (2014), 963–978.

34.

Morteza

Liu

Yuen

and Behera

, Remote sensing image fusion via compressive sensing, ISPRS Journal of Photogrammetry and Remote Sensing (152) (2019), 34–48.

35.

Peng

Chen

and Yu

, Sparse Recovery and Compressive Sampling Based on FFT and DWT, Proceedings of the International Conference on Information Engineering and Applications (IEA) 2012, Springer London, 2013.

Fusion coefficient		Mutual	Light spectrum
$\alpha$	$\beta$	entropy	torsion resistance
0.1	0.9	0.1152	4.6752
0.2	0.8	0.1075	6.9335
0.3	0.7	0.1032	9.1852
0.4	0.6	0.0952	11.9802
0.5	0.5	0.0758	14.4562
0.6	0.4	0.0651	17.2584
0.7	0.3	0.0456	19.8562
0.8	0.2	0.0325	22.4382
0.9	0.1	0.0184	25.3215