Research on fusion of GF-6 imagery and quality evaluation

Gaofen-6 (GF-6) has the advantages of wide coverage, multiple resolutions, and multiple bands. Image fusion method is the key process in high resolution remote sensing application. Beijing Daxing International Airport was selected as the experiment area and four image fusion methods of HPF, NND, Gram-Schmidt and Pansharp were employed to process panchromatic and multispectral imaging. The results demonstrated that Pansharp was the best algorithm for image information and spectral fidelity of GF-6, taking into account the preservation of color effects of images and enhancement of spatial details, which can meet most fusion needs. HPF’s color retention is not as good as Pansharp algorithm. The contrast of the NND algorithm result is relatively high, which may cause the local image to be too bright and the texture to be lost. The GS algorithm has lower information entropy and average gradient. Compared with the other three algorithms, it has a worse effect on spatial details and texture expression. This conclusion can provide key reference for scientific research and engineering application using GF-6 satellite image．


Introduction
Since the implementation of the major special project of high-resolution earth observation system of China in 2010, 12 high-resolution satellites (from GF-1 to GF-12) have been launched in the past ten years. The all-weather, allday, global coverage earth observation capability greatly meets the needs of China's high-resolution remote sensing data applications and completely reverses the long-term dependence of high-scoring remote sensing monitoring in China on foreign commercial satellites.
For the satellite GF-1, Guo et al. (2015) compared the three traditional fusion methods of PCA, Gram-Schmidt (GS), Pansharp and the three fusion methods of Elbakary, Proxy-sharpening, SFIM, and found that Pansharp and SFIM are better than other methods [1]. Cui et al. (2015) selected 4 methods (Brovey, HSV, PCA, GS) to fuse GF-1 images, and considered that the GS indicators were the best [2]. Liu et al. (2015) evaluated the four fusion methods of GF-1 satellite panchromatic and multispectral, and considered that each type of fusion method has different advantages. Among them, the Pansharp method has the best comprehensive performance [3].For satellite GF-2, Sun et al. (2016) compared the five common fusion methods, and believed that the HCS and GS methods have good results in both visual and feature classification applications [4]. Jiang et al. (2017) believed that Pansharp method has better preservation effect in image information, details and spectrum, followed by GS and PCA, and the color distortion is more obvious of HSV and Brovey [5]. Chen et al. (2017) concluded that the fusion effect of HCS and NND methods was the best [6].At present, the above various remote sensing image fusion algorithms have been implemented in the mainstream commercial remote sensing software. Although they can improve the spatial resolution of the image, the remote sensing algorithms have different advantages and disadvantages for different sensor types. No one algorithm is universally recognized. Therefore, the applicability of the fusion method for different sensor type images needs further research. GF-6 satellite was successfully launched at Jiuquan Satellite Launch Center on June 2, 2018, and has the advantages of high resolution, wide coverage, high quality imaging, high performance imaging, and high localization rate. It has been running for more than a year and has acquired a large number of high-quality ground-observed remote sensing images. However, few studies have been conducted on the applicability of GF-6 satellite panchromatic and multispectral fusion algorithms.

Introduction of GF-6 satellite
GF-6 is the sixth satellite of China's high-resolution earth observation system, developed by China Aerospace Science and Technology Corporation. It was successfully launched by Jiuquan Satellite Launch Center on June 2, 2018 by the Long March 2D rocket. On March 21, 2019, GF-5 and GF-6 were officially put into use, marking that the high-spectrum special-purpose hyperspectral capability has been formed. GF-6 satellite carries a 2m panchromatic / 8m multispectral resolution camera with a width of 90km and a 16m multispectral resolution wide coverage camera with a width of 800km. It is a sunsynchronous orbit with a design life of 8 years. It is the first multi-spectral remote sensing satellite of china with red-edge spectral bands and domestic CMOS imaging devices.
GF-6 is a high-resolution optical satellite with high maneuverability in China's high-resolution groundobserving system that takes into account both census and detailed investigation capabilities. The satellite and GF-1 satellite network has realized the revisited observation of China's land area for 2 days, which greatly improves the scale and timeliness of the acquisition of remote sensing data, and effectively makes up for the existing domestic and foreign high-resolution multispectral satellite resources. And the self-sufficiency rate and application range of domestic remote sensing satellite data have been improved.

research area overview
In this paper, domestic multi-spectrum PMS sensor of GF-6 satellite (multi-spectrum 1-4 band (resolution 8m) and panchromatic band (resolution 2m) are selected for fusion experiments. The experimental image date is September 18, 2019. The multi-spectral data and panchromatic data were spatially registered before the experiment.
Combined with the application characteristics of highresolution remote sensing data, the study area was selected near Beijing Daxing International Airport, mainly for building terminals (including bubble windows), fivefinger corridors, boarding bridges, comprehensive service buildings, parking buildings, and roads. It can reflect the spatial recognition ability of the high-resolution satellite image after fusion. A small amount of woodland, grassland and unused land are distributed around the airport. Daxing International Airport is located at the top of the "New Seven Wonders of the World" released by the British "Guardian", achieving four "world's best": the world's largest single airport terminal, the world's most technically difficult terminal, the world's largest airport terminal with isolation bearings and the world's largest unstructured seam integrated terminal. The experimental image date is one week before the official opening of Daxing International Airport on September 25, 2019, and related work is ready.

Fusion algorithm
(1) HPF (high-pass filtering fusion) The HPF method is to perform a positive FFT transform on a multispectral image and replace the high-frequency part of the multispectral image after the FFT with a highresolution panchromatic band image. Spectral fusion image. The fusion effect is affected by the size of the filter [7]. This paper chooses a high-pass filter window of 3 * 3.
(2) NND (Nearest Neighbor Diffusion) The HPF method downsamples the panchromatic band to low resolution, performs linear regression on pixels in each band to obtain the contribution vector T, and calculates the neighboring pixel difference factor using the original panchromatic band to generate the initial high-resolution multi-Spectral images, and finally use T as the weight to normalize the spectrum.
(3) GS (Gram-Schmidt) The GS method first uses spectral resampling to simulate the first component, transforms the multispectral image to orthogonal space, and replaces the first component with a high spatial resolution panchromatic image. Then obtain the fused image by inverse transform. The GS fusion algorithm has no band limitation on the image, and can fuse all multi-spectral bands in one fusion process.
(4) Pansharp The Pansharp method uses the least squares method to best match the gray value relationship between the fusion of the original multispectral image, the panchromatic image and the fused multispectral image and the panchromatic image. And the use of mathematical statistics to improve the integration process automation. When applying this fusion method, it is best to designate all the multispectral bands as reference bands to reduce the deviation between the fused image and the original multispectral image.

Qualitative evaluation
Qualitative evaluation mainly performs subjective evaluation from visual effects, that is, visual method. Through visual comparison of image spectrum, edges, texture and overall brightness before and after fusion, it can intuitively and conveniently reflect the spatial resolution and clarity of different fusion images. However, this evaluation method is relatively subjective, and is greatly affected by the observer's experience and technology, image types, application occasions, and environmental conditions, lacks stability, and does not guarantee the repeatability of the evaluation method.

Quantitative evaluation
Quantitative evaluation method is an objective method to quantitatively analyze images based on the statistical characteristics of the images. It can overcome the influence of human subjective factors and make scientific and objective evaluations of the performance of various image fusions for further research. Quantitative evaluation indicators are mainly divided into two categories, one is single-factor evaluation, including image mean, standard deviation, and information entropy; the other is comprehensive evaluation indicators, including average gradient, correlation coefficient, and relative deviation. According to the characteristics of GF-6 image, this experiment selects the mean value, standard deviation, information entropy and average gradient as the quality evaluation index of the fusion image, and quantitatively describes the spectral fidelity, information amount and image clarity of the fusion image.
(1) The mean is the average of all the pixel values in the image and reflects the average reflection intensity of the features.
(1) u is the average value of the image, F (i, j) is the gray value at the position of the pixel (i, j), and M and N are the number of image rows and columns, respectively.
(2) The standard deviation reflects the discrete situation of the gray level of the image relative to the average gray level average value. It is generally believed that the larger the standard deviation, the more dispersed the gray level distribution, the better the separability between features, and the more image information.
std is the standard deviation of a certain band of the image, M and N are the number of rows and columns of the image; F (i, j) is the gray value at the position of the pixel (i, j); u is the average gray of the image after fusion.
(3) Information entropy reflects the richness of image information. The larger the information entropy, the more information the image contains.

= − ∑ ( ) 2 [ ( )]
(3) H is the information entropy of a certain band of the image, p (i) is the probability of the pixel whose gray value is i; it is expressed by the ratio of the number of pixels with gray value i and the total number of pixels in the image.
(4) The average gradient is the average value of the grayscale change rate of the image, which can reflect the detail expression ability of the image.
(4) G is the average gradient of a certain band of the image, F (i, j) is the gray value of the image at the point (i, j), ∂F (i, j) / ∂x, ∂F (i, j) / ∂y are the partial derivatives of the image at the point (i, j) in the row and column directions.

Results and analysis
A full-color image with a resolution of 2m and a multispectral image with a resolution of 8m are selected as experimental data from Beijing Daxing Airport GF-6 satellite. Orthorectification and geometric registration are performed on the image using the GXL software for geography imaging accelerators to correct errors. The correction error is within 1 pixel. Then, HPF, NND, GS, and Pansharp method were used to perform fusion experiments on the registered full-color and multispectral images. The fusion effect of each method is shown in Figures 1-5.

Qualitative evaluation
Through the comparison of Fig. 1-5, it can be seen that the four fusion algorithms all enhance the resolution of the multispectral image, and at the same time, the fusion image has no obvious color difference compared to the original multispectral image, and it better retains the spectral information. The partial enlarged views of vegetation, parking lot, and bubble window show that the HPF algorithm and Pansharp algorithm have distinct vegetation layers and clear parking lot boundaries, but bubble window has obvious halo phenomenon by the HPF algorithm. The vegetation of the GS algorithm and the NND algorithm lacks a sense of hierarchy and cannot easily distinguish the tree boundaries. From the perspective of visual interpretation, the Pansharp algorithm fuses the roads and buildings with the clearest boundary, the vegetation texture level is more prominent, followed by the NND algorithm, and the GS algorithm has a slight loss of image details.

Quantitative evaluation
In order to further compare the differences of several fusion algorithms, the mean value, standard deviation, information entropy, and average gradient of the four fusion results were calculated for quantitative evaluation. The results are shown in Table 1.  (1) Mean The mean value is the average gray value of all pixels in the image, and it reflects the average brightness of the image. The closer the average of the fused image to the original multispectral image, the better the fusion effect. It can be seen from Fig. 6 (a) that the fusion image obtained by the GS and Pansharp algorithms maintains good color brightness, the mean value of the HPF algorithm fusion image has the largest deviation, and the mean value of the NND algorithm fusion image is higher than the original multispectral image.
(2) Standard deviation It reflects the discrete situation of image gray relative to the average. The larger the standard deviation, the more scattered the grayscale distribution of the image and the larger the contrast; on the contrary, the image has a small contrast and a single hue. It can be seen from Fig. 6 (b) that the standard deviation of the fusion image of the NND algorithm is the largest, the fusion of the GS algorithm and the Pansharp image is the second, and the standard deviation of the HPF algorithm is the smallest.
(3) Information entropy As an important indicator for measuring the richness of image information, the larger the entropy value, the better the image quality and the larger the amount of information it contains. It can be seen from Fig. 6 (c) that the NND algorithm has the largest entropy of the fusion image information, the PSH algorithm and the HFP algorithm take the second place, and the GS fusion image has the smallest information amount.
(4) Average gradient It reflects the contrast of small details of the image and the change of texture, and represents the clarity and layering of the image. With the same spatial resolution, the larger the average gradient, the higher the sharpness of the image. It can be seen from Fig. 6 (d) that the average gradient of the four fusion images is higher than that of the original multispectral image. The average gradient of the fusion images of Pansharp algorithm and HPF algorithm is larger, and the spatial detail enhancement effect is better. GS and NND algorithms have smaller average gradients.

Conclusion
Remote sensing image fusion is an effective method to improve image resolution, enhance image utilization, and fully mine image information. In this paper, GF-6 satellite images from Beijing Daxing International Airport area are selected. HPF, NND, GS and Pansharp algorithms are used to perform fusion experiments. On the basis of subjective evaluation, from the perspective of spectral fidelity, image clarity, and image information, four objective evaluation indexes were selected to evaluate the fusion result. The experimental results show that, Pansharp and HPF algorithm perform well in enhancing spatial details, and are suitable for fusion tasks that require high boundary features, but the color retention of HPF is not as good as Pansharp algorithm. The NND algorithm has a high contrast image fusion, which is likely to cause local over-brightness and texture loss. The GS algorithm has lower information entropy and average gradient. Compared with the other three algorithms, the effect of GS algorithm on spatial details and texture expression is poor.
Therefore, Pansharp algorithm has the best comprehensive evaluation of the fusion effect of GF-6 satellites, taking into account the color effect preservation and spatial detail enhancement of the image, which can meet most fusion needs. However, in practical work, diversification needs are diversified, and specific problems should be analyzed in detail to select appropriate fusion methods.