A Novel Image Fusion Method for Water Body Extraction Based on Optimal Band Combination

2.1 Study area

The 157,380-hectare study area lies in the southeast of central China’s Hubei Province. With a monsoon-influenced humid subtropical climate, the study area has an annual maximum rainfall of 1,600mm. Each year can be clearly divided into a dry season (October to next February) and a wet season (March to September). This area was selected partially due to the abundance of water resource in Hubei Province.

Figure 2 presents a composite image of bands 4, 5, and 3 of Landsat 7 ETM+ images taken on August 19, 2008. The image contains various surface objects, including but not limited to buildings, farmlands, mountains, and water bodies. The lakes and the Yangtze River are quite legible on the composite image. However, small water bodies (e.g. ponds and small rivers) cannot be distinguished easily from the surroundings. Since the images were taken in summer, a large area of water bodies are covered by lotus and other plants, and thus not observable on the composite image.

2.png

Figure 2. The study area

Note: BAL- Baoan Lake; CL- Chaohu Lake; HML- Huama Lake; YLL-Yanlan Lake; SSL-Sanshan Lake; HLR- Huanglong Reservoir; SQR- Shiqiao Reservoir; CiL-Cihu Lake; YTR- Yangtze River; R1-River 1; R2-River 2.

2.2 Data

2.2.1 Data selection

The source data have a great impact on the result of water body extraction. Many factors should be considered in data selection: imaging mechanism, data availability, previous work, regional features, etc. This paper selects the source images based on two factors: previous work and imaging features.

(1) Previous work

In previous work on remote sensing, most image fusion tools target images captured by multispectral imager (MSI), a passive imaging system, and synthetic aperture radar (SAR), an active imaging system.  The empirical results show that the fusion of the two types of images indeed has advantages in applications like resource monitoring, environmental protection, and disaster investigation [46-55]. The features not observed in the MSI image can be found in the SAR image, and the inverse is also true [56-58]. Some scholars have proved that the fusion between MSI and SAR images can improve the classification accuracy in water body extraction [59, 60].

(2) Imaging features

The MSI operates similarly to the human eyes. The imaging mechanism of the MSI is easy to understand and apply to specific problems. There are multiple bands of the MSI. For the same object, its appearance and color on the MSI image vary with the band combinations. Thus, the MSI images provide multiple viewing angles and high recognizability, facilitating the classification of features. In addition, the MSI is prone to noise disturbance, which ensures the image integrity.

However, every silver lining has a cloud. The MSI performance depends heavily on the weather, especially on the presence or absence of clouds. In MSI images, the same object many have different spectra, and the same spectrum may correspond to different objects. In other words, different objects with similar spectral features are often indistinguishable, such as water bodies, mountain shadows, and cloud shadows. Furthermore, the water area covered by plants cannot be captured by the MSI.

By contrast, the SAR is an active imaging system with its own energy source. Thus, the SAR images are not affected by weather. Compared with the MSI images, the SAR images boast a high resolution, as the SAR is sensitive to moisture, even if it is covered by plants. The main drawback of the SAR is the inability to remove speckle noise [61], which inevitably affects the integrity of the extraction result.

2.2.2 Selected data

Based on the previous work and imaging features, this paper chooses Landsat 7 ETM+ images and ALOS PALSAR images as the source data, for these two types of images provide complementary information in water body extraction.

(1) Landsat 7 ETM+ data

The ETM+ is a multispectral scanner that passively senses the surface reflection of solar radiation and heat radiation of the earth. There are eight band sensors in the ETM+, covering the wavelengths from infrared to visible light. The ETM+ has a better resolution in the infrared band than the thematic mappers (TMs) on Landsat 4 and Landsat 5. Therefore, Landsat 7 outperforms Landsat 4 and Landsat 5 in the accuracy of feature identification.

As shown in Table 1, the image (Path 122, Row 39) taken on August 18, 2008 was collected and processed for this research. The image is 170km (106 miles) in the north-south direction and 183km (114 miles) in the east-west direction. 

Table 1. The Landsat 7 ETM+ image (path/row: 122/39)

Table 2. The bands of Landsat 7 ETM+

As shown in Table 2, different bands of the ETM+ record different reflection information. In the light of the actual situation, band 4 (NIR), band 5 (SWIR), and band 3 (R) were selected to be fused with ALOS PALSAR image. The three bands were collectively referred to as EMT+453.

(2) ALOS PALSAR data

PALSAR is the L-band synthetic aperture radar on the ALOS satellite. The radar supports all-weather observation. The PALSAR image collected by the HH channel contains more layers and information than the HV channel, for the former channel is more penetrable than the latter. Therefore, the PALSAR HH image taken on July 3, 2018 was selected for further fusion (Table 3).

Table 3. The ALOS PALSAR data

(3) Other data

The data of the Digital Elevation Model (DEM) of Wuhan were selected for terrain correction of PALSAR HH data and for registration with Landsat 7 ETM+ data. The extraction accuracy was registered and evaluated by the ground truth data from Crustal Movement Observation Network of China (CMONOC) and the GPS points of the study area.

The topographic maps of the study area were scanned into digital files, and the geographic coordinates were referenced to a common projection and coordinate system (WGS84, UTM 50N). These data were used in image classification to identify the appropriate calibration and verification sites.

2.3 Methodology

3.png

Figure 3. The workflow of this research

As shown in Figure 3, this research is carried out in the following steps:

Step 1. Preprocess Landsat 7 ETM+ data, including stripe removal and cloud removal.

Step 2. Preprocess PALSAR HH data, including radiometric calibration, terrain correction, geocoding, and speckle suppression.

Step 3. Registrant ETM+453 and PALSAR HH images.

Step 4. Extract water bodies from ETM+453 and PALSAR HH images, respectively, and generate a benchmark based on the extraction results.

Step 5. Compare the water body extraction results of traditional image fusion methods, namely, HIS, DWT, principal component analysis (PCA), high-pass filter (HPF), and Brovey transform (BT).

Step 6. Design and test a new image fusion method.

Step 7. Compare the accuracy of different methods in water body extraction.

2.3.1 Preprocessing of Landsat ETM+ data

The Landsat 7 ETM+ image was downloaded from Geospatial Data Cloud (http://datamirror.csdb.cn/). The stripes of the downloaded image were removed successfully, using the multi-image adaptive local regression (MIALR) tool provided by the website. Based on various satellite images on the same region, the MIALR method fills the gaps in the target image through local regression of the region with the maximum correlation and the minimum local area. The MIALR can effectively remove speckles, despite its high time consumption.

About 5.35% of the original image was covered by clouds, which may affect the extraction of water bodies. Hence, both clouds and cloud shadows were removed through the local index-based method proposed by Xiao et al. [62].

2.3.2 Preprocessing of ALOS PALSAR data

The ALOS PALSAR image, provided by the Alaska Satellite Facility (ASF), was on the Level 1.5 and could not be used directly. Therefore, the image was preprocessed by the ASF MapReady 3.0 software. The preprocessing operations include radiation correction using Sigma calibration coefficients, terrain correction based on Wuhan DEM information, terrain normalization, and geocoding to 30m pixel resolution (WGS84, UTM 50N).

The speckles on the ALOS PALSAR image greatly affect the image quality and the evaluation of image classification. To solve the problem, the original image was speckle suppressed by a gamma maximum-a-posteriori (GM) filter [63] before image fusion for water body extraction.

2.3.3 Image registration

Registration is a key step in the fusion of remote sensing images [64]. Since Landsat 7 ETM+ image and ALOS PALSAR image were taken by imaging systems with different sizes and resolutions, image resolution must be performed before image fusion.

If the registration effect is poor, the fused dataset will have a nominal spatial resolution, which is worse than any of the two input images. Ideally, the registration should be pixel to pixel for systems with the same spatial resolution, but pixel-to-pixel registration is basically impossible.

Here, 12 ground control points (GCPs) in the study area are employed to correct the ALOS PALSAR image to the Landsat 7 ETM+ coordinate system, using second-order transform and the nearest neighbor difference. The final root mean square error (RMSE) was 0.5 pixels.

This section fuses the two source images in turn by several traditional image fusion methods, and uses the fused images to extract water bodies. The image fusion methods include the PCA [65], BT [66], HPF [67], IHS [68], and DWT [69]. For comparison, the fuse images were all classified by the MLC.

4.1 PCA image fusion

The ETM+453 image was taken as the multispectral input file, and PALSAR HH image was regarded as the high-resolution input file. Then, the PCA was performed on the ETM+453 image to generate three principal components. The first principal component was replaced by the PALSAR HH image. After the replacement, the three principal components were inversely transformed to create a fused image (Figure 7). The water bodies extracted from the fused image are displayed in Figure 8. As shown in Figures 7 and 8, more water bodies of the PALSAR HH image were retained than those of the ETM+453, thanks to the replacement of the first principal component with the high-resolution PALSAR HH image. The lotus area in Baoan Lake was visible, while the speckles in PALSAR image were also kept.

The main defect of PCA image fusion is that the PCA only considers statistical features, but not the features of each band. In terms of performance, the PCA works as if it selects between the source images, rather than fuse the salient information of source images. During the fusion process, the PCA measures information saliency by global variance, and tends to assign a high weight to the source image with large variance. The weighting strategy is not scientific. Besides, the PCA is highly sensitive to interference information like dead pixels and noises. In the presence of interference information, the global variance of the image will surge up. That is why far more information from the PALSAR HH image was included in the fused image than that from the ETM+453 image. In actual fusion, the PCA has better effect if the source images are of low contrast.

7.png

Figure 7. The image fused by the PCA

8.png

Figure 8. The water bodies extracted from the PCA fused image

4.2 BT image fusion

To enhance image information, the BT first normalizes each band in the ETM+453 image, and then multiplies the normalized results with the PALSAR HH image. The red (R), green (G), and blue (B) of the fused image can be respectively computed by:

$R=\frac{T M 4}{(T M 4+T M 5+T M 3)} * H H$     (1)

$G=\frac{T M 5}{(T M 4+T M 5+T M 3)} * H H$     (2)

$B=\frac{T M 3}{(T M 4+T M 5+T M 3)}^{*} H H$     (3)

where, TMi/(TM4+TM5+TM3) is the spectral information of the image; HH is the spatial information of the image.

The image fused by the BT and the water bodies extracted from the BT fused image are presented in Figures 9 and 10, respectively. Like the PCA, the BT introduces too much information of the PALSAR HH into the fused image. Besides the lotus area, both speckles and small water bodies were retained.

9.png

Figure 9. The image fused by the BT

10.png

Figure 10. The water bodies extracted from the BT fused image

4.3 IHS image fusion

The ETM+453 image was transformed into the IHS space to produce I, H, and S components. Then, the I component was replaced by the PALSAR HH image. After that, the three components were inversely transformed to the RGB space. The image fused by the IHS and the water bodies extracted from the IHS fused image are presented in Figures 11 and 12, respectively.

As shown in Figures 11 and 12, there was no speckle in the fused image, but the lotus-covered areas in Baoan Lake and Huama Lake were not extracted. This means the spectral information of the ETM+453 image plays a dominant role in the classification process.

11.png

Figure 11. The image fused by the HIS

12.png

Figure 12. The water bodies extracted from the IHS fused image

4.4 HPF image fusion

The HPF image fusion involves the following steps: The HPF of the PALSAR HH image to extract the high-frquency information (e.g. linear features and edge features); the low-pass filtering (LPF) of the ETM+453 image to extract low-frequency information (i.e. spectral information); the weighted summation of the extracted high-frequency and low-frequency information to create a sharpened fused image. The HPF transform can be defined as:

$H P_{i}=\left(w_{a} * M S I_{i L P}\right)+\left(w_{b} * H H_{i H P}\right)$     (4)

13.png

Figure 13. The image fused by the HPF with high weight coefficient

14.png

Figure 14. The water bodies extracted from the HPF fused image with high weight coefficient

15.png

Figure 15. The image fused by the HPF with low weight coefficient

16.png

Figure 16. The water bodies extracted from the HPF fused image with low weight coefficient

The weight of the ETM+453 in the fused image is defined as the weight coefficient. The weight coefficient is positively corelated with the sharpness of the fused image, and negatively with the smoothness of the fused image. Figures 13 and 14 are the image fused by the HPF and the water bodies extracted from the HPF fused image at a high weight coefficient (>0.6), respectively; Figures 15 and 16 are the image fused by the HPF and the water bodies extracted from HPF fused image at a low weight coefficient (<0.5), respectively.

For the PALSAR HH image, the grayscales of speckles are much larger than those of water. Therefore, the speckle information was preserved and the water information was not well preserved after the HPF. Under a high weight coefficient, the speckle information was inevitably enhanced in the fused image, bringing more noises. By contrast, the water body information was not preserved well, and the plant-covered areas in Baoan Lake and Huama Lake were not retained.

Under a low weight coefficient, the speckles in the PALSAR HH image were suppressed and smoothed, but the water body information in that image were more severely suppressed. Therefore, more water body information of the ETM+453 image was retained. The plant-covered areas in Baoan Lake and Huama Lake were not retained, and some small water bodies were gone.

4.5 DWT image fusion

Using the db4 wavelet function, the ETM+453 and PALSAR HH images were decomposed by three-layer DWT. For the approximate coefficients, the fusion rule based on average wavelet coefficients was adopted; for the detail coefficients, the rule of obtaining maximum variance of a local moving window (3×3) was employed. The image fused by the DWT and the water bodies extracted from the DWT fused image are presented in Figures 17 and 18, respectively. It can be seen that the water bodies were not satisfactorily extracted: the lotus-covered water areas, small rivers, and reservoir were all lost.

17.png

Figure 17. The image fused by the DWT

18.png

Figure 18. The water bodies extracted from the DWT fused image

4.6 Comparison of traditional fusion methods

Table 7 compares the water body areas in the images fused by the traditional fusion methods, and Table 8 compares the extraction accuracies of these methods.

Table 7. The water body areas in the images fused by the traditional fusion methods

Table 8. The accuracies of the traditional fusion methods in water body extraction

From Tables 7 and 8, it can be seen that the BT achieved the best effect in water body area and extraction accuracy, followed by the IHS and PCA.  Except HPF image fusion with a high weight vector, the other fusion methods all outperformed the ETM+453 image alone in water body extraction, and all overshadowed by the PALSAR HH image alone (accuracy: 93.4%). The results show that most fusion methods can improve the water body extraction of Landsat 7 ETM+ image, but cannot improve that of ALOS PALSAR image.

Compared with the benchmark, all the traditional fusion schemes are not satisfactory in visual effect and accuracy, failing to fuse the complementary water information of the two source images. Based on the global information of the entire image, some of the fusion methods overemphasize on the Landsat 7 ETM+ image, and some overemphasize on the ALOS PALSAR image. These theoretical methods mainly evaluate the fusion result based on the spectral preservation of the whole image, the information entropy, the degree of distortion, etc. None of them considers the requirement of a specific recognition task. For specific applications like water body extraction, the image fusion methods completely ignore the salient information of water in both source images. As a result, the image fusion does little good to water body extraction. Therefore, it is necessary to design a fusion method that combine the complementary information of multi-source images for specific recognition tasks.

Table 11 compares the water body extraction results of different methods, and Figure 25 describes the results in line charts. From Table 11 and Figure 25, it can be concluded that:

(1) The extraction accuracy is proportional to the size of extracted water bodies: the higher the accuracy, the greater the area of the extracted water bodies as a percentage of the total area of water bodies in the study area.

(2) Our method, i.e. the OBC-based image fusion, achieved the highest extraction accuracy (97.2%), followed by the PALSAR HH and then several traditional methods. The lowest accuracy belongs to the water body extraction from Landsat 7 ETM+ image alone.

(3) For single-source image methods, the ALOS PALSAR image had a higher accuracy than the Landsat ETM+ image. The former enjoys great advantages in extracting small water bodies and water areas covered by plants. But the water area extracted from the ALOS PALSAR image alone had some holes, due to the presence of speckles. Meanwhile, the extraction from Landsat ETM+ alone cannot detect the lotus-covered water areas or small water bodies, resulting in a very low accuracy.

(4) The traditional image fusion methods were lower than the PALSAR HH image alone in the extraction accuracy. The accuracy of the HPF image fusion with a high weight vector was even below that of the Landsat ETM+ image alone. Thus, it is very important to choose a proper fusion method for a specific task. The complementary information of each source image must be considered during the image fusion. Our image fusion method fully considers the complementary information of the two source images, and thus achieves high accuracy in water body extraction.

Table 11. The area and accuracy of water bodies extracted by different methods

25.png

Figure 25. The line charts of the area and accuracy of water bodies extracted by different methods