Shuai Shao
| Da Yu | Hangfei Yuan | Chunxiang Liu | Xinran Chen*
© 2025 The authors. This article is published by IIETA and is licensed under the CC BY 4.0 license (http://creativecommons.org/licenses/by/4.0/).
OPEN ACCESS
In order to improve the recognisability of thin cloud multi-spectral remote sensing image, a method of removing thin cloud from remote sensing images based on thin cloud physical model is proposed. Generally, the contrast of the region polluted by thin clouds in remote sensing image is low, and the real scene can be restored by increasing the contrast. However, the excessive contrast enhancement will cause data loss. The relationship between the contrast of MSE (Mean squared error) and the transmission of cloud is inversely calculated, and the transmission is estimated by establishing a cost function. The number reduces the loss of data information and maximizes the contrast. The intensity of sunlight is estimated by using the weighted quadtree search method. Finally, the restored image is obtained according to the thin cloud physical model. Experimental results show that the entropy of image information increases from 5.961 to 6.65, the e increases from 0.356 to 0.767, and FADE reduce from 0.289 to 0.261. These data indicate that our method achieved better results in terms of data information retention and thin cloud removal.
thin cloud physical model, remote sensing images, MSE contrast, cloud transmittance
Optical remote sensing satellite images are widely used in the field of earth observation and remote sensing of the earth because of their wide coverage, large data volume and high resolution [1-3]. A significant portion of the atmospheric system is covered by clouds [4], which are composed of tiny droplets and variable-sized ice crystals, and solar radiation passing through the clouds is scattered and absorbed before it reaches the target on the ground, leading to changes in colour and brightness in different areas of the remote sensing image, which results in blurred images and reduced contrast. When the clouds are too thick will not be able to obtain the target image, thin clouds will cause uneven illumination, thin cloud removal for remote sensing images can improve the quality of remote sensing images and the accuracy of the application [5-8]. The signal-to-noise ratio of cloud remote sensing images is low, and the visibility of colour remote sensing images is poor, therefore, thin cloud removal for colour remote sensing images has very important practical significance.
The current methods for thin cloud removal from remote sensing images are homomorphic filtering [9], Tasseled Cap Transformation method [10], image transformation based methods (Haze optimized transform, HOT) [11], and HOT improvement based methods [6, 12], and cloud physical model based methods [5, 7].
The method of homomorphic filtering is based on the characteristics of the thin cloud spectrum concentrated in the low-frequency band, through the filter compression of the low-frequency band to eliminate the impact of thin clouds on the image, which will cause a certain amount of data loss, notification of a greater demand for computer resources. The Tasseled Cap Transformation is an orthogonal linear transform determined based on the data structure of remote sensing images, the fourth component of the transform is related to the clouds, and cloud removal was realised by discarding the fourth component, this method was not suitable for scenes with brighter ground targets. The HOT based method first generates a transformed image (HOT map), which could be used to calculate the strength of the impact on clouds and mist. Based on the HOT value, the image was divided into multiple layers, and then dark target subtraction was performed on each layer [13] to achieve the goal of removing clouds and mist.
The HOT-based method first generated a transformed image (HOT map), according to the size of the HOT value can be deduced from the strength of the degree of influence with the cloud, based on the HOT value, the image was divided into multiple layers, and then each layer of the dark target subtraction [13] to achieved the purpose of removing the cloud, the method in the vegetated area has a good result, but for man-made features and other types of ground cover would produce excessive correction results.
Chen et al. proposed a remote sensing image cloud removal method using Iterative Haze optimized Transformation (IHOT) and cloud trajectories [6]. The method estimated cloud thickness using iterative Haze optimized transformation (IHOT), then marked the shadow areas affected by the cloud. Similar pixels in neighbouring regions at different cloud thicknesses were fitted by cloud trajectories. The cloud-contaminated areas in the image were corrected according to the relationship between surface reflectance and IHOT, the visual effect of remote sensing image was improved, but the estimation of cloud trajectory would be affected by the shadow detection error, which led to inaccurate estimation of cloud trajectory and failed to achieve the expected effect. The method based on the cloud physics model achieved thin cloud removal by eliminating the effect of clouds in the light transmission process. The model includes the cloud transmittance, atmospheric light intensity, and the attenuation coefficient of sunlight in the atmospheric transmission process, which can remove the effect of clouds on the image while preserving the colour information of the image.
In this paper, we proposed a method for removing thin clouds from multi-spectral remote sensing images based on cloud physics model, which is based on the effective estimation of cloud transmittance, sunlight intensity, and atmospheric attenuation coefficient to derive the true reflectance of the ground target. Among them, the estimation of cloud transmittance is crucial, and the estimation of cloud transmittance is achieved by the method of optimal contrast enhancement. Increasing the contrast of the image could improve the visual effect of the image contaminated by clouds, but the overstretching of the contrast would cause the pixel values to be overflowed or truncated. The optimal contrast enhancement value is determined by establishing a cost function to ensure that the loss of data information is reduced while maximizing the contrast.
2.1 The physical model of thin cloud image
Thin clouds can lead to degradation of the acquired remote sensing images. Currently, and the more commonly used physical model for thin cloud image [5, 14] is shown in Figure 1.
Figure 1. The physical model of thin cloud image
The image received by the image detector is composed of two parts, one is the part of sunlight that reaches the detector after being reflected from the clouds, and the other is the part of sunlight that reaches the detector after being reflected from the ground target and passing through the clouds, and the imaging model is expressed as follows:
\boldsymbol{I}(x, y)=\boldsymbol{\alpha} \operatorname{Lr}(x, y) t(x, y)+\boldsymbol{L}(1-t(x, y)) (1)
where, \boldsymbol{I}(x, y) represents the image received by the detector, r(x, y) is the true reflectance of the ground target scenery which represents the expected image to be obtained, t(x, y) is the cloud transmittance, L is the intensity of sunlight, \boldsymbol{\alpha} represents the attenuation coefficient of sunlight during atmospheric transmission, \boldsymbol{\alpha} \operatorname{Lr}(x, y) t(x, y) represents the part of the ground scenery that passes through the cloud after reflection, \boldsymbol{L}(1-t(x, y)) represents the cloud background. The imaging model can be written as:
r(x, y)=\frac{\boldsymbol{I}(x, y)-\boldsymbol{L}(1-t(x, y))}{\boldsymbol{\alpha} \operatorname{Lt}(x, y)} (2)
2.2 Estimation of cloud transmittance
The contrast of image areas contaminated by thin clouds is usually low, and the estimation of transmittance is achieved by maximizing contrast. MSE (Mean Squared Error) contrast, which represents the degree of difference between pixels, is calculated as follows:
C_{M S E}=\sum_{q=1}^N \frac{\left(r_i(q)-\bar{r}_i(q)\right)^2}{N} (3)
Among them i \in\{r, g, b\} represents the three color channels of the image, N represents the total number of pixels in the region, and \overline{r_i}(q) is the mean of r_i(q). After estimating the intensity of sunlight L, r_i(q) depends on the selection of the transmittance t. For simplicity, Eq. (3) can be rewritten as:
C_{M S E}=\sum_{q=1}^N \frac{\left(I_i(q)-\bar{I}_i(q)\right)^2}{t^2 N} (4)
where, \overline{I_i}(q) is the mean of I_i(q), according to Eq. (4), C_{M S E} is a decreasing function of t. Therefore, the contrast can be improved by decreasing t. For an 8 -bit image, we assume that (a, b) is a valid range for input pixel values which lead to output pixel values within (0,255). When most of the input values are in the valid range, the output image have a high contrast. Otherwise, the output pixel values would be below 0 or above 255 , result in the loss of image information.
Consequently, the improvement of contrast will inevitably be accompanied by the loss of data information. In order to find the best t value which can achieve the best results of the recovered image, the cost function E_C is proposed to find the optimal value of t. The contrast cost function is given as:
E_C=\sum_{i \in\{r, g, b\}} \sum_{q \in W} \frac{\left(I_i(q)-\overline{I_i}(q)\right)^2}{(t)^2 N_W} (5)
where, E_C is the contrast cost function, N represents the total number of pixels in the region of W . The cost function for data loss is defined as:
\begin{aligned}E_s=\sum_{i \in\{r, g, b\}} \sum_{q \in W} & \left\{\left(min \left\{0, r_i(q)\right\}\right)^2\right.\left.+\left(max \left\{0, r_i(q)-255\right\}\right)^2\right\}\end{aligned} (6)
where, min \left\{0, r_i(q)\right\} represents the portion of data truncation caused by pixel values below 0, and \left.max \left\{0, r_i(q)-255\right\}\right) represents the portion of data overflow caused by pixel values above 255. The contrast cost function and data loss cost function have been defined, then the transmittance t is calculated by minimizing the total cost function E_T, the total cost function E_T is defined as:
E_T=\beta E_S+E_C (7)
where, β is the adjustable weight parameter, used to adjust the specific weight given to reducing the degree of data loss or increasing the degree of contrast. According to the imaging model we have:
t(x, y)=\frac{I(x, y)+L}{l(\alpha r(x, y)-1)} (8)
In order to minimise data loss, we set 0 \leq r_i(q) \leq 255, data truncation and data overflow were avoided when ordering, then the following limiting conditions could be obtained for the value range of:
t \geq min _{i \in\{r, g, b\}} min _{q \in W}\left\{\frac{I_i(q)-L_i}{-L_i}\right\} (9)
t \geq max _{i \in\{r, g, b\}} max _{q \in W}\left\{\frac{I_i(q)-L_i}{L_i(255 \alpha-1)}\right\} (10)
By minimizing t , we can maximize the C_{M S E}, according to the above qualification conditions Eq. (9) and Eq. (10), We define t_m as the minimum value of t, The t_m is given as:
t_m=\max \left\{\begin{array}{c}min _{i \in\{r, g, b\}} min _{q \in W}\left\{\frac{I_i(q)-L_i}{-L_i}\right\} \\ max _{i \in\{r, g, b\}} max _{q \in W}\left\{\frac{I_i(q)-L_i}{L_i(255 \alpha-1)}\right\}\end{array}\right. (11)
We achieve a balance between data loss and contrast enhancement by controlling the value of b.
2.3 Intensity of sunlight estimation
In this paper, we are estimating the intensity of sunlight based on the pixel value relationship associated with a single image. When t tends to 0, the portion reflected from the surface of the imaging target is attenuated in the atmosphere and the detector is only able to acquire the sunlight illuminance portion. Therefore, the intensity of the sunlight in the thickest region of the clouds is used as an approximate estimation of the sunlight intensity.
He et al. [15] calculated the maximum value of the pixel points with the brightness value in the first 0.1% of the dark channel image as the estimation of global ambient light, this method is robust, but it is not applicable to the bright regions in the image, and the estimation value will be larger for RGB colour images. The estimation method of sunlight intensity in this paper is proposed based on the research of He et al. Firstly, the minimum channel map of the thin cloud degradation remote sensing image is given as:
D(X)=min _{c \in\{r, g, b\}}\left(I_C(x)\right) (12)
Figure 2(a) is the input image, and Figure 2(b) is its minimum channel image. Once the minimum channel image is acquired, it is computed using a weighted quadtree search approach [16] as shown in Figure 3.
Figure 2. The minimum channel graph
Figure 3. The quadtree method to calculate L
Firstly, we divide the minimum channel map into four regions of the same size and calculate the score value K for each sub-region separately, the region with large K value is more suitable for estimating the solar intensity estimation, K value is calculated by subtracting the standard deviation of these pixels from the mean of the pixels in the region, then we have:
K_a=\left(\overline{D_a}-\sigma_a^2\right), a=1,2,3,4 (13)
where, a represents the index of each region, \overline{D_a} represents the mean of the region of a, and \sigma_a^2 represents the variance of the region of a. We select the region with the largest value of K as the region to continue the iteration, and then divide it into four regions, and this step continues until the size of this region is smaller than a set threshold, and the mean value of each channel of the thin cloud image corresponding to the final selected region is the estimation of the sunlight intensity L.
When the transmittance t and light intensity L are known, we can get the real image by calculation, and in order to suppress the noise generation, the transmittance t is limited to the interval of (0.1-0.95), then r(x) is given as:
r(x)=\frac{I(x)-L}{\alpha Lmin\{max \{t(x), 0.1\}, 0.95\}}+\frac{1}{\alpha} (14)
where, r(x) is the expected image, and \alpha represents the ability of the atmosphere to scatter light per unit volume. In this paper, we have assigned the value of \alpha as 0.96 , the \alpha can usually be regarded as a constant in a homogeneous region [17].
The current image evaluation methods are divided into subjective and objective evaluation methods, the subjective evaluation methods are mainly based on human visual perception, and the objective evaluation methods are based on quantitative indicators. The subjective evaluation part of this paper mainly consists of comparing the experimental results of the method based on thin cloud background removal proposed by Liu et al. [5], dark channel prior dehazing method proposed by He et al. [15], the Multi-Scale Retinex (MSR) algorithm, and the algorithm proposed in this paper. We select some classical objective evaluation assessment indices, including the image entropy [18], the structural similarity index (SSIM) [19], the ratio of new visible edges (e) [20], and the fog aware density evaluator (FADE) [21].
3.1 Experimental results
Due to the presence of clouds in the original thin cloud remote sensing images, the visibility of the scene is reduced. In order to verify the effectiveness of the algorithm in this paper, multiple remote sensing images were selected for comparative experimental analyses, and the image data were obtained from Google Earth and NASA Earth Observatory websites. Firstly, in order to determine the optimal value of the parameter \beta, which is used to balance contrast enhancement with data loss, we conducted a comparative analysis of the experimental effects for different \beta values. The results of these experiments are illustrated in Figure 4.
Different values of \beta correspond to distinct processing outcomes. As shown in Figure 4(b), when \beta is set to a smaller value of 1, the processed image exhibits higher contrast, but suffers from significant loss of detail. Conversely, as shown in Figure 4(f), when \beta is set to a larger value of 10, the image retains more detail, but the effectiveness of thin cloud removal is compromised. Setting \beta to 6 achieves an optimal balance between preventing information loss and effectively removing haze. As a result, we maintain \beta at 6 across all experiments.
Figure 4. Comparison of processing results using the proposed method with different \beta parameters
Figure 5. Comparison of experimental results between different algorithms
In the following experiments, according to the characteristics of cloud distribution and the differences in cloud thickness, representative remote sensing images U, V, Y, Z were selected, and the comparison of experimental results between different algorithms are as shown in Figure 5.
Figure 5(a) are the original image, Figure 5(b) are the results after processing by MSR algorithm, the processing result of MSR algorithm show that although the contrast has been improved, the image brightness is too bright and there is a colour distortion. Figure 5(c) are the result of Liu's method, a part of the cloud layer in the image are removed, and the clarity enhancement effect is insufficient. Figure 5(d) represent the experimental results based on the dark channel prior method, there are noticeable deviations in the color and texture information of the ground scenery compared to the original image.
Figure 5(e) present the processed results of the algorithm proposed in this paper. Due to the thick cloud cover in the original image U of Figure 5 and the alternating distribution of thick and thin clouds in the original image V, some residual clouds remain in the processed results when applying our method. For the original images Y and Z, the areas with thin clouds have been effectively improved. A subjective visual analysis indicates that our method outperforms the aforementioned methods in terms of texture information, color fidelity, and cloud removal. The thin cloud-covered areas in the original images have been effectively removed, while the contrast of the images has been enhanced, improving the visibility of remote sensing images affected by thin clouds and thereby increasing their application value.
3.2 Quantitative results comparison
Image information entropy, as a statistical feature of images, serves as one of the objective evaluation indicators in this paper. Pixels with varying brightness levels in an image exhibit distinct spatial shapes and contain differing amounts of information. The greater the uncertainty in the image's shape, the higher the amount of information it contains, and consequently, the greater the image's information entropy. Structural Similarity (SSIM) is an objective evaluation metric independent of image brightness and contrast, aligning with the characteristics of the human visual system. It reflects the algorithm's ability to preserve structural information in images. A higher SSIM value indicates a greater similarity between the cloud-removed image and the ground truth image. The ratio of newly visible edges (e) is an image evaluation metric used to assess the effectiveness of restoring invisible edges in blurred images. In this paper, it is employed to evaluate the changes in visible edges before and after thin cloud removal. The Fog Aware Density Evaluator (FADE) can be used to predict the visibility of blurred scenes. It predicts the visibility of objects in haze by measuring the deviation in statistical patterns between natural scene images and images affected by haze or fog. The smaller the value of FADE, the higher the visibility of the processed image.
The experiments demonstrate that our algorithm achieves better performance on image structure information preservation, visible edge enhancement and thin cloud removal, the results are shown in Table 1.
Table 1. The corresponding values of image entropy, SSIM, e and FADE
|
Image |
Image Entropy |
SSIM |
e |
FADE |
|
|
Original |
U V Y Z |
6.333 6.134 6.215 5.961 |
0.743 0.698 0.732 0.712 |
2.632 0.675 1.786 0.356 |
0.357 0.626 0.315 0.289 |
|
MSRCR method |
U V Y Z |
6.415 6.254 6.323 6.372 |
0.783 0.745 0.765 0.756 |
2.806 0.927 1.946 0.463 |
0.281 0.472 0.306 0.285 |
|
Liu’s method |
U V Y Z |
6.354 6.403 6.397 6.218 |
0.763 0.745 0.756 0.772 |
2.763 0.654 1.845 0.547 |
0.335 0.607 0.282 0.276 |
|
He’s method |
U V Y Z |
6.434 6.207 6.401 6.179 |
0.712 0.746 0.786 0.749 |
2.782 0.765 1.864 0.658 |
0.317 0.608 0.264 0.286 |
|
Ours |
U V Y Z |
6.658 6.754 6.801 6.675 |
0.748 0.716 0.763 0.748 |
2.788 0.842 1.985 0.767 |
0.305 0.593 0.256 0.261 |
To improve the clarity of multi-spectral remote sensing images, this paper proposes a thin cloud removal method based on a cloud physical model. By estimating the transmittance of the cloud layer and the intensity of sunlight in the thin cloud remote sensing image model, the true reflectance of ground objects is accurately reconstructed, effectively removing thin clouds from degraded remote sensing images. The experimental results demonstrate that the method proposed in this paper significantly improves metrics such as image entropy and SSIM. In conclusion, our method exhibits superior performance in both thin cloud removal and the preservation of image details, effectively enhancing the visual quality and practical utility of remote sensing images degraded by thin clouds. In future research, we will focus on removing clouds from multi-spectral remote sensing images under conditions of thicker cloud layers and heterogeneous cloud distributions using deep learning techniques.
This paper was supported by National Natural Science Foundation of China (Grant No.: 12073028) and National Natural Science Foundation of China (Grant No.: 12473084).
[1] Han, J., Zhou, Y., Gao, X., Zhao, Y. (2024). Thin cloud removal generative adversarial network based on sparse transformer in remote sensing images. Remote Sensing, 16(19): 3658. https://doi.org/10.3390/rs16193658
[2] Mani, A.B., Nagashree, T.R., Manavalan, P., Diwakar P.G. (2024). Removal of clouds from satellite images using time compositing techniques. arXiv preprint arXiv:2410.08223. https://doi.org/10.48550/arXiv.2410.08223
[3] Jang, J.H., Kim, S.D., Ra, J.B. (2011). Enhancement of optical remote sensing images by subband-decomposed multiscale retinex with hybrid intensity transfer function. IEEE Geoscience and Remote Sensing Letters, 8(5): 983-987. https://doi.org/10.1109/LGRS.2011.2146227
[4] Hagolle, O., Huc, M., Pascual, D.V., Dedieu, G. (2010). A multi-temporal method for cloud detection, applied to FORMOSAT-2, VENµS, LANDSAT and SENTINEL-2 images. Remote Sensing of Environment, 114(8): 1747-1755. https://doi.org/10.1016/j.rse.2010.03.002
[5] Liu, J., Wang, X., Chen, M., Liu, S., Zhou, X., Shao, Z., Liu, P. (2014). Thin cloud removal from single satellite images. Optics express, 22(1): 618-632. https://doi.org/10.1364/OE.22.000618
[6] Chen, S., Chen, X., Chen, X., Chen, J., et al. (2018). A novel cloud removal method based on IHOT and the cloud trajectories for Landsat imagery. Remote Sensing, 10(7): 1040. https://doi.org/10.3390/rs10071040
[7] Lv, H., Wang, Y., Shen, Y. (2016). An empirical and radiative transfer model based algorithm to remove thin clouds in visible bands. Remote Sensing of Environment, 179, 183-195. https://doi.org/10.1016/j.rse.2016.03.034
[8] Yu, X.C., Zhi, Y., Tang, S.J., Li, B.B., Gong, Q., Qiu, C.W., Xiao, Y.F. (2018). Optically sizing single atmospheric particulates with a 10-nm resolution using a strong evanescent field. Light: Science & Applications, 7(4): 18003-18003. https://doi.org/10.1038/lsa.2018.3
[9] Cai, W., Liu, Y., Li, M., Cheng, L., Zhang, C. (2011). A self-adaptive homomorphic filter method for removing thin cloud. In 2011 19th International Conference on Geoinformatics, Shanghai, China, pp. 1-4. https://doi.org/10.1109/GeoInformatics.2011.5980963
[10] Chen, C., Tang, P., Bian, Z. (2012). Tasseled cap transformation for HJ-1A/B charge coupled device images. Journal of Applied Remote Sensing, 6(1): 063575. https://doi.org/10.1117/1.JRS.6.063575
[11] Saritha, P.K., Sethuraman, J., Anjani, R.N., Rao, C.V. (2019). Quantification of particulate matter using haze optimized transform (HOT) of oceansat-2 OCM data. In 2019 Second International Conference on Advanced Computational and Communication Paradigms (ICACCP), Gangtok, India, pp. 1-5. https://doi.org/10.1109/icaccp.2019.8882981
[12] He, X.Y., Hu, J.B., Chen, W., Li, X.Y. (2010). Haze removal based on advanced haze-optimized transformation (AHOT) for multispectral imagery. International Journal of Remote Sensing, 31(20): 5331-5348. https://doi.org/10.1080/01431160903369600
[13] Vincent, R.K. (1972). An ERTS multispectral scanner experiment for mapping iron compounds. https://ntrs.nasa.gov/api/citations/19730001633/downloads/19730001633.pdf.
[14] Saxena, J., Jain, A., Krishna, P.R., Bothale, R.V. (2022). Cloud removal and satellite image reconstruction using deep learning based image inpainting approaches. In Rising Threats in Expert Applications and Solutions: Proceedings of FICR-TEAS 2022, pp. 113-121. https://doi.org/10.1007/978-981-19-1122-4_13
[15] He, K., Sun, J., Tang, X. (2010). Single image haze removal using dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(12): 2341-2353. https://doi.org/10.1109/TPAMI.2010.168
[16] Qin, B., Huang, Z., Zeng, F., Ji, Y. (2015). Fast single image dehazing with domain transformation-based edge-preserving filter and weighted quadtree subdivision. In 2015 IEEE International Conference on Image Processing (ICIP), Quebec City, QC, Canada, pp. 4233-4237. https://doi.org/10.1109/ICIP.2015.7351604
[17] Narasimhan, S.G., Nayar, S.K. (2002). Vision and the atmosphere. International Journal of Computer Vision, 48: 233-254. https://doi.org/10.1023/A:1016328200723
[18] Casanova, A., DI Gesù, V., Lo Bosco, G., Vitulano, S. (2005). Entropy measures in image classification. In Human And Machine Perception: Communication, Interaction, and Integration, pp. 89-103. https://doi.org/10.1142/9789812703095_0007
[19] Gonçalves, T., Lopes, R.J., Nunes, P. (2010). Using metadata in video quality assessment based on the structural similarity (SSIM) index metric. In International Workshop on Quality of Experience for Multimedia Content Sharing – QoEMCS, Tampere, Finland.
[20] Hautiere, N., Tarel, J.P., Aubert, D., Dumont, E. (2008). Blind contrast enhancement assessment by gradient ratioing at visible edges. Image Analysis and Stereology, 27(2): 87-95. https://doi.org/10.5566/ias.v27.p87-95
[21] Ali, U., Lee, I.H., Mahmood, M.T. (2022). Robust regularization for single image dehazing. Journal of King Saud University-Computer and Information Sciences, 34(9): 7168-7173. https://doi.org/10.1016/j.jksuci.2022.02.020
