Classification of Satellite Images Using a Deep Learning-Inspired Hybrid Novel Approach

Several picture datasets containing annotations, as well as related detection and classification efforts, have recently emerged. Land cover categorization and structure recognition have dominated deep learning's applications to remotely sensed images. For instance, the UC Merced Land Use Dataset has 2100 photographs of the United States taken from the air. Cartography of the Earth [15, 16]. The ground sample distance per pixel in these 256×256 photos is 0.3 metres. Among the 21 categories are storage tanks, tennis courts, and more typical land uses like agriculture, roads, and water. One study found a 98.5 percent accuracy rate in classifying UC Merced photos into land cover categories using the VGG, ResNet, and Inception CNNs [17-19]. Nevertheless, this dataset has severe limitations in terms of scope, classification depth, and geographic coverage.

High-resolution Digital Globe satellite photos of five cities and building footprints make up the SpaceNet dataset [20]. Convolutional neural networks (CNNs) have been taught to extract building footprints from photos [21]. In terms of training a classifier, this dataset has several severe limitations.

Further remote sensing data sets are included in the previous study [22]. They need the global corpus of hundreds of thousands of photos to train a robust image classification algorithm.

Satellite image classification organises pictures by an object or semantic meaning into three primary categories: techniques based on standard features, methods based on high-scene features, and hybrid approaches [23]. Mid-features-based methods work well for complex images [24]. Plans with many qualities best handle complex visuals. CNN is a popular deep-learning image-processing method [25].

"Deep Belief Network for classification" utilizing Convolutional Neural Networks achieves 97.946 SAT4 and 93.916 SAT6 accuracy [26]. Ju et al. [27] investigated image classification and identification ensemble techniques using deep convolutional neural networks. Saikat Basu, Sangram Ganguly, and others developed "DeepSat," a satellite image learning system. The super learner, majority voting, and Bayes Optimal Classifier are examples. Albert et al. [28] use deep CNN-based computer vision and large-scale satellite imagery to examine urban land use. A deep neural network does this with data. To find ground truth land, they carefully use open-source survey class designations. The Urban Atlas land categorization dataset comprises 300 European cities and 20 land use types. They also show that deep representations using satellite pictures of urban landscapes can compare cities' communities. Metropolitan satellite photographs proved this. Robinson et al. [29] created high-resolution population estimates using satellite data and a deep-learning convolutional neural network model. CNN algorithm trained on one year of composite Landsat pictures predicts the US population on a 0.01 by 0.01 grid. CNN model validation used quantitative and qualitative methodologies. The quantitative validation compared the proposed model's grid cell estimates to many US Census county-level population estimations. Qualitative validation directly evaluated model predictions for satellite image inputs. The model illustrates how machine learning algorithms can tackle social issues using unstructured and remotely sensed data. The literature review is described in Table 1.

Table 1. Summarized results of literature review

The suggested ensemble model combines layered Convolutional Neural Network and SVM algorithm [30-34]. Figure 2 depicts the whole model.

Convolutional Neural Network (CNN) is used for classification, one of the most popular technologies, as shown in Figure 3. Each layer of CNN contains the following sub-layers:

(1). Convolutional Layer (2). Completely Networked Layer (3). The Pooling Layer (4). The Drop-out Layer (5). Linked Dataset Classification Layer.

• Convolutional Layer

The convolution procedure is crucial to the convolutional layer, which maps the input picture with a filter of size mm to produce feature maps for the output. In equation form, the result of the convolutional layer may be represented as

$A_n^m=f\left(\sum_{k \in L_n} A_{k n}^{m-1} * M_{k n}^m+C_n^m\right)$            (5)

where,

Results-based feature maps;

C_n: Bias term;

L_n: Input maps;

M_kn: Convolution kernel.

The final feature map's sophistication may be described as:

$N=\frac{(\mathrm{X}-\mathrm{M}-2 \mathrm{Y})}{\mathrm{T}}$             (6)

Calculating Output Height/Length (N).

Height/Length Input X.

Filter size (M), padding (Y), and stride length (T).

The data may be saved via padding in this case. Eq. (7) describes the padding:

$Y=\frac{(M-1)}{2}$            (7)

where, M is the size of the filter.

• The ReLU Layer

The convolutional process becomes more linear due to this layer's contributions. Hence, a ReLU layer is connected to each convolutional layer in the network. The most important thing that needs to be done in this layer is to ensure that all negative activations are set to zero and that the thresholding is set to maximum (0, p).

2.png

Figure 2. Ensemble deep neural network

3.png

Figure 3. CNN architecture

• The Max-Pooling Layer

This layer is responsible for producing the output in a smaller size after the components of each block have been maximized.

• Dropout Layer

During the training phase, this layer is used to remove input components whose probability is less than a predetermined threshold.

• The Batch Normalization Layer

To standardize the value of the activation layer, this layer does a variety of mathematical operations, including subtraction, division, shifting, and scaling. The Eqs. (8)-(11) may be used to represent the batch normalized result, also known as B_k:

$\begin{aligned} B_k & =\mathrm{D} O_{\theta_\alpha} \times\left(A_k\right)\equiv \theta \widehat{A_k}+D\end{aligned}$         (8)

where, $\hat{A}_k$ is the settling down of activation A_k.

$\hat{A}_k=\frac{A_k+U_D}{\left(\sigma_D^2+\varepsilon\right)^{1 / 2}}$              (9)

where,

ε: constant in nature;

U_D: Mini-batch average;

σ_D²: Minimal batch variance given by:

$U_D=\frac{1}{d} \sum_{k=1}^d A_k$         (10)

$\sigma_D^2=\frac{1}{\mathrm{~d}} \sum_{\mathrm{k}=1}^{\mathrm{d}}\left(\mathrm{A}_{\mathrm{k}}-\mathrm{U}_{\mathrm{D}}\right)^2$        (11)

• Completely Connected Layer

This layer links the neurons of the next layer to those of the layer below it, creating a vector. The vector's dimensions indicate class numbers.

• The Output Layer

At this layer, the softmax algorithm is used. The equation that defines the softmax is as follows:

$P\left(v_r \mid A, \theta\right)=\frac{P\left(A, \theta \mid v_r\right) P\left(v_r\right)}{\sum_{n=1}^M P\left(A, \theta \mid v_r\right) P\left(v_r\right)}$        (12)

where, $0 \leq P\left(v_r \mid \mathrm{A} \theta\right) \leq 1$ and $\sum_{n=1}^M P\left(v_r \mid A, \theta\right)=P\left(A, \theta \mid v_r\right)$ are the conditional and class prior probabilities. Eq. (13) may also be:

$P\left(v_r \mid A, \theta\right)=\frac{\exp \left[\mathrm{d}_{\mathrm{r}}(\mathrm{A}, \theta)\right]}{\sum_{n=1}^M \exp \left[d_n(A, \theta)\right]}$          (13)

written as follows:

$d_r=\ln \left(P\left(A, \theta \mid v_r\right) P\left(v_r\right)\right)$           (14)

The output of the layered CNN is used as inputs for the regression model in the following way: The logistic regression model is described as follows: In this section of the model, the feature vector is represented by the letter x, and the outputs are probabilities:

$\hat{y}=P(y=1 \mid x)$             (15)

The feature vector denotes each instance of an item that belongs to the class, and each model is represented by one of the RGB channels outlined in Eq. (16):

$n_x=n_h+n_w+3$           (16)

$\hat{y}=\sigma(z)$          (17)

where,

$\sigma(z)=\frac{1}{1+e^z}$              (18)

The logistic function is expressed as z:

$z=w^T x+b$           (19)

Eqs. (20) and (21) express the loss and cost functions, respectively:

$\begin{aligned}L(\hat{y}(i), y(i)))= & -[y(i) \log \hat{y}(i)+(1 -y(i)) \log (1-\hat{y}(i))]\end{aligned}$           (20)

$J(w, b)=\frac{1}{m} \sum_{i=1}^m(L(\hat{y}(i), y(i)))$             (21)

where, m represents training examples.

The classification is provided by:

$\frac{\delta L}{\delta w}=(\hat{y}(i)-y(i)) x_j(i)$ and $\frac{\delta L}{\delta b}=\hat{y}(i)-y(i)$         (22)

The feature vector is represented by j =1, 2, ..., nx.

The size of each picture is first normalized here in Figure 4 preprocessing so that it conforms to the criterion of 256 pixels by 256 pixels. Python libraries are used to carry out identical operations with the highest possible degree of precision. When the data have been preprocessed, the appropriate acronyms are affixed. Then the data are separated into the various classes that will be utilized for testing afterwards.

4.png

Figure 4. Segmentation and classification

The input layer depicts the picture fed into the CNN at the beginning of the process. The image loaded into the computer is denoted by the formula [height * width * many color channels]. The value of the color channel indicates the kind of picture; for example, the value channel=3 indicates an RGB image. The same input is then run through a data argumentation before being sent to the CNN for processing. The argumentation is carried out using various procedures, including cropping, rotation, and so on. Since the CNN model requires a substantial quantity of data to provide accurate results, the input data are augmented using some process that generates more data to meet its requirements.

The investigation is done with GPU-based Google Co-Lab and karas libraries written in Python. Experiments are carried out within the scope of this study, with the batch size, epoch, and learning rate all being subjected to change. The investigation uses two different epoch sizes, 50 and 100, and three different learning rates, namely 0.1, 0.001, and 0.0001.

Figure 5 shows the training sample images. The comparison of the training loss to the validation loss, as shown in Figures 6 (a) and 6 (b), and the comparison of the training accuracy to the validation accuracy, using 50 epochs and 0.0001 as the learning rate, are shown. Figure 6 (a) shows the error rate inversely proportional to the amount of model learning. When there is an increase in model learning, there is a corresponding drop in the error rate. As shown in Figure 6 (b), the model's accuracy and the amount of learned information improve. The accuracy of the model is around 98.97% of the time.

5.png

Figure 5. Sample training images

6a.png

(a)

6b.png

(b)

Figure 6. (a) Loss associated with a learning rate of 0.0001 and 50 epochs; (b) Accuracy with learning rate 0.0001, 50 epochs

7a.png

(a)

7b.png

(b)

Figure 7. (a) Loss during 100 epochs at a learning rate of 0.0001; (b) Accuracy with learning rate 0.0001, 100 epochs

Figures 7 (a) and 7 (b) show the difference between the training loss and the validation loss, as well as the difference between the training and validation accuracy, with a learning rate of 0.0001 epochs per second. Figure 7 (a) shows the error rate inversely proportional to the amount of model learning. When there is an increase in model learning, there is a corresponding drop in the error rate. Figure 7 (b) illustrates that the model's accuracy improves as the amount of learned information increases. The accuracy of the model is around 98.90% of the time.

8a.png

(a)

8b.png

(b)

Figure 8. (a) Loss with learning rate 0.001, 50 epochs; (b) Accuracy with learning rate 0.001, 50 epochs

These results show that the computing algorithm acts as a primary role. For example, the weight correction used in the learning rate is computed using a training parameter. Figures 8 (a) and 8 (b) demonstrate the contrast between training loss/validation loss and training accuracy/validation accuracy on 50 epochs and a 0.001 learning rate. Figure 8 (a) depicts the error rate inverse to the learning of the model. Whenever model learning increases, the error rate decreases. Figure 8 (b) illustrates that the model accuracy increases as knowledge increases. The accuracy of the model reaches 98.98%.

Figures 9 (a) and 9 (b) show the difference between training loss and validation loss, as well as between training accuracy and validation accuracy, with a learning rate of 0.001 on 100 iterations. The error rate inversely related to the learning of the model is shown in Figure 9 (a). When there is an increase in model learning, there is a corresponding drop in the error rate. Figure 9 (b) illustrates that the model's accuracy improves as the amount of learned data increases. The accuracy of the model is around 98.99% of the time.

Figures 10 (a) and 10 (b) compare training loss/validation loss and training accuracy/validation accuracy on 50 epochs and 0.001 learning rate. Figure 10 (a) depicts the error rate inverse to the learning of the model. Whenever model learning increases, the error rate decreases. Figure 10 (b) illustrates that the model accuracy increases as learning rises. The accuracy of the model reaches 98.98%.

9a.png

(a)

9b.png

(b)

Figure 9. (a) Loss with learning rate 0.001, 100 epochs; (b) Accuracy with learning rate 0.001, 100 epochs

Figures 11 (a) and 11 (b) show the comparison between training loss and validation loss at 100 epochs and 0.01 learning rate. Figure 11 (a) shows the error rate as a function of model learning. Error rates go down as model learning speeds up. Figure 11 (b) shows that the model becomes more accurate with more training. The model achieves a 98.99% degree of accuracy.

10a.png

(a)

10b.png

(b)

Figure 10. (a) Loss with learning rate 0.001, 50 epochs; (b) Accuracy with learning rate 0.01, 50 epochs

11a.png

(a)

11b.png

(b)

Figure 11. (a) Loss with learning rate 0.01, 100 epochs; (b) Accuracy with learning rate 0.01, 100 epochs

Table 3 shows that a dataset and meaningful epoch and learning rate settings were used to train the model. The dataset and the relevant epoch value affect the experiment's results. An exact outcome may be seen in the value of the critical epoch. Several epochs and learning rates are used in the investigation. This paper provides results for two different period lengths and three different learning rate settings. The experiment results with two epochs and three learning rate settings are shown in Table 3. The envisioned paradigm contrasts with the gold standard classification model and CNN based hybrid model. The results, summarized in Table 4, show that the suggested model is superior to the alternatives. Table 5 shows the comparison with literature and other state of art algorithms.

The results show that the suggested neural network ensemble performs better than competing models. Accuracy, F1, Recall, and precision value are only a few of the metrics that have been compared.

Table 3. Tuning parameter values and accuracy

Table 4. Comparative analysis

Table 5. Comparative analysis with literature and other state of art algorithms