Optimizing Hyperspectral Image Classification Through Advanced Deep Learning Models Enhanced by Adam

High-spatial and spectral-resolution images with hundreds of data bands could be produced with hyperspectral sensors. HSI is an influential tool in several industries, including environmental mining [1], meticulous agriculture [2], and environmental monitoring [3], as a consequence of the abundance of spatial and spectral data. The most important method of HSI extraction is HSI categorization. Nonetheless, because of the complicated features of HSI data, HSI classification is still difficult. The Hughes phenomenon may be due to the high dimensionality of HSI and the massive number of spectral bands [4]. Thus, it may not be appropriate to classify HSIs directly using spectral fingerprints [5]. However, the classification of hyperspectral data presents arduous challenges due to innate complexities. In a high-dimensionality problem, each pixel in an HSI scene is characterized by a spectrum with hundreds of bands, resulting in data of high dimensionality and redundancy. Traditional classification techniques scuffle to effectively handle this complex data space, often leading to issues of overfitting, computational inefficiency, and difficulty in discriminating elusive spectral differences between classes. HSI data can be susceptible to noise, atmospheric interference, or artifacts caused by sensor limitations. These disturbances can degrade the quality of spectral signatures, affecting classification accuracy and making it challenging to distinguish a true signal from noise. Addressing these challenges needs innovative methods that can handle high-dimensional, complex, and noisy data effectively.

Over the past ten years, several feature extraction techniques have also been implemented to address this issue. The dimension of HSI reduction is a familiar concept. Principal component analysis (PCA) is covered [6]. For HSI classification, other nonlinear dimension reduction techniques are also used, such as manifold learning [7, 8]. Developed sparse subspace clustering is the foundation of the band selection method [9]. A kernel-based feature choice approach to finding a portion of the actual HSI is provided [10]. Many academics have worked on spectral-spatial feature extraction to enhance classification performance further. A lengthy morphological outline is presented in study [11] to merge spatial and spectral data. Surveys use a spectral-spatial categorization approach based on feature outlines [12]. For HSI classification, a discontinuity-preserving relaxation technique is created [13]. It clearly talks about the issue of pixels that are mixed up [14] and gives an overview of the spectral and spatial HSI categorization [15].

Deep-learning procedures have recently been performed in various disciplines, including image classification and data dimensionality reduction [16, 17]. Deep learning technology even outperformed human ability in face recognition in 2014 [18]. Deep-learning techniques try to learn representative and discriminative characteristics from the data in a hierarchical form. A methodological tutorial on utilizing deep learning in remote sensing may be found [19]. For the first time, HSI classification is handled using deep learning [20]. Somewhere, a stacked autoencoder (SAE) is applied to extract the characteristics in the HSI dataset. A few of the enhanced autoencoder-based techniques that are presented as a result of this work [21-24] According to research, CNN may also provide helpful deep features for HSI categorization. Nevertheless, deep learning-based methods frequently require training networks with complicated structures, which makes the training process time-consuming. A condensed deep-learning baseline known as a PCA network (PCANet) is presented [25]. PCANet is a significantly less complex network than CNN. Each layer's convolution filter bank comprises PCA filters, the nonlinear layer comprises binary quantization, and the feature pooling layer is altered to a layer consisting of block-wise histograms of the binary codes. In many images’ classification tasks, investigations show that PCANet is already highly competitive with and frequently outperforms conventional deep-learning-based features, despite being relatively straightforward [26-29]. HSI produces extensive data volumes due to the several spectral bands captured per pixel. Storing, processing, and transmitting this data can be resource-intensive, requiring efficient compression techniques. Balancing spectral and spatial resolution in HSI sensors is a challenge. Analyzing hyperspectral datasets involves computationally intensive tasks such as preprocessing, feature extraction, classification, and visualization. Another major limitation in HSI data classification is obtaining accurately labeled ground truth data for training and validation purposes. Addressing these limitations involves advancements in sensor technology, data processing algorithms, computational efficiency, and the progress of user-friendly tools for HSI data interpretation and analysis.

However, with few training examples, deep learning-based approaches cannot perform well [30]. Reducing the amount of training datasets needed for deep learning-based HSI categorization is considered one of the future works [31]. To build a deep-learning approach with strong feature depiction capabilities, hundreds of thousands to one crore of training databases are typically required [32], but this is practically unattainable for the job of HSI categorization. The researcher knows some well-known datasets where conventional classical methods consistently compete with deep learning-based ones when training samples are constrained. Although deep learning-based techniques show promise, the issue of small sample sizes needs to be fixed.

The rationale for choosing specific CNN architectures like 3D-CNN, R-3D-CNN, 2D-CNN, and R-2D-CNN methods typically depends on the nature of the data, its dimensions, and the spatial-spectral characteristics of the images. The novel 3D-CNN, R-3D-CNN, 2D-CNN, and R-2D-CNN methods utilized for HSI categorization are shown in this section. We initially extract a tiny patch focused on every picture element to create the categorization approaches for these methods. Our proposed approach, the 3D-CNN and R-3D-CNN methods, uses the spectral correlations and spatial information of picture elements, but the 2D-CNN and R-2D-CNN methods only use spatial contexts.

3.1 The 2D convolutional neural network (2D-CNN)

Our 2D-CNN method is separated into three sections, as exposed in Figure 1: patch extraction, feature extraction, and label identification. We initially extract a tiny patch focused on every pixel from a hyperspectral image to serve as the essential feature. The feature maps of these patches are then developed with a built-in deep learning method. As a concluding phase, the feature map of the similar patch is utilized to categorize every picture element label. We don't include the pooling layers for any of the four methods to retain as much pixel information as possible. The following diagram depicts the 2D-CNN methods of three-phase processing.

1.png

Figure 1. 2D convolutional process of 2D-CNN method

Assuming that a hyperspectral image of size W×H×C is provided to us, where H and W stand for the image's height and width, respectively, and C stands for the number of spectral bands, Predicting the labels for every pixel in the image is our goal. Given that labels for spatially nearby pixels frequently match, the proposed approach should consider "spatial coherence." The initial step in the presented method of processing is to extract a P×P×C patch for every picture element. A pixel, the patch's center, is the location about which every patch is built. There might not be enough data for the picture elements close to the edge of the image to create a patch that is the expected size. As a result, we use the mirror-padding process of picture elements to generate the spatial context.

The next processing stage handles every extracted patch as a separate image with multiple channels. To extract the feature maps for the patch, we can thus use a deep CNN method with 2D layers. The 2D-CNN operator at every layer is more appropriately stated in the subsequent Eq. (1):

$u_{i j}^{x y}=G\left(d_{i j}+\sum_s \sum_{a=0}^{H_i-1} \sum_{b=0}^{W_j-1} \vartheta_{i j}^{s a b} u_{i-1}^{(x+a)(y+b)}\right)$         (1)

Here, uⅈjxy stands for the outcome at point (x, y) of the j is amount of feature map at the i layer, dij denotes the bias term, G(.) represents the activation function of the layer, and s indexes over the set of feature maps of the (ⅈ-1)th layer, which are the inputs to the ith layer. Where i denotes the specific layer under consideration. Hi and Wj are the width and height of this kernel, and ϑijsab is the point (a,b) of the convolution kernel that connects the ith feature map to the jth feature map. Use the ReLU function as the initiation G function for the proposed approach, which is described in Eq. (2) below:

$G(x)=\max (0, x)$       (2)

Three convolutional layers are used in our 2D-CNN model. We ignore the pooling layers from the 2D-CNN method to protect the essential data of every picture element. The prediction is then built using a fully connected layer that inputs the feature maps from the previous 2D-CNN. In this case, we use the soft-max function to calculate the probability for every class. For multiple categorizations, the soft-max function extends the sigmoid function. The cross-entropy process was also chosen as the actual function that will guide the Backpropagation created training operation.

Here, M and c stand in for every variable in our 2D-CNN method. We use the following soft-max function to convert the scores g_d (I_i,j,l; (M,c)) of every class of interest dϵ{1,…,N} into the qualified possibilities as we train the 2D-CNN method by maximizing the likelihood (3):

$q\left(\left.d\right|_{i, j, l} ;(M, c)\right)=\frac{e^{g_d\left(I_{i, j, l} ;(M, c)\right)}}{\sum_{r \in\{1, \ldots, N\}} e^{g_r\left(I_{i, j, l} ;(M, c)\right)}}$     (3)

By reducing the negative log-likelihood used in the training

process, the parameters (M,c) are learned (4).

$L(M, c)=-\sum_{I_{i, j, l}} \ln q\left(I_{i, j, l} \mid I_{i, j, l} ;(M, c)\right)$       (4)

When the pixel at the position (i,j) in the image I_l has the proper class label, denoted by the notation I_i,j,l. Adam optimizer with Backpropagation, maximizes the objective function. Using the argmax process, the outcome layer of the presented approach forecasts the label of the picture element at (i,j) of a data point I at testing time (5).

$\widehat{l_{l, j}}=\arg \max q\left(\left.d\right|_{i, j, l} ;(M, c)\right) \cdot d \epsilon\{1, \ldots, N\}$     (5)

3.2 The 3D-CNN method (3D-CNN)

The previous is taken by examining the identical area with various spectral bands. Whereas the final is not, this is one of the key distinctions between a HSI and a common image. It is preferable to consider hyperspectral correlations since the data created by hyperspectral bands has some connections; for example, nearby hyperspectral bands provide the same images. The spatial context can be utilized by the 2D-CNN method, but the hyperspectral correlations are discarded. To create a 3D-CNN process to deal with this problem.

The process specifics of the 3D-CNN method are comparable to the 2D-CNN approach, as illustrated in Figure 2. The primary distinction is that the 3D-CNN process has an additional reordering phase. The C hyperspectral bands are reordered in this phase in an increasing direction. This sequential ordering of images of associated spectral bands can maintain their correlations in a spectral context. The two methods' patch extraction and label recognition phases are remarkably comparable. Rather than using a 2D convolution operator for the feature extraction stage, the 3D-CNN method is utilized. The 3D convolution operation is explicitly written in the following Eq. (6):

$\begin{aligned} & u_{i j}^{x y}=G\left(d_{i j}+\sum_s \sum_{a=0}^{H_i-1} \sum_{b=0}^{W_i-1} \sum_{k=0}^{C_i-1} \vartheta_{i j s}^{k a b} u_{(i-1) s}^{(x+a)(y+b)(z+k)}\right)\end{aligned}$          (6)

Here, ϑijskab is the point at the (a,b,k)th location of the kernel associated with the sth feature map of the prior layer, where Ci is the size of the 3D kernel together with the spectral dimension and j is the number of kernels in the ⅈth layer. Once more, activation function G is assumed to be the ReLU function.

2.png

Figure 2. 3D convolutional process of 3D-CNN method

Figure 2 depicts the 3D convolution technique. We can see that a 3D patch is subject to the 3-D convolution process in stages, such as inner to outer, top to bottom, and left to right. A convolution layer is created at every stage and positioned in the appropriate location on the feature map. As a result of this technique, a feature map is a smaller 3D cube. Similar to training a 2D-CNN method, training a 3D-CNN method involves computing the possibility of every class using the soft-max function. Moreover, by expanding the log-likelihood of the training set, we express the analysis procedure as an optimization issue. Adam optimizer with Backpropagation is additionally used for network training.

3.3 The recurrent 2D-CNN method (R-2D-CNN)

As the categorization of a picture element depends on the properties of a tiny patch adjacent to the picture element rather than the features instantly committed to the picture element, the 2D-CNN method, as previously mentioned, may generate undesirable noise even if it can take advantage of the spatial context. To create an R-2D-CNN method to utilize the spatial environment more effectively. A multiscale deep neural network is used by the R-2D-CNN method to fuse numerous shrinking patches into multiple instances, which it then uses to make predictions.

To make things clear, we refer to the instances as the first phase, the second phase, and the q^th phase, correlating after the larger patches to the minor patches, here the P^th phase frequently correlates to the picture element for categorization, i.e., a 1×1 patch. A R-CNN structure, in which a fundamental 2D-CNN block is repeatedly reused, makes up the R-2D-CNN deep neural network. It extracts the feature maps for the initial-phase instances using the essential 2D-CNN block. The similar 2D-CNN block extracts the following-phase feature maps by concatenating these feature maps with the second-phase representatives. Up until the q^th level instances are fused, this process is repeated. The probability of every class is calculated after applying a soft-max layer. We can examine the spatial context data and concentrate additional on the data closest to the picture element for categorization by using the numerous shrunk patches. As a result, the undesirable noises might decrease.

3.png

Figure 3. R-2D-CNN method contains two fundamental 2D-CNN

Figure 3 represents the foremost framework of the R-2D-CNN method. At the q^th phase, the system is sustained with the original "feature image" G^q of F +C, which consists of H feature maps for the (q-1)^th instances, C is HSI for the q^th occurrences, and 1≤ q≤ Q. F means the number of feature maps created by the 2D-CNN method. The method is described in formal terms as follows (7):

$G^q=\left[G\left(G^{q-1}, I_{i, j, l}^q\right)\right], G^1=\left[0, I_{i, j, l}\right]$         (7)

where, I_i,j,l refers to the actual patch that encloses the picture element at coordinates (i,j) on the training image l. Since there is no occurrence since a prior to build the feature maps, the network just accepts the actual image as input in the first phase. Despite having multiple phases, the R-2D-CNN method problem does not improve with the number of phases. The explanation is that the parameters for many levels are expected, as shown in Figure 3.

The gradients are generated utilizing the Backpropagation through time (BPTT) procedure throughout the Backpropagation method, just like the 2D-CNN model during model training. More specifically, we train the method using the BPTT algorithm after first unfolding the network, as depicted in Figure 3. Contrary to the 2D-CNN method, the recurrent multilevel architecture forces us to acquire the network limitations (W, b) through a novel loss function (8). The loss function is established using (7):

$I(G)+I(G \circ G)+\cdots+I\left(G \circ{ }^q G\right)$       (8)

I(G) is the log-likelihood of the 2D-CNN model established in (3), ◦q stands for the composition operation carried out q times. In order to provide the appropriate label at the location, each network instance is trained (i,j). The R-2D-CNN method can learn from its errors and fix them in subsequent iterations. The R-2D-CNN model can also categorize dependences, which involves predicting an instance's label based on the label of an earlier occurrence centered on location (i,j).

It is important to note that the sizes of the tiered sample in order for the R-2D-CNN method to be appropriately planned for the instances to be combined by the feature maps of the prior instances. To achieve this, we must initially determine how a feature map's size variations at what time is employed in a 2D convolution layer. Here rz_s-1 represents the (s-1)th convolution layer's feature map size. Then, the formula (9) below is utilized to calculate the size of the feature map generated by the mth convolution layer:

$r Z_s=\frac{r Z_{s-1}-c V_s}{t V_s}+11$    (9)

where, tVs is the stride size, and cVs is the size of the sth layer's convolution kernel. To determine the size of a feature map created by the 2D-CNN block (8). As a result, we can calculate the instances' proper sizes for various categories.

3.4 The recurrent 3D-CNN method (R-3D-CNN)

We created the R-3D-CNN method to use hyperspectral images' spatial and spectral contexts more effectively. Like the R-2D-CNN model, multilevel RNNs that gradually shrink a patch into multiple instances also support the R-3D-CNN paradigm. R-2D-CNN method and the R-3D-CNN method differ primarily in two ways. The primary distinction is that the prior uses the 3D convolution process, while the last utilizes their 2D equivalents. As a result, the R-3D-CNN method could be a recurrent addition to the 3D-CNN process. The second distinction is that feature maps produced at the current level must be concatenated with instances from the following level once they have been pre-processed. We use 3D convolution layers for this purpose, which causes the spectral bands to have variable lengths. In order to adjust to the shifting sizes, we must pre-process the instances of the following level using specific 3D spectral channel convolution techniques.

An illustration of the proposed R-3D-CNN method is shown in Figure 4. A multi-layer RNN with P-tiered instances makes up the model. With the R-2D-CNN method, the matching feature maps are extracted using a "simple" 3D convolution network and then concatenated with the instance from the following phase to develop novel feature maps at every phase. This approach is repetitive until completely tiered samples have been included. A pre-processing step is added to the spectral networks to maintain consistency between the sizes of feature maps at the present phase and the sizes of the occurrences at the following phase. The cross-entropy objective function is next applied, and a soft-max layer is lastly added.

The BPTT method is once again used in the optimization process. The R-3D-CNN approach's complexity is comparable to the R-2D-CNN approach since the recurrent structure uses the same network parameters at several stages. We must reorder the hyperspectral images in the 3D-CNN method by spectral band ordering. As with the R-2D-CNN paradigm, the size of tiered instances must also be wisely calculated.