A Hybrid Firefly with Particle Swarm Optimization Based Hyperspectral Image Classification and Segmentation Using Structured Support Vector Machine

K nearest neighbor (KNN) based spectral–spatial hyperspectral image categorization is presented [24]. This method, SVM is used for classification where the mapping of different classes is attained based on probability. Next, averaging nonlocal neighborhoods and matching is obtained for pixel-wise probability maps based KNN filtering approach. Then, the KNN approach is applied to use the original image's nonlocal principle fully, and sophisticated segmentation and optimization strategies weren’t used in this method. This method uses two datasets and is compared with many classification results. A new method for creating segmentation benchmarks is presented and employed to develop hyperspectral training-test data segments ready for usage [25]. It can be used for better validation of both new and old techniques without leaking training or test data.

Resource-efficient quantized convolutional neural networks are presented and drastically reduce their size without degrading the capacity to classify data [26]. The tests using two hyperspectral benchmarks demonstrated that the quantization procedure can be used effectively while training, producing significantly smaller yet broadly applicable deep learning methods. The watershed segmentation technique is presented to segment hyperspectral data to determine spatial structure information [27]. The segmentation maps are subsequently included in a classifier, illuminating the accuracy of the watershed algorithms. Based on pixel-wise categorization using SVM, succeeded by majority voting within watershed areas, an original spectral-spatial categorization strategy for hyperspectral data is presented. The outcomes of investigational segmentation and classification on two hyperspectral data are exposed. Investigations have shown that feature extraction and multidimensional gradients are preferable to vectorial gradients as spectral bands rise. The classification accuracy is increased, and categorization maps with additional similar areas are produced due to the inclusion of spatial data from the watershed segmentation in the hyperspectral data clustering.

A unique method for unsupervised categorization of land cover use case utilizing multi-stage thresholding using the highest entropy to better the separation between objects and background [28]. A widely used segmentation tool is multi-level thresholding, which divides a gray-level image to numerous identical areas. The computing efficiency and robustness of the presented approach are enhanced by using Differential Evolution (DE), a popular and straightforward evolution process. By contrasting DE's performance with that of other well-known, environment-global optimization strategies, the effectiveness of DE is also thoroughly examined. The findings of the MRE-based thresholding were also applied to train an SVM classifier using a combined kernel technique to increase the classification accuracy.

A multi-stage algorithm suitable for performing thematic categorization of hyperspectral satellite data using minimal training sets of data chosen from identified image sections is presented [29]. Without sensor condition data, this approach is necessary and is used relatively frequently in real-world situations. The technique of choice comprises the steps of pixel and spatially-based pre-processing, spatial post-processing of the categorization outcomes, and image categorization using spatial and spectral parameters. The trials revealed that the methodology based on merging pixel-level segmentation results from employing k-means, linked elements labeling, and nonlinear pre- and post-processing techniques produced the best outcomes.

A deep learning neural network is utilized in this study to determine techniques for satellite remote sensing data [30]. Before applying the image information to the segmentation step using a fuzzy-relevant vector machine, pre-processing is required. A Gaussian Filter approach based on Cellular Automata removes noise from satellite data. The pre-processed satellite data is next segmented utilizing the Fuzzy- Relevance vector machine Segmentation method to identify reverse shapes with the minimum energy. The recommended technique's results show an extraordinary accuracy of 98.9 percent compared to other techniques. The classification of hyperspectral images obtained from satellites is actualized in this work using a practical technique [31]. This approach is founded on enhancing SVM utilizing Self Organizing Maps (SOM). Then, inside and outside Picture element can be classified by comparing the Posterior Probability of every pixel intensity. This approach contains two steps, the initial step is to optimize the SVM utilizing self-organizing maps to extract the important characteristics from the image. The next step is determining the inside and outside pixels and associating the optimal threshold and probability. The presented methodology better than the traditional methods based on parameters such as, kappa, confusion matrix and accuracy.

To classify land cover, researchers provide a unique framework that accounts for the spectral and spatial image included in the data [32]. Employ the Gaussian Maximum Likelihood and CNN models for the pixel-by-pixel spectral categorization. Next, utilizing segmentation maps produced by the Watershed algorithm, integrate the spatial context data into this paper to use an altered maximum vote approach. The investigational evaluations on two standard datasets, the Pavia University and the Indian Pines datasets, show that recommended technique outperforms the existing techniques by reaching an accuracy of 99.52% and 98.31% on the corresponding datasets. A brand-new hyperspectral band selection method combines novel attention-based CNNs to weigh the bands [33]. The recommended attention-based method is data-driven, utilizes convolutional activations at various depths of a deep model, and identifies the spectrum's maximum valuable parts. Moreover, it can be trained using gradient descent from beginning to end and is modular, simple to use, and seamlessly transferable to any convolutional network. This approach testing, conducted across benchmark datasets and supported by statistical analyses, demonstrated that the deep systems with the consideration process are good with the traditional band selection methods.

The lightweight CNN method [34] considerably decreases computational costs by spreading spatial-spectral feature extraction over a lighter framework with pre-processing to enhance the classification performance. Five Hyperspectral benchmark datasets are employed for investigational estimation. The investigational outcomes demonstrate that the presented pipeline performed better than the traditional approaches regarding statistical significance, and computational complexity.

The proposed approach contains of two stages of hyperspectral image processing, including categorization and segmentation. The binary OTSU threshold approach is combined with the SSVM algorithm for classification. The Hybrid Firefly with Particle Swarm Optimization algorithm (Hybrid FF with PSO) uses the weighted function analysis. The image pixels are iteratively optimized using the modified optimization technique. We can increase the swarming variety based on the fitness value. The image's noise is reduced using the weighted least squares filter.

The workflow for the proposed approach, which includes the two primary methods of image classification and image segmentation, is shown in Figure 1. The SSVM technique is utilized to classify the images in this case. A HSI image dataset captured by an AVIRIS sensor makes up the input dataset. With its actual values, the Indian Pines dataset is applied for imaging techniques. This dataset is imported to obtain an image that has been transformed into binary form. With this binary image, threshold classification is performed. The weighted least squares (WLS) filter is applied to classify the image using the OTSU threshold approach. Now, we can determine the fitness value for each epoch using a Hybrid FF with PSO approach, which will decrease the computing cost. The segmentation of images uses the threshold method. Finding the ideal correlated value for image classification through principal component evaluation lowers the dimensionality and error rate of the images. It is applied to extract a limited collection of uncorrelated variables from a massive dataset of Indian pine.

1.png

Figure 1. Proposed approach workflow

The hyperspectral satellite images are pre-processed using the median filter to increase image quality and make categorization more efficient. The input data is processed into a gray-level image, which serves as the matrix value for the image during the feature extraction process using GLCM. These extracted images contain pixel values that have been improved classification results using a Hybrid FF with PSO approach. The data is segmented in this case utilizing the binary categorization process by the OTSU's threshold model. The SSVM with different image classes is used for the classification procedure. The performance evaluation of proposed approach is obtained based on sensitivity, specificity, accuracy, recall, precision and F1-score once we have the train and test datasets from the SSVM. The proposed approach is compared to traditional approaches is enhanced the system performance.

2.1 Pre-processing using median filter

The median filter replaces the window's center value with the median of all the pixels, making it a sliding window spatial filter. Although this filter reduces noise, it loses fine details. Because median filters may effectively reduce many types of noise with less blurring, research frequently utilizes them. In addition to the signal, time series, and image processing, median filters are frequently employed as smoothers. The ability of the median filter to eradicate the effect of input noise with huge magnitudes is a significant advantage over linear filters. A non-linear filtering method that helps eliminate noise; the median filter has advantages over the mean filter. It can eliminate "impulse" noise (outlying values either high or low). Since step edges should be preserved without blurring, it was also frequently called "edge-preserving.” Even though, it does slightly blur image edges when noise is present. The formula for the common median filter is Eq. (1), where A and B are the input and output, respectively, at position n of the filter. The {v_n}_ss=1, ……, 2N+1, s^th order statistic of the datasets inside the window {v_n}₁<{v_n}₂<{v_n}₃<……. {v_n}2N+1. The filters v_n is window size for this job, however, is 3×3. This size was chosen since rise in mask size will outcome in an increase in RMSE (Root Mean Square Error).

$B_n=\operatorname{med}\left\{V_n\right\}=\operatorname{med}\left\{A_n+\mathrm{s}: \mathrm{s} \in \mathrm{V}\right\}$               (1)

2.2 GLCM feature extraction

Texture analysis has many uses, from texture categorization in remote sensing to biometric concerns to pattern identification in image retrieval. It is necessary to extract essential features that accomplish the texture features, which can be retrieved using GLCM to assist in extracting those features for each image processing procedure. GLCM, which has grown most common, is a statistical method for eliminating textural details from images. Hara lick defines fourteen textural features, measured from the probability matrix by the co-occurrence matrix, to extract the texture statistics of remote sensing images. The GLCM shows how frequently various sets of greyscale pixels appear in an image. It is produced for a particular pair of distance and angle. In Figure 2, relative recurrences are calculated for a given space and angle between every pixel and its neighbor.

This study recovers four significant characteristics: the Angular Second Moment, Inertia Moment, Correlation, Entropy, and Inverse Difference Moment.

2.png

Figure 2. Pixel interest and it surrounding region for angle relative

2.2.1 Angular second moment

Another name for an Angular Second Moment (ASM) is uniformity or energy. It is the squared sum of all GLCM entries. The graphic similarity is measured by the ASM. When an image has excellent homogeneity or its pixels are substantially similar, the angular second moment is high. Eq. (2) is Gray tone is represented by k_g, where m and n are the spatial coordinates of the function p (m, n).

$\mathrm{ASM}=\sum_{m=0}^{k_{g-1}} \sum_{n=0}^{k_{g-1}} Q_{m n}^2$            (2)

2.2.2 Inverse difference moment

Local similarity is defined as the inverse difference moment (IDM). This value is high when both the inverse GLCM and the local grey level are high. The value of the IDM weight is the opposite of the Contrast weight shows in Eq. (3).

$\operatorname{IDM}=\frac{\sum_{m=0}^{k g-1} \sum_{n=0}^{k_g-1} Q_{m n}}{1+(n-m)^2}$            (3)

2.2.3 Entropy

Entropy displays the amount of image information required for image compression. Entropy evaluates the image information and the data or message loss in a sent signal.

Entropy $=\sum_{m=0}^{k_{g-1}} \sum_{n=0}^{k_{g-1}} Q_{m n}{ }^* \log Q_{m n}$            (4)

2.2.4 Correlation

The linear association between the grey levels of nearby pixels is computed based on correlation. Image correlation is a visual approach that uses tracing and image registration approaches to quantify image modifies in 2D and 3D effectively. Widely utilized in many fields of science and engineering and is frequently applied to detect deformation, displacement, strain, and optical flow. Computing the motion of an optical mouse is one prevalent application.

Correlation $=\frac{\sum_{m=0}^{k_{g-1}} \sum_{n=0}^{k_{g-1}} Q(m, n)-\mu_x \mu_y}{\sigma_x \sigma_y}$            (5)

2.3 Hybrid FF with PSO approach

The brighter firefly always draws the less bright one in iterations of the firefly algorithm. This algorithm performs an excellent job of findings the answer in the random movement of the Neural Computing and Applications 123 location. Still, it ignores the firefly's ideal position and impacts how the problem is solved on a larger scale. The movement of fireflies in Firefly Algorithm aims to explore the feature space efficiently, leading to the identification of relevant features for classification tasks. Mathematically, the movement of firefly i towards firefly j at iteration t+1 can be expressed as:

$a_i(t+1)=a_i(t)+\beta e^{-\gamma r_{i j}^2}\left(a_j(t)-a_i(t)\right)+\alpha \varepsilon(t)$            (6)

where, a_i(t) is the position of firefly i at iteration t; r_ij is the distance between fireflies i and j; β is the attractiveness coefficient; γ is a constant; αε(t) is a randomization term.

3.png

Figure 3. Hybrid FF with PSO algorithm flowchart

The drawback of specific PSO and FF algorithms are mitigated by hybrid approach of FF with PSO. In the global search space, the PSO algorithm discovers better solutions. The Ibest and Tbest numbers produced via PSO are utilized by every firefly. This method strikes a balance between exploitation and exploration to improve search results. Every firefly's movement improves solution of accuracy and boosts convergence. Figure 3 shows the working process of hybrid FF with PSO optimization algorithm. The hybrid algorithm's position vector is provided by:

$\begin{gathered}z_i^{l+1}=z_i^l+\mu_o e^{-\lambda r_{I z}^2}\left(\text { Ibest }_i-z_i\right) \mu_o e^{-\lambda r_{I z}^2}\left(\text { Tbest }_i\right)+\tau(\text { rand-0.5 })\end{gathered}$            (7)

where, Ibest is denote the Individual best position, Tbest is denotes the Total best position, µ is denotes Initial attractiveness, τ is denotes Randomized parameter, λ is denotes Light absorption coefficient.

The distance between z_i and Ibest is calculated using Euclidean distance. This can be described as:

$r_{I z}=\sqrt{\sum_{l=1}^m\left(b e s t_{i j}-z_{i j}\right)^2}$             (8)

where, m is Number of populations.

The distance between z_i and Tbest is computed using Euclidean distance. This can be described as:

$r_{T z}=\sqrt{\sum_{l=1}^m\left(\text { Tbest }_{i j}-z_{i j}\right)^2}$             (9)

When an FF can't find an FF that is brighter than it, it randomly shifts positions in order to reach the global optimum.

$z_i^{q+1}=z_i^q+\tau \varepsilon_i$             (10)

where, q is denotes the Number of iterations.

2.4 Structured support vector machine is a machine learning (SSVM)

A Structured Support Vector Machine (SSVM) is a machine learning algorithm that extends the traditional support vector machine to handle structured output spaces. This is an approach that uses an SVM. for categorization of the HSI image is performed using the SSVM. The efficiency of the SSVM is enhanced through the optimization procedure. Regression, multiclass classification, and binary classification are the three basic concepts that employ this. Class labels are used in the train and test functions. After the network has processed test datasets, the datasets sent predicted using the training process. Here, the sources images are matched into respective labels using structured responses to produce complicated outcomes. The SSVM practices the kernel function for data transformations to find optimal results. The SSVM categorization logic is provided below:

$d^{\prime}(c ; m)=\operatorname{sign}(c, m)$             (11)

$d^{\prime}(c ; m)=\operatorname{sign}(\rho(c, m))$             (12)

where, m is the weighted vector, c is an input array, ρ(c) is the function of feature map, and d is output array. The regression equation given below:

$H(Z) \sim(m,(\rho(c))$             (13)

In SSVM method, the relationship between input and output is mentioned below equation.

$d^{\prime}(c ; m)=\operatorname{argmax}\{m, \alpha(c, d)\} d \in D$             (14)

Equation for regularization given below:

$F(M)=\frac{\lambda}{2}\|m\| 2+1 / s \sum_{i=1}^5 \Delta\left(d_i \cdot d^{\prime}\left(c_i ; m\right)\right)$             (15)

where, ∆ is loss function, c_i is actual value of input and d_i is output. The pattern verification procedure attains the lesser space area in linear classification. The non-linear classification method is used for achieving a high dimensional area. Random array regularisation with high-dimensional data uses the Kernel function in this case. A process called "G" is used to denote the higher feature space, and it is provided in the equation:

$\mathrm{G}: \mathrm{c} \rightarrow \varphi(\mathrm{c}) \epsilon$             (16)

2.5 OTSU’s binary threshold approach

The structured output space defines the cost function during SSVM training. The testing procedure is processed using train datasets, which optimizes the outcome by choosing greater than zero values. The post-processing phase offers the categorized HSI image without an error signal.

Otsu's approach is a binary-valued image threshold technique. With this technique, grayscale images are transformed into binary images. The binary threshold method minimizes the intra-class variance, the weighted sum of variance with various classes.

$\omega_m^2(\mathrm{~s})=b_o(\mathrm{~s}) \omega_o^2(\mathrm{~s})+b_1(\mathrm{~s}) \omega_1^2(\mathrm{~s})$             (17)

where, b₀ and b₁ is probability of each classes value, s is threshold factor, and ω₀² and ω₁² is variance of each classes value. The equations listed below estimate the histogram of these two classes.

$b_o(\mathrm{~s})=\sum_{i=0}^{s-1} r(i)$             (18)

$b_1(\mathrm{~s})=\sum_{i=s}^{s-1} r(i)$             (19)

Otsu’s measurement procedure is implemented systematically with its respective classes. The algorithm first determines the variance's maximum and minimum values, determined by a threshold value, and the histogram's and every particle's probability. The intraclass interval is constrained to [-1, 1]. We can run an image threshold method based on the maximum variance of the classes.