Improving Classification Performance of Medicinal Plants with StyleGAN2-ADA

Iraq's rich medicinal plant diversity, deeply rooted in Sumerian, Babylonian, and Assyrian traditions, presents classification challenges due to undocumented species, morphological similarities, and environmental variations. For instance, the diverse landscapes of Sulaymaniyah support a unique traditional healthcare system that relies heavily on oral knowledge. However, habitat destruction, overharvesting, and a lack of scientific validation threaten these valuable resources [1].

Medicinal plants are the plants or parts of plants that are used for therapeutic and nutritional purposes and from the beginning of human existence, plant materials with medicinal qualities have been used extensively to cure human illnesses [2]. Conserving biodiversity is essential due to the extinction risk encountered by several plant species, and conventional medicinal systems extensively depend on a wide variety of plants, providing an alternative to synthetic pharmaceuticals and fostering wellness. Notwithstanding the importance of these plants, records for medicinal herbs are not easily accessible [3]. Given their vital role in healthcare and biodiversity, medicinal plants have been the focus of extensive research. However, classifying and identifying these species remains a complex and time-consuming task, even for experienced botanists. This study aims to document medicinal plants to support conservation efforts and facilitate future phytochemical research.

Consequently, a vision-based method like neural networks may assist scientists and the general populace in identifying plant species with higher speed and precision [4]. Especially since neural networks have shown significant progress in multiple fields. They have been used for object detection [5], design and implementation of monitoring robotic systems [6], malware detectors [7], predictive maintenance [8], and many applications in the medical field, including diagnosing diseases like COVID-19 [9]. However, deep learning models are data-intensive, requiring significant amounts of training data for optimal efficacy, and a limited dataset leads to overfitting. To address this issue, various approaches like dropout, data augmentation, and semi-supervised learning were implemented [10]. On the other hand, amassing a substantial training dataset requires significant time and effort, especially since the collection of plant samples is tough due to the changing environmental conditions and the extensive diversity of plants. Samples must be obtained under diverse weather circumstances and growth stages, encompassing differences in shape, size, and color [11].

Recently, Generative Adversarial Networks (GAN) developed by Ian Goodfellow in 2014 [12], are progressively utilized for the production of image data. Research demonstrates that GAN has superior capabilities for producing picture data for training compared to conventional data augmentation techniques in image processing [13]. Generative Adversarial Networks (GANs) are a form of deep learning models including two main components: the generating network and the discriminative network. The primary neural network is the generator, which creates novel synthetic data instances that mimic the training data. The discriminator seeks to distinguish between real data and fake data generated by the generator network [14].

Collecting a sufficiently large dataset of images for a certain application is tough due to constraints related to subject type, quality of picture, geographical position, period of time, privacy, copyright status, and other factors. The challenges are intensified in applications involving the collecting of a new, distinct dataset: obtaining, processing, and distributing the images needed to train an effective, high-quality, high-resolution GAN [12] is a cost-intensive operation. This restricts the increasing utilization of generative models in domains such as medicine. Since a significant amount of real training data is still required to train GAN models for generating samples with appropriate quality, in 2020, developers at NVIDIA released StyleGAN2-ADA [15], which significantly advances the effective training of GANs with limited data. This was achieved by the use of an Adaptive Discriminator Augmentation (ADA) mechanism, which stabilizes training in scenarios that have limited data, hence minimizing common overfitting concerns associated with the discriminator [16].

This study employed the StyleGAN2-ADA model to generate more samples for the ten medicinal plants dataset. Then compare the results of classification using six transfer learning models when using the conventional augmentation techniques versus StyleGAN2-ADA. Figure 1 shows the real data with the data generated by StyleGAN2-ADA of ten medicinal plant species.

1.png

Figure 1. The results of StyleGAN2-ADA on the Iraqi Medicinal Plant Dataset, (a) Real images, (b) StyleGAN2-ADA

This study made the following significance contributions:

1. To the best of the authors' knowledge, this is the first kind of Iraqi medicinal dataset created by the authors, where a total of 438 images are divided into 10 classes of medicinal plant images.
2. Successfully improve the accuracy of classification of limited data using StyleGAN2-ADA.
3. Comparison of performance of trained images of six models.
4. Adjusting the hyperparameters of classification models.

The rest of the study is organized as follows: Section 2 is composed of the most recent related work in the field. Section 3 explains the methodology in detail. Section 4 presents all experiments and results. Section 5 concludes the study.

The main objective of this paper is to improve the performance of medicinal plants identification especially on limited or unbalanced dataset. To achieve this goal multiple stages have been done as shown in the framework in Figure 2.

2.png

Figure 2. Overview of the proposed methodology

Starting with the stage of forming a dataset for medicinal plants in Iraq by taking pictures of medicinal plants. Since plants have specific growth stages and seasons in different regions of Iraq, between north and south, east, and west, it takes a long time and effort to obtain a sufficient number of pictures. Therefore, additional pictures were collected from reliable flora sites. After that verify the collected images by a botanist to make sure that all the images are belonging to the right class and labelled with right scientific name. Then resizing all the images into 256 × 256 pixels and change their format to jpg. Next comes the most important stage that can improve the accuracy of classifier and solve overfitting problem, which is the augmentation. Two types of augmentation were applied, the conventional augmentation which includes a python code do rotation, translation and zoom processing and the other is a GAN-based augmentation by StyleGAN2-ADA model. After training StyleGAN2-ADA model on the dataset, it generates good quality images depending on the size of the training dataset. Both types of augmentation results will be classified by pretrained deep learning models.

3.1 Dataset description

This dataset is a worldwide attempt to document and study medicinal plants that support traditional medicinal systems internationally. A detailed description is provided in Table 2. Given the widespread distribution of medicinal plants across various provinces in Iraq some of which are difficult to access the ten species with the highest number of available images were selected to enhance accessibility and dataset quality. The collection has 438 photos across 10 species of medicinal plants: Hibiscus rosa-sinensis, Brassica oleracea var. italica, Ricinus communis, Lactuca sativa, Datura innoxia, Capparis spinosa, Hibiscus sabdariffa, Senna sulfurea, Matricaria chamomilla, and Silybum marianum. The majority of these images were independently obtained using an iPhone 13 camera at the Research Unit for Palms Lab, College of Agricultural Engineering Sciences, University of Baghdad. Meanwhile, the remaining data is gathered from botanical image repositories such as Plant of The World, all were scaled at 256 × 256 pixels.

Table 2. The Iraqi Medicinal Plant Dataset samples

The Scientific Name	No. of Samples	Sample Image
Hibiscus rosa-sinensis	59	a.jpg
Brassica oleracea var. italica	57	b.jpg
Ricinus communis	56	c.jpg
Lactuca sativa	55	d.jpg
Datura innoxia	42	e.jpg
Capparis spinosa	37	f.jpg
Hibiscus sabdariffa	35	g.jpg
Senna sulfurea	34	h.jpg
Matricaria chamomilla	32	i.jpg
Silybum marianum	31	j.jpg
Total	438

3.2 Proposed model

The proposed model includes two main deep learning components. The first component is GAN, which is used for the preprocessing phase, while the deep transfer model is used in the training, validation, and testing phases. The proposed GAN/deep transfer learning framework enhances image data preprocessing and augmentation using GANs, which boosts the dataset's quality and diversity. This data is fed into a CNN model for training, with performance assessed through validation and testing phases. The approach ensures robust model training and evaluation, improving generalizability and accuracy in image-based tasks.

3.2.1 Training StyleGAN2-ADA

The StyleGAN2-ADA algorithm is an enhanced version of StyleGAN2, with ADA signifying Adaptive Discriminator Augmentation. The primary objective is to address the issue of potential overfitting of the discriminator in StyleGAN2 during training whereas the training dataset is limited [21]. This may prevent the generator from successfully adjusting the model parameters, resulting in reduced training efficiency.

The up-sampling network in Figure 3 comprises a latent input $z$, often a style picture, which is normalized, encoded, and non-linearly transformed into a latent space $w$ using a multi-layer perceptron (MLP) with 8 fully linked layers. The generator network comprises a constant input vector with a mean of zero and a standard deviation of one, which is employed in the first layer and acts as the input for the first Adaptive Instance Normalization (AdaIN) operation. The styles from the encoded latent space $w$ are applied via the AdaIN operation at the corresponding resolutions of the generator network. The output of the corresponding convolutional layers is dictated by the style imparted by the AdaIN operation. Gaussian-distributed noise is introduced per pixel following the corresponding convolutional layers to effectively mitigate distortions that may manifest in the picture, like water droplets in areas such as wrinkles and hair on human faces. This method successfully incorporates stochastic variance into the produced pictures. While the discriminator is a down sampling network shown below in Figure 4, that aligns the picture dimensions with the final output of the generating network. The picture is inputted into the discriminator network and subsequently enhanced with prescribed augmentation transformations. Additionally, these input pictures are down sampled to the appropriate resolutions of the generator network utilizing residual or skip connections. The discriminator network performs binary classification, differentiating between authentic and counterfeit pictures [22].

3.png

Figure 3. StyleGAN2-ADA generator network architecture [21]

4.png

Figure 4. StyleGAN2-ADA discriminator network architecture [21]

Training StyleGAN2_ADA on Iraq Medicinal Plant Dataset duration was 18 hours, 11 minutes, and 27 seconds on a Google Colaboratory Pro Plus equipped with 52 GB of RAM and a P100 GPU. The dataset produced the model (.pkl) file following a designated training period. The (.pkl) file will be created every 100 epochs. Upon achieving training results of 256×256 pixels, the training process terminated and utilized the final (.pkl) file, after saving it. The initial training iteration seemed irregular as they shifted from the original photos to a new set. Subsequently, the maximum number of iterations significantly differed from the initial iterations, and then images were generated.

Even though the Iraq medicinal plants dataset is challenging because it is limited and unbalanced, the images of the ten classes dataset increased nearly two times to 856 medicinal plant images. The classes with the highest number of images get better quality images. Table 3 displays StyleGAN2-ADA parameters, where the model was trained with images at a resolution of 256×256, using a batch size of 16 and a learning rate of 0.0025. The training progressed over 416 ticks, corresponding to 1,664 kimg (1,664,000 images processed). These settings balance model stability and computational efficiency, allowing the generator and discriminator to learn effectively at this resolution.

Table 3. Parameters for StyleGAN2-ADA

3.2.2 Classifiers

In the second component of the methodology, this study employs the same transfer learning models that were utilized in previous study [23]. Selected deep learning models differ in architecture, efficiency, and feature extraction capabilities, making them suitable for various aspects of medicinal plant classification. Bit_Sr50x1 leverages strong transfer learning from large datasets, ensuring good generalization with minimal fine-tuning. ResNet_V1_152 and Inception_ResNet_V2 offer deep feature extraction and improved learning through residual connections, making them ideal for fine-grained plant classification. Inception_V3 balances accuracy and computational efficiency by capturing multi-scale features, while MobileNetV2_130_224 is optimized for real-time classification on resource-limited devices. Finally, Nasnet Large, designed via Neural Architecture Search, dynamically optimizes performance, making it well-suited for complex plant datasets. This diverse selection enables a comprehensive evaluation of plant classification under different constraints and requirements.

(1) Bit_Sr50x1

A deep learning model that has been pre-trained [24]. The acronym "Bit_s" denotes the Big Transfer family-s, while "r50x1" implies ResNet-50x1. This implies that the architecture is a ResNet-50 that has been pretrained on ImageNet-1k. Bit_s-r50x1 is recognized for its outstanding performance across a wide range of image classifying task, which is a direct result of its exhaustive pre-training on large datasets. It was selected for its capacity to generalize effectively to new tasks with minimal fine-tuning.

(2) Resnet_V1_152

A deep convolutional neural network design was developed known as ResNet_v1_152, which is an abbreviation for Residual Network version 1 with 152 layers [25]. It is a member of the Res Net family, which was created to address the issue of very deep neural network degradation by implementing residual learning. Known for its exceptional performance in deep learning tasks, it was attributed to its residual connections, which effectively mitigated the vanishing gradient problem.

(3) Inception_V3

Inception-v3 is a convolutional neural network (CNN) architecture that belongs to the Inception family. Inception-v3 [26] has been widely employed in several computer vision applications, including object recognition, picture segmentation, and image classification. It has exhibited superior performance on benchmarks such as the ImageNet Large Scale Visual Recognition Challenge (ILSVRC).

(4) MobileNetV2_130_224

MobileNetV2 architecture has a special variant: 130_224. It has been developed for efficiency in picture classification [27]. "130" in MobileNetV2_130_224 is the width multiplier, which regulates the number of channels in each network layer, and "224" represents the input resolution. It is effective and lightweight in resource-constrained environments, such as IoT, mobile, and edge computing platforms. Depth wise separable convolutions, inverted residual blocks, and skip connections make the MobileNetV2_130_224 architecture efficient and accurate. The lightweight architecture was designed to evaluate plant recognition models on resource-limited devices.

(5) Inception_Resenet_V2

The Inception-ResNet-v2 deep convolutional neural network architecture combines the advantages of the Inception module with residual connections. This advanced deep neural network reaches superior performance across multiple computer vision tasks, such as object detection, image segmentation, and image classification [28]. Inception-ResNet-v2 shares the same architecture as Inception-ResNet-v1 but varies in its foundational elements.

(6) Nasnet_Large

NASNet-A is a convolutional neural network architecture designed for image classification. The configuration of its convolutional layers has been determined by Neural Architecture Search (NAS). NASNets, as revealed via researches [29, 30], are available in multiple sizes, including Nasnet_large of NASNet-A for ImageNet, which initiates with 168 convolutional filters and utilizes 18 Normal Cells, Nasnet_large is an architecture developed by Neural Architecture Search (NAS), which enhances the network configuration for improved performance. It was incorporated to assess whether this automated design approach might surpass hard designed models.

The classifiers were trained with a batch size of 16, a learning rate of 0.001, and a validation split of 20% over 10 epochs, with fine-tuning enabled. The Categorical Crossentropy loss function, incorporating label smoothing (0.1), and the SGD optimizer with a momentum of 0.9 ensured stable and effective model optimization. This consistent setup allowed for a robust comparison across models under both conventional and StyleGAN2-ADA-augmented datasets.

In this study, we introduced an innovative approach to address the challenge of limited datasets in medicinal plant classification by utilizing StyleGAN2-ADA for synthetic data augmentation. The results highlight the effectiveness of augmenting the dataset with synthetic images, leading to significant improvements in the performance and generalization capabilities of several state-of-the-art transfer learning models. Notably, the Bit_Sr50x1 model achieved an impressive 96.47 F1-score, with models like ResNet_V1_152 and Inception_V3 also demonstrating substantial gains over traditional augmentation methods. This approach not only mitigates overfitting but also strengthens model robustness, showcasing StyleGAN2-ADA's potential as a valuable tool for improving plant classification with limited data.

The novelty of this work lies in its application of advanced synthetic augmentation techniques to a domain with historically limited image datasets, providing a meaningful step toward more accurate and efficient automated plant recognition systems.

Future research could expand on this approach by exploring the extension of StyleGAN2-ADA to larger, more complex datasets, including a wider range of plant species. Further investigations into the combination of synthetic data generation with other deep learning techniques may open new possibilities for enhanced plant classification in diverse environmental and geographical contexts.