A Lightweight Convolutional Neural Network-Based Method for Cotton Mosaic Disease Identification

Cotton is a critical resource for the national economy and people's way of life. The total area planted with cotton was 2.952 million hectares as of May 2022, according to China's National Cotton Market Monitoring System. Illnesses that drastically reduce cotton yield, throughout its development cycle, cotton is susceptible to a number of diseases, which can cause losses of 10-20% annually, up to 30-50% in severe years, and even no harvest in some regions. Although between 80% and 90% of cotton illnesses may be detected by looking at the leaves. However, the signs of leaf diseases can vary depending on the stage of plant development and can affect both the front and rear of the leaf [1]. These types of diseases frequently call for specialized knowledge and tools, which cotton farmers frequently lack. To lessen output loss, enhance cotton quality, and ensure the safety of the national cotton industry, it is crucial to accurately identify the different types of cotton diseases and implement the necessary control measures in a timely manner.

In recent years, with the development of data technology and artificial intelligence technology, Convolutional Neural Network (CNN) based image classification [2], project detection [3] and other technologies have developed rapidly. Moreover, with the development of computer hardware and the increase of computing power, neural network models of deep learning are slowly moving from theory to application [4]. CNN is increasingly used to identify agricultural diseases due to its efficiency in processing large amounts of data and capacity to learn its data properties. Mohanty et al. [5] used the deep convolution neural network architecture Alexnet [6] and Googlenet [7], trained within the PlantVillage data set of 54,306 images containing 38 classes of 14 crop species and 26 diseases (or absence thereof), The trained model achieves an accuracy of 99.35% on a held-out test set. Rahman et al. [8] Since large scale architectures are not suitable for mobile devices, a two-stage small CNN architecture has been proposed, and compared with the state-of-the-art memory efficient CNN architectures, the proposed architecture can achieve the desired accuracy of 93.3% with a significantly reduced model size. The existing methods on tea leaf blight severity estimation are relatively few, Hu et al. [9] proposes an estimation method combines semantic segmentation with a fully connected CRF model to segment tea leaf blight leaves and reduce the effect of complex backgrounds, experimental results show that the proposed method has higher estimation accuracy and stronger robustness against occluded and damaged tea leaf blight leaves .In order to realize the rapid and accurate identification of apple leaf disease, Li et al. [10] proposed a new lightweight convolutional neural network RegNet. A series of comparative experiments had been conducted based on 2141 images of 5 apple leaf diseases (rust, scab, ring rot, panonychus ulmi, and healthy leaves) in the field environment. Although intelligent identification technology is frequently employed in the disease recognition of various crops, there are very few studies on the recognition of cotton leaf diseases. There are barely a few references available, Latif et al. [1] used ResNet101 along with transfer learning to recognize three major diseases in cotton leaves, and genetic algorithm is applied to the combined matrix to select the best points for final recognition, the highest achieved accuracy was 98.8% using Cubic SVM. Caldeira et al. [11] use of deep learning models GoogleNet and Resnet50 [12] to identify lesions on cotton leaves on the basis of images of the crop in the field, a precision of 86.6% and 89.2%, respectively, was obtained. Much higher than traditional approaches for the processing of images. To achieve the classification of cotton leaf spots by small sample learning, Based on the main features of the classical convolutional neural network classifier to classify the disease spots, Liang [13] constructed the structure and framework of S-DenseNet convolutional neural network, and the classification accuracy of this method is 7.7% higher than DenseNet [14] on average for different number of steps.

Above research is to achieve accurate identification and classification of cotton diseases by building CNN models of various types and structures. They provide a solution for the application of CNN to cotton disease identification, but there are still some unresolved issues. Firstly, the models suggested by the aforementioned research require a lot of computer power and expensive hardware to run because they are very complicated. Second, the model proposed in the above experiment was trained on a dataset with a single background and prominent disease features, ignoring the complex background that exists in a real natural environment. So, in this paper, we present the CSGNet model, which has a greater recognition accuracy and a faster convergence rate. ShuffleNet V2 [15] is the primary framework used in this study, by using cotton disease images with a natural background as a training data set, and adding attention mechanisms, modify the activation function, and modify the optimization algorithm in order to increase the effectiveness and precision of cotton disease identification in the natural environment. While having fewer parameters, it may nonetheless function effectively in complicated backgrounds.

3.1 The base networks

Beijing Megvii Co., Ltd. has developed ShuffleNet V2, a more effective convolution model structure, for low-power mobile devices. It significantly reduces the computational complexity of the model while maintaining high recognition accuracy by using pointwise group evolution and channel shuffle operations. Its advantages over the conventional architecture, which runs each channel's feature map directly, are light weight, high precision, and ease of deployment.

In this work, ShuffleNet V2 1× architecture is used as the model skeleton to make it easier to apply the illness identification model in the mobile terminal. The structural parameters are shown in Table 2. However, the data set of diseases affecting cotton leaves utilized in this experiment was collected in the natural setting, which is characterized by a variety of backdrops, covered disease spots, and varying sizes of disease spots. ShuffleNet V2 frequently encounters issues while detecting these data sets, such as fragmented regions of interest and a single feature extraction scale. This research has improved ShuffleNet V2 to address these issues and produce a network model with greater recognition accuracy.

Table 2. Architecture of ShuffleNet v2 1×

3.2 Shuffle attention

At present, attention mechanism has become an important part of improving the performance of deep neural networks. So that the neural network can accurately focus on all the relevant elements of the input, and focus the attention of network model to identify diseases, while the influence of irrelevant background is reduced. Shuffle Attention (SA) [16] module, which adopts Shuffle Units to combine two types of attention, spatial attention and channel attention, The simplest single-layer transformation is used in the SA channel attention, and the formula is as follows:

$s=\mathcal{F}_{g p}\left(X_{k 1}\right)=\frac{1}{H \times W} \sum_{i=1}^H \sum_{j=1}^W X_{k 1}(i, j)$          (1)

Furthermore, a compact feature is created to enable guidance for precise and adaptive selection. This is achieved by a simple gating mechanism with sigmoid activation. Then, the final output of channel attention can be obtained by

$X_{\mathrm{k} 1}^{\prime}=\sigma\left(\mathcal{F}_{\mathrm{c}}(\mathrm{s})\right) \cdot \mathrm{X}_{\mathrm{k} 1}=\sigma\left(\mathrm{W}_1 \mathrm{~s}+\mathrm{b}_1\right) \cdot \mathrm{X}_{\mathrm{k} 1}$           (2)

where, $W_1 \in \mathbb{R}^{C / 2 G \times 1 \times 1}$ and $b_1 \in \mathbb{R}^{C / 2 G \times 1 \times 1}$   are parameters used to scale and shift s, At the same time, the idea of part spatial attention is also very simple, First, SA use Group Norm (GN) over $X_{\mathrm{k} 2}$ to obtain spatial-wise statistics. Then, $\mathcal{F}_{\mathrm{c}}(\cdot)$ is adopted to enhance the representation, The final output of spatial attention is obtained by, where only $\mathrm{W}_2$ and $\mathrm{b}_2$ are parameters.

$X_{k 2}^{\prime}=\sigma\left(W_2 \cdot G N\left(X_{k 2}\right)+b_2\right) \cdot X_{k 2}$          (3)

Because of these simple computations and the associated channel shuffle operations, which ensure the interaction between each set of sub-features, the SA module becomes a reasonably small module and avoids computational costs. To create the CSGNet unit, Unit, we combine the SA module with the ShuffleNet V2 unit. The two various units are depicted in Figure 2. This lightweight unit makes it possible for the model to more accurately detect the disease region in the image without growing the model's size or the number of calculations required.

2.png

Figure 2. ShuffleNet V2 Unit and CSGNet Unit

3.3 Gaussian error liner units

The activation function used in the original model of ShuffleNet V2 is ReLU, ReLU is a piecewise linear function that changes all negative values to 0, while the positive ones remain unchanged, such unilateral inhibition makes the neurons in the neural network also have sparse activation, which makes the model better able to mine relevant features and fit training data. However, when the input is negative, the learning speed of the ReLU activation function may become very slow, or even make the neuron directly invalid, because the input is negative and the gradient is 0, so that its weight cannot be updated.

3.png

Figure 3. ReLU and GELU

The proposed model, which weighs the input by its own value, GELU [17], replaces ReLU with Gaussian Error Linear Units in order to eliminate this issue. In contrast to GELU, which has a specific probability that the output is not 0, ReLU has an output of 0 for values less than 0, as seen in Figure 3. This makes GELU possible to avoid the gradient disappearance problem while avoiding the aforementioned difficulties, which enhances the robustness of the model training process instead of gating the input by its symbol like in ReLU. GELU can be approximated by:

$\operatorname{GELU}(x)=0.5 \times \left(+\tanh \left(\sqrt{\frac{2}{\pi}}\left(x+0.044715 x^3\right)\right)\right)$          (4)

3.4 Adaptive momentum

The Stochastic Gradient Descent (SGD) was used by the original ShuffleNet, but because each parameter in SGD maintains the same learning rate, it is challenging to choose an appropriate learning rate and SGD is simple to converge to the local optimum during the training process, the average recognition accuracy of pests in the model is not optimal.

This study employs Adam to address the aforementioned issues. Adam not only enhances the network model's generalization performance but also qualifies it for situations with high noise or sparse gradients and improves the model's ability to recognize objects in complicated surroundings. Adam also satisfies the criteria for the lightweight model in this study because of his efficient calculation and low memory needs. This study employs Adam to address the aforementioned issues. Adam not only enhances the network model's generalization performance but also qualifies it for situations with high noise or sparse gradients and improves the model's ability to recognize objects in complicated surroundings. Adam also satisfies the criteria for the lightweight model in this study because of his efficient calculation and low memory needs.

Adam (Adaptive momentum) is a stochastic optimization method based on adaptive momentum, which combines AdaGrad (Adaptive Gradient), Reserve a learning rate for each parameter to improve performance on sparse gradients (i.e., natural language and computer vision problems), at the same time, it uses RMSProp (Root Mean Square Prop) to make the algorithm has excellent performance in non-stationary and on-line problems. The update formula is as follows:

$m_t=\beta_1 m_{t-1}+\left(1-\beta_1\right) g_t$

$v_t=\beta_2 v_{t-1}+\left(1-\beta_2\right) g_t^2$

$\hat{m}_t=\frac{m_t}{1-\beta_1^t}$

$\hat{v}_t=\frac{v_t}{1-\beta_2^t}$

$\theta_{t+1}=\theta_t-\eta \cdot \frac{\widehat{m}_t}{\sqrt{\hat{v}}_t+\epsilon}$           (5)

This study is dedicated to putting the identification model into practice in order to aid in the growth of the cotton sector in underdeveloped regions. It is based on the detection of cotton leaf diseases in the natural background environment. We add an attention mechanism to the ShuffleNet V2 unit to create the CSGNet unit, which allows the model to concentrate on the region with the leaf illness, enhancing its capacity to extract disease characteristics in the natural environment backdrop. The suggested model also replaces the ReLU of the original model with GELU in order to further minimize the number of calculations required by the model; lastly, the network model is trained using the Adam optimization method. The suggested model continues to function well, with an average illness detection accuracy rate of 99.1%. By comparing the proposed model CSGNet to the traditional CNN model, it can be seen that it strikes a good balance between the quantity of parameters, the amount of calculation, and the accuracy. It can also be used in low-performance equipment for lightweight networks and quick disease identification by reasoning, and it offers a suggestion for applying a deep learning model to a mobile terminal in the field for disease detection.

In the future, we will focus on research to determine the severity of crop diseases based on the shape, colour and area of disease symptoms in pest images. This is because the use of pesticides depends to a large extent on the level of disease, with different levels of disease requiring different doses of pesticides, and unrestricted use of pesticides can cause considerable damage to the environment.

Also, for the purpose of in-field evaluation, we also aim to implement our proposed algorithm on mobile phone，So that it can really serve the cotton growers.