An IoT- Enabled Augmented Reality Framework for Plant Disease Detection

Augmented Reality (AR) is a futuristic technology that provides an interactive experience with the real-world environment. AR defines a simple combination of a real and virtual environment. Specifically, AR feeds the input data from the live environment to the application, which automatically presents them to the eyeglass display. The end- user automatically receives the supervised information without any other need to search for information. AR system has three basic features, first is a fusion of near-real worlds environment, second, it provides an effective and precise 3D registration of virtual as well as real objects and the third feature is live interaction [1].

Enhancement in display technology includes wearable devices and Augmented Reality. This paper proposed situational analytics-supported decision-making and merged with visual analytics augmented reality based on supermarket shopping context [2]. This module evaluated on the simulated supermarket. It supports analytical reasoning in physical space and AR detailed information supports multi-dimensional data analysis about the object decision making. This system is less prone to error and Statistical analysis was faster than the traditional manual methodology.

AR allows users to gain information on image right on the spot. It gives a 3-Dimensional view ignoring the difficulty and hardness nature. By combining image composition and image special effects, the AR system validated and produced a valid result on 3D tracking camera images [3].

Virtual reality technology applied to the agriculture field is called as virtual agriculture. It helps researchers to explore in finding various soil stress conditions, agricultural machinery design, and manufacturing [4]. It can aid the farmers in different ways, like inspect the field, identifying the species of insects, and also determining the ways to handle it.

In smart agriculture, AR wearable glass makes the farmers to view the virtual portion of the field with their naked eye and also provides flexibility to operate by their hands. It serves as a piece of distinct hardware equipment for the farmers [5]. Combining IoT with AR deploys live environment makes superimposed IoT data into physical objects and delineation of such information [6]. A supporting computer graphics system is developed to visualize direction for farming operations and overlay them on a field. For example, it records the driver’s state and viewpoints during their operations [7].

A drone imaging method with AR is proposed by Huuskonen and Oksanen [8] to automatically find the position of soil sampling based on details from the soil map. Samples are taken from the field soil via worn AR smart glass and analyzed the nutrient contents such as pH, Mg (Magnesium), field soil type, Ca (calcium), K(Potassium), phosphorous, and S(Sulphur). Augmented reality wearable was used to guide the farmer based on the generated soil sample points.

Agricultural land will not always be perfect and smooth for the tractor driver's naked eye perception while tractor navigation is carried out. This developed tractor navigation AR model gives superimposed images of virtual graphics (3D) three-dimensional images, and area captured by AR camera [9]. This system is able to do effective and uniform plowing on the soil. This system model will supervise the driver on the tractor to wrap the field areas completely in a uniform manner, by the way, which prepares the soil for more productive cultivation. Regular interval observation is made to measure the positioning accuracy of the navigation system by way of altering the pitch, roll and yaw angle of the tractor. The 3D model is generated by utilizing the tractor position and direction, which can be identified by inertial measurement segment and on-time GPS kinematic unit. Finally, this AR-based navigation system could work for less than 3 cm positioning error and also within 3 m distance from the tractor front end. Another guidance system suggested in which the tractor operator provided with eye monitor glass can able to visualize the live plot and differentiate between the treated and the untreated zones in a different color [10]. AR technology was used to adjust the magnitude of the steering wheel and to control the steering of the tractor wheel rotation to move on the correct trajectory concerning the treated area maps.

Another multi-objective path planner system was suggested, in which the problem of spraying pesticides in greenhouses was addressed as a vehicle routing issue [11]. The probabilistic roadmap path planner was designed to solve the vehicle routing problem, and the routes are generated between the plants for spraying pesticides based on the virtual environment.

An AR-vino system was developed on the AR platform for visualizing 3D data in outdoors, using a mobile tripod-based computer [12]. The combination of GIS (Geographic Information System) and augmented reality provided a new way to visualize viticulture (grape yard/vineyard) and to monitor the yield and quality of grapes. The AR system solves the long-flat view problem to view field or agricultural data over the uniform ground.

One more work with AR aid for large-scale crop distribution is given for the crops planting in uneven areas [13]. It solves the planting problem by adapting the design of visually altering zonal edges to identify the area of sow crops in the virtual surroundings. It is also suggesting a fast building method of the virtual scene which is based on the Even-Odd algorithm and Catmull-Rom spline.

This paper proposes a fixed-point triggering method [14] to address the issue of synchronizing voice and video, which is difficult in the 3D site. The batch loading model on VR says that comparing with the manual batch loading method; the VR method achieved good accuracy. The video, audio, and roaming points are precisely synchronized and also effectively promote multimedia resources. This proposed model provides an interactive experience for the VR module through the HTC vibe.

Augmented Reality on location [15] demonstrated for in site visualization as well as fieldwork navigation. This model enables the farmers to track their crop growth. It also visualizes the sensor data with the point of interest. This system enables the non-expert user’s to make use of this demonstrated module.

Pest identification and insect management system developed [16] with AR for aiding the farmers to control the pests. The designed prototype able to detect the bugs and suggest treatments and pesticide usages to control the growth of pests and to kill the harmful bugs in flora and fauna.

AR image recognition algorithm [17] promotes to identify and quantify weeds by species and also control the weed growth. By integrating AR with weed economic thresholds software, the AR provides results on year-wise crop production, yield loss due to weed, herbicide need for the next season can be known.

Since AR set hardware having limitations on computational proficiency, and on-field image data processing are not processed within the AR set. So, in this paper, we suggest real-time data analysis on the cloud computing platforms. In this article, the design of an AR system models using machine learning and cloud computing are furnished for the application of on-field plant analysis. This study aids in exploring the elaborate architecture of Augmented Reality in smart farming, which helps in determining the diseases on plants.

Major contributions of the article are as follows:

• Development of Augment reality system for on-field plant disease identification.
• Development of cloud-based machine learning framework for plant disease analysis.

The remaining part of the article is organized as follows; Section 2 provides the system model and cloud data processing; section 3 presents the result and discussion with the data collection mechanism. Finally, section 4 concludes the article with a summary of work and future direction.

In this paper, input image data are captured from the SRM urban farm center of SRM IST, Faculty of agriculture Science. The Farm view is shown in Figure 4. Tomato leaf samples of the raw data are collected on the field from 340 different healthy plants and for each plant at five different places. Samples collectively give spatial variability. Totally 2000 samples of healthy leaves are collected. Similarly, 2000 Samples of sick leaves also obtained. During data acquisition, plant pathology expert from SRM agriculture science is involved in guiding to collect the proper unhealthy plant image samples.

4.png

Figure 4. Data collection Centre-SRM urban farm centre

Table 1. Number of samples taken for experimentation

The unhealthy leaves with different varieties of diseases, namely bacterial, viral infections, lesion spots, blight, are covered during data acquisition.

The varieties of healthy and unhealthy leaf images are equally distributed from the total sample that we have taken. Table 1 shows the total no of collected healthy, viral infected, bacterial, blight and lesion spot leaves.

First, the samples are pre-processed in the AR HMD unit, then analyzed in the cloud module using the Convolution Neural Network (CNN). Preprocessing method of ROI detection and segmentation is applied. Before processing the entire image of the leaves is taken. During preprocessing, the green portion of the histogram is extracted by employing ROI detection and segmentation mechanism and the irrelevant background noise is eliminated.

This will result in the reduction of data size, which will improve the transmission speed of the data to the cloud. On the other hand, it will also improve the accuracy of disease detection since the background area is removed. The collected samples are divided into 70% for training, 15% for validation, and 15% for testing purposes.

The convolution neural network architecture employed in the proposed work is shown in Figure 5: 1. The input layer size 256* 256*3. The image size is 256*256 and 3 represents the RGB color space; 2. Convolution layers size of 20 with 5*5 kernel. The 5*5 is a feature identifier applied to select the features by removing the backgrounds; 3. ReLU layer of activation; 4. Pooling layer this layer is probably utilized for resizing image; 5. The fully connected layer, this layer peer for the high-level features; 6. Flatten layer for convert 2D data into 1D 6; 7. Softmax layer (Compute the probability of classes) and Classification.

5.png

Figure 5. General CNN architecture

The convolutional neural network architecture is given in Table 2. with its relevant parameters.

The different data set consist of about 500 samples, 1000 samples, and 2000 samples are executed for analysis of the performance of the system with 56 iterations with eight epochs. Figure 6 shows the accuracy of classification with respect to the number of iterations for 2000 samples on training. From Figure 7, it is evident that the maximum accuracy of about 93% is obtained in classification. Inferred from Figure 6, the loss values are minimized during the training and attained a minimum amount of 0.001 at the last iteration.

Table 2. CNN architecture design

Table 3. Performance of CNN

6.png

Figure 6. Testing accuracy

7.png

Figure 7. Loss / error value on training

Table 3 shows the accuracy and time complexity of the proposed system for three different data sizes. The hardware involved in this system is Arm-based Amazon EC2 M6g gravitation 2 processor. We run the program in the AWS cloud server. The computational complexity result is valid for the Platform of Intel i5 8th gen- 8250U 8GB RAM, CPU at 1.8 GHz in Windows 10 OS. From the table, it is evident that the proposed method is suitable for the practical deployment for the on-field disease analysis. The maximum computational training time is 10 minutes12 seconds for 2000 samples and testing time is 10ms. This computational time shows that the system can support and meet the real-time response demand of the AR system.

The following comparison charts shows the accuracy of literature work surveyed.

Table 4. Comparison of literature works

The comparison of the proposed work with literature literary works are presented in Table 4. The images for those literature works are obtained only from the internet sources for tomato leaf disease detection and employed a Convolutional Network algorithm to achieve better accuracy.

Our proposed work involved our own dataset, which is executed on cloud-enabled Virtual Reality headset. The developed system is portable, ensures the real-time analysis for the identification of diseases on the tomato plant leaves.