Keywords

1 Introduction

Machine learning is the field of research that enables the computer to learn itself without explanatory programming. Machine learning is the application of artificial intelligent (AI). The process is divided into two types; training and testing. The user provides the training data to the system which helps it to get trained. The trained system is tested using test data. A large amount of training data gives good accuracy to the results. When the system is given images as the input, the images have to be processed with the help of image processing techniques. The identification of crops requires images of the plants in real-time conditions. These images can be either laboratory-based or field-based images [1]. Machine learning has radically changed how the world could process, model, and interprets data [2]. The machine learning systems run in the cloud and are computationally intensive and require significant memory. Mobile devices are projected to hit 5.6 billion by 2020 [3]. Machine learning (ML) is now widely used in mobile applications for object detection, health monitoring, malware detection, language translation, and so on [4,5,6]. Machine learning is thought to play an important part in the development of mobile devices [7]. This on-device machine learning system developed by the engineering team expander, allows technologies to be used, without being linked to the cloud for any use. A large selection of training data is used during the training process to fine-tune the hyperparameters in the deep neural network (DNN) using gradient descent algorithms [8]. The wide DNN models surpass the memory of mobile devices, and thus the off-chip memory has to accommodate them, which significantly consumes more energy. To reduce the intricacy in computing, the fast Fourier transform-based multiplication is used [9, 6, 10]. This paper focuses on the identification of the crop using the field image of the crop. The mobile camera is used to capture the field image of the crop. The captured image is then processed using machine learning algorithms (CNN). A mobile application that automatically recognizes the plant is important for real-world ecological surveillance [11,12,13], invasive exotic plant control [14, 15], popularization of ecological science, and so forth. To upgrade the efficiency of these models, plant identification draws the attention of engineers and scholars.

2 Literature Survey

There are various studies and research done on crop identification using machine learning. Deshmukh and Manjusha Deshmukh detected paddy leaf using the K-means algorithm. This paper deals with the image acquisition, image preprocessing and segmentation, selection, and description of the features using ANN [16]. Segmentation of images is built with the K-means algorithm. These papers result in the fast detection of paddy leaves. Images of paddy leaf captured by using cameras are handled via the remote server. Yu Sun, Yuan Liu, et al. have proposed “Deep Learning for Natural Plant Recognition” [11]. In this system, the dataset which consists of 10,000 pictures of 100 ornamental plants is collected from the University of Beijing Forestry campus by cell phone in a natural scene. For plant classification, a deep learning model is built consisting of 26 layers and 8 residual building blocks. This model achieves a 91.78 percent recognition rate. H. Gosau, P. Bonnet, and A. Joly have proposed “Plant identification in an open-world (life clef 2016) [17].” The main aim of this system is to evaluate plant identification methods and very large scale schemes, similar to real-world biodiversity monitoring scenario conditions. The main novelty is that the identification task is strong enough to handle the unseen categories. N. Kumar et al. ha proposed “A Computer Vision Device for Automated recognition of Plant Species” [18]. This system explains the detection of plant species using automated object detection in a mobile app. Removing non-leaf photos, segmenting the leaf from the context, extraction characteristics reflecting the curvature of the contour of the leaf over multiple scales, and description of the species from a dataset of the 184 trees are done using computer vision components [19]. S. H. Lee, Y. L. Chang, C. S. Chan, and P. Remagnino have proposed an approach for plant recognition which uses neural convolution network [20]. This system identified 1000 forms of plants photograph referring to 7 different plant organs and even invasive species automatically identified from unknown groups. They used the CNN model to blend the habitat and organ characteristics for the plant classification task. The results of this system are limited when matched with VGGNet. While working, automatic plant taxonomy has yielded fruitful results. On observations, these trained models meet the requirements of completely automated environmental monitoring scenarios [14]. The deep convolutionary neural networks proposed in [21] had shown exceptional conduct in ILSVRC-2012’s large scale image classification task [22]. The conventional classification models depend on preprocessing to remove highly complicated backgrounds to boost desirable functionalities and features.

3 Proposed Model

The following are the major components in this system: (1) dataset preparation, (2) implementation of CNN using TensorFlow, (3) on-device machine learning, and (4) working on mobile application (Fig. 1).

Fig. 1
figure 1

Technology stack of this system

Full size image

3.1 Dataset Preparation

Data preparation is the first significant step in a machine learning project. It is the act of manipulating raw data in a manner that can be analyzed readily and precisely by data scientists and analysts, who can run it through machine learning algorithms to make predictions. The dataset of 30 different classes of crops, each of which 200 images have been collected from the natural scene. The most special feature of this dataset is that it contains different images for various cycles of growth for a given crop.

3.2 Implementation of CNN Architecture

Convolution neural network (CNN), a deep learning model used for classification and detection, has been increasingly developed with the advancement in deep learning and large machine hardware, showing good accuracy on different crops [23]. A neural convolutionary network (CNN) is a deep learning algorithm, which is specifically used for analyzing images. Multilayer perceptrons are regularized copies of CNNs. Multilayer perceptrons are fully connected network, that is, every neurons in the previous layer is mapped to each neuron in that layer. This multilayered CNN is built using TensorFlow (Fig. 2).

Fig. 2
figure 2

Deep learning neural network

Full size image

CNN architecture used in this project is MobileNet V2. MobileNet V1 is an architecture used mainly to run a trained model embedded in the mobile application. This architecture optimizes the trained model and makes it effective to run on the mobile application. It uses very few parameters and addition, multiplication of convolution to optimize the size and complicacy. MobileNet V2 uses depthwise convolution which is fast and efficient, whereas MobileNet V1 uses pointwise convolution which uses the same channels or uses double of it, and that makes it more costly. MobileNet V2 uses advanced features like linear bottlenecks between the layer and shortcut connections between the bottlenecks. But these advanced features are not found in MobileNet V1.

Figure 3: It explains the comparison of accuracy and latency between MobileNet V1 and MobileNet V2. This proves that MobileNet V2 is more efficient when compared to MobileNet V1.

Fig. 3
figure 3

MobileNet V1 versus MobileNet V2

Full size image

Steps involved in creating a neural network using TensorFlow are:

  1. A.

    Importing Datasets: The crops dataset along with its labels are imported into our model.

  2. B.

    Resizing and recoloring the images: The images in the dataset are resized and recolored in the RGB format.

  3. C.

    Creating layers: A stack of convoluted layers along with dense and flatten layers are created.

  4. D.

    Training the dataset.

  5. E.

    Obtain a trained model.

On-Device Machine Learning:

On-device machine learning enables the machine learning attributes to execute on the user’s personal devices. The TensorFlow lite makes inference on-device machine learning with small latency and limited binary size. The above-trained model is converted into a mobile compactable form using TensorFlow lite converter. The interpreter on TensorFlow lite runs the specially optimized models on mobile phones.

Working on Mobile Application:

In the mobile application, first, authentication and approval are done. After this process, the user is navigated to the home page, where the user can capture the image of the crop along with its geo-location. Now, the process of image recognition is efficiently and effectively performed. As a result, the bottom sheet displays the recognized image in text format along with the accuracy rate and inference time. The main advantage of this app is that it operates even without the presence of an Internet connection, which is achieved by learning machines on-device. By default, the recognition is carried out by the CPU. We have given an option to change the processor from CPU to GPU. After recognition, the details of the crops are displayed to the user based on the label with the help of Wikipedia. Wikipedia API is used to set up a connection between Wikipedia and the application. The text-to-speech (TTS) option is provided for user convenience. This application will also provide weather data of the place using the geo-location. The recognized crop image along with its information and geo-location is mailed to the admin in the presence of the Internet.

Figure 4: Whenever the user captures the crop image using the mobile camera, automatically, the deep learning algorithm recognizes the image even in the absence of the Internet and renders the output to the user. Once the Internet connection is established, the crop’s image with its geo-location and the information about the crop will be stored in the cloud and also e-mailed to the respective person (Table 1 and Fig. 5).

Fig. 4
figure 4

Overall architecture of our method

Full size image
Table. 1 Training accuracy rate of different class which has been collected during training the model
Full size table
Fig. 5
figure 5

Comparison between training and testing accuracy rate of our deep learning model

Full size image

4 Results and Discussion

The dataset is prepared from the live field images and is not taken from the Internet which therefore helps in attaining high accuracy. The dataset also consists of mobile captured images so as to ease the process of recognizing the images from end user's device. The dataset is prepared at three stages of crops, the early stage, the ripened stage, and the parched stage, which in turn helps in providing a consistent accuracy for any crop. Successions of experiments were conducted to check the accuracy of our model. The testing accuracy rate is increased by multiplying the number of neural layers in our model. During different experiments, we have tuned the important parameters of the training model to improve the accuracy of our model.

Figure 6 is taken during testing of our application. This image shows the accuracy rate attained and information displayed in the application during the detection of maize.

Fig. 6
figure 6

Output of maize prediction

Full size image

Figure 7 is taken during the testing of our application. This image shows the accuracy and the information of the wheat that are displayed in the application. The accuracy rate is 100%.

Fig. 7
figure 7

Output of wheat prediction

Full size image

The accuracy rate varies from crop to crop and device to device. This trained model produces an average accuracy rate of about 97% on a series of testing under different situations and circumstances.

5 Conclusion

Agriculture plays an important role in global economy of the country. Owing to the population growth, the farm industry is constantly being forced to increase the productivity and produce more crops. Nowadays, artificial intelligence and machine learning systems are used in various number applications related to agriculture. The usage of machine intelligence in agriculture resulted in more healthy crops and also increased the overall productivity. Many systems used images dataset for identification of the crops. Our dataset is not collected from Google or other Web sites; instead, it is collected from the fields at various stages of the growth period of each crop, which leads to the development of more accurate image analysis tool. The model is trained under different methodologies and analyzed, and the best methodology is chosen to achieve high accuracy. In this system, the trained machine learning algorithms provide good and better results. Most of the currently used agricultural-based machine learning application used Internet to process the results. This application does not rely on the Internet connection, which allows explorers to identify any crops irrespective of the region. Pests and diseases in crops lead to the decrease in the productivity of crops. In future, this application will also include the pest detection.