Keywords

1 Introduction

With the continuously increasing world population, the global hunger index has also increased rapidly. This can be dealt with the means of digital farming which in turn will help in production of disease free crops and can provide high nutritional food [32]. By 2050, the global population will reach 9 billion according to reports by the United Nations. So automating the farming techniques can help in feeding the human population in the near future [33]. Climate crisis is also impacting the agriculture, and with extreme rainfall to severe shortage of rains in the past few years have also degraded the agriculture output. Extreme rainfall leads to plant diseases and threats from pests and weeds [18]. Researchers and farmers for past many years have been trying to overcome the provocations posed by the weeds, initially by introducing herbicides and then using mechanical means to remove weeds.

Weeds can appear anywhere in the field, and they compete for essential nutrients with crops, if not controlled, it could result in lower crop yield and quality [22]. Numerous techniques were tried to control weeds, and some of them includes manual weeding with hands or by simple hand tools. These methods were used for many years and still being used in small fields [31]. Herbicides used for weeds management may destroy weeds but are mistrusted for being problematic to the environment. For example, arsenical-based herbicides like monosodium methyl arsenate (MSMA) may be the reason for groundwater contamination [21]. Mechanical way of removing weeds is efficient and cost-effective but they are not able to remove intra-row weeds and may damage the crops, though efforts are being made to design mechanical hoes [34]. Intra-row weeding is still a challenging task which needs different assisting methods, like using real-time kinematic GPS (RTGPS) for measurement of diameter and height of agriculture crops.

Agriculture ecosystem is complex, unpredictable, and multivariate and can be understood in a greater way by constant observation and analyzing various traits related to it. This means need of big agriculture data and obtaining new information from it [3]. Data can be collected from satellites, unmanned aerial vehicles (UAV), airplanes, drones, field robots, providing large dataset of images of the agriculture environment. Raw images provide variety of challenges and need to be passed through various image processing techniques for decision-making purposes [12]. There are various techniques used for analyzing images, some popular techniques include machine learning (ML) (artificial neural networks (ANN), K-means clustering, support vector machines (SVM) amongst others) [38].

Besides the above-mentioned techniques, deep learning is gaining momentum in the past few years [29]. DL belongs to machine learning class and is similar to artificial neural network (ANN). However, it differs from conventional neural networks as they are deeper and have complex layers of interconnection with the requirements of strong computational power for extraction of desired parameters [27]. There are various network architectures used in deep learning (i.e., recursive neural networks (RNN), pre-trained networks, recurrent neural network, convolutional neural network (CNN) [15] (Fig. 1).

Fig. 1
figure 1

Source caffe.berkeleyvision.org [15]

An example of CNN architecture: CaffeNet.

Full size image

2 Methodology

The review presented aims to identify weeds in crops using artificial intelligence and deep learning methods. A procedure was established to interpret and compare the results relevant to the research using a systematic literature analysis. As deep learning in agriculture is relatively new, focus was on the research papers published from 2015 to 2020 with areas related to removal of weeds. The papers were selected from databases like ScienceDirect, Springer, IEEE Xplore, and Google Scholar. An expression-based research was initiated with keyword: (“weed detection” OR “deep learning in agriculture”) and (“deepweeds”). For each study, information was obtained and classified into: year of publication, problems described, architecture of the deep learning framework used, labels, classes, and data used (Figs. 2 and 3; Table 1).

Fig. 2
figure 2

Images from Grass–Broadleaf dataset. Upper row shows images of broadleaf weeds and lower row shows images of soil class. Adapted with permission from Elsevier, Copyright 2019. Source [5]

Full size image
Fig. 3
figure 3

Images from DeepWeeds dataset showing two examples of each weed species. Adapted with permission from Elsevier, Copyright 2019. Source [5]

Full size image
Table 1 Deep learning methodology used by authors for weed detection
Full size table

3 Deep Learning

DL learning is an extension of classical ML with more depth and complexity in the neural network which allows data to be represented in hierarchical way, through abstraction at several levels [27]. Feature learning is the biggest advantage of deep learning in comparison with ML. In ML, the input image has to go through preprocessing, segmentation, and feature extraction steps, in which normalization, size reduction, spectral property, etc., are done and the image is then fed to a machine learning model [33]. In DL due to the hierarchical representation, the features at higher level are formed by the lower level features.

The papers reviewed showed convolutional neural network (CNN) used as a class of deep learning framework to identify weeds from crops. CNN layers represent data with general features at the first layer, then becoming more specific going to the deeper layer [17]. Maxpooling was used to reduce the dimensionality of the layers, and fully connected layers were placed near the output of the network which acts as a classifier to classify input image or to make predictions.

3.1 DL Architectures and Frameworks

There are various prevalent architectures which were used by researchers reviewed in this paper either for using as pre-trained weights or for comparison of accuracy with their own DL model. AlexNet, VGG, CaffeNet (Fig. 1), GoogleNet among others were used for comparison. Various tools which researchers have utilized to experiment with their model were TensorFlow, Py-Torch, Caffe, Theano, Keras (programming interface which is integrated with TensorFlow 2.0), and DL MATLAB Toolbox.

4 Deep Learning in Weeds

The papers presented in the review were classified according the problem they addressed, sources of data with classes and labels, DL architecture implemented, whether data segmentation or preprocessing done, performance according to metrics chosen and comparison with other DL architectures.

4.1 Data Sources

DL model works best when fed with lots of image data containing thousands of images either captured through a camera, UAV, satellite, or mounted on a movable platform [25]. Some authors have synthetically increased the number of images by altering the images through various operations [2, 12]. Some researchers have used publically available dataset for weeds identification purpose like DeepWeeds [5, 14, 23], while other have generated their own dataset for their required research [35]. Papers dealing with weed identification who generated their own dataset used small dataset of images to be fed into DL model [25]. In general, problems in which large number of weeds species were to be identified required large number of data [26]. Many researchers have used camera mounted on a moving platform to catch images of weeds, and it turned to be most economical way to collect data [24, 26].

4.2 Data Preprocessing and Augmentation

Many researchers opted to do some image preprocessing or segmentation steps, before the specific features of the image were fed as input to the deep learning model. The most common was image resize, mostly converting into an image of smaller sizes of 128 × 128 or of 60 × 60 pixels [15]. Image segmentation was done to alter the size of the dataset or for expediting the learning process by feature enhancement [4, 8]. Pixel extraction [2, 12], background removal [26, 29] of images were done. The work under study utilized data augmentation to synthetically increase the dataset of training images. This was done when the datasets available were smaller in size and to increase the performance and the learning procedure of the deep learning model [9]. This process became important for authors who trained their model on artificial images and tested them on original ones.

4.3 Technical Details

Many researchers employed faith in CNN architectures like Inception-ResNet, GoogleNet, VGG16, and AlexNet, and some also developed their own architecture to compare their performance metrics with them [8, 13, 16]. Some of the research work also compared deep learning models with traditional machine learning models like support vector machines (SVM), artificial neural networks (ANN) [19] to evaluate performance between them.

Regarding deep learning framework used, papers published from 2015 to 2019 have employed Caffe followed by TensorFlow. Some authors also developed their own framework on top of Keras/Theano, Keras/TensorFlow. In [32], TensorFlow 2.0 was released which used Keras as API this prompted authors to use this as their programming framework [14]. Many studies divided their dataset into training/cross-validation and testing in the ratio of 90–10 or 95–5, respectively, with learning rate varying from 0.001 and 0.005 up to 0.01 [15]. Moreover, many authors used pre-trained weights as transfer learning to influence the learning efficiency [20].

4.4 Overall Performance

Measuring the performance of the deep learning model is to check with what percentage of accuracy the weeds have been correctly identified [19]. Authors have used different performance metrics related to their study. Kounalakis et al. [6] dos Santos Ferreira et al. [17] used precision and recall as their performance metrics with accuracy more than 90% indicating good performance. Jiang et al. [14] compared F1 score of their self-developed CNN architecture with other deep learning architectures. In nearly all of the cases reviewed DL approach outperformed the traditional ML approach (SVM, ANN, K-Means) when comparison was done to identify weed [29].

5 Discussion

The analysis of the research published in last five years shows the superiority of Deep learning in identifying weeds. It offered better accuracy and performance when compared to other ML approaches. Traditional ML approach required various preprocessing steps for feature extraction from the image such as histogram, scale-invariant feature transform (SIFT), texture- and shape-based algorithms and many more, whereas in DL, the hierarchical representation helps the features to get automatically extracted. Many of the author’s preferred CNN models to perform classification and some modified their model for complex problems like plant disease detection. Dyrmann et al. [7] used DetectNet CNN to detect weeds in cereal crops using bounding boxes. This establishes a very promising future in the research to identify weeds in real time as it will be helpful for an automated machine or a robot to detect and remove weeds (Table 2).

Table 2 Performance comparison of deep learning models with different framework used
Full size table

The biggest advantage of using deep learning to identify weed was the reduced effort to extract features from the images as it can be very time-consuming and requires considerable time to put the images in a shape that can be fed into a traditional machine learning algorithm [37]. A considerable shortcoming in using DL is that it needs large amount of data for training purpose and even longer time to train the model than ML approaches. There is an immense potential for the application of DL in agriculture and specifically in weed identification.

6 Conclusions

In this paper, a review of weed detection in crops using artificial intelligence and deep learning was summarized. Papers were identified with the technical details of the DL models employed, DL framework used, data preprocessing and augmentation techniques utilized, performance of the model according to the classification accuracy and comparison with other ML models were done. The finding indicates that DL deals with better classification accuracy, performance and offers better confidence in finding weeds than other image processing techniques. For future work, DL could be applied to complex agriculture problems like plant disease detection and weeds removal. AI and DL can significantly improve farming practices and could lead to smarter and sustainable farming.