Introduction

The agricultural industries are facing several problems including deficiency in the growth of products like fruits, vegetables, etc. (Chen et al., 2019), unpredictable soil contents (Padarian et al., 2019), improper application of pesticides (Sladojevic et al., 2016), herbicides, fungicides or insecticides to reduce crop/plant diseases and shortage of trained/skilled labour (Zhao et al., 2016a), etc. It is very important to address these issues as advancements in agriculture play a vital role in the economy of a country. Just like other fields of research including medical science, mechanical/automation, and business industries, etc., agriculture can also benefit from the use of robots to complement the human workforce. Therefore, in recent years, several attempts have been made to resolve agricultural issues through robotic platforms (Ebrahimi et al., 2017; Wspanialy & Moussa, 2016; Zhao et al., 2016a). Many state-of-the-art approaches have been introduced/modified to perform various agricultural tasks like fuzzy logic/classifier (Cho, Chang, et al., 2002; Cho, Lee, et al., 2002; Sujaritha et al., 2017), combined radar-vision system (Milella et al., 2011), HIS colour model (Feng et al., 2015), improved Otsu threshold algorithm (Wei et al., 2014), integration of various sensors (Milella, Reina, et al., 2019; Reina et al., 2016), self-supervised scheme (Reina et al., 2016), etc. In this regard, Artificial Intelligence (AI) has been proven to have great potential towards agricultural applications by the implementation of robotic systems with machine learning (ML)/deep learning (DL) algorithms (Ebrahimi et al., 2017; McCool et al., 2017; Zhang, Jia, et al., 2018; Zhang, Qiao, et al., 2018). Some advanced visualization techniques are prominent: saliency map visualization (Brahimi et al., 2018), hyperspectral imaging (Mahlein et al., 2017; Wang, Vinson, et al., 2019; Wang, Zhang, et al., 2019), multispectral imaging (Patrick et al., 2017; Pourazar et al., 2019; Slaughter et al., 2008) and thermal imaging (Azouz et al., 2015; Ishimwe et al., 2014), etc., have also been applied with ML/DL models for agricultural tasks. Therefore, with the progress in AI, the performance of many complex agricultural operations has improved as compared to the earlier approaches. This led us to present an overall review of research outcomes that have been obtained for agricultural applications by the implementation of ML/DL algorithms through robotic systems.

Some review articles have been published incorporating only a particular type of agricultural application with/without a robotic system by considering AI/computer vision/other advanced vision control techniques. For example, a recent review addressed the crop water stress by the machine learning approach (Virnodkar et al., 2020). A review article summarized the statistical ML algorithms, which have been implemented for various agricultural operations (Rehman et al., 2019). In (Huang et al., 2010), soft computing techniques including fuzzy logic, neural network, genetic algorithm, decision tree, and support vector machine (SVM) were presented for the analysis of soil, precision agriculture, and management of crops. A comprehensive review was conducted for precision agriculture by Unmanned Aerial Systems (UAS) and important future directions were also provided in the article (Zhang & Kovacs, 2012). In (Kamilaris & Prenafeta-Boldú, 2018), the DL architectures were reviewed for several agricultural operations. The review presented in (Zhao et al., 2016a) indicated the algorithms/schemes developed for vision control of harvesting robots. Another review paper outlined the harvesting robots to show their performance along with the procedures of robotic designs, and adaptive algorithms for harvesting purposes. Some interesting future recommendations including modification in the environment of crops, innovative robotic designs, and other important factors like safety and economy were also summarized (Bac et al., 2014). For the harvesting purpose, the advancement in sensors was summarized in (Zujevs et al., 2015), by dividing them into four classes: chemical, tactile, proximity sensors, and computer vision. The issues like an in-camera sensor, design of the filter, and image segmentation methods for the identification of fruits through harvesting robots were presented in Li et al. (2011). Another review article presented the development of sensors for the detection/localization of fruit; it also described the AI-based classification methods and highlighted loopholes in those approaches (Gongal et al., 2015). The applications of machine vision with AI for agricultural tasks like detection of disease/pests in crops, evaluation of the quality of the grain, and automatic detection of plant phenotyping were studied in (Patrício & Rieder, 2018). The procedure for weed detection by various classification methods including machine learning and deep learning was reviewed in (Wang, Vinson, et al., 2019; Wang, Zhang, et al., 2019). The supervision of plant pathology by the robotic system while utilizing AI and machine vision techniques were presented in (Ampatzidis et al., 2017). Various sensing technologies and advanced cameras along with their limitations to categorize fruit/plant and analyse the physical structure of plants were summarized in (Narvaez et al., 2017). Another review article outlined the latest smart methodologies like internet of things (IoT), ML, and DL for agricultural purposes including crops/plant disease, pesticide and weed control, and storage and water management. (Jha et al., 2019). A review paper summarized ML algorithms for addressing weed detection, plant disease/pest detection tasks (Behmann et al., 2015). A recent review article presented the DL-based techniques for various agricultural applications (Santos et al., 2019). Another review paper explained and summarized deep learning models for the identification and classification of plant disease along with the application of DL with advanced imaging techniques including hyperspectral/multispectral imaging and some interesting future directions were also provided (Saleem et al., 2019). Moreover, the application of Big Data for agriculture was reviewed by Wolfert et al. (2017).

To the best of the authors’ knowledge, there is no systematic review in a single article presenting the performance of robotic systems by machine/deep learning algorithms considering the major agricultural operations including detection of plant disease, identification of crop/plant, fruit counting, fruit recognition, identification of weed, crop/weed discrimination, and classification of agricultural land cover. Therefore, this review article will be useful to advance the agricultural field of research by studying machine and deep learning techniques that have been implemented on various intelligent agricultural systems. It will also be helpful to understand the research gaps in several complex agricultural applications to save cost related to agricultural protection and increase the growth of several agricultural products. To understand an overall idea of a robotic system for agricultural operations by implementing an ML/DL algorithm, Fig. 1 can be a good resource. First, the agricultural application should be selected, which would lead to the selection of a certain robotic platform that can be primarily used for the collection of datasets. Then, the Machine Learning/Deep Learning model would be proposed and trained into a robot that will perform the agricultural task, and finally record the accuracy of the models in terms of various performance metrics, like classification accuracy, F1-score, detection/failure rate, etc.

Fig. 1
figure 1

Block diagram of the implementation of robotic system through ML/DL algorithms

Full size image

On top of that, during this review, the following questions were addressed that will guide the researchers of agricultural automation about many aspects of ML/DL algorithms employed through robotic platforms specifically in agricultural fields.

  • Which agricultural operations have majorly implemented machine/deep learning algorithms through automated systems and what are the robotic platforms adopted for these agricultural tasks?

  • Which agricultural products/fruits/vegetables have been included in the previous studies when considering the implementation of ML and DL algorithms for robotic systems?

  • Which ML/DL algorithms have been applied frequently for agricultural operations?

  • How much has deep learning outperformed traditional machine learning algorithms for various agricultural tasks?

  • Which performance metrics have been considered in the previous studies for the evaluation of ML and DL models that were used to perform agricultural tasks?

  • What are the research gaps which could be filled to achieve better performance of various agricultural operations by ML/DL-based automated systems?

The remainder of the paper is further divided into the following sections: “Application of Traditional Machine Learning Algorithms in Agricultural Robots” presents machine learning models for various agricultural applications applied on robotic systems along with the research gaps; “Deep Learning Approach for Agricultural Operations by Robotic Platforms” elaborates the deep learning architectures for several agricultural operations implemented through robotic platforms along with the performance plots, and “Conclusion and Future Directions” concludes the review along with some future directions which will be helpful to achieve higher accuracy and great advancements in several agricultural tasks.

Application of Traditional Machine Learning Algorithms in Agricultural Robots

In this era of automation, artificial intelligence (AI) has complemented the human workforce in many real-life applications. Similarly, the agricultural industries also require smart solutions to address important issues like saving cost, better production of agricultural products like fruits/vegetables, shortage of trained labour, etc. In this regard, machine learning (ML) as a subset of AI produced a significant contribution to agricultural automation. The ML has further subcategories such as deep learning (DL), which is an emerging technology to perform various agricultural operations intelligently. A general representation of AI, ML, and DL is presented in Fig. 2.

Fig. 2
figure 2

A hierarchy of artificial intelligence (AI) according to which machine learning is typically a subset of AI and similarly deep learning is the subcategory of machine learning

Full size image

Before the evolution of deep learning architectures, prominent among which is AlexNet (Krizhevsky et al., 2012), machine learning (ML) algorithms produced many state-of-the-art results for various agricultural tasks. In these algorithms, Support Vector Machines (SVM), K-Nearest Neighbour (KNN), Random Forest (RF) classifier, and Decision Tree (DT) are the most prominent models. Although the ML algorithms like SVM have also been used to perform various complex tasks like classification and mapping of agricultural terrain (Reina et al., 2017), this review is focused on the studies which applied ML models/algorithms to perform five agricultural tasks through robotic systems as described in the next sub-sections. Also, their limitations are summarized which could help to advance upcoming future research in the field of agricultural automation. The general flow for the implementation of an ML algorithm is presented in Fig. 3.

Fig. 3
figure 3

A general representation of steps to implement machine learning algorithms

Full size image

Plant Disease/Pest Detection and Classification

The diseases/pests on plant species produce a significant impact on the growth of agricultural products. Therefore, their detection and classification are a necessity, particularly by an automated approach. In this regard, ML algorithms have been applied to perform this important agricultural task. For example, a multi-support vector machine (M-SVM) was proposed for the detection of disease in citrus fruit and its performance was compared with state-of-the-art approaches like Weighted K-Nearest Neighbour (W-KNN), Decision Tree (DT), Linear Discriminant Analysis (LDA), and Ensemble Boosted Tree (EBT). However, the model were not compared with DL architectures to prove the effectiveness of the model more clearly (Sharif et al., 2018). Another research was conducted to detect and classify the healthy and diseased leaves of vine by Local Binary Patterns and One-Class Classifiers (Pantazi et al., 2019).

The ML algorithms have also been implemented through robotic platforms for the detection of plant disease. For example, an Unmanned Aerial Vehicle (UAV) was used for the detection of Citrus greening and well-known ML algorithms like linear SVM, coarse gaussian SVM, standard gaussian SVM, K-Nearest Neighbour, and simple and complex Decision Tree were implemented to obtain the best-suitable model. From this study, a research gap can be filled by comparing the performance of ML models with well-known DL models like AlexNet, ResNet-50, VGG-16, etc. for the classification between healthy and diseased leaves (Sarkar et al., 2016). A mobile robot was implemented in a strawberry greenhouse to detect its disease; an SVM algorithm was applied for this purpose and achieved a considerably lower prediction error (Ebrahimi et al., 2017).

Plant/Leaves Recognition and Classification

Another important task of plant recognition has been done by state-of-the-art ML techniques through robotic platforms. A mobile robot was implemented to find its best route for a plantation in a real agricultural farm (Jodas et al., 2013) and for that purpose, SVM and ANN were evaluated and achieved 93% and 90% accuracy respectively. A critical task of the classification of grapevines was performed by SVM and ANN models through an all-terrain vehicle (Gutiérrez et al., 2018). This research could have a more interesting analysis if the performance of these two ML algorithms were compared with some successful CNN models like AlexNet, VGG, ResNet-50, etc. Another plant classification-related task was described in Huang et al. (2016) which proposed and designed a stand-still imaging system consisting of a hyperspectral camera, and a Least Squares Support Vector Machine (LSSVM) model was selected to classify the maize seed. Although the technique implemented in this research achieved good classification accuracy (CA), still the effectiveness of LSSVM should be proved by comparing its performance with the other ML classifiers like RF. Furthermore, more diversity in data samples should be included to prove the robustness of the applied model. In a research, as a prerequisite for an agricultural robot in a practical field, an SVM-based classification method was applied to distinguish eight different plant species (Dyrmann et al., 2018); the method proposed in this research improved the classification accuracy which showed the significance of the work. Another research performed the classification among six different plant species by a BoniRob mobile robot through the implementation of well-known ML algorithms. Their comparison brought simple logistic regression, SVM, and neural network, with the best results (Weiss et al., 2010). A research used a UAV-based system for the task of tobacco plant recognition. An SVM model was implemented for this purpose and further enhancement in the performance was suggested by the candidate region extraction and feature extraction (Xie et al., 2016).

Crop/Weed Discrimination and Classification of Weeds/Crops

A critical agricultural task of crop and weed discrimination is also important to address as it is useful for determining the amount of herbicide required to control the weeds. Most of the studies were conducted for sugar beet fields; several were performed for carrot, rice, maize, and cereal farms. The ML algorithms including RF and SVM were prominently applied to robots for this agricultural operation. For example, the discrimination between crop and weed in a carrot farm was performed by an autonomous system through the implementation of the Random Forest classifier (Haug et al., 2014). For a more comprehensive assessment of the proposed system, there should be a comparative evaluation of this method with other state-of-the-art techniques like SVM. In (Cheng & Matson, 2015), an autonomous robot was used for the discrimination between crop (rice) and weed by featuring a base system consisting of a Harris Corner Detection algorithm along with ML algorithms that were compared in terms of precision and recall. A research was conducted for the discrimination between sugar beet and weeds by a BoniRob robotic platform; the classification was done by the Random Forest classifier, and the results were improved by MRF (Markov Random Field). Due to the successful classification outcomes, it is suggested that the RF would be useful for multiple-weed class problems (Lottes et al., 2016). Another research was conducted for the discrimination between a sugar beet crop and weed by a UAV; using information from RGB images; classification was done by RF classifier to achieve high precision and recall (Lottes et al., 2017). For an autonomous detection of weeds in a sugar beet field, two well-known algorithms—SVM and ANN—were implemented. The ANN achieved a considerably higher classification accuracy than SVM which proved the usefulness of the neural network-based technique (Bakhshipour & Jafari, 2018). A research was conducted to show the effectiveness of near-infrared mosaic hyperspectral imaging for crop and weed discrimination in a maize field. In the domain of machine learning, a random forest classifier was used (Gao et al., 2018); the higher precision and recall percentages showed that the applied method should be tested in a real-time robotic system. In (Tellaeche et al., 2011), the weeds were identified in a cereal crop by SVM and the classification accuracy was measured by Correct Classification Percentage (CCP) and Yule coefficient; the novelty of this work was shown by the evaluation of spray applied in the field. A research used the UAV for the detection/mapping of Silybum marianum weeds on hyperspectral images by comparing the performance of various ML techniques out of which One Class Support Vector Machine (OC-SVM) achieved the highest accuracy (Alexandridis et al., 2017).

Harvesting/Recognition of Fruits and Vegetables

The agricultural task of fruit harvesting has been addressed in recent studies that implemented well-known ML algorithms through robotic systems. However, the modified versions of ML models have also been proposed in a few research articles to perform this agricultural operation. For example, to identify tomatoes according to their maturity, a pixel and blob-based segmentation methods were applied along with a machine learning algorithm named X-means clustering which was derived from the famous K-means clustering method (Yamamoto et al., 2014). In (Ji et al., 2012), a harvesting robot was tested in a real field environment by applying the vector median filter for the removal of noise, then image segmentation was applied for the extraction of the features of apples. An SVM-based method was applied to get improvement in recognition accuracy and some research gaps were also provided in the paper like addressing the unrecognized apples and reduction in timing of fruit recognition for a practical system. The apple harvesting system was developed in another research which consisted of a manipulator, end-effector, and vision system, whereas the SVM with RBF (radial basis function) was used for the recognition of apples, and the effectiveness of the system was shown by performing the experiments in the laboratory and real agricultural farms (De-An et al., 2011). To harvest tomatoes, an ML approach named RVM (Relevance Vector Machine) was introduced based on Bayesian inference, and a higher accuracy was obtained which has provided the motivation to use an RVM model for upcoming research (Wu, Zeng, et al., 2019; Wu, Zhang, et al., 2019). In addition to the applied methods, a comparative analysis should be provided in these studies with the other ML algorithms to show the usefulness of the proposed approach.

However, some research articles have considered various ML algorithms and compared their performance for the recognition/classification of fruits. For example, a conveyor belt-based system was proposed to evaluate various conditions of biscuits by implementing a Radial-based SVM classifier with Wilk’s λ method which achieved a higher classification accuracy as compared to the Polynomial SVM and discriminant analysis (DA) (Nashat et al., 2011). In, (Tao & Zhou, 2017), the authors used the Colour-FPFH 3D descriptor to extract the features of apples. For the classification purpose, the Genetic Algorithm SVM classifier (GA-SVM) was used and its performance was compared with other classifiers like SVM, KNN, and RF. A complete study of Broccoli was presented in Kusumam et al. (2017), which incorporated the important steps from detection to size estimation and level of growth by a robotic tractor system and the SVM algorithm was again used along with a viewpoint feature histogram and temporal filter; a comparison was also done between KNN and SVM algorithms and the detection accuracy can be further improved by considering texture features. Another research used a tractor system for the localization and detection of Broccoli by using a method composed of VFH (ViewPoint Feature Histogram) and SVM (Support Vector Machine), its performance was increased when temporal filtering (TF) was included, and the proposed method was compared with ANN (Kusumam et al., 2016). Another study used the SVM classifier (Liu, Mao, et al., 2019; Liu, Pi, et al., 2019), which applied the HOG descriptor for the training of SVM and False Colour Removal (FCR) and Non-Maximum Suppression (NMS) were proposed for the removal of false positives and merge the overlapped detections. This research has practical importance for the future robotic system as the images were taken at 500–1000 mm distance which is quite feasible for an actual robotic platform. Therefore, the proposed method can be used in a real-time robotic system. Moreover, the proposed method was compared with the other approaches like AdaBoost (Zhao et al., 2016b), YOLO model (Redmon et al., 2016), Circular Gabor Filter, and Eigen Fruit (Kurtulmus et al., 2011).

Another approach used SVM for texture classification along with Canny edge detection with a graph-based connected component algorithm and the Hough line detection method for the removal/reduction of false positives of green citrus fruit (Sengupta & Lee, 2014). An SVM-based approach was proposed in (Mao et al., 2020) to recognize cucumbers in a farm; the method consisted of Iterative-RELIEF which was used for the extraction of colour components, background pre-processing being done by Median filter, Otsu algorithm, and Maximally Stable Extremal Regions (MSER); a fine-tuned DL model was proposed for feature extraction and finally PCA was used for the reduction of the dimension which eventually became useful for SVM classification.

Land Cover Classification

Several researchers used well-known ML algorithms and compared their performance for the selection of the best-suited model to classify different classes of agricultural land covers. For example, the classification among agricultural lands was performed and compared by implementing DT, RF, and SVM and it was concluded that object-based SVM got the highest Overall accuracy (Duro et al., 2012). A research was conducted to classify 16 classes divided into ten agricultural and six non-agricultural landscapes; a comprehensive comparison was provided between six state-of-the-art ML techniques including Multilayer Perceptron (MLP), Support Vector Regression (SVR), the Least-Squares (LS)-SVM, Bagged Regression Trees (BaRTs), Boosted Regression Trees (BoRTs), and the Random Forest (RF) by using EPR- (Eenmalige perceels registratie—in the Dutch language) based data and CORINE Land-Cover 2006 dataset. It was found that SVM classifiers (SVR and LS-SVM) outperformed other classifiers in terms of pixel-level Nash–Sutcliffe (NS) index and some future directions were provided in the article including the selection of input variables and the implication of fractional abundance constraints (Heremans & Van Orshoven, 2015). For the land cover classification, three state-of-the-art methods were applied including Support Vector Machine (SVM), Neural Network (NN), and Classification and Regression Trees (CART). It was found that the SVM classifier achieved the highest classification accuracy (Shao & Lunetta, 2012). Another research performed a comparative study between RF, kNN, and SVM to classify six different classes (including agricultural landscape) by using images taken through Sentinel-2 satellite (Thanh Noi & Kappas, 2018). A study was conducted for the classification of croplands and this time TerraSAR-X satellite data was used; the significance of RF was noted by comparing its performance with Classification and Regression Tree (CART) (Sonobe et al., 2014). In (Peña et al., 2014), nine important crops were classified by considering input from images of ASTER satellite and state-of-the-art ML approaches like DT, LR, SVM, and MLP were utilized for this purpose. Among all of them, SVM and MLP outperformed the others and the authors implemented an SVM + SVM algorithm that achieved slightly higher accuracy than SVM and MLP models.

On the other hand, a few articles implemented only RF classifier for the classification of the landscape. For example, the Landsat-5 Thematic Mapper data was used to classify complex landscapes by RF algorithm and achieved 92% overall accuracy (Rodriguez-Galiano et al., 2012). Similarly, the research presented in Eisavi et al. (2015) showed the significance of the RF classifier by taking the images of 13 agricultural landscapes via Landsat 8 satellite.

The Random Forest (RF) classifier and Maximum Likelihood Classification (MLC) were implemented on images taken from SPOT 5 satellite for the classification of various agricultural cropland fields. The outcome of this research favoured RF classifier by a significant margin (Ok et al., 2012). A research performed the classification of four croplands by classical ML algorithms such as SVM and RF through images taken by time series UAV. The novelty of this work was proved by considering the effect of textural features through the Grey-Level Co-occurrence Matrix (GLCM) along with the spectral features. Moreover, DL architectures could also be applied for further improvement in the classification task (Kwak & Park, 2019).

Overall Presentation of ML Algorithms for Agricultural Operations by Robots

Few important research gaps/future directions related to each agricultural operation from this section are presented in Table 1. Moreover, a summary of the performance of ML algorithms is shown in Table 2.

Table 1 Research gaps in some of the articles implemented ML algorithms to perform the respective agricultural tasks
Full size table
Table 2 A summary of machine learning approaches used for various agricultural tasks by robotic systems/platforms along with their performance indicators and agricultural products
Full size table

Deep Learning Approach for Agricultural Operations by Robotic Platforms

After the development of deep learning (DL), many state-of-the-art models were implemented for various real-life applications. Among those models, Convolutional Neural Network (CNN) produced significant improvement for many image recognition/classification tasks. Similarly, agricultural operations have also been performed by the implementation of CNN architectures through robots.

Previously, some review articles were focused on DL with respect to certain agricultural operations. For example, a comprehensive review of DL in agriculture was presented in (Kamilaris & Prenafeta-Boldú, 2018), in which all the major agricultural tasks were summarized. In contrast, this review article presents deep learning approach for major agricultural operations implemented through robotic platforms. Moreover, few research articles are also included in this review which showed the effectiveness of proposed DL-based models for upcoming agricultural robotic projects. Furthermore, some important research gaps are mentioned to address agricultural issues by automation through CNN architectures. The performance plots are also drawn to indicate the significance of DL architectures over traditional/well-known ML models for the respective agricultural tasks.

The implementation of DL to perform agricultural operations through robots involves few steps as presented in FigS. 4, and 5 further explains all the three steps of Fig. 4 more clearly.

Fig. 4
figure 4

The basic steps of a robotic platform for an agricultural task by DL approach; (A) indicates the input dataset like images of various plants/fruits, (B) presents that the robotic system needs a brain to perform certain tasks and here DL models act as a brain to the robot, and, finally, (C) represents the output of agricultural robots to show the significance of applied DL architecture

Full size image
Fig. 5
figure 5

A clearer explanation of (A, B, and C) (presented in the previous figure) require to implement an agricultural task by DL-based robot

Full size image

Plant Disease/Pest Detection and Classification

In recent times, DL has been considered a better method to perform agricultural tasks. These tasks are performed by implementing well-known CNN architectures or by proposing some modifications to those well-known models. A complex task of plant disease identification has been addressed by the DL techniques (Esgario et al., 2020; Li et al., 2020; Liu, Abd-Elrahman, et al., 2018; Liu, Zhang, et al., 2018; Singh et al., 2019). An overall review can be referred to (Saleem et al., 2019) related to plant disease identification by DL. However, in this section, a summary of the DL approaches applied through automated systems (like mobile robot, robotic arm, etc.) is provided for plant disease and pest identification.

An imaging system was proposed to detect the powdery mildew disease by the implementation of a famous CNN architecture named GoogLeNet and the accuracy was compared with experts’ performance (Bierman et al., 2019). A research was conducted for the comparative evaluation of the performance of DL and ML algorithms for the detection of pests on tomato and pepper crops for autonomous robots and concluded with the superior accuracy of DL architecture, principally Faster RCNN (more accurate but requires more computation time) and SSD (less accurate than RCNN and requires less training time). Moreover, the characteristics of the automatic feature extraction of DL were presented that helped to achieve better accuracy/classification as compared to an ML approach which required complex feature engineering works, and some innovative future directions were also presented like data augmentation technique and an inclusion of dataset images having pests present in the plants to generate diversity in a dataset (Gutierrez et al., 2019). Another research was conducted to prove the significance of deep learning in terms of its ability to automatically detect disease on fruits through an automatic sorting machine; the performance of the proposed system could be improved by deep autoencoder (da Costa et al., 2020). For the detection of a crop virus, a Fully Convolution Neural Network (FCN) model was deployed on the hyperspectral images through a tractor-shaped system containing a push broom (Polder et al., 2019). A 6 degree-of-freedom (DoF) robotic arm was used to automatically detect the diseased leaves by the implementation of Faster-RCNN with ResNet-101 model (Joffe et al., 2018). An Unmanned Aerial Vehicle (UAV)—based system was developed for the identification of vineyard disease by a deep learning algorithm applied to multispectral images, but it is suggested that false detections could be reduced by applying and testing various DL models (Kerkech et al., 2019). The UAV was also used for the detection of Fusarium wilt in a radish farm; the well-known VGG model performed well and achieved comparatively higher accuracy than the K-means clustering ML algorithm. Some recommendations were provided to improve the performance such as the combination of RGB and infrared images; a comprehensive analysis was recommended based on the severity of the disease (Ha et al., 2017). Another approach used UAV technology for the detection of disease in a radish farm by the implementation of K-means clustering along with GoogLeNet architecture (trained by a fine-tuning technique). The performance of the DL model was significantly better than the SVM model (Dang et al., 2018).

From this section, it can be concluded that various DL architectures were implemented for the detection of disease in plant leaves. However, the future research should be conducted to detect and classify the disease present in all the defected parts of the plant species including leaves, stem, fruit, and flowers, by utilizing the adaptive nature of DL. This is one of the most important research gaps provided in this review. Moreover, very few studies have been conducted to perform this task by a real-time automated system, therefore there is a need of a robotic platform than can address this agricultural problem. Furthermore, the chemical sprays like fungicide/herbicide/pesticides should be applied intelligently after the successful detection of plant disease, which would be helpful to generate a cost-effective crop protection system.

Plant/Leaves Recognition and Classification

Just like traditional ML algorithms, deep learning models have also applied for the plant recognition task. For instance, a research was conducted for the classification of several plant species by proposing/implementing CNN models (Dyrmann et al., 2016). A very important study was done in (Lee et al., 2017) to understand the concept and capability of deep learning models to extract the characteristics/features of several plant species. The development of deep plant phenomics (DPP) created a major contribution to the community of plant phenotyping (Ubbens & Stavness, 2017). Another state-of-the-art approach was proposed for the classification of plants with their multiple organs through CNN and RNN models (Lee et al., 2018). The classification of four different plant species was done by proposing a CNN model which outperformed the approaches like Scale-Invariant Feature Transform (SIRF) and speeded up robust features (SURF) (Kazerouni et al., 2019). A study was conducted for the classification of plant seedlings by CNN. This research also compared its performance with well-known ML techniques like SVM and KNN which proved the significance of the approach (Nkemelu et al., 2018).

Few of the studies were conducted to perform the task of plant recognition through robotic systems/platforms. To extract the stalk count and stalk width of the plant, a deep convolutional neural network and a semantic segmentation-based ground mobile robotic platform were proposed and validated the performance of the robot with two humans which showed the effectiveness of the proposed idea. The Faster Recurrent Convolution Neural Network (Faster RCNN) model was used for generating the bounding box and the binary output was obtained by Fully Convolution Network (FCN) to classify images as either stalk or background. As compared to human performance, the robot performed the stalk count task 30 times faster and stalk width measurement task 270 times faster (Baweja et al., 2018). A study was conducted for the plant phenotyping by the deep learning technique based on a recently developed Point Cloud Network (PCN) model through multi-robotic systems (Wu, Zeng, et al., 2019; Wu, Zhang, et al., 2019). A research used the UAV to count corn plants through a DL model named U-Net. The successful result of this approach provided a future motivation to implement this type of system for other crops (Kitano et al., 2019). A robotic manipulator was used to recognize seven vegetables by famous DL feature extraction/detection architectures, but the recognition accuracy should be further increased by proposing some key modifications to the models applied in this research (Zheng et al., 2018). To recognize Legacy Blueberries plants, a Computer Numerical Control (CNC)—based system was developed and a CNN was proposed to achieve a good performance in terms of precision, recall, F1-score, and accuracy. Although this proposed scheme can also be tested for pest detection, more improvement in the system’s performance is recommended in the future by the implementation of generative adversarial networks (GANs) for the generation of synthetic images (Quiroz & Alférez, 2020). The segmentation of Fig plants was done by UAV, and a CNN model was inspired by SegNet encoder-decoder architecture CNN. The code of the CNN model and dataset were published online for the research community. A good thing was that the complex and variable/original background of images were considered and it was suggested that the orthomosaic images could improve the system proposed in the paper (Fuentes-Pacheco et al., 2019). Another research used a UAV for the collection of datasets to detect Tobacco plants by the CNN models. Although the proposed CNN architecture achieved good accuracy, there is still room to improve the performance further by the implementation of various available well-known CNN models, the advanced training techniques like transfer learning/fine-tuning techniques could be utilized and some other crops should also be considered in the future (Fan et al., 2018). A research addressed the problem that occurs due to the critical distribution of heads of sorghum by a CNN model named RetinaNet on the UAV images; the system can achieve better performance by including diversity in the dataset (Ghosal et al., 2019). A hybrid approach consisting of SLIC (Simple Linear Iterative Clustering) and Hue properties was combined with a CNN model for the detection of flowers in a Soybean field and a single axis robot was used for this purpose, and the authors also provided a future direction—that seed pod counting should also be considered (Yahata et al., 2017). A research utilized the UAV for the detection, classification, counting of trees, and evaluation of varieties of citrus by the implementation of a famous DL detection model named YOLO-v3 (Ampatzidis & Partel, 2019).

In summary, the plant recognition task by DL models achieved considerably good performance. Some future works are recommended like diversity in datasets and considering different crops to prove the effectiveness of CNNs. And stalk count/width should also be addressed in more detail.

Crop/Weed Discrimination and Classification of Weeds/Crops

Another complex agricultural task of discrimination between crop and weed has been reported by the DL approach through real-time robotic systems. In (Adhikari et al., 2019), the authors presented a deep convolutional encoder-decoder neural network and achieved a higher mIoU (mean Intersection of Union) which was significantly higher than the previously-used models like UNet (Ronneberger et al., 2015), FCN (Long et al., 2015) and DeepLabV3 (Chen et al., 2017). A mobile platform was designed and implemented for the detection of weed in a radish farm by implementing an ANN model that showed the obvious effectiveness of a neural network for the detection of weeds (SeI Cho, Chang, et al., 2002; Cho, Lee, et al., 2002). A CNN-based semantic segmentation for real-time crop and weed classification was done in a sugar beet field (Milioto et al., 2018). Another research was conducted to classify crop and weed by the implementation of a lightweight & deeper CNN on a mobile robot and the novelty of the work was that these CNN architectures were applied on the RGB along with Infra-red images (Potena et al., 2016). A research was conducted to propose a class-wise stem and pixel-wise semantic segmentation-based system for the stem and crop/weed classification. This research achieved state-of-the-art results through a mobile robot and UAV, and they outperformed conventional approaches like Random Forest, baseline-stem (Lottes et al., 2018a). An FCN model having an encoder-decoder structure was proposed and implemented in sugar beet fields through a mobile robot named BoniRob containing RGB and NIR cameras for the collection of datasets (Lottes et al., 2018b). An automated ground robotic system was implemented for crop and weed discrimination through a simple ANN model by considering the natural environment, ignoring plants having incomplete features, and maximizing the pixels of weeds (Jeon et al., 2011). The classification between crop and weed was also performed by the UAV platform through the famous CNN model ResNet and two agricultural fields were considered to show the effectiveness of the proposed system; supervised labelling was done to improve the AUC on both fields. Moreover, an improvement in background segmentation by using multispectral images and graphical interface to generate an infestation map was suggested to reduce costs while applying herbicide in the fields (Bah et al., 2018). A smart sprayer system was designed for the management of weeds and the system’s performance was analysed by using two different Graphical Processing Units (GPUs); real/artificial plants were also considered which clearly proved the usefulness of the DL model for the detection of weeds. As a future direction, an algorithm should be deployed that can vary the amount of chemical spray required to control the weed and its performance should be compared with the traditional sprayers (Partel et al., 2019). A wheeled robot named AgBotII was implemented on a cotton field to manage the weeds and proposed an image locking system for a clustering algorithm. The value of the work was shown by introducing a new performance metric named DScore and discrimination of weed was done successfully without previous knowledge of the field (Hall et al., 2017). A Micro Aerial Vehicle (MAV) was also implemented for the treatment of weed in a sugar beet field. The images were taken by multispectral imaging technique and a recently-developed SegNet model was trained and tested for the classification of weed and crop which could achieve higher accuracy by training the model on a larger dataset (Sa et al., 2017). In (dos Santos Ferreira et al., 2017), a quadcopter was used for the collection of crop and weed images, whereas the classification task was done by a very famous CNN architecture called AlexNet on Caffe software. Classification accuracy was compared with ML-based state-of-the-art approaches like SVM, Random Forest, and AdaBoost, and the results were obtained under a controlled environment which leads to a research gap that can be filled by considering a real environment with a larger dataset. A research was conducted to generate a publicly available dataset for the classification of eight types of weeds that were trained and tested through well-known CNN models like Inception-v3 and ResNet-50 (Olsen et al., 2019). A mobile robot was used to generate the dataset for crop/weed detection (Di Cicco et al., 2017) and higher accuracy could be obtained by the use of NIR spectroscopy and hyperspectral imaging. A research was evaluated the robustness of the two models (JULE and DeepCluster) on the datasets developed in Olsen et al. (2019) and (dos Santos Ferreira et al., 2017) by unsupervised clustering algorithms (dos Santos Ferreira et al., 2019). A mobile robot was designed and implemented to classify crop and weed, and implemented popular CNN architectures like AlexNet, VGG, ResNet, and Inception-v3, while the training was performed by the transfer learning technique through ImageNet dataset (Suh et al., 2018). It is recommended to use Multiple classes to better prove the strength of the applied method. In (Dyrmann et al., 2017), the DetectNet model proposed in Barker et al. (2016) was used for the detection of weeds in a wheat field and an all-terrain vehicle was operated to generate the dataset. The proposed DL model should be tested in a real-time system to show the effectiveness of DetectNet architecture. Another article was published in the domain of weed detection by comparing the performance between SVM and ResNet models through a UAV in sugar beet fields; the obtained results favoured the ResNet model (Bah et al., 2019).

To conclude, many UAVs and ground robots have been implemented on fields like sugar beet, corn, etc., for performing complex tasks of weed/crop discrimination by state-of-the-art DL models. These successful DL models should be tested on other crops by UAVs or other robotic platforms. Moreover, smart chemical sprayers should be deployed to control the weeds in agricultural fields.

Harvesting/Recognition of Fruits and Vegetables

Some of the recent studies were conducted for the fruit detection/harvesting task by well-known DL architectures or by proposing an improved version of a DL model for forthcoming agricultural robotic projects (Sa et al., 2016; Zhang et al., 2019). A few of them focused on designing a gripper for a fruit harvesting robot, like (Zhang, Harrison, et al., 2020; Zhang, Huang, et al., 2020) proposed a harvesting system consisting of a low-cost robotic gripper and manipulator; the detection of the fruits/vegetables was done by the state-of-the-art Mask-RCNN model. A research proposed an improved version of Faster-RCNN to detect fruits and the effectiveness of the proposed model was proved by comparing its performance with other well-known and successful DL image detection architectures including YOLO, Fast RCNN, and Faster RCNN (Wan & Goudos, 2020). A recent article proposed an apple recognition system by pulse couple neural network and genetic Elman neural network and achieved a higher recognition rate (Jia et al., 2020). In (Liu, Mao, et al., 2019; Liu, Pi, et al., 2019), the authors proposed an improved version of a DenseNet model (Huang et al., 2017) to recognize and harvest tomatoes in a real environment. That research used a complex/actual environment that proved the novelty of the research as many of the previous studies used plain/controlled background and the rate of detection was comparatively better when compared to popular CNN models like ResNet, DenseNet, and SSD architectures. Another recent research was conducted for the classification of date fruits by well-known AlexNet and VGG-16 models trained through the transfer learning technique and comparing the performance of these models with previously published work (Altaheri et al., 2019). For the tomato harvesting robot, a wavelet transform-based image processing technique was applied along with two hidden layer feed-forward neural network models (Arefi & Motlagh, 2013). Another research implemented DL architecture by proposing a CNN model to harvest tomatoes and obtained 91.9% accuracy in a short period of time (Zhang, Jia, et al., 2018; Zhang, Qiao, et al., 2018).

In this article, those studies which used different robotic platforms for fruit harvesting/recognition purpose are extensively summarized. A novel research was conducted in which a robotic manipulator consisting of four arms was designed and implemented in a kiwifruit orchard for harvesting; the novel end-effector for each arm was designed to pick kiwifruit safely and dynamic scheduling was also done. The detection of kiwifruits was done by proposing a fully convolutional network named FCN-8S and real-field testing was performed which gave 51% successful harvesting results. Moreover, it was also determined that with the applied approach, the rate of successful harvesting can be increased to 70% and that a greater degree of freedom could increase cycle time (Williams et al., 2019). Few recent articles considered an important agricultural task of segmentation of fruit clusters in the real agricultural environment. In this regard, the first attempt was made for the estimation of canopy volume, counting, and detection of grape clusters. The images were taken by an RGBD camera placed on a mobile platform. Four pre-trained DL models were implemented; the VGG-19 model attained the highest accuracy (Milella, Marani, et al., 2019). Another recent study reported the segmentation of clusters of grapes by pre-trained DL architectures. Moreover, a novel method to improve the segmentation of cluster pixels was proposed. Due to high segmentation accuracy (Marani et al., 2020), this research could be adopted for the future research. A study implemented a simple backpropagation neural network on a sorting system consisting of a conveyor belt to classify the date fruit, and the future work should comprise an impact sensor, and feature a distribution-based method which should be introduced for better grading of the fruit (Al Ohali, 2011). A six DoF robotic manipulator was implemented for an ice lettuce farm and achieved a good success rate for harvesting, but the average cycle was comparatively slower than a human’s performance due to the weight of the end-effector. Also, the damage rate is required to be reduced in future studies (Birrell et al., 2019). For the harvesting robot, the Mask-RCNN with ResNet-50 model was used to detect strawberries and achieved higher mean Intersection over Union (mIoU). But it is suggested that the real-time implementation can be improved by proposing a lightweight model and the sample size could also be increased to improve the performance (Yu et al., 2019). To perform two tasks simultaneously (detection of fruits and estimation of their ripeness), a CNN-based system was proposed in Halstead et al. (2018). An important task of fruit detection on a coffee crop was performed by UAV in which a simple ANN model was used and compared with well-known ML techniques like K-nearest neighbour and random forest classifier; ANN outperformed the ML techniques in terms of F-score (Carrijo et al., 2017). An Unmanned Ground Vehicle (UGV) system was deployed in an orchard for the detection of fruits and their yield was estimated by CNN, MLP, and WS algorithms. Future studies can be conducted to use the transfer learning technique and various labelling methods should be implemented to advance the performance (Bargoti & Underwood, 2017a, b). A comprehensive research was conducted for the detection of fruits in an orchard by UGV through Faster-RCNN with ZFNet and VGG-16. The secondary contribution of this research was the evaluation of the transfer learning method, the conclusion being that for the fruits detection task of the dataset used in that research, this approach was not very useful in terms of average precision, and the transfer learning strategy was suggested with variation/diversity in the dataset images (Bargoti & Underwood, 2017a, b). A robotic arm along with its grippers and recognition system was designed to harvest tomatoes; a YOLO model was used for the detection of tomatoes. It is to be noted that, following hardware design and obtaining good recognition and harvesting results, the applied YOLO method should be compared with other DL architectures like Faster-RCNN, SSD, etc. (Yeshmukhametov et al., 2019). A mobile robot was designed that consisted of robotic arms for the detection of tomatoes according to their maturity level. For that purpose, the MobileNet model with SSD architecture was selected due to its best performance in terms of classification accuracy after a comprehensive comparison of state-of-the-art DL architectures like YOLO-v3 and ResNet-152 with Faster-RCNN (Horng et al., 2019). An UR3 robotic arm was used for harvesting apples and Single Shot Multibox Detector (SSD) was used, although the implemented DL network achieved more than 90% detection accuracy. But other efficient DL models (Faster RCNN or RFCN) should still be tested to further investigate the effectiveness of SSD for that task (Onishi et al., 2019). Another research article used a robotic manipulator to recognize seven vegetables by well-known DL feature extraction/detection architectures, but the recognition accuracy should be further increased by proposing some key modifications on the models applied in this research (Zheng et al., 2018).

From the explanation provided above, it is evident that several robotic manipulators have been proposed in many studies for the recognition/harvesting of fruits and vegetables, and various DL architectures have been implemented to perform these tasks in real-time. Nonetheless, only a few suggestions have been highlighted for the improvement in accuracy such as training the DL models by transfer learning technique and some modifications in famous DL models, etc.

Land Cover Classification

Land cover classification is a vast topic. Many studies have been conducted to classify land covers of various types to perform an overall analysis of one or more areas by DL-based techniques, specifically CNN architectures (Huang et al., 2018; Luus et al., 2015; Zhang, Harrison, et al., 2020; Zhang, Huang, et al., 2020). Some researchers used publicly available datasets by considering important lands of an area and performed classification studies by CNN models (Helber et al., 2019). In this review, only those studies were considered which incorporated agricultural land covers. For example, a single hidden layer neural network based on the extreme learning machine (ELM) method was proposed for this task and achieved comparable performance with a backpropagation neural network (BPNN) (Pal, 2009). An Unmanned Aircraft System (UAS) was implemented to classify land covers by Fully Convolutional Network (FCN), Support Vector Machines (SVM), Random Forest (RF), and Deep Convolutional Neural Network (DCNN) and concluded that DCNN and FCN have substantially higher accuracy than other classifiers, and authors suggested that multi-view data taken from the UAS can work with the DNN without needing a huge amount of training data (Liu, Abd-Elrahman, et al., 2018; Liu, Zhang, et al., 2018). To monitor forest cover, the CNN approach was adopted for the images taken from airborne and LiDAR. The weights can be optimized and other agricultural lands should be considered to prove the effectiveness of the method (Suzuki et al., 2018). The detection of citrus along with other crops’ trees was performed by UAV through the implementation of a simple CNN model consisting of only one hidden layer (Csillik et al., 2018). A research letter was published to show the significance of the Deep Recurrent Neural Network (DRNN) for the task of land cover classification on the satellite images, found that one set of images achieved the highest accuracy by RF (LSTM) model while other datasets obtained best results by SVM (LSTM) model (Ienco et al., 2017). Another research was conducted to classify satellite images of 11 different crops’ land by RNN predominantly Long Short Term Memory (LSTM) and Gated Recurrent Unit (GRU). This analysis was important to get the spatial information of the crops. It was suggested that multi-source data such as optical and SAR radar could be implemented in the future (Ndikumana et al., 2018). Another research was conducted to project the dynamics of forest cover by LSTM-based DL architecture, and relative explanatory variables to be included in future work and a robust deep learning model for different forest covers could be proposed (Ye et al., 2019). A study proposed the 3D-VGG model to show its effectiveness for the classification of crops’ lands from the images taken by two satellites (Ji et al., 2018). In (Kussul et al., 2017), a study presented the classification of various land cover types and crop types by the application of 1-D and 2-D CNNs for the first time specifically for the multisource satellite images, and the performance of CNNs was compared with RF and MLP. A study utilized hyperspectral images for the classification of various categories of natural vegetation and evaluated the performance of CNN, RF, and SVM. Moreover, accuracy can be increased by proposing an improved version of DL architecture (Guidici & Clark, 2017). A research was conducted by an airborne imaging system considering three kinds of datasets out of which two datasets were related to agricultural crops and the third dataset was related to various buildings; the applied CNN model was highly accurate to classify different crops for the first two locations (Song & Kim, 2017). Some well-known and successful DL models were trained and tested on wetland classes and found that the Inception ResNet-v2 model outperformed the other DL models including VGG, ResNet, Xception, DenseNet, Inception-v3 (Mahdianpari et al., 2018). The classification of 14 agricultural landscapes was performed by proposing six DL architectures and comparing the best CNN model with RF algorithm to prove its effectiveness in terms of its spatial feature extraction capability (Xie et al., 2019).

Various state-of-the-art DL models performed the task of agricultural land cover classification, especially CNNs and RNNs, which opens future research opportunities to manage various agricultural landscapes in a better way.

Overall Presentation of DL Algorithms for Agricultural Operations by Robots

The performance of DL/ML algorithms described in this section for the various agricultural operations is presented by bar plots in Figs. 6, 7, 8, and 9. In these plots, the DL/ML models are grouped by their respective research articles (denoted by D1, D2, D3, and so on) which are cited in Figs. 10, 11, 12, and 13 respectively. These figures are addressing the research questions mentioned in “Introduction” regarding the performance metrics/indicators, robotic platforms, and agricultural products, that have been commonly used during the implementation of deep learning architectures. It is also to be noted from the plots that some of the articles have shown the superiority of DL/ANN over traditional ML algorithms, like (Gutierrez et al., 2019) evaluated that RCNN outperformed the KNN model for the task of plant disease detection; in (Fan et al., 2018), the CNN achieved slightly better performance than SVM and RF for plant recognition purposes. To perform the crop/weed discrimination task, ResNet outperformed SVM and RF models (Bah et al., 2018), and, as described in (dos Santos Ferreira et al., 2017), a Convnet achieved better precision than RF. For the recognition of coffee, ANN obtained better results than KNN and RF (Carrijo et al., 2017). Moreover, several studies were conducted which proved the significance of DL models as compared to ML algorithms for the classification of agricultural land cover (Liu, Abd-Elrahman, et al., 2018; Liu, Zhang, et al., 2018), (Kussul et al., 2017), (Guidici & Clark, 2017) and (Xie et al., 2019). Similarly, from bar plots, it can also be observed that the deep learning-based image classification algorithms like AlexNet (Krizhevsky et al., 2012), ResNet (He et al., 2016), VGG (Simonyan & Zisserman, 2014), Xception (Chollet, 2017), MobileNet-v2 (Sandler et al., 2018) and object detection algorithms including Fast RCNN (Girshick, 2015), Faster RCNN (Ren et al., 2015), SSD (Liu et al., 2016), various versions of YOLO (You Only Look Once) models like YOLO-v1 (Redmon et al., 2016), YOLO-v2 (Redmon & Farhadi, 2017) and YOLO-v3 (Redmon & Farhadi, 2018) have been commonly used for various agricultural tasks. Therefore, upcoming research should incorporate any of the agricultural tasks by using successful deep learning models or proposing modifications in the form of cascaded or hybrid versions, essential changes in convolutional layers, the number of filter, stride, etc. (Liu, Abd-Elrahman, et al., 2018; Liu, Zhang, et al., 2018; Singh et al., 2019; Zhang, Jia, et al., 2018; Zhang, Qiao, et al., 2018). Its performance should then be tested offline before its implementation on real robotic platforms. Some important research gaps/future directions from this section are provided in Table 3.

Fig. 6
figure 6

Performance plots (in %) of ML/DL models used in robotic systems for plant disease detection (horizontal bars) and plant recognition (diagonal bars) tasks

Full size image
Fig. 7
figure 7

Performance plots (in %) of ML/DL models used in robotic systems for crop/weed discrimination task

Full size image
Fig. 8
figure 8

Performance plots (in %) of ML/DL models used in robotic systems for fruit recognition and harvesting tasks

Full size image
Fig. 9
figure 9

Performance plots (in %) of ML/DL models used in robotic systems for the land cover classification

Full size image
Fig. 10
figure 10

The corresponding reference of research articles (D1–D15) linked to the bar plot (Fig. 6), and performance metrics along with robotic platforms and agricultural products; a and b present plant disease detection task, whereas, c and d present plant recognition task

Full size image
Fig. 11
figure 11

The corresponding reference of research articles (D16-D28) linked to the bar plot (Fig. 7) for crop/weed discrimination task, and performance metrics along with robotic platforms and agricultural products

Full size image
Fig. 12
figure 12

The corresponding reference of research articles (D29-D39) linked to the bar plot (Fig. 8) for fruit recognition and harvesting tasks, and performance metrics along with robotic platforms and agricultural products

Full size image
Fig. 13
figure 13

The corresponding reference of research articles (D40-D49) linked to the bar plot (Fig. 9) for land cover classification task, and performance metrics along with robotic platforms and agricultural products

Full size image
Table 3 Research gaps from some of the articles implemented DL architectures with their respective agricultural operations
Full size table

Conclusion and Future Directions

In this review, robotic solutions are presented for the major agricultural tasks by machine and deep learning algorithms. Moreover, the performance of machine learning models is summarized along with selected agricultural products and robotic platforms for certain agricultural operations. Furthermore, the performance plots are drawn to indicate the effectiveness of deep learning models for the respective agricultural tasks. From the plots, it can be concluded that the DL architectures outperformed traditional ML algorithms. Although significant developments have been observed in recent studies, still some important research gaps are identified to further advance the agricultural field of research.

A brief summary of prominent results to indicate the significance of the DL architectures as compared to the ML-based techniques applied through the robotic system for five selected agricultural applications, and few future works is presented as follows:

  • Plant disease detection: RCNN achieved 82.51% detection rate, which was better than the other methods including SSD, MLP, and KNN with a difference of 13.08%, 17.61%, and 18.75%, respectively.

  • Plant recognition: CNN attained 0.84 F-measure, which was greater than the Viola-Jones’ method that achieved 0.80 F-measure.

  • Crop/weed discrimination: A well-known DL model named ResNet (94.84%) outperformed the traditional ML algorithms including SVM (60.6%) and RF (70.16%) in terms of area under the curve.

  • Fruit recognition/harvesting: An ANN-based model achieved 0.5553 F-measure, which was slightly better than the ML models like KNN (0.5284) and RF (0.5255).

  • Agricultural land cover classification: Three studies revealed that the performance of DL models was better than the ML-based techniques as listed below:

    • FCN got 83.9% overall accuracy, which was greater than DCNN (76.9%), SVM (67.6%), and RF (65.6%) models.

    • 2D-CNN attained a higher accuracy (94.6%) as compared to 1D-CNN (93.5%), ENN (92.7%), and RF (88.7%) models.

    • CNN (89.9%) performed better than SVM (89.5%) and RF models (82.2%) in terms of overall accuracy.

  • Out of five major agricultural operations, plant disease detection and classification lack a comprehensive study. Although these agricultural tasks have been addressed by offline approaches in many research articles, these should be performed by a robotic manipulator/mobile robot through deep learning meta-architectures.

  • After the successful application of DL algorithms for the detection/classification of plant disease by the robot, a combined effort by engineers and agronomists is required to implement a chemical spraying system that would apply fungicide/herbicide spray to the defected parts of the plant. It will be useful to reduce the cost of the crop protection system for agricultural farms.

  • Most of the approaches were detected/classified disease in plant leaves, but the defects in other parts of the plant species should also be detected like stems/flowers.

  • The adaptive nature of DL models should be utilized to show its automatic feature extraction capability for performing the various agricultural tasks by an efficient DL-based robot. For this purpose, the diversity in datasets must also be presented.

  • To improve the performance of various complex agricultural tasks, the modified/cascaded version of DL models should be proposed which can show their effectiveness by visualizing their convolutional layers.

  • A multi-purpose robot should be designed to show its adaptive behaviour in a sense of its physical structure to perform various operations in a farm (a tractor is the best example of a robotic platform that can be used for various purposes like plowing, planting, and similar tasks).

  • To visualize the complex agricultural tasks like crop/weed discrimination and fruit detection, advanced visualization techniques such as saliency map should be applied.

  • Some articles have previously presented to understand the factors affecting the performance of ML/DL algorithms for agricultural tasks, but a comprehensive study is still required for further development in agricultural automated systems.

  • Improvement in land cover classification could be done by proposing an improved version of CNN/RNN.

  • For better growth of agricultural products, an automated system should be proposed for the prediction of soil moisture content through robotic platforms.

  • A recent topic like Internet of Robotic Things should also be deployed for agricultural purposes so that a new research area would be able to be explored.