Skip to main content
SpringerLink
Log in

A Deep Learning-Based Detection System of Multi-class Crops and Orchards Using a UAV

  • Chapter
  • First Online:
Computer Vision and Machine Learning in Agriculture, Volume 2

Part of the book series: Algorithms for Intelligent Systems ((AIS))

Abstract

Agriculture is vital to the country’s economy. A 70% increase in agricultural consumption is expected by 2050 as the world’s population approaches 9 billion people. However, obtaining this figure is difficult due to weather and natural disasters. Modernizing this field’s technologies can help achieve the desired outcome. Precision agriculture is a type of ICT-based technology that has significantly increased global productivity. Since their introduction in agriculture, unmanned aerial vehicles (UAVs) and other robots have also offered the potential to be used in various PA applications such as health monitoring, yield estimation, and spraying. In this context, a real-time deep learning detection system based on an improved faster-RCNN is proposed to detect multi-class crops and orchards accurately. The proposed framework was implemented and evaluated in two different croplands (garlic and coriander) and two different orchards (loquat and peach). The developed system outperformed its competitors with 91.3% mean average precision (mAP) and a processing time of 0.235 s. Thus, the proposed framework provided an excellent potential to be deployed on autonomous systems (UAVs, robots, etc.) for various PA applications.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 179.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Tian Y, Zhao C, Lu S, Guo X (2011) Multiple classifier combination for recognition of wheat leaf diseases. Intell Autom Soft Comput 17(5):519–529. https://doi.org/10.1080/10798587.2011.10643166

    Article  Google Scholar 

  2. Escalante HJ, Rodríguez-Sánchez S, Jiménez-Lizárraga M, Morales-Reyes A, De La Calleja J, Vazquez R (2019) Barley yield and fertilization analysis from UAV imagery: a deep learning approach. Int J Remote Sens 40(7):2493–2516. https://doi.org/10.1080/01431161.2019.1577571

    Article  Google Scholar 

  3. Di Girolamo-Neto C et al (2019) Assessment of texture features for Bermudagrass (Cynodon dactylon) detection in sugarcane plantations. Drones 3(2):36. https://doi.org/10.3390/drones3020036

    Article  Google Scholar 

  4. Srivastava K, Bhutoria AJ, Sharma JK, Sinha A, Chandra Pandey P (2019) UAVs technology for the development of GUI based application for precision agriculture and environmental research. Remote Sens Appl Soc Environ 16:100258. https://doi.org/10.1016/j.rsase.2019.100258

  5. Ahmed F, Al-Mamun HA, Bari ASMH, Hossain E, Kwan P (2012) Classification of crops and weeds from digital images: a support vector machine approach. Crop Prot 40:98–104. https://doi.org/10.1016/j.cropro.2012.04.024

    Article  Google Scholar 

  6. Pajares G (2015) Overview and current status of remote sensing applications based on unmanned aerial vehicles (UAVs). Photogramm Eng Remote Sens 81(4):281–329. https://doi.org/10.14358/PERS.81.4.281

    Article  Google Scholar 

  7. Tri NC et al (2017) A novel approach based on deep learning techniques and UAVS to yield assessment of paddy fields. In: Proceedings—2017 9th international conference on knowledge and systems engineering, KSE 2017, vol 2017, pp 257–262. https://doi.org/10.1109/KSE.2017.8119468

  8. Mekuria W (2018) The link between agricultural production and population dynamics in Ethiopia: a review. Adv Plants Agric Res 8(4):348–353. https://doi.org/10.15406/apar.2018.08.00336

    Article  Google Scholar 

  9. Long S. Drones and precision agriculture: the future of farming. [Online]. Available: https://www.microdrones.com/en/content/drones-and-precision-agriculture-the-future-of-farming/. Accessed 25 May 2021

  10. Radoglou-Grammatikis P, Sarigiannidis P, Lagkas T, Moscholios I (2020) A compilation of UAV applications for precision agriculture. Comput Netw 172:107148. https://doi.org/10.1016/j.comnet.2020.107148

    Article  Google Scholar 

  11. Lv Z (2019) The security of Internet of drones. Comput Commun 148:208–214. https://doi.org/10.1016/j.comcom.2019.09.018

    Article  Google Scholar 

  12. Tokekar P, Vander Hook J, Mulla D, Isler V (2016) Sensor planning for a symbiotic UAV and UGV system for precision agriculture. IEEE Trans Robot 32(6):1498–1511. https://doi.org/10.1109/TRO.2016.2603528

  13. Stefas N, Bayram H, Isler V (2019) Vision-based monitoring of orchards with UAVs. Comput Electron Agric 163:104814. https://doi.org/10.1016/j.compag.2019.05.023

  14. Lottes P, Behley J, Milioto A, Stachniss C (2018) Fully convolutional networks with sequential information for robust crop and weed detection in precision farming. IEEE Robot Autom Lett 3(4):2870–2877. https://doi.org/10.1109/LRA.2018.2846289

    Article  Google Scholar 

  15. Ahmadi A, Nardi L, Chebrolu N, Stachniss C (2019) Visual servoing-based navigation for monitoring row-crop fields, no iii

    Google Scholar 

  16. Hayat S, Yanmaz E, Muzaffar R (2016) Survey on unmanned aerial vehicle networks for civil applications: a communications viewpoint. IEEE Commun Surv Tutor 18(4):2624–2661. https://doi.org/10.1109/COMST.2016.2560343

    Article  Google Scholar 

  17. Sampedro C, Rodriguez-Ramos A, Bavle H, Carrio A, de la Puente P, Campoy P (2018) A fully-autonomous aerial robot for search and rescue applications in indoor environments using learning-based techniques. J Intell Robot Syst Theory Appl 95(2):601–627. https://doi.org/10.1007/s10846-018-0898-1

    Article  Google Scholar 

  18. More A. Drone market size 2021 to 2025 segmentation at region level Incl—WBOC TV. [Online]. Available: https://www.wboc.com/story/44081263/drone-market-size-2021-to-2025-segmentation-at-region-level-including-market-revenue-share-and-price-analysis. Accessed 19 Aug 2021

  19. Choi H, Geeves M, Alsalam B, Gonzalez F (2016) Open source computer-vision based guidance system for UAVs onboard decision making. In: IEEE aerospace conference proceedings, Mar 2016. https://doi.org/10.1109/AERO.2016.7500600

  20. Ward S, Hensler J, Alsalam B, Gonzalez LF (2016) Autonomous UAVs wildlife detection using thermal imaging, predictive navigation and computer vision. In: IEEE aerospace conference proceedings, pp 1–8. https://doi.org/10.1109/AERO.2016.7500671

  21. Hazim B, Alsalam Y, Campbell D, Morton K, Gonzalez F (2017) Autonomous UAV with vision based on-board decision making for remote sensing and precision agriculture. In: IEEE aerospace conference

    Google Scholar 

  22. Hunt ER, Cavigelli M, Daughtry CST, McMurtrey JE, Walthall CL (2005) Evaluation of digital photography from model aircraft for remote sensing of crop biomass and nitrogen status. Precis Agric 6(4):359–378. https://doi.org/10.1007/s11119-005-2324-5

    Article  Google Scholar 

  23. Xiang H, Tian L (2011) Method for automatic georeferencing aerial remote sensing (RS) images from an unmanned aerial vehicle (UAV) platform. Biosyst Eng 108(2):104–113. https://doi.org/10.1016/j.biosystemseng.2010.11.003

    Article  Google Scholar 

  24. Lelong CCD, Burger P, Jubelin G, Roux B, Labbé S, Baret F (2008) Assessment of unmanned aerial vehicles imagery for quantitative monitoring of wheat crop in small plots. Sensors 8(5):3557–3585. https://doi.org/10.3390/s8053557

    Article  Google Scholar 

  25. Gonzalez-Dugo V et al (2013) Using high resolution UAV thermal imagery to assess the variability in the water status of five fruit tree species within a commercial orchard. Precis Agric 14(6):660–678. https://doi.org/10.1007/s11119-013-9322-9

    Article  Google Scholar 

  26. Felderhof L, Gillieson D (2012) Near-infrared imagery from unmanned aerial systems and satellites can be used to specify fertilizer application rates in tree crops. Can J Remote Sens 37(4):376–386. https://doi.org/10.5589/m11-046

    Article  Google Scholar 

  27. Von Bueren S, Yule I (2013) Multispectral aerial imaging of pasture quality and biomass using unmanned aerial vehicles (UAV)

    Google Scholar 

  28. Khan S, Tufail M, Khan MT (2021) Deep learning-based identification system of weeds and crops in strawberry and pea fields for a precision agriculture sprayer. Precis Agric 0123456789. https://doi.org/10.1007/s11119-021-09808-9

  29. Khan S, Tufail M, Khan MT, Khan ZA, Anwar S (2021) Deep learning based spraying area recognition system for unmanned aerial vehicle based sprayers. Turk J Electr Eng Comput Sci 29(2021):241–256. https://doi.org/10.3906/elk-2004-4

    Article  Google Scholar 

  30. Khan S, Tufail M, Khan MT, Khan A, Iqbal J, Wasim A (2021) Real-time recognition of spraying area for UAV sprayers using a deep learning approach. PLoS ONE 16(4):1–17. https://doi.org/10.1371/journal.pone.0249436

    Article  Google Scholar 

  31. Khan S, Tufail M, Khan MT, Khan ZA, Iqbal J, Wasim A (2021) A novel framework for multiple ground target detection, recognition and inspection in precision agriculture applications using a UAV. Unmanned Syst 10(1):1–12. https://doi.org/10.1142/S2301385022500029

    Article  Google Scholar 

  32. Bah MD, Hafiane A, Canals R (2020) CRowNet: deep network for crop row detection in UAV images. IEEE Access 8:5189–5200. https://doi.org/10.1109/ACCESS.2019.2960873

    Article  Google Scholar 

  33. Hung C, Xu Z, Sukkarieh S (2014) Feature learning based approach for weed classification using high resolution aerial images from a digital camera mounted on a UAV. Remote Sens 6(12):12037–12054. https://doi.org/10.3390/rs61212037

    Article  Google Scholar 

  34. Rebetez J et al (2016) Augmenting a convolutional neural network with local histograms—a case study in crop classification from high-resolution UAV imagery. In: ESANN 2016—24th European symposium on artificial neural networks, Apr 2016, pp 515–520

    Google Scholar 

  35. Milioto A, Lottes P, Stachniss C (2017) Real-time blob-wise sugar beets vs weeds classification for monitoring fields using convolutional neural networks. ISPRS Ann Photogramm Remote Sens Spat Inf Sci 4:41–48. https://doi.org/10.5194/isprs-annals-IV-2-W3-41-2017

    Article  Google Scholar 

  36. Lottes P, Khanna R, Pfeifer J, Siegwart R, Stachniss C (2017) UAV-based crop and weed classification for smart farming. In: Proceedings—IEEE international conference on robotics and automation, pp 3024–3031. https://doi.org/10.1109/ICRA.2017.7989347

  37. Fan Z, Lu J, Gong M, Xie H, Goodman ED (2018) Automatic tobacco plant detection in UAV images via deep neural networks. IEEE J Sel Top Appl Earth Obs Remote Sens 11(3):876–887. https://doi.org/10.1109/JSTARS.2018.2793849

  38. de Castro AI, Torres-Sánchez J, Peña JM, Jiménez-Brenes FM, Csillik O, López-Granados F (2018) An automatic random forest-OBIA algorithm for early weed mapping between and within crop rows using UAV imagery. Remote Sens 10(2):1–21. https://doi.org/10.3390/rs10020285

    Article  Google Scholar 

  39. Wei H, Mao J (2018) The recognition of rice area images by UAV based on deep learning. MATEC Web Conf 232:1–5. https://doi.org/10.1051/matecconf/201823202057

    Article  Google Scholar 

  40. Valente J, Doldersum M, Roers C, Kooistra L (2019) Detecting Rumex obtusifolius weed plants in grasslands from UAV RGB imagery using deep learning. ISPRS Ann Photogramm Remote Sens Spat Inf Sci 4:179–185. https://doi.org/10.5194/isprs-annals-IV-2-W5-179-2019

    Article  Google Scholar 

  41. Gao P, Zhang Y, Zhang L, Noguchi R, Ahamed T (2019) Development of a recognition system for spraying areas from unmanned aerial vehicles using a machine learning approach. Sensors (Switzerland) 19(2). https://doi.org/10.3390/s19020313

  42. Pérez-Ortiz M, Peña JM, Gutiérrez PA, Torres-Sánchez J, Hervás-Martínez C, López-Granados F (2016) Selecting patterns and features for between- and within-crop-row weed mapping using UAV-imagery. Expert Syst Appl 47(2016):85–94. https://doi.org/10.1016/j.eswa.2015.10.043

    Article  Google Scholar 

  43. Huang H, Deng J, Lan Y, Yang A, Deng X, Zhang L (2018) A fully convolutional network for weed mapping of unmanned aerial vehicle (UAV) imagery. PLoS ONE 13(4). https://doi.org/10.1371/journal.pone.0196302

  44. Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, Montréal, QC, pp 91–99. https://doi.org/10.1109/TPAMI.2016.2577031

  45. Sa I, Ge Z, Dayoub F, Upcroft B, Perez T, McCool C (2016) DeepFruits: a fruit detection system using deep neural networks. Sensors (Switzerland) 16(8). https://doi.org/10.3390/s16081222

  46. Deng X, Tong Z, Lan Y, Huang Z (2020) Detection and location of pine wilt disease induced dead pine trees based on faster R-CNN. AgriEngineering 51(7):228–236. https://doi.org/10.6041/j.issn.1000-1298.2020.07.026

    Article  Google Scholar 

  47. Zheng YY, Kong JL, Jin XB, Wang XY, Su TL, Zuo M (2019) CropDeep: the crop vision dataset for deep-learning-based classification and detection in precision agriculture. Sensors (Switzerland) 19(5). https://doi.org/10.3390/s19051058

  48. Ammar A, Koubaa A, Ahmed M, Saad A (2019) Aerial images processing for car detection using convolutional neural networks: comparison between faster R-CNN and YoloV3, pp 1–28

    Google Scholar 

  49. Girshick R (2015) Fast R-CNN. In: Proceedings of the IEEE international conference on computer vision, vol 2015, pp 1440–1448. https://doi.org/10.1109/ICCV.2015.169

  50. Thuan D (2021) Evolution of Yolo algorithm and Yolov5: the state-of-the-art object detection algorithm

    Google Scholar 

  51. Liu W et al (2016) SSD: single shot multibox detector. In: European conference on computer vision, pp 21–27. https://doi.org/10.1007/978-3-319-46448-0

Download references

Author information

Authors and Affiliations

  1. Department of Mechatronics Engineering, University of Engineering and Technology Peshawar, Peshawar, Pakistan

    Shahbaz Khan, Muhammad Tufail, Muhammad Tahir Khan & Zubair Ahmad Khan

  2. Advanced Robotics and Automation Laboratory, National Center of Robotics and Automation (NCRA), Peshawar, Pakistan

    Shahbaz Khan, Muhammad Tufail & Muhammad Tahir Khan

Authors
  1. Shahbaz Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Muhammad Tufail
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Muhammad Tahir Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zubair Ahmad Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shahbaz Khan .

Editor information

Editors and Affiliations

  1. Department of Computer Science and Engineering, Jahangirnagar University, Dhaka, Bangladesh

    Mohammad Shorif Uddin

  2. Department of Mathematics, South Asian University, New Delhi, India

    Jagdish Chand Bansal

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Khan, S., Tufail, M., Khan, M.T., Khan, Z.A. (2022). A Deep Learning-Based Detection System of Multi-class Crops and Orchards Using a UAV. In: Uddin, M.S., Bansal, J.C. (eds) Computer Vision and Machine Learning in Agriculture, Volume 2. Algorithms for Intelligent Systems. Springer, Singapore. https://doi.org/10.1007/978-981-16-9991-7_3

Download citation

Publish with us

Policies and ethics

Search

Navigation