Skip to main content
SpringerLink
Log in

AIM in Endoscopy Procedures

  • Reference work entry
  • First Online:
Artificial Intelligence in Medicine

  • 372 Accesses

Abstract

Artificial intelligence (AI) is revolutionizing the way medicine is practiced. In this context, the application of AI algorithms in endoscopy is gaining increasing attention so that modern endoscopy is moving towards more and more assisted/automatic solutions. Several approaches have been carried out in order to improve accuracy in diagnosis and surgical procedures. In this chapter, a general overview of the main contributions in the field is surveyed. Four main categories of applications were identified, namely, (i) detection and diagnosis during endoscopic procedure, (ii) informative frame selection, (iii) mosaicking and surface reconstruction, (iv) augmented reality systems for intraoperative assistance and surgeon training. Discussions on future research directions and implementation in clinical practice are provided.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 699.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 1,199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Rahim T, Usman MA, Shin SY. A survey on contemporary computer-aided tumor, polyp, and ulcer detection methods in wireless capsule endoscopy imaging. Comput Med Imaging Graph. 2020;85:101767.

    PubMed  Google Scholar 

  2. Bergen T, Wittenberg T. Stitching and surface reconstruction from endoscopic image sequences: a review of applications and methods. IEEE J Biomed Health Inform. 2014;20(1):304–21.

    PubMed  Google Scholar 

  3. Sharma P, Pante A, Gross SA. Artificial intelligence in endoscopy. Gastrointest Endosc. 2020;91(4):925–31.

    PubMed  Google Scholar 

  4. El Hajjar A, Rey JF. Artificial intelligence in gastrointestinal endoscopy: general overview. Chin Med J. 2020;133(3):326.

    PubMed  PubMed Central  Google Scholar 

  5. Togashi K. Applications of artificial intelligence to endoscopy practice: the view from Japan Digestive Disease Week 2018. Dig Endosc. 2019;31(3):270–2.

    PubMed  Google Scholar 

  6. Dougherty G. Medical image processing: techniques and applications. Springer Science & Business Media; 2011.

    Google Scholar 

  7. Karkanis SA, Iakovidis DK, Maroulis DE, Karras DA, Tzivras M. Computer-aided tumor detection in endoscopic video using color wavelet features. IEEE Trans Inf Technol Biomed. 2003;7(3):141–52.

    PubMed  Google Scholar 

  8. Iakovidis DK, Maroulis DE, Karkanis SA. An intelligent system for automatic detection of gastrointestinal adenomas in video endoscopy. Comput Biol Med. 2006;36(10):1084–103.

    PubMed  Google Scholar 

  9. Bernal J, Tajkbaksh N, Sánchez FJ, Matuszewski BJ, Chen H, Yu L, et al. Comparative validation of polyp detection methods in video colonoscopy: results from the MICCAI 2015 endoscopic vision challenge. IEEE Trans Med Imaging. 2017;36(6):1231–49.

    PubMed  Google Scholar 

  10. Mori Y, Kudo SE, Mohmed HE, Misawa M, Ogata N, Itoh H, et al. Artificial intelligence and upper gastrointestinal endoscopy: current status and future perspective. Dig Endosc. 2019;31(4):378–88.

    PubMed  Google Scholar 

  11. Min JK, Kwak MS, Cha JM. Overview of deep learning in gastrointestinal endoscopy. Gut Liver. 2019;13(4):388.

    PubMed  PubMed Central  Google Scholar 

  12. Alagappan M, Brown JRG, Mori Y, Berzin TM. Artificial intelligence in gastrointestinal endoscopy: the future is almost here. World J Gastrointest Endosc. 2018;10(10):239.

    PubMed  PubMed Central  Google Scholar 

  13. Gulati S, Emmanuel A, Patel M, Williams S, Haji A, Hayee B, et al. Artificial intelligence in luminal endoscopy. Therapeut Adv Gastrointest Endos. 2020;13:2631774520935220.

    Google Scholar 

  14. Lazăr DC, Avram MF, Faur AC, Goldiş A, Romoşan I, Tăban S, et al. The impact of artificial intelligence in the endoscopic assessment of premalignant and malignant esophageal lesions: present and future. Medicina. 2020;56(7):364.

    PubMed Central  Google Scholar 

  15. de Souza Jr LA, Palm C, Mendel R, Hook C, Ebigbo A, Probst A, et al. A survey on Barrett’s esophagus analysis using machine learning. Comput Biol Med. 2018;96:203–13.

    Google Scholar 

  16. van der Sommen F, Zinger S, Curvers WL, Bisschops R, Pech O, Weusten BL, et al. Computer-aided detection of early neoplastic lesions in Barretts esophagus. Endoscopy. 2016;48(07):617–24.

    PubMed  Google Scholar 

  17. Swager AF, van der Sommen F, Klomp SR, Zinger S, Meijer SL, Schoon EJ, et al. Computer-aided detection of early Barretts neoplasia using volumetric laser endomicroscopy. Gastrointest Endosc. 2017;86(5):839–46.

    PubMed  Google Scholar 

  18. de Groof AJ, Struyvenberg MR, van der Putten J, van der Sommen F, Fockens KN, Curvers WL, et al. Deep-learning system detects neoplasia in patients with Barretts esophagus with higher accuracy than endoscopists in a multistep training and validation study with benchmarking. Gastroenterology. 2020;158(4):915–29.

    PubMed  Google Scholar 

  19. Marks R. Squamous cell carcinoma. Lancet. 1996;347(9003):735–8.

    CAS  PubMed  Google Scholar 

  20. Guo L, Xiao X, Wu C, Zeng X, Zhang Y, Du J, et al. Real-time automated diagnosis of precancerous lesions and early esophageal squamous cell carcinoma using a deep learning model (with videos). Gastrointest Endosc. 2020;91(1):41–51.

    PubMed  Google Scholar 

  21. Ohmori M, Ishihara R, Aoyama K, Nakagawa K, Iwagami H, Matsuura N, et al. Endoscopic detection and differentiation of esophageal lesions using a deep neural network. Gastrointest Endosc. 2020;91(2):301–9.

    PubMed  Google Scholar 

  22. Itoh T, Kawahira H, Nakashima H, Yata N. Deep learning analyzes Helicobacter pylori infection by upper gastrointestinal endoscopy images. Endoscopy Int Open. 2018;6(2):E139.

    Google Scholar 

  23. Redondo-Cerezo E, Sánchez-Capilla AD, De La Torre-Rubio P, De Teresa J. Wireless capsule endoscopy: perspectives beyond gastrointestinal bleeding. World J Gastroenterol: WJG. 2014;20(42):15664.

    PubMed  PubMed Central  Google Scholar 

  24. Ashour AS, Dey N, Mohamed WS, Tromp JG, Sherratt RS, Shi F, et al. Colored video analysis in wireless capsule endoscopy: a survey of state-of-the-art. Curr Med Imaging. 2020;16:1074.

    PubMed  Google Scholar 

  25. Soffer S, Klang E, Shimon O, Nachmias N, Eliakim R, Ben-Horin S, et al. Deep learning for wireless capsule endoscopy: a systematic review and meta-analysis. Gastrointest Endosc. 2020;92:831.

    PubMed  Google Scholar 

  26. Liu Y, Zhao Z. Review of research on detection and tracking of minimally invasive surgical tools based on deep learning. J Biomed Eng. 2019;36(5):870–8.

    Google Scholar 

  27. Colleoni E, Moccia S, Du X, De Momi E, Stoyanov D. Deep learning based robotic tool detection and articulation estimation with spatio-temporal layers. IEEE Robot Automat Lett. 2019;4(3):2714–21.

    Google Scholar 

  28. Park SY, Sargent D, Spofford I, Vosburgh KG, Yousif A, et al. A colon video analysis framework for polyp detection. IEEE Trans Biomed Eng. 2012;59(5):1408–18.

    PubMed  Google Scholar 

  29. Iddan G, Meron G, Glukhovsky A, Swain P. Wireless capsule endoscopy. Nature. 2000;405(6785):417.

    CAS  PubMed  Google Scholar 

  30. Fan Y, Meng MQH, Li B. A novel method for informative frame selection in wireless capsule endoscopy video. In: 2011 Annual international conference of the IEEE engineering in medicine and biology society. IEEE; 2011. p. 4864–7.

    Google Scholar 

  31. van der Sommen F, de Groof J, Struyvenberg M, van der Putten J, Boers T, Fockens K, et al. Machine learning in GI endoscopy: practical guidance in how to interpret a novel field. Gut. 2020;69:2035.

    PubMed  Google Scholar 

  32. Alizadeh M, Sharzehi K, Talebpour A, Soltanian-Zadeh H, Eskandari H, Maghsoudi OH. Detection of uninformative regions in wireless capsule endoscopy images. In: 2015 41st annual northeast biomedical engineering conference. IEEE; 2015. p. 1–2.

    Google Scholar 

  33. Szeliski R. Computer vision: algorithms and applications. Springer Science & Business Media; 2010.

    Google Scholar 

  34. Moccia S, Vanone GO, De Momi E, Laborai A, Guastini L, Peretti G, et al. Learning-based classification of informative laryngoscopic frames. Comput Methods Prog Biomed. 2018;158:21–30.

    Google Scholar 

  35. Patrini I, Ruperti M, Moccia S, Mattos LS, Frontoni E, De Momi E. Transfer learning for informative-frame selection in laryngoscopic videos through learned features. Med Biol Eng Comput. 2020;58:1225–38.

    PubMed  Google Scholar 

  36. Miranda-Luna R, Hernandez-Mier Y, Daul C, Blondel WC, Wolf D. Mosaicing of medical video-endoscopic images: data quality improvement and algorithm testing. In: (ICEEE). 1st international conference on electrical and electronics engineering, 2004. IEEE; 2004. p. 530–5.

    Google Scholar 

  37. Weibel T, Daul C, Wolf D, Rösch R, Guillemin F. Graph based construction of textured large field of view mosaics for bladder cancer diagnosis. Pattern Recogn. 2012;45(12):4138–50.

    Google Scholar 

  38. Behrens A, Stehle T, Gross S, Aach T. Local and global panoramic imaging for fluorescence bladder endoscopy. In: 2009 Annual international conference of the IEEE engineering in medicine and biology society. IEEE; 2009. p. 6990–3.

    Google Scholar 

  39. Iakovidis DK, Spyrou E, Diamantis D. Efficient homography-based video visualization for wireless capsule endoscopy. In: 13th IEEE international conference on bioinformatics and bioengineering. IEEE; 2013. p. 1–4.

    Google Scholar 

  40. Szeliski R. Image alignment and stitching: a tutorial. Found Trends® Comput Graph Vis. 2006;2(1):1–104.

    Google Scholar 

  41. Maier-Hein L, Mountney P, Bartoli A, Elhawary H, Elson D, Groch A, et al. Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery. Med Image Anal. 2013;17(8):974–96.

    CAS  PubMed  Google Scholar 

  42. Münzer B, Schoeffmann K, Böszörmenyi L. Content-based processing and analysis of endoscopic images and videos: a survey. Multimed Tools Appl. 2018;77(1):1323–62.

    Google Scholar 

  43. Prados E, Faugeras O. Shape from shading. In: Handbook of mathematical models in computer vision. Springer; 2006. p. 375–88.

    Google Scholar 

  44. Wang R, Price T, Zhao Q, Frahm JM, Rosenman J, Pizer S. Improving 3D surface reconstruction from endoscopic video via fusion and refined reflectance modeling. In: Medical imaging 2017: image processing, vol. 10133. International Society for Optics and Photonics; 2017. p. 101330B.

    Google Scholar 

  45. Zhao Q, Price T, Pizer S, Niethammer M, Alterovitz R, Rosenman J. The endoscopogram: a 3D model reconstructed from endoscopic video frames. In: International conference on medical image computing and computer-assisted intervention. Springer; 2016. p. 439–47.

    Google Scholar 

  46. Phan TB, Trinh DH, Wolf D, Daul C. Optical flow-based structure-from-motion for the reconstruction of epithelial surfaces. Pattern Recogn. 2020;105:107391.

    Google Scholar 

  47. Park J, Hwang Y, Yoon JH, Park MG, Kim J, Lim YJ, et al. Recent development of computer vision technology to improve capsule endoscopy. Clin Endosc. 2019;52(4):328.

    PubMed  PubMed Central  Google Scholar 

  48. Qiu L, Ren H. Endoscope navigation with SLAM-based registration to computed tomography for transoral surgery. Int J Intell Robot Appl. 2020;4(2):252–63.

    Google Scholar 

  49. Turan M, Pilavci YY, Ganiyusufoglu I, Araujo H, Konukoglu E, Sitti M. Sparse-then-dense alignment-based 3D map reconstruction method for endoscopic capsule robots. Mach Vis Appl. 2018;29(2):345–59.

    Google Scholar 

  50. Yoon JH, Park MG, Hwang Y, Yoon KJ. Learning depth from endoscopic images. In: 2019 International conference on 3D vision. IEEE; 2019. p. 126–34.

    Google Scholar 

  51. Mahmood F, Durr NJ. Deep learning and conditional random fields-based depth estimation and topographical reconstruction from conventional endoscopy. Med Image Anal. 2018;48:230–43.

    PubMed  Google Scholar 

  52. Rau A, Edwards PE, Ahmad OF, Riordan P, Janatka M, Lovat LB, et al. Implicit domain adaptation with conditional generative adversarial networks for depth prediction in endoscopy. Int J Comput Assist Radiol Surg. 2019;14(7):1167–76.

    PubMed  PubMed Central  Google Scholar 

  53. DeTone D, Malisiewicz T, Rabinovich A. Deep image homography estimation. arXiv preprint arXiv:160603798. 2016.

    Google Scholar 

  54. Gomes S, Valério MT, Salgado M, Oliveira HP, Cunha A. Unsupervised neural network for homography estimation in capsule endoscopy frames. Procedia Comput Sci. 2019;164:602–9.

    Google Scholar 

  55. Bano S, Vasconcelos F, Tella-Amo M, Dwyer G, Gruijthuijsen C, Vander Poorten E, et al. Deep learning-based fetoscopic mosaicking for field-of-view expansion. Int J Comput Assist Radiol Surg. 2020;15:1807–16.

    PubMed  PubMed Central  Google Scholar 

  56. Tokuyasu T, Okamura W, Kusano T, Inomata M, Shiraishi N, Kitanou S. Training system for endoscopic surgery by using augmented reality and forceps control devices. In: 2014 Ninth international conference on broadband and wireless computing, communication and applications. IEEE; 2014. p. 541–4.

    Google Scholar 

  57. Bhushan S, Anandasabapathy S, Shukla R. Use of augmented reality and virtual reality technologies in endoscopic training. Clin Gastroenterol Hepatol. 2018;16(11):1688–91.

    PubMed  Google Scholar 

  58. Mahmud N, Cohen J, Tsourides K, Berzin TM. Computer vision and augmented reality in gastrointestinal endoscopy. Gastroenterol Report. 2015;3(3):179–84.

    Google Scholar 

  59. Shuhaiber JH. Augmented reality in surgery. Arch Surg. 2004;139(2):170–4.

    PubMed  Google Scholar 

  60. Bertrand LR, Abdallah M, Espinel Y, Calvet L, Pereira B, Ozgur E, et al. A case series study of augmented reality in laparoscopic liver resection with a deformable preoperative model. Surg Endosc. 2020;34:5642. https://doi.org/10.1007/s00464-020-07815-x.

    Article  PubMed  Google Scholar 

  61. Hussain R, Lalande A, Marroquin R, Guigou C, Grayeli AB. Video-based augmented reality combining CT-scan and instrument position data to microscope view in middle ear surgery. Sci Rep. 2020;10(1):1–11.

    Google Scholar 

  62. Yoon JW, Chen RE, Kim EJ, Akinduro OO, Kerezoudis P, Han PK, et al. Augmented reality for the surgeon: systematic review. Int J Med Robot Comput Assist Surg. 2018;14(4):e1914.

    Google Scholar 

  63. Al Janabi HF, Aydin A, Palaneer S, Macchione N, Al-Jabir A, Khan MS, et al. Effectiveness of the HoloLens mixed-reality headset in minimally invasive surgery: a simulation-based feasibility study. Surg Endosc. 2020;34(3):1143–9.

    PubMed  Google Scholar 

  64. Qian L, Deguet A, Kazanzides P. ARssist: augmented reality on a head-mounted display for the first assistant in robotic surgery. Healthc Technol Lett. 2018;5(5):194–200.

    PubMed  PubMed Central  Google Scholar 

  65. Neumann H, Bisschops R. Artificial intelligence and the future of endoscopy. Dig Endosc. 2019;31(4):389–90.

    PubMed  Google Scholar 

  66. Namikawa K, Hirasawa T, Yoshio T, Fujisaki J, Ozawa T, Ishihara S, et al. Utilizing artificial intelligence in endoscopy: a clinicians guide. Expert Rev Gastroenterol Hepatol. 2020;14:689.

    CAS  PubMed  Google Scholar 

  67. Bilal M, Brown JRG, Berzin TM. Incorporating standardised reporting guidelines in clinical trials of artificial intelligence in gastrointestinal endoscopy. Lancet Gastroenterol Hepatol. 2020;5:962.

    PubMed  Google Scholar 

  68. Li Y, Richter F, Lu J, Funk EK, Orosco RK, Zhu J, et al. SuPer: a surgical perception framework for endoscopic tissue manipulation with surgical robotics. IEEE Robot Automat Lett. 2020;5(2):2294–301.

    Google Scholar 

  69. Turan M, Almalioglu Y, Araujo H, Konukoglu E, Sitti M. A non-rigid map fusion-based direct SLAM method for endoscopic capsule robots. Int J Intell Robot Appl. 2017;1(4):399–409.

    PubMed  PubMed Central  Google Scholar 

  70. Shimada S, Golyanik V, Theobalt C, Stricker D. IsMo-GAN: adversarial learning for monocular non-rigid 3D reconstruction. In: IEEE conference on computer vision and pattern recognition workshops. 2019. p. 0–0.

    Google Scholar 

  71. Bozic A, Zollhofer M, Theobalt C, Nießner M. Deepdeform: learning non-rigid RGB-D reconstruction with semi-supervised data. In: IEEE/CVF conference on computer vision and pattern recognition. 2020. p. 7002–12.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics and Computer Science, University of Calabria, Rende, Italy

    Aldo Marzullo & Francesco Calimeri

  2. Department of Excellence in Robotics & AI, Scuola Superiore Sant’Anna, Pisa, Italy

    Sara Moccia

  3. Department of Electronics, Information and Bioengineering, Politecnico di Milano, Milan, Italy

    Elena De Momi

  4. The BioRobotics Institute, Scuola Superiore Sant’Anna, Pisa, Italy

    Sara Moccia

Authors
  1. Aldo Marzullo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sara Moccia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Francesco Calimeri
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Elena De Momi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aldo Marzullo .

Editor information

Editors and Affiliations

  1. Department of Women’s and Children’s Health, Karolinska Institutet, Stockholm, Sweden

    Niklas Lidströmer

  2. Imperial College London, London, UK

    Hutan Ashrafian

Electronic Supplementary Materials

(MP4 203757 kb)

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this entry

Check for updates. Verify currency and authenticity via CrossMark

Cite this entry

Marzullo, A., Moccia, S., Calimeri, F., De Momi, E. (2022). AIM in Endoscopy Procedures. In: Lidströmer, N., Ashrafian, H. (eds) Artificial Intelligence in Medicine. Springer, Cham. https://doi.org/10.1007/978-3-030-64573-1_164

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-64573-1_164

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-64572-4

  • Online ISBN: 978-3-030-64573-1

  • eBook Packages: MedicineReference Module Medicine

Publish with us

Policies and ethics

Search

Navigation