Skip to main content
SpringerLink
Log in

A Survey of Deep Learning Based Natural Language Processing in Smart Healthcare

  • Conference paper
  • First Online:
Advances in Machine Intelligence and Computer Science Applications (ICMICSA 2022)

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 656))

  • 255 Accesses

Abstract

Natural language processing (NLP) is the subfield of artificial intelligence that has the potential to make human language analyzable by computers. NLP is increasingly proving its importance in the medical field where a huge amount of data remains unstructured (free text) stored as electronic medical records (EMR); discharge summaries, lab reports, clinical notes, pa-thology reports, etc. Traditional Machine learning (ML) based approaches have been widely used for medical NLP tasks, but these methods require a set of manual work and still suffer in terms of accuracy. However, deep learning (DL) based methods have made significant improvement. The main goal of this study is to present the state-of-the-art DL based NLP tech-niques in healthcare. We started by presenting word embedding techniques and popular deep learning models used in this area, and then reviewed ap-plications of NLP tasks in medical domain such as classification, predic-tion, and information extraction. We concluded our study with analyzing cited architectures and showing the promising results of CNN and BiLSTM and BERT fine-tuning.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Liu, Z., Lin, Y., Sun, M.: Representation learning and NLP. In: Representation Learning for Natural Language Processing, pp. 1–11. Springer, Singapore (2020). https://doi.org/10.1007/978-981-15-5573-2_1

    Chapter  Google Scholar 

  2. Liu, Z., Lin, Y., Sun, M.: Word representation. In: Representation Learning for Natural Language Processing, pp. 13–41. Springer, Singapore (2020). https://doi.org/10.1007/978-981-15-5573-2_2

    Chapter  Google Scholar 

  3. Plate, T.: Distributed Representations. Cognitive Science, pp. 1–15 (2003)

    Google Scholar 

  4. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv, arXiv:1301.3781 (2013)

  5. Banerjee, I., Chen, M.C., Lungren, M.P., Rubin, D.L.: Radiology report annotation using intelligent word embeddings: applied to multi-institutional chest CT cohort. J. Biomed. Inform. 77, 11–20 (2018). https://doi.org/10.1016/j.jbi.2017.11.012

    Article  Google Scholar 

  6. Soriano, I.M., Castro, J.L., Fernandez-Breis, J.T., Román, I.S., Barriuso, A.A., Baraza, D.G.: Snomed2Vec: Representation of SNOMED CT Terms with Word2Vec, pp. 678–83. IEEE Computer Society (2019). https://doi.org/10.1109/CBMS.2019.00138

  7. Pennington, J., Socher, R., Manning, C.D.: Glove: Global Vectors for Word Representation, vol. 14, 1532–1543 (2014). https://doi.org/10.3115/v1/D14-1162

  8. Kalyan, K.S., Sangeetha, S.: SECNLP: A Survey of Embeddings in Clinical Natural Language Processing. J. Biomed. Inform. 101, 103323 (2020). https://doi.org/10.1016/j.jbi.2019.103323

  9. Khattak, F.K., Jeblee, S., Pou-Prom, C., Abdalla, M., Meaney, C., Rudzicz, F.: A survey of word embeddings for clinical text. J. Biomed. Inf. 100(1-4) 100057 (2019). https://doi.org/10.1016/j.yjbinx.2019.100057

  10. Habib, M., Faris, M., Alomari, A., Faris, H.: AltibbiVec: a word embedding model for medical and health applications in the Arabic language. IEEE Access 9, 133875–88 (2021). https://doi.org/10.1109/ACCESS.2021.3115617

    Article  Google Scholar 

  11. Vaswani, A., et al.: Attention Is All You Need. arXiv (2017). https://doi.org/10.48550/arXiv.1706.03762

  12. Liu, Y., et al.: RoBERTa: a robustly optimized bert pretraining approach. arXiv (2019). https://doi.org/10.48550/arXiv.1907.11692

  13. Lee, J., et al.: BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinform. 36, btz682 (2019). https://doi.org/10.1093/bioinformatics/btz682

  14. Rasmy, L., Xiang, Y., Xie, Z., Tao, C., Zhi, D.: Med-BERT: PRetrained contextualized embeddings on large-scale structured electronic health records for disease prediction. npj Digit. Med. 4(1), 1–13 (2021). https://doi.org/10.1038/s41746-021-00455-y

  15. Huang, K., Altosaar, J., Ranganath, R.: ClinicalBERT: modeling clinical notes and predicting hospital readmission. ArXiv (2019)

    Google Scholar 

  16. Jia, Y., Kaul, C., Lawton, T., Murray-Smith, R., Habli, I.: Prediction of weaning from mechanical ventilation using convolutional neural networks. Artif. Intell. Med. 117, 102087 (2021). https://doi.org/10.1016/j.artmed.2021.102087

  17. Borjali, A., Magnéli, M., Shin, D., Malchau, H., Mu-ratoglu, O.K., Varadarajan, K.M.: Natural language processing with deep learning for medical adverse event detection from free-text medical narratives: a case study of detecting total hip replacement dislocation. Comput. Biol. Med. 129, 104140 (2021). https://doi.org/10.1016/j.compbiomed.2020.104140

  18. Guan, M., Cho, S., Petro, R., Zhang, W., Pasche, B., Topaloglu, U.: Natural language processing and recurrent network models for identifying genomic mutation-associated cancer treatment change from patient progress notes. JAMIA Open 2(1), 139–149 (2019). https://doi.org/10.1093/jamiaopen/ooy061

  19. Bayrak, S., Yucel, E., Takci, H.: Epilepsy radiology reports classification using deep learning networks (2022). https://doi.org/10.32604/cmc.2022.018742

  20. Chaib, R., Azizi, N., Schwab, D., Gasmi, I., Chaib, A.: GL-LSTM Model for multi label text classification of cardiovascular disease reports (2022). https://easychair.org/publications/preprint/BMRx

  21. Mao, C., Yao, L., Luo, Y.: AKI-BERT: a pre-trained clinical language model for early prediction of acute kidney injury. arXiv (2022). https://doi.org/10.48550/arXiv.2205.03695

  22. Liu, X., Chen, Y., Bae, J., Li, H., Johnston, J., Sanger, T.: Predicting heart failure readmission from clinical notes using deep learning. In: 2019 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 2642–48 (2019). https://doi.org/10.1109/BIBM47256.2019.8983095

  23. Schneider, E.T.R., et al.: BioBERTpt - a portuguese neural language model for clinical named entity recognition. In: Proceedings of the 3rd Clinical Natural Language Processing Workshop. Association for Computational Linguistics (2020). https://doi.org/10.18653/v1/2020.clinicalnlp-1.7

  24. Khalafi, S., Ghadiri, N., Moradi, M.: Hybrid deep learning methods for phenotype prediction from clinical notes. arXiv (2022). https://doi.org/10.48550/arXiv.2108.10682

  25. Mahajan, D., Liang, J.J., Tsou, C.-H.: Extracting daily dosage from medication instructions in EHRs: an automated approach and lessons learned. arXiv (2021). https://doi.org/10.48550/arXiv.2005.10899

  26. Jouffroy, J., Feldman, S., Lerner, I., Rance, B., Burgun, A., Neuraz, A.: MedExt: combining expert knowledge and deep learning for medication extraction from french clinical texts (Preprint) (2020). https://doi.org/10.2196/preprints.17934

  27. Shen, Y., Zhang, Q., Zhang, J., Huang, J., Lu, Y., Lei, K.: Improving medical short text classification with semantic expansion using word-cluster embedding (2018). https://doi.org/10.48550/arXiv.1812.01885

  28. Hsu, E., Malagaris, I., Kuo, Y.-F., Sultana, R., Roberts, K.: Deep learning-based NLP data pipeline for EHR scanned document information extraction. arXiv (2021). https://doi.org/10.48550/arXiv.2110.11864

  29. Olthof, A.W., et al.: Machine learning based natural language processing of radiology reports in Orthopaedic trauma. Comput. Methods Programs Biomed. 208, 106304 (2021). https://doi.org/10.1016/j.cmpb.2021.106304

  30. Ren, G.R., et al.: Differentiation of lumbar disc herniation and lumbar spinal stenosis using natural language processing-based machine learning based on positive symptoms. Neurosurg. Focus 52(4), E7 (2022). https://doi.org/10.3171/2022.1.FOCUS21561

    Article  Google Scholar 

  31. https://www.deeplearningbook.org/contents/rnn.html

  32. Kumar, A.: Different types of CNN architectures explained: examples. Data Analytics (blog) (2022). https://vitalflux.com/different-types-of-cnn-architectures-explained-examples

  33. Kumar, E.S., Jayadev, P.S.: Deep learning for clinical decision support systems: a review from the panorama of smart healthcare. In: Deep Learning Techniques for Biomedical and Health Informatics (2020) https://doi.org/10.1007/978-3-030-33966-1_5

  34. Sandeep Kumar, E., Satya Jayadev, P.: Deep learning for clinical decision support systems: a review from the panorama of smart healthcare. In: Dash, S., Acharya, B.R., Mittal, M., Abraham, A., Kelemen, A. (eds.) Deep Learning Techniques for Biomedical and Health Informatics. SBD, vol. 68, pp. 79–99. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-33966-1_5

    Chapter  Google Scholar 

  35. Adnan, K., Akbar, R., Khor, S.W., Ali, A.B.A.: Role and challenges of unstructured big data in healthcare. In: Sharma, N., Chakrabarti, A., Balas, V.E. (eds.) Data Management, Analytics and Innovation. AISC, vol. 1042, pp. 301–323. Springer, Singapore (2020). https://doi.org/10.1007/978-981-32-9949-8_22

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. ENSIAS, Mohammed V University in Rabat, Rabat, Morocco

    Zineb El M’hamdi & Mohamed Lazaar

  2. ENSA, Abdelmalek Essâadi University, Tetuan, Morocco

    Oussama Mahboub

Authors
  1. Zineb El M’hamdi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohamed Lazaar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Oussama Mahboub
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohamed Lazaar .

Editor information

Editors and Affiliations

  1. National School of Applied Sciences of Khouribga, Sultan Moulay Slimane University, Khouribga, Morocco

    Noureddine Aboutabit

  2. ENSIAS, Mohammed V University, Rabat, Morocco

    Mohamed Lazaar

  3. National School of Applied Sciences of Khouribga, Sultan Moulay Slimane University, Khouribga, Morocco

    Imad Hafidi

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

El M’hamdi, Z., Lazaar, M., Mahboub, O. (2023). A Survey of Deep Learning Based Natural Language Processing in Smart Healthcare. In: Aboutabit, N., Lazaar, M., Hafidi, I. (eds) Advances in Machine Intelligence and Computer Science Applications. ICMICSA 2022. Lecture Notes in Networks and Systems, vol 656. Springer, Cham. https://doi.org/10.1007/978-3-031-29313-9_9

Download citation

Publish with us

Policies and ethics

Search

Navigation