Abstract
With the availability of voice-enabled devices such as smartphones, mental health disorders such as depression could be detected and treated earlier, particularly post-pandemic. The current methods involve extracting features directly from audio signals. In this paper, two methods are used to enrich voice analysis for depression detection: the transformation of voice signals into a visibility graph and the natural language processing of the transcript text based on representational learning. The results of processing text and voice with different features are fused to produce final class labels. Experimental evaluation with the DAIC-WOZ dataset suggests that integrating text-based voice classification and learning from low-level and graph-based voice signal features can improve the detection of mental disorders like depression. Our text-based method has achieved %72.7 F1-score, which is higher than other single-modal scores. The fusion of all prediction models based on voice and text has resulted in %82.4 F1-score that outperforms other models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Amsalem, D., Dixon, L.B., Neria, Y.: The coronavirus disease 2019 (COVID-19) outbreak and mental health: current risks and recommended actions (2021)
Gong, Y., Poellabauer, C.: Topic modeling based multi-modal depression detection. In: AVEC 2017, pp. 69–76. Association for Computing Machinery, Inc (2017)
Sun, B., et al.: A random forest regression method with selected-text feature for depression assessment. In: AVEC 2017, pp. 61–68. Association for Computing Machinery, Inc (2017)
Toto, E., Tlachac, M.L., Stevens, F.L., Rundensteiner, E.A.: Audio-based depression screening using sliding window sub-clip pooling. In: Proceedings - 19th IEEE International Conference on Machine Learning and Applications, ICMLA 2020, pp. 791–796. IEEE (2020). https://doi.org/10.1109/ICMLA51294.2020.00129
Dubagunta, S.P., Vlasenko, B., Magimai-Doss, M.: Learning voice source related information for depression detection. In: ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 6525–6529. IEEE Inc (2019)
Alhanai, T., Ghassemi, M., Glass, J.: Detecting depression with audio/text sequence modeling of interviews. In: INTERSPEECH. September 2018, pp. 1716–1720 (2018)
Lam, G., Dongyan, H., Lin, W.: Context-aware deep learning for multi-modal depression detection. In: ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing – Proceedings, pp. 3946–3950. IEEE Inc (2019)
Ye, J., et al.: Multi-modal depression detection based on emotional audio and evaluation text. J. Affect. Disord. 295, 904–913 (2021)
Yela, D.F., Stowell, D., Sandler, M.: Spectral visibility graphs: application to similarity of harmonic signals. In: European Signal Processing Conference. September 2019 (2019)
Peng, Z., Hu, Q., Dang, J.: Multi-kernel SVM based depression recognition using social media data. Int. J. Mach. Learn. Cybern. 10(1), 43–57 (2017). https://doi.org/10.1007/s13042-017-0697-1
Skaik, R., Inkpen, D.: Using social media for mental health surveillance: a review. ACM Comput. Surv. 53, 1–31 (2021)
Lin, C., et al.: SenseMood: depression detection on social media. In: ICMR 2020 - Proceedings of the 2020 International Conference on Multimedia Retrieval, pp. 407–411. ACM, Inc (2020)
Ma, X., Yang, H., Chen, Q., Huang, D., Wang, Y.: DepAudioNet: an efficient deep model for audio based depression classification. In: AVEC 2016, pp. 35–42. ACM, Inc (2016)
Tlachac, M.L., Sargent, A., Toto, E., Paffenroth, R., Rundensteiner, E.: Topological data analysis to engineer features from audio signals for depression detection. In: ICMLA 2020, pp. 302–307. IEEE Inc (2020)
Zhang, L., Duvvuri, R., Chandra, K.K.L., Nguyen, T., Ghomi, R.H.: Automated voice biomarkers for depression symptoms using an online cross-sectional data collection initiative. Depress. Anxiety. 37, 657–669 (2020)
Bailey, A., Plumbley, M.D.: Gender bias in depression detection using audio features. In: EUSIPCO 2021. University of Surrey (2020)
Shin, D., et al.: Detection of minor and major depression through voice as a biomarker using machine learning. J. Clin. Med. 10, 3046 (2021)
Williamson, J.R., et al.: Detecting depression using vocal, facial and semantic communication cues. In: AVEC 2016, pp. 11–18. Association for Computing Machinery, Inc (2016)
Yang, L., Jiang, D., Sahli, H.: integrating deep and shallow models for multi-modal depression analysis-hybrid architectures. IEEE Trans. Aff Com 12, 239–253 (2021)
Lacasa, L., Luque, B., Ballesteros, F., Luque, J., Nuño, J.C.: From time series to complex networks: The visibility graph. Natl. Acad. Sci. 105, 4972–4975 (2008)
Zhang, Z., Qin, Y., Jia, L., Chen, X.: Visibility graph feature model of vibration signals: a novel bearing fault diagnosis approach. Materials (Basel), 11, 2262 (2018)
Eyben, F., Wöllmer, M., Schuller, B.: OpenSMILE - the Munich versatile and fast open-source audio feature extractor. In: MM’10 - Proceedings of the ACM Multimedia 2010 International Conference, pp. 1459–1462 (2010)
Huang, K., Altosaar, J., Ranganath, R.: ClinicalBERT: modeling clinical notes and predicting hospital readmission (2019)
Gratch, J., et al..: The distress analysis interview corpus of human and computer interviews. In: LREC 2014, pp. 3123–3128 (2014)
DeVault, D., et al.: SimSensei Kiosk: a virtual human interviewer for healthcare decision support. In: AAMAS 2014, pp. 1061–1068 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Ghadiri, N., Samani, R., Shahrokh, F. (2023). Integration of Text and Graph-Based Features for Depression Detection Using Visibility Graph. In: Abraham, A., Pllana, S., Casalino, G., Ma, K., Bajaj, A. (eds) Intelligent Systems Design and Applications. ISDA 2022. Lecture Notes in Networks and Systems, vol 646. Springer, Cham. https://doi.org/10.1007/978-3-031-27440-4_32
Download citation
DOI: https://doi.org/10.1007/978-3-031-27440-4_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-27439-8
Online ISBN: 978-3-031-27440-4
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)