Abstract
Social media sites such as YouTube and Facebook have become an integral part of everyone’s life, and in the last few years, hate speech in the social media comment section has increased rapidly. Detection of hate speech on social media Web sites faces a variety of challenges including small imbalanced datasets, the finding of an appropriate model and also the choice of feature analysis method. Furthermore, this problem is more severe for the Bengali speaking community due to the lack of gold standard labeled datasets. This paper presents a new dataset of 30,000 user comments tagged by crowdsourcing and verified by expert. All the user comments collected from YouTube and Facebook comment section and classified into seven categories: sports, entertainment, religion, politics, crime, celebrity and TikTok and meme. A total of 50 annotators annotated each comment three times, and the majority vote was taken as the final annotation. Nevertheless, we have conducted baseline experiments and several deep learning models along with extensive pretrained Bengali word embedding such as Word2Vec, FastText and BengFastText on this dataset to facilitate future research opportunities. The experiment illustrated that although all the deep learning model performed well, SVM achieved the best result with 87.5% accuracy. Our core contribution is to make this benchmark dataset available and accessible to facilitate further research in the field of Bengali hate speech detection.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Awal MA, Rahman MS, Rabbi J (2018) Detecting abusive comments in discussion threads using Naïve Bayes. In: 2018 international conference on innovations in science, engineering and technology. ICISET, pp 163–167. https://doi.org/10.1109/ICISET.2018.8745565
Banik N (2019) Toxicity detection on Bengali social media comments using supervised models. https://doi.org/10.13140/RG.2.2.22214.01608
Baruah A, Das K, Barbhuiya F, Dey K (2020) Aggression identification in English, Hindi and Bangla text using bert, roberta and svm. In: Proceedings of the second workshop on trolling, aggression and cyberbullying, pp 76–82
Ben-David A, Fernández AM (2016) Hate speech and covert discrimination on social media: Monitoring the facebook pages of extreme-right political parties in spain. International Journal of Communication 10:27
Chakraborty P, Seddiqui MH (2019) Threat and abusive language detection on social media in Bengali language. In: 1st international conference on advances in science, engineering and robotics technology 2019, ICASERT 2019. ICASERT, pp 1–6. https://doi.org/10.1109/ICASERT.2019.8934609
Chakravarthi BR, Arcan M, McCrae JP (2018) Improving wordnets for under-resourced languages using machine translation. In: Proceedings of the 9th global wordnet conference (GWC 2018), p 78
Davidson T, Warmsley D, Macy M, Weber I (2017) Automated hate speech detection and the problem of offensive language. In: Proceedings of the 11th international conference on web and social media, ICWSM 2017 (ICWSM), pp 512–515
Emon EA, Rahman S, Banarjee J, Das AK, Mittra T (2019) A deep learning approach to detect abusive Bengali text. In: 2019 7th international conference on smart computing and communications. ICSCC, pp 1–5. https://doi.org/10.1109/ICSCC.2019.8843606
Farha IA, Magdy W (2020) Multitask learning for Arabic offensive language and hate-speech detection. In: Proceedings of the 4th workshop on open-source Arabic corpora and processing tools, with a shared task on offensive language detection, pp 86–90
Gambäck B, Sikdar UK (2017) Using convolutional neural networks to classify hate-speech (7491), 85–90. https://doi.org/10.18653/v1/w17-3013
Hearst MA, Dumais ST, Osuna E, Platt J, Scholkopf B (1998) Support vector machines. IEEE Intel Syst Appl 13(4):18–28
Ishmam AM, Sharmin S (2019) Hateful speech detection in public facebook pages for the Bengali language. In: Proceedings—18th IEEE international conference on machine learning and applications, ICMLA 2019, pp 555–560. https://doi.org/10.1109/ICMLA.2019.00104
Joulin A, Grave E, Bojanowski P, Douze M, Jégou H, Mikolov T (2016) Fasttext.zip: compressing text classification models. arXiv:1612.03651
Joulin A, Grave E, Bojanowski P, Mikolov T (2017) Bag of tricks for efficient text classification. In: 15th conference of the European chapter of the association for computational linguistics, EACL 2017 Proc Conf 2(2), 427–431. https://doi.org/10.18653/v1/e17-2068
Karim MR, Chakravarthi BR, McCrae JP Cochez M (2020) Classification benchmarks for under-resourced Bengali language based on multichannel convolutional-LSTM network. http://arxiv.org/abs/2004.07807
Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J (2013) Distributed representations of words and phrases and their compositionality. In: Advances in neural information processing systems, pp 3111–3119
Mondal M, Silva LA, Benevenuto F (2017) A measurement study of hate speech in social media. In: Proceedings of the 28th ACM conference on hypertext and social media, pp 85–94
Rezaul Karim M, Raja Chakravarthi B, Arcan M, McCrae JP, Cochez M (2020) Classification benchmarks for under-resourced Bengali language based on multichannel convolutional-lstm network, pp arXiv-2004
Sambasivan N, Batool A, Ahmed N, Matthews T, Thomas K, Gaytán-Lugo LS, Nemer D, Bursztein E, Churchill E, Consolvo S (2019) They don’t leave us alone anywhere we go, gender and digital abuse in south Asia. In: Proceedings of the 2019 CHI conference on human factors in computing systems, pp 1–14
Schmidt A, Wiegand M (2017) A survey on hate speech detection using natural language processing. In: Proceedings of the fifth international workshop on natural language processing for social media, pp 1–10
Waseem Z (2016) Are you a racist or am I seeing things? annotator influence on hate speech detection on twitter, pp 138–142. https://doi.org/10.18653/v1/w16-5618
Zhang Z, Luo L (2019) Hate speech detection: a solved problem? The challenging case of long tail on Twitter. Semantic Web 10(5):925–945. https://doi.org/10.3233/SW-180338
Acknowledgements
This work would not have been possible without the kind support from SUST NLP Research Group and SUST Research Center. We would also like to express our hearfealt gratitude to all the annotators and volunteers who made the journey possible.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Romim, N., Ahmed, M., Talukder, H., Saiful Islam, M. (2021). Hate Speech Detection in the Bengali Language: A Dataset and Its Baseline Evaluation. In: Uddin, M.S., Bansal, J.C. (eds) Proceedings of International Joint Conference on Advances in Computational Intelligence. Algorithms for Intelligent Systems. Springer, Singapore. https://doi.org/10.1007/978-981-16-0586-4_37
Download citation
DOI: https://doi.org/10.1007/978-981-16-0586-4_37
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-0585-7
Online ISBN: 978-981-16-0586-4
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)