Abstract
Social Media Networks are one of the main platforms to express our feelings. The emotions we put in text tell a lot about our behavior towards any topic. Therefore, the analysis of text is a need for detecting one’s emotions in many fields. This paper introduces a deep learning model that classify sentiments from tweets using different types of word embeddings. The main component of our model is the Convolutional Neural Network (CNN) and the main used features are word embeddings. Trials are made on randomly initialized word embeddings and pretrained ones. The used pre-trained word embeddings are of different variants such as Word2Vec, Glove and fastText models. The model consists of three CNN streams that are concatenated and followed by a fully-connected layer. Each stream contains only one convolutional layer and one max-pooling layer. The model works on detecting positive and negative emotions from Stanford Twitter Sentiment (STS) dataset. The accuracy achieved is 78.5% when using the randomly initialized word embeddings and achieved a maximum accuracy 84.9% using Word2Vec word embeddings. The model not only proves that randomly initialized word embedding can achieve good accuracy, it also showing the power of the pretrained word embeddings that helps to achieve a higher competitive accuracy in sentiment classification.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Polignano, M., De Gemmis, M., Basile, P., Semeraro, G.: A comparison of word-embeddings in emotion detection from text using BiLSTM, CNN and Self-Attention. In: The 27th Conference on User Modeling, Adaptation and Personalization, pp. 63–68 (2019). https://doi.org/10.1145/3314183.3324983
Gaind, B., Syal, V., Padgalwar, S.: Emotion detection and analysis on social media. arXiv preprint arXiv:1901.08458 (2019)
De Choudhury, M., Gamon, M., Counts, S., Horvitz, E.: Predicting depression via social media. In: Seventh International AAAI Conference, pp. 128–137 (2013)
Deng, X., Li, Y., Weng, J., Zhang, J.: Feature selection for text classification: a review. Multimed. Tools Appl. 78(3), 3797–3816 (2019)
Cheng, H., Yang, X., Li, Z., Xiao, Y., Lin, Y.: Interpretable text classification using CNN and max-pooling. arXiv preprint arXiv:1910.11236 (2019)
Kim, Y., Jernite, Y., Sontag, D., Rush, A.M.: Character-aware neural language models. In: Thirtieth AAAI Conference on Artificial Intelligence (2016)
Lecun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)
Word embeddings and their use in sentence classification tasks. arXiv preprint arXiv:1610.08229 (2016)
Pennington, J., Socher, R., Manning, C.D.: GloVe: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space, pp. 1–12. arXiv preprint arXiv:1301.3781 (2013)
Bojanowski, P., Grave, E., Joulin, A., Mikolov, T.: Enriching word vectors with subword information . Trans. Assoc. Comput. Linguist. 5, 135–146 (2017)
Go, A., Bhayani, R., Huang, L.: Twitter sentiment classification using distant supervision. Processing 150(12), 1–6 (2009)
Pang, B., Lee, L., Vaithyanathan, S.: Thumbs up? Sentiment classification using machine learning techniques. In: Proceedings of ACL-02, Conference on Empirical Methods in Natural Language Processing, vol. 10, pp. 79–86. Association for Computational Linguistics (2002)
Parikh, R., Movassate, M.: Sentiment analysis of user-generated twitter updates using various classification techniques. CS224N Final Report, vol. 118 (2009).
Zhang, Y., Pezeshki, M., Brakel, P., Zhang, S., Bengio, C.L.Y., Courville, A.: Towards end-to-end speech recognition with deep convolutional neural networks. arXiv preprint arXiv:1701.02720 (2017)
Qian, Y., Woodland, P.C.: Very deep convolutional neural networks for robust speech recognition. In: IEEE Spoken Language Technology Workshop (SLT), San Diego, CA, USA, vol. 1, no. 16, pp. 481–488 (2016)
Purwins, H., Li, B., Virtanen, T., Schlüter, J., Chang, S.Y., Sainath, T.: Deep learning for audio signal processing. IEEE J. Sel. Top. Signal Process. 13(2), 206–219 (2019)
Ren, S., He, K., Girshick, R., Zhang, X., Sun, J.: Object detection networks on convolutional feature maps. IEEE Trans. Pattern Anal. Mach. Intell. 39(7), 1476–1481 (2016)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition, pp. 1–14. arXiv preprint arXiv:1409.1556 (2014)
Shang, R., He, J., Wang, J., Xu, K., Jiao, L., Stolkin, R.: Dense connection and depthwise separable convolution based CNN for polarimetric SAR image classification. Knowl. Based Syst. 105542 (2020)
Zhang, X., Zhao, J., LeCun, Y.: Character-level convolutional networks for text classification. In: Advances in Neural Information Processing Systems, pp. 649–657 (2015)
Kim, Y.: Convolutional Neural Networks for Sentence Classification. arXiv preprint arXiv:1408.5882 (2014)
Severyn, A., Moschitti, A.: UNITN: training deep convolutional neural network for twitter sentiment classification. In: Proceedings of the 9th International Workshop on Semantic Evaluation, pp. 464–469 (2015). https://doi.org/10.18653/v1/S15-2079
Kim, H., Jeong, Y.S.: Sentiment classification using Convolutional Neural Networks. Appl. Sci. 9(11), 2347 (2019)
Johnson, R., Zhang, T.: Semi-supervised convolutional neural networks for text categorization via region embedding. In: Advances in Neural Information Processing Systems, pp. 919–927 (2015)
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)
Tharwat, A.: Classification assessment methods. Appl. Comput. Inform. (2018). https://doi.org/10.1016/j.aci.2018.08.003
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Hamdi, E., Rady, S., Aref, M. (2021). A Deep Learning Architecture with Word Embeddings to Classify Sentiment in Twitter. In: Hassanien, A.E., Slowik, A., Snášel, V., El-Deeb, H., Tolba, F.M. (eds) Proceedings of the International Conference on Advanced Intelligent Systems and Informatics 2020. AISI 2020. Advances in Intelligent Systems and Computing, vol 1261. Springer, Cham. https://doi.org/10.1007/978-3-030-58669-0_10
Download citation
DOI: https://doi.org/10.1007/978-3-030-58669-0_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-58668-3
Online ISBN: 978-3-030-58669-0
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)