Abstract
The public opinion expressed on review or blogging sites and social networking platforms can be the source for the extraction of very critical information related to feelings and emotions of mass towards the subject matter in the field of commerce and governance. Natural Language Processing (NLP) and Artificial Intelligence can be used for sentiment analysis of this textual information. For text processing, NLP applications nowadays rely on pre-trained embeddings derived from large corpora such as news collection and web crawlers. There are many pre-trained word embeddings available. However, no study found which compares the accuracy achieved using these embeddings. In this paper, we worked on different kinds of word embeddings (pre-trained and untrained) and derived a comparison concerning accuracy for sentiment analysis applications using Deep Learning (DL) models. We found that the deep learning models perform better with pre-trained embeddings compared to Keras default (untrained) embedding.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
References
O. Kolchyna, T.T. Souza, P. Treleaven, T. Aste, Twitter sentiment analysis: Lexicon method, machine learning method and their combination. arXiv preprint arXiv:1507.00955 (2015)
P. Bojanowski, E. Grave, A. Joulin, T. Mikolov, Enriching word vectors with subword information. Trans. Assoc. Comput. Linguist. 5, 135–146 (2017)
A.L. Maas, R.E. Daly, P.T. Pham, D. Huang, A.Y. Ng, C. Potts, Learning word vectors for sentiment analysis, in Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-vol. 1 (Association for Computational Linguistics, 2011), pp. 142–150
R. Socher, A. Perelygin, J. Wu, J. Chuang, C. D. Manning, A. Ng, C. Potts, Recursive deep models for semantic compositionality over a sentiment treebank, in Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1631–1642 (2013)
T. Mikolov, I. Sutskever, K. Chen, G.S. Corrado, J. Dean, Distributed representations of words and phrases and their compositionality, in Advances in Neural Information Processing Systems, pp. 3111–3119 (2013)
J. Pennington, R. Socher, C. Manning, Glove: global vectors for word representation, in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)
T. Mikolov, E. Grave, P. Bojanowski, C. Puhrsch, A. Joulin, Advances in pre-training distributed word representations, in Proceedings of the International Conference on Language Resources and Evaluation (LREC 2018) (2018)
A. Hassan, A. Mahmood, Convolutional recurrent deep learning model for sentence classification. IEEE Access 6, 13949–13957 (2018)
Y. Bengio, R. Ducharme, P. Vincent, C. Jauvin, A neural probabilistic language model. J. Mach. Learn. Res. 3, 1137–1155 (2003)
T. Mikolov, K. Chen, G. Corrado, J. Dean, Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)
Y. Kim, Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882 (2014)
M. Sundermeyer, H. Ney, R. Schlüter, From feedforward to recurrent lstm neural networks for language modeling. IEEE/ACM Trans. Audio Speech Language Process. 23(3), 517–529 (2015)
S. Hochreiter, J. Schmidhuber, Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
X. Wang, W. Jiang, Z. Luo, Combination of convolutional and recurrent neural network for sentiment analysis of short texts, in Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pp. 2428–2437 (2016)
Y. Xiao, K. Cho, Efficient character-level document classification by combining convolution and recurrent layers. arXiv preprint arXiv:1602.00367 (2016)
Acknowledgements
The authors would like to acknowledge and thank NVIDIA for their support provided through the GPU grant for carrying out this research work.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Santosh Kumar, P., Yadav, R.B., Dhavale, S.V. (2021). A Comparison of Pre-trained Word Embeddings for Sentiment Analysis Using Deep Learning. In: Gupta, D., Khanna, A., Bhattacharyya, S., Hassanien, A.E., Anand, S., Jaiswal, A. (eds) International Conference on Innovative Computing and Communications. Advances in Intelligent Systems and Computing, vol 1165. Springer, Singapore. https://doi.org/10.1007/978-981-15-5113-0_41
Download citation
DOI: https://doi.org/10.1007/978-981-15-5113-0_41
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-5112-3
Online ISBN: 978-981-15-5113-0
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)