Abstract
Analyzing stock trends based on the sentiment of social media provides a novel direction for investors to analyze the stock market. Behavioral financial theory and social psychology indicate that irrational behaviors related to financial decisions could result in stock fluctuations. Taking representative 20 stocks on Shanghai Stock Exchange as an example, user generated contents from January 31, 2017 to January 31, 2019 are obtained from Sina and Fortune.com. TF-IDF and TextRank algorithms are applied to extract keywords, based on which 2000-word-level financial sentiment lexicon is generated. In addition, the LSTM model is built and 23,152 comments were analyzed based on the lexicon. Eventually, relationships between sentiment scores and the trend of stock fluctuation are explored by applying the correlation coefficient parameter and Apriori algorithm. Results show that LSTM has a great advantage in sentiment analysis, which presents a higher accuracy (99.87%) than the sentiment lexicon-based method (94.57%). Taking the delay impact of stockholders’ sentiments on the stock trend into account, this research discusses the correlation between current investor sentiments and stock markets in the next few days. The paper finds that current emotional tendency has a deeper influence on the stock trend at the third day afterwards. Thus, this study extends financial sentiment lexicons, explores applications of LSTM machine learning in financial fields, and discusses the influence of investor sentiments on the stock market based on social media platforms. Processes of Web crawling, keyword extraction, sentiment analysis, correlation analysis and result visualization are coded in Python programming language, code packages are contributed through the Github website.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Li, D., Wang, Y., Madden, A., Ding, Y., Tang, J., Sun, G.G., Zhang, N., Zhou, E.: Analyzing stock market trends using social media user moods and social influence. J. Assoc. Inf. Sci. Technol. 70(9), 1000–1013 (2019)
Aggarwal, U., Saxena, A., Herald, S.: Artificial intelligence review in stock markets. Int. J. Res. Eng. Sci. Manag. 2(11), 92–95 (2019)
Vachhani, H., Obiadat, M.S., Thakkar, A., Shah, V., Sojitra, R., Bhatia, J., Tanwar, S. : Machine learning based stock market analysis: a short survey. In: International Conference on Innovative Data Communication Technologies and Application, pp. 12–26. Springer, Cham (2019)
Rossi, M., Gunardi, A.: Efficient market hypothesis and stock market anomalies: empirical evidence in four European countries. J. Appl. Bus. Res. (JABR) 34(1), 183–192 (2018)
Kumar, H., Jawa, R.: Efficient market hypothesis and calendar effects: empirical evidences from the Indian stock markets. Bus. Analyst 37(2), 145–160 (2017)
Shah, D., Isah, H., Zulkernine, F.: Stock market analysis: a review and taxonomy of prediction techniques. Int. J. Finan. Stud. 7(2), 26 (2019)
Dash, M.: Testing the random walk hypothesis in the Indian stock market using ARIMA modelling. J. Appl. Manag. Investments 8(2), 71–77 (2019)
Nasr, N., Farhadi Sartangi, M., Madahi, Z.: A fuzzy random walk technique to forecasting volatility of Iran stock exchange index. Adv. Math. Finan. Appl. 4(1), 15–30 (2019)
Shaik, M., Maheswaran, S.: Random walk in emerging Asian stock markets. Int. J. Econ. Finan. 9(1), 20–31 (2017)
Liu, B., Zhang, L.: A survey of opinion mining and sentiment analysis. In: Mining Text Data, pp. 415–463. Springer, Berlin (2012)
Pang, B., Lee, L.: Opinion Mining and Sentiment Analysis (Foundations and Trends (R) in Information Retrieval). Now Publishers Inc. (2008)
Schumaker, R.P., Chen, H.: A quantitative stock prediction system based on financial news. Inf. Process. Manag. 45(5), 571–583 (2009)
Schumaker, R.P., Chen, H.: Textual analysis of stock market prediction using breaking financial news: the AZFin text system. ACM Trans. Inf. Syst. (TOIS) 27(2), 12 (2009)
Bollen, J., Mao, H., Zeng, X.: Twitter mood predicts the stock market. J. Comput. Sci. 2(1), 1–8 (2011)
Mittal, A., Goel, A.: Stock Prediction Using Twitter Sentiment Analysis. Standford University, CS229 (2012). Available online. http://cs229.stanford.edu/proj2011/GoelMittalStockMarketPredictionUsingTwitterSentimentAnalysis.pdf. Cited 23 June 2021
Lee, H., Surdeanu, M., MacCartney, B., Jurafsky, D.: On the importance of text analysis for stock price prediction. In: The 9th International Conference on Language Resources and Evaluation. LREC 2014, pp. 26–31. Reykjavik, Iceland (2014)
Kalyanaraman, V., Kazi, S., Tondulkar, R., Oswal, S.: Sentiment analysis on news articles for stocks. In: The 2014 8th Asia Modelling Symposium (AMS), pp. 23–25. Taipei, Taiwan (2014)
Cakra, Y.E., Trisedya, B.D.: Stock price prediction using linear regression based on sentiment analysis. In: The 2015 International Conference on Advanced Computer Science and Information Systems (ICACSIS), pp. 10–11. Depok, Indonesia (2015)
Gao, Z., Feng, A., Song, X., Wu, X.: Target-dependent sentiment classification with BERT. IEEE Access 7(1), 154290–154299 (2019)
Pagolu, V.S., Reddy, K.N., Panda, G., Majhi, B.: Sentiment analysis of Twitter data for predicting stock market movements. In: The 2016 International Conference on Signal Processing, Communication, Power and Embedded System (SCOPES), pp. 3–5. Paralakhemundi, India (2016)
Xu, Y., Cohen, S.B.: Stock movement prediction from tweets and historical prices. In: The 56th Annual Meeting of the Association for Computational Linguistics, pp. 15–20. Melbourne, Australia (2018)
Mohammed, M., Omar, N.: Question classification based on Bloom’s taxonomy cognitive domain using modified TF-IDF and word2vec. PloS One 15(3) (2020). https://doi.org/10.1371/journal.pone.0230442
Kazemi, A., Pérez-Rosas, V., Mihalcea, R.: Biased: TextRank: Unsupervised Graph-Based Content Extraction (2020). arXiv preprint arXiv:2011.01026. Available online. https://arxiv.org/pdf/2011.01026.pdf. Cited 23 June 2021
Ombabi, A.H., Ouarda, W., Alimi, A.M.: Deep learning CNN-LSTM framework for Arabic sentiment analysis using textual information shared in social networks. Soc. Network Anal. Min. 10(1), 1–13 (2020)
Beckman, M.D., Çetinkaya-Rundel, M., Horton, N.J., Rundel, C.W., Sullivan, A.J., Tackett, M.: Implementing version control with Git and GitHub as a learning objective in statistics and data science courses. J. Stat. Educ. 29(Sup 1), 1–35 (2020)
Bo, Y., Liu, Y., Li, H.: Sentiment classification in Chinese microblogs: lexicon-based and learning-based approaches. Int. Proc. Econ. Dev. Res. 68(1), 1–5 (2013)
Fulian, Y., Wang, Y., Liu, J., Lin, L.: The construction of sentiment lexicon based on context-dependent part-of-speech chunks for semantic disambiguation. IEEE Access 8(1), 63359–63367 (2020)
Acknowledgements
This work is in part supported by the national key research project [YFE0101000], 2020 Key Technology R&D Program of GuangDong Province ZH01110405180056PWC] and Zhuhai Technology and Research Foundation [TC200802D4]. Thanks for the funding of mentioned projects.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Wang, H. et al. (2021). The Effect of Online Investor Sentiment on Stock Movements: An LSTM Approach. In: Lee, R. (eds) Computer and Information Science 2021—Summer . ICIS 2021. Studies in Computational Intelligence, vol 985. Springer, Cham. https://doi.org/10.1007/978-3-030-79474-3_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-79474-3_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-79473-6
Online ISBN: 978-3-030-79474-3
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)