Trends on Sentiment Analysis over Social Networks: Pre-processing Ramifications, Stand-Alone Classifiers and Ensemble Averaging

Abstract

Technology advancements gave birth to social networks during the last decade. Many people tend to increasingly use them in order to share their personal opinion on current topics of their everyday life as well as express their emotions about situations in which they are interested. Hence, the emotions that are expressed in social networks can be positive, negative or neutral. To this direction, the analysis of people’s sentiments has drawn the attention of many scientists worldwide and offers a fertile ground for increasing research. Furthermore, a social network that is adopted by an ever growing percentage of people is Twitter. Twitter is an online news and social networking service where users post and interact with messages; such messages conceal people’s feelings and sentiments. Therefore, Twitter can be seen as a source of information and holds a vast amount of data that can be exploited for sentiment analysis research. In view of above, the purpose of this paper is to provide a guideline for the decision of optimal pre-processing techniques and classifiers for sentiment analysis over Twitter. In this context, three well-known Twitter datasets (OMD, HCR and STS-Gold) were used and a set of experiments was conducted. In particular, firstly, an extended comparison of sentiment polarity classification methods for Twitter text and the role of text preprocessing in sentiment analysis are discussed in depth. Secondly, four well-known learning-based classifiers (Naive Bayes, Support Vector Machine, k-Nearest Neighbors and C4.5) have been evaluated based on confusion matrices. Thirdly, the most common ensemble methods (Bagging, Boosting, Stacking and Voting) are examined and compared to base classifiers’ results. Finally, a case study concerning the application of Twitter sentiment analysis in an e-learning context is presented. The main result of the utilization of the Twitter-based learning application is that the exploitation of students’ emotional states can be used to enhance adaptivity in the learning content as well as deliver recommendations about activities and provide personalized assistance. Concerning data pre-processing, the experimental results demonstrate that feature selection and representation can affect the classification performance positively. Regarding the selection of the proper classifier, the superiority of Naive Bayes and Support Vector Machine, regardless of datasets, is proved, while the use of ensembles of multiple base classifiers can improve the accuracy of Twitter sentiment analysis.