Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

Today, social media are emerging as a new platform for information exchange, discussions, and as a source of news. Many companies utilize social media platforms as marketing tools or for promotional purposes as a part of a marketing mix. Today, also online brokers provide online discussion forums, blogs, networking tools, educational material, or networking platforms as part of their marketing mix in order to directly reach the end consumer. Thus, today, social networks provide a pool of collaborative knowledge, which also allows the understanding of collaborative pricing behavior on stock markets. This includes e.g. specialized platforms providing collaborative knowledge around the topic of stock exchange trading via blogs, wikis, or communities. It’s obvious that social media therefore provide insights into market sentiments and investing behavior, due to the collaborative knowledge and ‘shared mind’ of many investors. Social media therefore might act as a market sentiment indicator far beyond the currently existing sentiments based e.g. on questionnaires or quantitative consumer spending indexes. The analysis of the content can provide more insights into behavioral finance on stock markets and might lead to more real-time and accurate sentiments. Existing examples show the relation e.g. between discussions on social media and movie sales. It’s obvious that social media based sentiment analysis provides new insights into the relation between stock exchange pricing and investors’ sentiments and eventually into new behavior financing models. Example services, as e.g. StockTwits demonstrate the efficiency of social media as a tool (StockTwits, n.d.).

Currently the trend of social media attempts to measure brand value, or market sentiments of movies or products. However, their impact as sentiment measurement tool for stock exchange trading is rather poorly researched or leads to completely irrational conclusions that twitter feeds can predict the stock market with an accuracy of 87 % (Bollen et al. 2011), a prediction success rate which even the most experienced analysts are dreaming of. Within the scope of this chapter, the potentials of social media as a tool for measuring investor sentiment are analyzed. However, within this text, investor sentiment is defined as “belief about future cash flows and investment risks that are not justified by the facts” (Baker and Wurgler 2007). Therefore a model of measuring sentiment should include variables for describing financial facts, perception of risk, and perception of potential future returns.

Fig. 1
figure 00331

Social media applied for the financial market [extended list from (Lombardi et al. 2011)]

Full size image

1.1 Potentials and Application Scenarios

The aim of this paper is to provide a framework for the potentials of applying social media to the wider scope of the stock exchange. It provides insights into how social media can be utilized as sentiment indicators to determine stock exchange pricing and price changes—and which models might exist to determine stock exchange behavior. The paper provides an overview of tools, methods, and implications in the potential usage of social media within the financial sector far beyond the currently existing probabilistic models assisting investors in their decision making process. In Fig. 1 the potentials of the usage of social media in conducts around financial markets is shown. The application of social media can be clustered into four distinct application areas: marketing and promotion; surveillance, monitoring, and analysis; investment support and visualization; and technical indicator for investment decisions. In the following section these application scenarios are enlisted. Many of the below mentioned scenarios have are mentioned in (Lombardi et al. 2011), but the list below is a more comprehensive list of possible application scenarios:

  • Marketing, Customer Management, and Promotion

    • Marketing and promotion of services from investors, brokers, or financial institutions to end-consumer or professionals;

    • Customer analysis and evaluation of products, services, and offers and effectiveness of marketing, sales, and provided information services;

  • Market Surveillance, Monitoring, and Analysis (Lombardi et al. 2011)

    • Market surveillance and analysis to gather insights, trends, forecasts, equity prices, volumes, money flows, and information;

    • Detections of events and happenings impacting portfolios, investment decisions, and financial strategies;

    • Risk management and monitoring of reputational risk to secure investments and monitor insider trading, anomalies, market habits, business relationships, and self-assessment;

  • Investment Support, Management, and Visualization (Lombardi et al. 2011)

    • Market monitoring and investment support tool for portfolio management, in-depth analysis, and crowed sourced knowledge not available otherwise;

    • Answering of investors questions or queries to gain knowledge and information about securities, market, scenario development, trends, and background;

    • Data visualization of textual data existing on the web to gain easily access to crowed sourced information and knowledge;

  • Technical Indicator for Investment Decisions

    • Sentiment indicators for stock exchange trading and investments identifying market events, market turns, correlations with equity prices, sentiment values, and portfolio management [extended from (Lombardi et al. 2011)]

1.2 Chapter Structure

This book chapter is divided into four sections. The first section “The Stock Exchange Environment Market Theories” gives a general overview of market theories especially focusing on behavioral economics. Theories in behavioral economics seem to match with the phenomenon of social media. The section also gives an outline of market theories, the general stock exchange environment, and key-measures of market valuation. The second section “Sentiment Analysis and Indicators on the Stock Exchange” explores sentiment indicators that are currently used for measuring investor sentiment on the stock exchange. The third section “Framework for Social Media in the Context of Stock Exchange Investments” provides an overview of social media, and introduces a very basic framework for applying social media at the stock exchange. It also shows the potentials and possibilities of social media for sentiment analysis. The fourth section, “Analyzing Social Media and Creating a Social Media Sentiment Indicator Platform” gives a more technical description of potential methods, algorithms, and other implementation details. The last section of the book “Discussion—How can Social Media Predict the Stock Market?” is a round-up chapter, which revises the potentials of social media. It includes also a discussion of the potential of social media under the aspect of how social media can be utilized to predict the stock market.

2 The Stock Exchange Environment and Market Theories

For many decades academic and investment theories focused on the meaningful predictability of stock exchange market prices to increase the probability of correct investment decisions—simply which models can be established to predict the stock market price? An outline of the development of these theories can be found in Fig. 2.

Fig. 2
figure 00332

History of stock exchange theories

Full size image

2.1 A Brief History of Theories to Explain the Stock Exchange

A very good and brief explanation for the history of theories explaining stock exchange behavior and its scholars can be found in (Lawrence et al. 2007), upon which this chapter and its references is based on. The introduction section of this article compiles in a few lines the most important scholars, theories, and thoughts. This section of the book chapter extends the basic description of (Lawrence et al. 2007) and provides a wider context. Before the 1950s many researchers believed in models that were capable of predicting stock markets, and in the fact that the market moves gradually, rather than immediately. One of the classical models describes the macroeconomic level of markets as (Keynes 2008), in which business cycles with over-production and under-production lead either to over-employment and under-employment—attempting to equilibrate the market and eventually requiring additional market intervention (Keynes 2008). Nevertheless, these theories describe a market on macroeconomic level and are of little help for specific decisions in stock exchange investments.

Between the 1950s and 1960s these theories were challenged. The believe was strengthened that stock market changes are random and behave in a more immediate way to new information and analysis. Main scholars during these time-period including Mandelbrot (1966) or Samuelson (1965). Fama (Fama and French 1988a; Fama 1970) introduced the Efficient Market Theory (EMT), which basically notes that stock market prices act according to rational behavior of investors. The EMT claims, that the prices of a stock exchange fully reflect the available market information at a certain point in time. The market is efficient and underlays rational principles. This theory was supported by other scholars such as Jensen (1978), Thaler (1999) and Malkiel (2012) argued with this statement. However, the theory was not long lasting and started to be scrutinized from the 1980s onwards, when scholars started to show that stock markets are by far less rational expected, especially when considering stock volatility, reactions on new information, market crashes, or under/overreactions on the stock exchange (Lawrence et al. 2007).

A few of the phenomena, that the stock exchange is by far not reacting rationally have been researched and are compiled in the following [as referenced and listed in (Lawrence et al. 2007)]:

  • Extreme losers outperform extreme winners after their low period (Bondt and Thaler 1985);

  • Under-reaction to news (Rouwenhorst 1998);

  • Overreactions to long streams of news in bullish/bearish direction (Fama and French 1988b);

  • Dividend yields predict the performance and variance (Fama and French 1988b);

  • Low price/earnings and/or price/book ratios outperform returns (Campbell and Shiller 1998);

  • Investors trade to market noise rather than hard facts (Black 1986);

  • Cognition and perception influence pricing (Neal and Wheatley 1998);

  • An increase/decrease in consumer confidence lead to bullish/bearish investments (Fisher and Statman 2002);

  • Theoretical explanations for anomalies as e.g. high trading volumes, bubbles, and volatility (Thaler 1999);

  • …

Thus, stock exchange prices seemed to follow other principles as well. In contrast to what pure rational pricing theorists were trying to underpin. Their research works lead to the emergence of behavioral finance theory, providing new insights into the way of thinking of how stock exchange prices react. Thus, market sentiment and individual emotions seem to play a more crucial role in the development of models describing the behavior of stock exchange prices.

As all the scholars mentioned above have stated, investor sentiment is a driving force in describing stock exchange anomalies and either individual or market emotions and biases. Thus sentiment based stock pricing seems to provide an answer especially to stock exchange anomalies as high volume, bursting bubbles, or volatility (Lawrence et al. 2007). However, even today very specific and probability based models assist investors in their decision making process. And one of the tools to explore the dependence between investor sentiment and the stock exchange are social media.

2.2 Revising the Year 2011 on the Example of the DAX Performance Index

This book chapter is based on examples of the DAX Performance Index 2011, which shows a major stock exchange crash in summer, with a loss of about 1/3 of its total value. Macroeconomic indicators, the financial crisis in Europe, and the Fukushima natural disaster in Japan are only partial explanations for the loss on such a high level. Thus, such a severe crash is not only explained by ‘rational’ investors, who identify realistic investment risks and realistic valuation of equities. Therefore also emotional decision making, and irrational investors led to such a sever stock exchange crash (Zouaoui et al. 2010)—the investor’s sentiment as an explanation for irrational behavior on investment decisions are not fully based on facts (De Long et al. 1990). However the notion that it’s better to follow the herd than going against the flow might cause high losses (Shleifer and Vishny 1997) maintains a self-proficiency—or a closed circuit. A decrease of sentiment seems to result into an eventually unjustified magnified decrease of returns, while an increase in sentiment is leading to an eventually unjustified magnified increase of returns.

The year 2011 represented a black year for several stock exchanges worldwide. Three events and crises led to a major crash of several stock exchanges in summer 2011, followed by another crash due to computerized trading systems. The following three major crises dominated the year: the explosion of the nuclear power plant Fukushima in Japan followed by a natural disaster; the European debt crisis caused by an almost default of Greece; and the debt crisis in the USA. Figure 3 illustrates the DAX 30 Performance Index in the year 2011 as a candle stick chart. It shows several events within the scope of its chart patterns: Fukushima in March 2011 and the stock exchange crash in late July/early August due to the European debt crisis that almost led to a Greek default. Within the charts several events are associated with high trading volumes.

Fig. 3
figure 00333

Daily DAX 30 Performance Index and its Returns for the 257 Trading Days in 2011. The DAX 30 Performance Index is depicted with its Moving Average (Period 10) as Candlestick Graph. The Figure includes the Trading Volume in Billions together with a Moving Average for a period of 5 to indicate Volume Trends. The Figure below the DAX 30 Performance Index presents the Return Series of the Performance Index, and the Changes of Trading Volume Series in Percentages

Full size image

Trading volumes represents the simplest indicator for investor sentiment, as it reflects the emotional involvement of market participants: buyers and sellers of stocks commit to their investment actions, either as losers or as winners of a trade. This correlation is described in Elder (1993). From a chart technical perspective high volumes indicate trend confirmations or potential retesting of market lows. Shrinking volumes indicate market turns of the current trends (Elder 1993). Extraordinary changes in volume can easily be spotted in Fig. 3: Fukushima showed high trading volumes, as the crash in summer 2011 does. However, the turn of the market at the end of 2011 is indicated with very low trading volumes, which is partially also due to the low investor activities during the Christmas season. Figure 3 as well shows the DAX 30 returns for the year 2011, and shows major losses in the range of almost up to 6 % on one trading day.

3 Sentiment Analysis and Indicators on the Stock Exchange

A general distinction can be made between explicit and implicit sentiment indicators to evaluate the expectations of the stock market. Explicit sentiment indicators imply a direct sentiment measurement e.g. via questionnaires or polls conducted on market participants. These measurements allow direct insights into investor’s sentiment and their current market behaviour. Examples are polls to gain insight knowledge on the current sentiment conducted by the stock exchange or economic research institutes as e.g. the Deutsche Börse (Boerse-Frankfurt, n.d.) or the Centre for European Economic Research (CEER) (Centre for European Economic Research, n.d.a). However, implicit sentiment indicators are indicators whereas other measurements or proxies lead to conclusions on the current market behaviour or sentiment. Methods deriving from statistics, technical chart analysis, or other economic methods allow to estimate and explain the current market behaviour. A prominent example of implicit sentiment is the volatility (Baker and Wurgler 2007). Volatility reveals information of the current possible price changes on the stock market. A high volatility indicates a highly uncertain investor environment, as e.g. a high volume ratio of short selling vs. long selling on the derivate market (Elder 1993).

However, the introduction of social media requires a benchmark or measurement of how potential social media indicators perform. Within the scope of this chapter several indicators combined with index metrics can be considered as an option for benchmarking eventual social media sentiment indicators. The total set of indicators can be seen as a list of sentiment indicators for the DAX 30. Table 1 compiles several indicators [extended from the indicators listed in Baker & Wurgler (2007) and Zouaoui et al. (2010)].

Table 1 Potential variables for analysing sentiment [extended and adapted set of indicators from Baker and Wurgler (2007) and Zouaoui et al. (2010) to suit the DAX 30 performance index]
Full size table

3.1 Explicit Sentiment Indicators

Finding a suitable quantitative measurement for investor sentiment over time is a rather tricky task. Market sentiment data is difficult to obtain, and fluctuation happens rapidly based on emerging news, daily happenings and long-term perspectives changes. Let’s consider signals from rating agencies, such as Fitch, Moodies, and S&P which have direct impact on immediate changes in equity prices and the attitude of investors in buying and selling. There are also many other examples for potential sentiment indicator proxies, as e.g. the retail investor sentiment. It is clear that each new event causes a chain of reactions on stock exchanges worldwide—starting with the first stock exchange that opens with the Nikkei in Japan. There are many examples for such a chain reaction in 2011, like the catastrophe in Fukushima, after which stock exchanges worldwide crashed. We might consider stock exchange returns as measurement for investor’s sentiment—however, as we would like to identify factors that impact on stock exchange returns, it’s a poor measurement for this purpose. Even though returns might be an easy quantifiable measure, they underlay also other principles and factors such as company performance. Returns reflect sentiment as outcome of investor sentiment and are a direct impact from changes in sentiment. Therefore it’s a rather poor indicator for stock exchange sentiment changes. Also common indicators such as surveys or market analysis from investment companies might not lead to an objective sentiment analysis, due to the fact, that this data is not sufficiently representative as well as biases and certain agendas can’t be ruled out.

However, a previous research work (Baker and Wurgler 2007) evaluated a set of possible proxies for measuring investor sentiment and developed a sentiment index based on the consideration of the following possible sentiment indexes [extended and excerpted from (Baker and Wurgler 2007)] and adapted to specific DAX Performance Index variables):

  • Surveys and Questionnaires: there are many examples for questionnaires and surveys on consumer, investor, and economic levels based on market surveys. These give insights into various market aspects, such as consumer sentiment, investor sentiment, or retail sentiment. For the German market the following indicators are of relevancy:

    • Börse Frankfurt Sentiment Index: an example for this type of index is the sentiment analysis contracted by the Börse Frankfurt, Xetra (Boerse-Frankfurt, n.d.). The index is collected on a weekly basis by Cognitrend (n.d.), and shall predict if DAX or TecDAX increases, decreases, or is neutral during the following 30 days based on the sentiment of 150 professional investors. It reflects the ratio of optimists (bulls) and pessimists (bears) on the stock exchange.

    • Economic Sentiment Index (ESI): an index representing sentiment from an economic perspective is the Economic Sentiment Index (ESI) published by Eurostat (n.d.), composed of indicators for consumer, industrial, service, construction, and retail sentiments. This index reveals a rather general view towards market sentiment, however, does not indicate the direct impacts on stock exchange prices.

    • ZEW Indicator: another important index for sentiment analysis is published by the Centre for European Economic Research (CEER) (Centre for European Economic Research, n.d.a) entitled ZEW Indicator for Economic Development (Centre for European Economic Research, n.d.b). The index is evaluated on a monthly basis by surveying 350 financial experts about the economy in Germany, total Europe, Japan, Great Britain, and the USA resulting into a ratio between optimists and pessimists as economic sentiment outlook for the next 6 months.

    • Ifo Business Climate Index: the Ifo (CESifo GmbH, n.d.) publishes a series of sentiment indexes, where the most significant is the Ifo Business Climate Index published on a monthly basis. However, the institute as well publishes a wide range of other indicators relevant for the German market (e.g. employment barometer, credit constraint indicators, investment surveys, manager survey) or indicators for the global market.

  • Newsletters and Market Journalist Opinion: there is a wide range of specialized media, newsletters, email distributors, or online forums that reflect current investor sentiment [see e.g. finanzen.net, (n.d.)]. However, professional investors argue that journalistic opinion on sentiment is rather ambiguous, as journalists ought to provide true insights requiring a high degree of interpretation. Also turns in opinion follow rather slowly in trend changing markets (Elder 1993).

  • Endogenous Sentiment Indexes: endogenous sentiments relate to events not directly related to market data which might impact the performance of investors such as football game results, seasons, or urban legends about the behaviour of stock exchanges at a certain constellation. It has been e.g. shown by (Edmans et al. 2007) that there might be an impact on stock exchange performance; however, the impact of these sentiment indicators can be considered as rather questionable. They might only be considered with a very low weight in the calculation of the total sentiment of investors or rather be ignored due to their statistical insignificance.

3.2 Implicit Sentiment Indicators

There is a wide set of tools available to measure implicit sentiment through proxies and technical analysis. Implicit sentiment variables are proxies that reflect the current investor sentiment and are based on measurements, chart analysis, and macroeconomic indicators among other quantifiable metrics. In the following the most prominent indicators are enlisted. The description of technical indicator is based on Stockcharts (n.d.), which is also an excellent resource for further investigation of technical indicators on the stock exchange.

  • Volatility: The most prominent indicator for sentiment on the stock exchange is the volatility indicator—the investor’s “panic gauge” (Elder 1993). In the case of Germany the VDAX-New is the volatility index for the DAX performance index. Volatility gives information on how risky the current market is for investments and moves inverse to the current stock market prices. A high volatility indicates a bearish and risky market, whereas a low volatility indicates a bullish market. However, volatility extremes often indicate excessive sentiments (Stockcharts n.d.). Figure 4 shows the VDAX-New for the year 2011 nicely reflecting the risk attitude at the stock market in summer 2011, and its recovery period in fall.

    Fig. 4
    figure 00334

    DAX volatility index for 2011

    Full size image

  • Call/Put Volume Ratio: The call/put (or long/short) (Scoach n.d.; Baker and Wurgler 2007; Stockcharts, n.d.) volume ratio of options and derivatives is a good indicator for market sentiment. A high amount of put (short) options indicate a declining market, whereas a high amount of call (long) options indicate an increasing market. The ratio is calculated by dividing “market optimism” (volume of buying calls plus sales of puts) through “market pessimism” (volume of buying puts plus sales of calls). Thus, the higher the number over 0.5, the more bullish market sentiment; whereas the lower the number under 0.5, the more bearish market sentiment. The indicator measures market emotions, as it reflects if investors fear a declining market by hedging their positions by puts to protect their equities, or if they speculate on a declining market. Figure 5 illustrates the call/put ratio for the year 2011, as published by Scoach on a daily basis (Scoach, n.d.) and shows the low call/put ratio especially during summer 2011 and an increasing number during the recovery period.

    Fig. 5
    figure 00335

    Call/Put Volume Ratio as published by Scoach for 2011 (Scoach, n.d.)

    Full size image

  • Trading Volume: Trading volume (Elder 1993) is a rather simple sentiment indicator reflecting investor emotions—depending on the market scenario, high or low volume indicates a change or continuation of a trend. From the investor sentiment point of view volume always reflects the “emotion” of the market as it represents a market action of buying or selling. Thus the money is transferred between winners and losers of the specific interaction (Elder 1993). Figure 3 shows e.g. nicely extraordinary high volumes indicating panic sales during the natural disaster in Japan and during the stock exchange crash in summer 2011.

  • Technical Indicators: There are several technical indicators deriving from chart analysis that indicate the current market state (thus either bull, bear, or neutral market). There is a wide range of further technical indicators that can be utilized as implicit sentiment indicators to indicate investor’s sentiment, as there is a wide range of further implicit sentiment proxies. However, this chapter contains the most utilized ones in studies. An excellent resource for exploring technical indicators can be found on (Stockcharts, n.d.), which acted as resource to explain these within the scope of this book chapter. The most prominent indicators utilized in research works examining sentiment are ARMS and RSI:

    • ARMS (TRIN) Indicator: the ARMS (or Short-term TRading INdex) was developed by Richard Arms in 1967. It is a very simple indicator that represents the relationship between the ratio of advancing/declining equity’s and the ratio of traded volume of advancers/decliners. The indicator reflects an inverse market movement. Values above 1 show a market decline and values below 1 indicate a rising of the market. Above or below equity specific thresholds the indicator reflects an overbought or oversold equity price [as defined in (Stockcharts, n.d.)].

    • Relative Strength Indicator (RSI): The RSI was developed by Welles Wilder in 1978 and measures the acceleration or deceleration of price movements. It ranges from 0 to 100 and a value above or below a certain threshold indicates an overbought or oversold equity price. Per default these values are set at above 70 and below 30 and the indicator is calculated for 14 periods. However, the threshold is as well equity specific, and can be utilized to reduce or increase the number of trading signals [as defined in (Stockcharts, n.d.)].

    • Other Market Indicators: There is a wide set of other market indicators that allow identifying a market trend that could be utilized: Commodity Channel Indicator (CCI), High-Low Index, Net New Highs, or the Bullish Percentage Index. As their description would go beyond the scope of this book chapter, it is referred to (Stockcharts, n.d.) for further investigation.

  • Market Turnover and Return: another well-suited sentiment indicator is the market return, describing daily returns on the stock exchange, as e.g. presented in Fig. 3. Market returns indicate the change of a stock exchange index, thus its performance. The turnover on a stock exchange indicates the total price of the total shares traded within a certain period. The turnover ratio gives information how many times equities are exchanged within a certain period (Sheu et al. 2009).

  • Open Interest: open interest (Eurex n.d.; Stockcharts, n.d.) is an indicator for the number of put and calls positions of options or futures expressed in the number of contracts or total price. The ratio between puts and calls is a well suited indicator for investor sentiment—the higher the amounts of puts, the more pessimistic investors are. Open interests are directly proportional to the level of interest in an option or future. Historical data is e.g. available at Eurex (n.d.).

  • Initial Public Offerings (IPO): the amount of IPOs (thus companies entering the stock market) (Baker and Wurgler 2007), and their first day returns are well suited sentiment indicators, as first day returns show the level of interest of investors in a particular equity. The interest level can be interpreted as directly proportional to the sentiment of investors (Baker and Wurgler 2007). Statistical data of IPOs in Germany is e.g. available at (Deutsche Boerse Group, n.d.). An excellent example for a failed IPO is Facebook, reflecting a negative sentiment towards the underlying asset.

  • Mutual Fund Flows: mutual fund flows (Baker and Wurgler 2007) are an indicator where investors are allocating their capital. A sentiment indicator based on mutual fund flows allows visualizing where capital is currently allocated and gives insights in which asset classes investors believe (Baker and Wurgler 2007).

3.3 Macroeconomic Sentiment Indicators

A further category of sentiment proxies are macroeconomic sentiment indicators. However, a full enumeration of these indicators would be beyond the scope of this book chapter. The most prominent indicators are: interest rate [as published by the ECB (European Central Bank, n.d.)], inflation, domestic credit, GDP, term spread, industrial investments, industrial production, growth of durable goods, etc.

4 Framework for Social Media in the Context of Stock Exchange Investments

The role of social media in stock exchange trading is based on theories of behavioural finance, and the investor’s sentiment reflected in social media. Figure 6 gives a general overview of the relationship for a framework including social media in stock exchange trading.

Fig. 6
figure 00336

Social media as support tool for stock exchange investments

Full size image

4.1 Social Media as Marketing Tool

The application of Social Media for online brokers and financial firms shows similarities to other market segments, especially when focusing on social media as a tool for marketing and promotion and the application of social media as a customer analysis tool. Social Media have increased influence on the consumer, especially on the young generation in their decision making in buying products. This is also valid for applying social media in the context of investments. Brokers can actively use social media market channels for branding, providing product advertisements, and getting in touch with the consumer. Thus the three main reasons to deploy social media tools can be compiled as follows (Vukanovic 2011):

  • Reduced costs for advertising, distribution of information, and targeted offers;

  • Low-cost possibility for brand building, consumer engagement, and communication;

  • Active and rapid engagement with peers, consumers, and the public;

  • Building knowledge, learning professional knowledge, and instant information.

While these application areas are currently under heavy investigation, another opportunity for the application of social media in the context of stock exchange pricing is provided—merging social media with the theories of behavioral finance.

4.2 Behavioral Finance and Social Media

Social media as sentiment indicator allow providing new insights into investor’s behavior, especially from a behavioral finance viewpoint. As the market is seen to reflect the real value of securities, the direct impact of social media based sentiment analysis of investors has to consider several variables. To establish the connection between stock exchange prices and the sentiment of investors a correlation to stock exchange prices has to be established. The following stock market behaviors and indicators can be utilized as basic measures:

  • General market knowledge: consumer confidence indicators (e.g., consumer indices), expected market or market segment growth rates, market environment, chart analysis and backtracking theories, and general investment principles;

  • Principle market state: bull market (increasing), bear market (decreasing), neutral market (equilibrium);

  • Anomalies: stock market bubbles, stock market crashes, unusual trading volumes, increased or high volatility;

  • Market perception and reaction: reactions of the market on news, new information, rumors, general believe, current market perception, filtering of news, and discussions;

  • Equity price: future cash flows, growth rate, discount rate, company data and their interpretation, investment risk rate, and fluctuation of these parameters over time.

4.3 Potential Forms of Social Media

As content source for the principle data analysis exist various different social media genres that adapted to the needs of investors and traders. A wide variety of tools is available either to support marketing and promotion, as well as they can act as source for the evaluation of market sentiments. The following enumeration gives an overview and was published in Lietsala & Sirkkunen (2008):

  • Collaborative networks and content creation: common creation and publishing of information, and knowledge (e.g. blogs, wikis, podcasts, twitter);

  • Content sharing and knowledge exchange: sharing of information with similar minded people on dedicated networks (e.g. Flickr);

  • Dedicated social networks: creation of community, exchanging knowledge, and communication of information (e.g. Facebook);

  • Virtual worlds: creation of a digital simulated environment for common specialized activities (e.g. virtual trading rooms);

  • Features and add-ons: usage of social media in dedicated environments for specific purposes (e.g. mobile applications).

4.4 Sentiment Analysis Results

As sentiment analysis deals with the analysis of “an attitude, thought, or judgment prompted by feeling[s]” (Merriam-Webster, n.d.) and is most commonly based on textual analysis techniques. Thus, the key idea is the analysis of social media data either in (1) real-time to gain more knowledge about the sentiment of the status of the market at a specific time; (2) as assisting tool to evaluate the current market state on a longer time-scale; (3) obtaining specific pieces of knowledge and information concerning actual issues; or (4) sentiment towards future events, such as IPOs, performance, or political event. These four possible directions have been analyzed within the scope of the IST project FIRST (Lombardi et al. 2011). Thus, sentiment analysis can be performed based on qualitative methods as e.g.:

  • Dimensions of mood: categorized representations of the current state of social media sentiments (e.g. calm, angry, pessimistic);

  • Predictors for stock indicators: quantitative predictors for stock pricing with associated probabilities and reliabilities;

  • Qualitative indicators: acquisition of knowledge and information about successful investment strategies and future trading behavior.

4.5 Additional Framework Requirements and Impact Factors

The previous sections discussed many aspects of applying social media around financial services. Within the scope of this section, other considerations that have not been discussed as far are mentioned. It shall give an overview of research works and framework requirements and additional impact factors:

  • Semantic data framework: semantic data framework for describing social media and investment data and knowledge mining as e.g. based on methods from the semantic web (e.g. ontologies) as evaluated in (Lombardi et al. 2011);

  • Sentiment impact and causalities: impact and causalities between sentiment and stock market returns under various market scenarios and causalities between different sentiment indicators. E.g. (Sheu et al. 2009) showed, that the “causal relationships between sentiment indicators and returns are mixed, if the market scenario is not classified according to investors’ sentiments”, which is supported by other studies [e.g. (Zouaoui et al. 2010)].

  • Networking and social media propagation: measurement and propagation models of social media information and how online communities diffuse information (Garg et al. 2011);

  • Stock exchange networks: based on the thought that social media propagate differently across online communities, the European stock exchange structure might impact on sentiment, as there might be correlations on how news and sentiment spread (Sakalyte 2009);

  • Benchmarking of social media sentiment indicators: social media sentiment indicators require a benchmarking with existing quantitative market data or other sentiment indicator to validate their reliability;

  • Data source quality and weighting: one of the major issue is the quality of social media data, as well as weighting source data according potential impact;

  • Predictive investment model: models and algorithms for predicting the future behavior of the stock exchange considering social media information;

  • Financial decision support: integration of the social media framework into existing financial management systems and investment support systems;

  • Model for sentiment impact on various equity types: different forms of equities react differently on sentiment changes. A theoretical model describing different equities under first different market scenarios [e.g. Sheu et al. (2009)]; and second relating these to different equity types [e.g. Baker and Wurgler (2007)];

5 Analysing Social Media and Creating a Social Media Sentiment Indicator Platform

Most of existing sentiment analysis on stock exchanges is based on qualitative evaluation methods as e.g. questionnaires as e.g. performed by Xetra, Börse Frankfurt [see Cognitrend (n.d.); Boerse-Frankfurt, (n.d.)]. The analysis is addressed to professional investors and financial institutions, and gives insights into the attitude if they believe in a downward (bear) or upward (bull) trend of the DAX performance index. However, this method has several major drawbacks:

  • Information is not gathered and analyzed in real-time and provided to the consumer;

  • It reflects a general trend of the stock market, rather than current happenings;

  • Poor illustration of the correlation between current sentiment and stock prices;

  • The too scarce updates do not provide sufficient information for guiding daily investments.

Within the scope of this section very briefly existing sentiment analysis techniques and methods are discussed. Existing works already show the importance and application of social media in industries, as e.g. the work of (Asur and Huberman 2010) which demonstrates the effect of social media on motion industries. It analyses sentiments extracted from social media platforms and demonstrates how these sentiments predict real-world outcomes, as e.g. movie sales in this particular example. Existing experiments show e.g. a direct correlation between sentiments extracted from Twitter messages and stock market prices. The accuracy of predicting basic bull and bear markets is obtained via an accuracy of 86.7 % (Bollen et al. 2011).

For individual trading decisions a higher granularity is required to place correct investments. Other research e.g. focused on the analysis and effect of events such as sport events on stock exchange prices, demonstrating that the result of sport events has an effect on stock returns and the impact of investors’ mood (Edmans et al. 2006). Other research focuses on the impact of the ‘weekend effect’ on the stock exchange, and how stock pricing is affected by investors’ sentiment (French 1980).

However, neither of these studies will help to make concrete investment decisions. They are only showing some sort of statistical correlation rather than concrete issues. Within the scope of this chapter, a principle workflow based on social media input data is presented in Fig. 7. The key-component of this workflow is taking place in the steps semantic annotation, in which sentiment extraction is taking place; and within the step knowledge mining, in which e.g. correlations between social media data and stock exchange data are performed. A more technical view for the utilization of social media information on the stock exchange is presented in Fig. 8. The presented architecture is extended from the principal architecture in (Lombardi et al. 2011).

Fig. 7
figure 00337

Principle workflow for performing a social media data analysis based on social media input data

Full size image
Fig. 8
figure 00338

Principal architectural blocks for a system for analysis of social media data as investor support system [extended and generalized from the architecture presented in (Lombardi et al. 2011)]

Full size image

5.1 Sentiment Dictionary

The central component of a social media based sentiment analysis is a sentiment dictionary. There are many English language based dictionaries. German language dictionaries that could be utilized for sentiment analysis for stock exchange sentiment analysis are rather scares. However e.g. SentiWS (Remus et al. 2010) provides a solution for general data sets and input formats with an extensive set of approximately 16.000 positive and negative word forms weighted within the interval [−1;1]. As the dictionary solely provides general words, an extension of the dictionary with stock exchange words and their weight is essential.

5.2 Data Sources and Acquisition

The data source and acquisition part has been well described by the IST project FIRST (Lombardi et al. 2011): in addition to real-time data streams containing the actual market information (e.g. equity quotes) social and online data streams have to be aggregated. These data feeds containing information on the web, proprietary resources, or social media are provided in a certain timeframe (e.g. real-time, daily, weekly) and range from simple RSS based web-feeds up to complex web conversations taking place in e.g. discussion forums or blogs. Thus, in principle we can distinguish between structured or unstructured data formats. Structured data follows certain syntax and encoding, and enables easy machine readable processing. Unstructured data such as conversations or other textual data requires much higher efforts in gaining the desired results. However, different data sources impact equity prices in a different level—a simple blog entry has rather no or minor effects on the equity market. But central bank interest changes, credit ratings from rating agencies, unemployment data from governmental institutions, or events such as natural disasters can be considered as having high impact on equity valuation. Therefore it is useful to attribute data sources with certain relevancy information (e.g. very high, high, neutral, low and very low) (Lombardi et al. 2011). An overview of potential data sources is given in Table 2.

Table 2 Overview of web and social media data sources suitable for gaining sentiment information [widely extended from the ideas in Lombardi et al. (2011)]
Full size table

5.3 Analysis Methods and Techniques

To provide an insight into the total set of analysis methods and techniques is a rather complex task. However, the three major techniques for the analysis of social media data are based on financial economic analysis; statistical methods; and the core method for social media data analysis is natural language processing.

5.3.1 Text Mining and Natural Language Processing

The core of the analysis of social media data streams is textual analysis of mostly unstructured data, especially tools for Natural Language Processing (NLP). There is a wide variety of methods and techniques to perform this task, which have been e.g. revised in (Pang and Lee 2008). Textual information is the data source to obtain information on investor sentiment. Thus textual input from various social media platforms is analysed, and by the application of probabilistic models or analysis software additional insight and support about investment decisions can be gained (e.g. via advanced software such as recommendation systems, information extractions tools, summarization, etc.) (Pang and Lee 2008). Examples for such models are e.g. Facebook APIs, or more sophisticated Probabilistic Latent Semantic Analysis (PLSA).

5.3.2 Effect of Sentiment on Investors’ Equity Valuation

One of the major concerns in modelling a sentiment based investment system is how sentiment affects equity prices. Sentiment changes have different effects on equities. As discussed in (Baker and Wurgler 2007), the valuation level of equities is dependent on the level of speculation of specific equities (e.g. high sentiment lead to high valuation levels of highly speculative equities). However, to obtain a more general model it can be useful to cluster sentiment according equities representing company types (e.g. medicals, consumables). As has been especially seen in the DAX crash in 2011, sentiment impacts equity prices differently. Whereas e.g. the valuation level of shares of consumables is higher in market turbulences than the one of production industry.

5.3.3 Benchmarking and Correlation of Social Media Sentiment

Another issue is the evaluation and benchmarking of social media sentiment indicators and especially finding the variables that correlate to allow predictions and other more advanced models. Benchmarking and correlation analysis evolves according three different modalities:

  • Causality of social media sentiment indicators and live-market data;

  • Causality between social media sentiments and other sentiment indicators;

  • Prediction methods for explaining the impact of social media on market returns.

In (Sheu et al. 2009) a method for benchmarking sentiment indicators such as put/call volumes, put/call opened interest ratio, volatility, and a technical indicator as e.g. ARMS in various market scenarios is introduced. This method has high potential in describing the causality of social media sentiment indicators and their statistical relevancy. Other methods based on the selection of market variables and non-social media related sentiment data are presented in (Chen et al. 2010).

5.3.4 Creating a Social Media Sentiment Index

To create a sentiment index reflecting the current social media sentiment is a far more complex task. Simple models exist on the market, as e.g. Thomson Reuter’s Social Pulse (Thomson Reuters, n.d.) providing sentiment indexes for single equities together with additional market information based on social media. The main challenge in providing a social media sentiment index is to find the correct variables influencing equity sentiment but there are many examples for methods how a social media sentiment index can be calculated based on sentiment index calculation methods for other sentiment indexes [e.g. (Baker and Wurgler 2007)].

5.3.5 Stock-Exchange Predication Models

The development of statistically valid predictive models for the stock exchange is rather tricky. Similar to benchmarking social media sentiment, a prediction model requires the selection of correct correlating variables. The underlying questions therefore are:

  • Which variables correlate with social media data?

  • Can liquidity and volume be forecasted?

  • What is the correlation between volatility and sentiment?

  • What is the relation between noise (sentiment) trading and correct valuation?

  • How can the overreaction to news be forecasted?

  • Can the rational/irrational behaviour of investors be predicted?

  • How can influences from markets be removed from sentiment indicators?

However, there are a few approaches that are based on indirect crowd sourcing models as e.g. web-search engine queries (Bordino et al. 2011). The study examines the relation between the volume of web queries for a specific equity and its trading volume. The study also demonstrates that web queries can be utilized as early warning system for changes of investment sentiment. Another interesting study determines that sentiment has high impact on volatility forecasting (Sheu and Wei 2011). An additional aspect is that market changes imply a change in sentiment indicators. Thus the power of sentiment indicators to predict the market can be increased by purifying sentiment indicators from such influences, as shown in (Aronson and Wolberg 2009). Thus a purification of social media sentiment indicators impacts also the predictive power these.

6 Discussion: How Can Social Media Predict the Stock Market?

Social media affect the stock market as e.g. discussed by (Eler 2011) and first communities are created [e.g. StockTwits, (n.d.)]. Many brokers already offer online service portals that include social media features such as blogs, wikis, podcasts, and discussion forums—many of these even free of charge. The main aim of these forums is the creation of a community to sell the products associated with the current discussions and trends. Still, one thing is very obvious—social media will definitely not be able to predict the stock price with 100 % accuracy. Also in social media, the social media customer is not always right (Read 2011). Customers complain, and especially when investors loose, it’s easy to shift a failure trade to wrong information online platforms and published trading aids. This implicates clear advices for consumers that investments cannot be predicted as many would believe and a clearer communication method with consumers to avoid misunderstandings. Especially inexperienced private investors might take discussions and advices as granted facts rather than guides.

However, what this publication definitely was presenting was a wide set of application scenarios of social media in investment. Especially the application of social media in marketing of financial products will gain of importance with the advent of online brokerage and opening of the market to everyday consumers (Weinhardt et al. 2000). For the professional trader social media are already considered as serious form of media in investment decisions, as social media platforms as e.g. StockTwits (n.d.) show.

But there still one major question remains: Can social media predict the stock market? Since decades people attempt to develop models to explain stock market behaviour. Many of these have been listed within this publication. From the current point of view the answer to this question is rather simple: no. Social media sentiment analyses are just one additional tool to assist investors in their decision process. Especially financial management and investment support systems will benefit from the analysis of social media data as an additional data source for various purposes as e.g. risk management, event detection, or to assist in forecasting markets. Social media are simply another analysis tool to help increasing performance together with traditional investment analysis tools. For everyday consumer social media allow to gain access to information, which was as far solely in the hands of professional. It empowers the consumer, to understand the markets, and it assist in risk management and give an edge to survive on the market. Only together with traditional investment analysis tools social media will help in increasing performance.

Social media are a tool especially reflecting market crowd psychology, and existing applications of social media in other domains demonstrate the power of the crowd. Also in investments, social media will show their power, and be a major factor in decision making. Social media as sentiment indicator could act as tool for fundamental and technical analysis. Eventually new indicators assisting investors are emerging. However, as for many indicators, also this indicator will lose its power or trading secret, as many other indicators did, when they have been revealed to a large community. But they will give new insights and potential new possibility to bet on the right investment, thus stay on the market longer. The field is still rather unexplored, and new indicators might emerge rather quickly and are part of professional analysis software packages.

For event-, topic-, and current news detection social media is already proven to be efficient in many domains. Thus it can be predicted that social media will also show its powers and strength as financial support tool. Also in rating and gaining additional information about equities social media will definitely be a very helpful tool to gain additional knowledge and information.

However, this research work is on-going, and has been arising many new questions requiring a more thorough examination and research. As a next step a series of social media data will be examined and causalities on predictability tested.