Predicting the Brazilian Stock Market with Sentiment Analysis, Technical Indicators and Stock Prices: A Deep Learning Approach

Abstract

Recent advances in Machine Learning and, especially, Deep Learning, have led to applications of these areas in different fields of knowledge, with great emphasis on stock market prediction. There are two main approaches in the literature to predict future prices in the stock market: (1) considering historical stock prices; and (2) considering news or social media documents. Despite the recent efforts to combine these two approaches, the literature lacks works in which both strategies are performed with Deep Learning, which has led to state-of-art results in many regression and classification tasks. To overcome these limitations, in this work we proposed a new Deep Learning-based approach to predict the Brazilian stock market combining the use of historical stock prices, financial technical indicators, and financial news. The experiments were performed considering the period from 2010 to 2019 with the Ibovespa index and the historical prices of the following Brazilian companies: Banco do Brasil, Itaú, Ambev, and Gerdau, which have significant contribution to the Ibovespa index. Our results show that the combination of stock prices, technical indicators and news improves the stock market prediction considering both the prediction error and return-of-investment.

1 Introduction

Stock market prediction (i.e., predicting the value or the movement of an asset in the future) is a topic that has attracted much attention over the last decades, with much discussion in the scientific community on this issue. Malkiel and Fama (1970) presented the Efficient Market Hypothesis (EMH) stating that, in an efficient market, the available information is already contained in the market price, implying that the stock market movement is unpredictable considering the past stock prices. On the other hand, Khadjeh Nassirtoussi et al. (2014) address this issue, showing that it is possible to empirically contradict the market efficiency. Thus, some predictable markets exist, at least partially. In fact, as presented by the Adaptive Market Hypothesis (AMH) (Lo, 2004), which combines evolutionary and behaviorial aspects to economic exchanges, it is feasible to profit from a portfolio of stocks by exploiting flaws in the market efficiency.

Furthermore, the Dow Theory (Rhea, 1993) added more information about the comprehension of the stock market. According to this theory, the movements of the prices in the stock market occur according to trends. Thus, in order to predict the market trends, investors developed two strategies: technical and fundamental analysis. While technical analysts try to predict future prices in the stock market according to past stock prices and mathematical (technical) indicators, fundamental analysts base their trading decisions on the company’s financial conditions and macroeconomic indicators. As showed in Atsalakis and Valavanis (2009), it is known that Technical Analysis is the most frequent approach presented in the published works. Most investors, however, naturally try to observe both technical and fundamental data.

Current financial analysis have moved towards data science due to the increasing power of machine learning and Deep Learning techniques (Picasso et al., 2019). Advances in Web technologies have also led to large amounts of collected information, attracting attention from investors to platforms such as news websites, social media, and blogs. Many works studied the influences of these online textual data in the stock market (Bollen et al., 2011; Li et al., 2020). As presented in Ozbayoglu et al. (2020), financial text mining has recently gathered a lot of attention, mostly focusing on obtaining the investor’s sentiment and correlating it with the stock market. Fundamental data (i.e., textual data), however, is commonly unstructured and its use is computationally challenging (Khadjeh Nassirtoussi et al., 2014). There is still an open challenge on how to integrate as many information sources as possible and develop a useful stock market prediction approach.

Existing works frequently focus on one type of information to predict the stock market, such as: (1) textual data, such as news and social media posts (Schumaker et al., 2012; Kraus & Feuerriegel, 2017); or (2) historical stock prices and technical indicators (Nelson et al., 2017; Pang et al., 2018). Other works integrate several market information sources. Both Li et al. (2020) and Picasso et al. (2019) present a system to predict the stock market combining news, stock prices, and technical indicators.

Thus, to the best of our knowledge, and as stated in (Li et al., 2020), are still recent in the literature the efforts to combine textual data alongside stock prices and technical indicators to predict the stock market. However, one disadvantage is that these works generally perform sentiment analysis in unstructured data using lexical methods, which rely on knowledge about the language and, generally, presents inferior results when compared to Machine Learning techniques (Taboada et al., 2011; Avanço et al., 2014).

To address these issues, we propose an approach to predict the Brazilian stock market, incorporating news sentiments, technical indicators, and stock prices into a single system. The whole process is performed with deep neural networks, considering both: (1) the news sentiments; and (2) the prediction of the stock market.

We used a deep learning approach to perform Sentiment Analysis (unlike the works mentioned above, Li et al. (2020) and Picasso et al. (2019), which use a lexical approach) of Brazilian news (thus, written in Portuguese). Then, the result feeds another deep Artificial Neural Network (ANN), alongside with technical indicators and historical stock prices. The following deep ANNs were considered: Multilayer Perceptron (MLP), Long Short-Term Memory (LSTM) and Bidirectional Long Short-Term Memory (Bi-LSTM). The baseline technique was a Support Vector Machine (SVM). These different ANNs architectures were compared considering that they are the most commonly used architectures in the financial field, according to the work Ozbayoglu et al. (2020).

We would also like to highlight that few works address the Brazilian stock market. Recently, Ozbayoglu et al. (2020) reviewed 140 papers and only three used data from the Brazilian stock market; Bustos and Pomares-Quimbaya (2020) reviewed of 53 papers and only two used Brazilian data; also, Kumbure et al. (2022); Ashtiani and Raahmei (2023) show that North American and Asia are the most referenced markets in the literature. The São Paulo Stock, Commodities and Futures Exchange is the official stock exchange in Brazil (B3). Bovespa is the second largest stock exchange in the world by market value. Many factors make Bovespa attractive, including the high volatility of the negotiated stock prices.¹ Thus, our approach was tested in data from the Brazilian financial market, from January, 2010 to December, 2019, through the Ibovespa index and some of its representative companies: Banco do Brasil (BBAS3), Itaú (ITUB4), Ambev (ABEV3) and Gerdau (GGBR4).

As indicated by Picasso et al. (2019), the evaluation of a Machine Learning model in finance might be complicated; as such, our experiments were divided in two stages: (1) evaluation of metrics related to the Machine Learning field; (2) evaluation of an investment simulation according to the financial metrics: final Return of Investment (ROI), Maximum Drawdown (MDD) and the Sharpe Ratio (SR). According to Picasso et al. (2019), the first step proves the effectiveness of the model and the second step demonstrates the quality of the model predictions.

The main contributions of our work are: (1) the use of Deep Learning in the whole process of stock market prediction (both applied to Sentiment Analysis and to the stock market historical data); and (2) a case study of the Brazilian stock market, which has few published works. Our results show that the use of news improves the stock market prediction, considering both Machine Learning metrics and investment simulation.

The remainder of the paper is organized as follows: Sect. 2 introduces the main topics of this work and related works; Sect. 3 underlines the methodology of our approach and presents the datasets used; Sect. 4 reports the results obtained for each of our experiments; and finally, Sect. 5 presents the conclusion and future works.

2 Theoretical Foundation

Advances in Web technologies allowed for easy access to a great amount of news articles. Thus, the investigation of the sentiment associated with each publication attracts investors’ attention, in order to exploit sentiment-based investment strategies and profit in the stock market. However, as there is too much information available, it is difficult to explore it without a computational system. Besides, stock market prediction is a difficult task, since the market is influenced by many factors, such as national policies, regional and global economics, and even human behavior. In this sense, there is a need to develop an approach to predict the stock market that combines as many sources of information as possible.

There is a wide variety of models and techniques in the literature that seek to predict the stock market. Bustos and Pomares-Quimbaya (2020) indicate that Machine Learning models, like Support Vector Machines (SVM) and, more recently, Artificial Neural Networks (ANNs) are the most used. These models can be used both to predict the stock market and to perform Sentiment Analysis. Indeed, ANNs with multiple layers (Deep Learning) present even better capabilities, overcoming the classic Machine Learning models, such as SVM, Linear Regression, and Naive Bayes, among others (Ozbayoglu et al., 2020; Bustos & Pomares-Quimbaya, 2020; Kraus & Feuerriegel, 2017).

In this work, we evaluate several ANNs architectures in order to predict the stock market: MLP, LSTM and Bi-LSTM; we also used a Convolutional Neural Network to perform Sentiment Analysis in financial news.

This section presents the theoretical foundation of this work. Section 2.1 describes the models used in our proposed architecture, while Sect. 2.2 presents the related works.

2.1 Artificial Neural Networks

Artificial Neural Networks (ANNs) are composed of linked units, i.e., artificial neurons, that simulate the behavior of its biological counterparts. As presented in Ozbayoglu et al. (2020), Deep Learning models (ANNs with multiple layers) are receiving increasing attention due to their capability of overcoming the classic models. In this work, we consider some of the most used ANN architectures in the financial field: Multilayer Perceptron (MLP), Long Short-Term Memory (LSTM), Bidirectional Long Short-Term Memory (Bi-LSTM), and Convolutional Neural Network (CNN).

MLP was one of the pioneer ANN models in the literature, commonly used both to perform regression and classification tasks (Haykin, 1994). This model is composed of three types of layers, namely: input, output, and hidden layers. Each layer contains one or more artificial neurons, called Perceptron, which have as parameters inputs, weights, bias, and activation functions. Figure 1 presents an MLP architecture with one input layer, two hidden layers, and one output layer. The Perceptrons of the MLP architecture receive the input signals, apply the weights and bias and propagate its result to the next layer through an activation function until the output layer is reached. Some authors differentiate the MLP of the DMLP (Deep Multilayer Perceptron), considering the last one similar to the MLP, but with higher number of hidden layers (Ozbayoglu et al., 2020).

Fig. 1

Multilayer Perceptron (MLP) neural network composed by: input layer; two hidden layers; output layer. Adapted from Carosia et al. (2020)

ANNs can also be organized as a Recurrent Neural Network (RNN), which deals with the idea of time steps in the model. RNNs are normally used to predict time series, such as stock market prediction, and to perform Natural Language Processing (NLP) tasks, such as Sentiment Analysis (Li et al., 2020; Lipton et al., 2015). One of the most common RNN is the Long Short-Term Memory (LSTM) (Hochreiter & Schmidhuber, 1997), which was designed to learn long-term dependencies contained in the training data set, using neurons specially created to solve this issue.

The LSTMs’ neurons can be connected in a similar fashion to the MLPs. However, its main innovation is the memory cell, whose basic structure is presented in Fig. 2. The memory cell receives the input provided by a previous time step and can block or pass the information according to its relevance to the model. This neuron behavior has four steps:

• 1. Forgets disposable information, $f_{t}$, which is performed by the Forget Gate;

• 2. Generates the cell’s internal state using both the Input Gate, $x_t$, and the output of the cell in the previous time step $h_{t-1}$;

• 3. Calculates the new cell’s state value, $C_t$, considering the values generated by the aforementioned steps;

• 4. Calculates the cell’s output, $h_{t}$, which is used as input to the memory cell in the following time step.

Fig. 2

LSTM’s Memory Cell. Adapted from Zhang et al. (2018)

Another RNN-based model is the Bidirectional LSTM (Bi-LSTM), which extends the traditional LSTM neuron, as presented before, adding a feature that considers both the past and future input sequences, while the original LSTM only deals with the data provided by the previous time step. The Bi-LSTM is based in the principles proposed by the Bidirectional RNN (Schuster & Paliwal, 1997), in which two layers are connected to process the input data: while the first layer operates considering the past input sequences (i.e., $t-1$, t, $t+1$,...), other layers operate considering the opposed time step direction (i.e., $t+1$, t, $t-1$,...). Finally, the results of both layers are merged, as presented in Fig. 3. Some works show the effectiveness of the Bi-LSTM over the traditional LSTM (Althelaya et al., 2018; Graves & Schmidhuber, 2005).

Fig. 3

Representation of the layers of a Bi-LSTM model. Adapted from Althelaya et al. (2018)

Finally, the CNN model (LeCun et al., 1989, 2015) is mostly used for image classification and recognition. Recently, though, their potential for Natural Language Processing, such as in text classification, has also been highlighted (Kim, 2014; Johnson and Zhang, 2015; Qian et al., 2020). In this architecture, the input data is scanned with filters and passed through various types of layers: convolutional (to generate characteristics maps), pooling (to apply the average or maximum functions to reduce the dimensionality of the characteristics map), and a fully connected layer (similar to an MLP). Figure 4 presents a generic CNN-based model containing the aforementioned layers.

Fig. 4

Generic CNN-based neural network. Adapted from de Faria (2018)

As presented in Ozbayoglu et al. (2020), there is a dominance of RNNs, mainly LSTMs, in works related to stock market prediction in finance. In fact, the nature of the stock data is ideal to the RNNs, since they present temporal dependency. Other models are also considered in the works of this area; among them, the MLP (including DMLP) is being widely used to perform classification tasks. Finally, the CNN just recently received more attention in the financial field, being used both as a classifier and a regressor. However, its disadvantage is that the model requires a data transformation to represent the financial data considering a 2-D image view.

Thus, in this work we used the MLP, LSTM and Bi-LSTM in order to predict the stock prices, and a CNN to perform Sentiment Analysis, considering the traditional use of each of them in the literature.

2.2 Related Work

Although past works studied stock market prediction, works that consider both news and historical stock prices represent a recent research topic, with increasing interest only in the last years. Generally, the published works focused on only one of these data sources. In this section, we presented related works according to three data sources used as input: (1) news; (2) stock prices; (3) news and stock prices.

News as Input Data: According to Ashtiani and Raahmei (2023), although the use of news in the market forecasting literature has increased in recent years, the use of news is still scarce, being surpassed by the use of data from social networks. Thus, Bollen et al. (2011) developed a seminal work regarding this topic, using the sentiments in the social media platform Twitter to predict the stock market movement with a Fuzzy Neural Network. The data used comprises Tweets from February to December 2008 and the results presented 86.7% precision in the stock market movement prediction of the Dow Jones Industrial Average index (DJIA). Schumaker et al. (2012) proposed the Arizona Financial Text System to predict the stock market movement using financial news of the companies listed in the S &P 500 index considering the period between October and November, 2005. The results showed that news can predict the stock market with a 59% hit and lead to a profit of 3.3%.

Yoshihara et al. (2014) presented a Sentiment Analysis-based strategy with Recurrent Neural Networks to predict the Japanese stock market. The data used was news from ten companies of the Nikkei index, considering the period between 1999 to 2008. The results reported that the prediction error was reduced when compared to the baselines. Also, Li et al. (2014) showed a methodology to measure the online media impact on the stock market. The work performed Sentiment Analysis in news with an Emotion Dictionary, and as a predictive model, a SVM technique was used. The data used in the study was from China Securities Index (CSI 100) companies considering the year of 2011. The results showed that the public sentiment has an important role in investors’ decision and that news articles can improve investors’ action in the market.

Kraus and Feuerriegel (2017) compared the use of Deep Learning with RNNs, LSTMs and traditional Machine Learning algorithms considering financial disclosures as a data source to build a Decision Support System. The dataset comprises 13,135 disclosures from German companies. They also showed that ANNs improve 6.8% of the results when compared to the baselines. Furthermore, Hájek (2018) used an MLP neural network applied to annual reports to predict stock prices returns. They used a dictionary-based technique to perform Sentiment Analysis in the news of 1,402 companies presented in NASDAQ and NYSE. The results showed that the proposed method has similar results to the Naive Bayes algorithm, but overcomes the remaining baselines, namely SVM, C4.5 decision tree, and k-NN.

Jena and Majhi (2023) examined how COVID-19 sentiments on Twitter, about the health impacts of the disease, and other macroeconomic indicators can be combined to predict stock indices in the United States and India with LSTM neural networks. The results showed that the LSTM model outperforms the time-series statistical models in predicting stock indices. Swathi et al. (2022) proposed the use of LSTM neural networks for stock price prediction using Twitter data. The authors used the neural network to process sentiments of tweets and a Reinforcement Learning method to predict the stock market. The results suggest that the approach with a learning-based model is a promising tool for stock price prediction using Twitter data.

Stock Prices as Input Data: Nelson et al. (2017) presented the use of LSTMs to predict the stock price movement. The data source used was the historical stock prices of the Ibovespa index considering the period from 2008 to 2015. The results showed, on average, 55.9% of accuracy in the movements of the stock market. In the same way, Fischer and Krauss (2018) presented the use of LSTMs to predict the stock market movement. The data source comprised historical data from the S &P 500 index from 1992 to 2015 and the results showed that the LSTMs overcomes the baselines and can bring a 23% profit with an investment strategy.

Sang and Di Pierro (2019) also used LSTMs to improve investment algorithms with Technical Analysis with data of the S &P 500 index. The results showed that LSTMs with Technical Analysis indicators can improve the stock market prediction when compared to the Technical Analysis itself. In Chen et al. (2021) the authors present a CNN-based neural network to perform stock trend prediction in the Chinese stock market. The experiments were performed considering six Chinese companies in the period between 2010 and 2020. The results showed that the proposed model outperforms the baseline models and stock trading strategies.

Bhandari et al. (2022) used an LSTM neural network to predict the S &P 500 index combining fundamental market data and technical indicators. The results showed that the performance of a single-layer model is found to overcome the prediction accuracy of the multilayer models, concluding that the LSTM model is a promising tool for stock price prediction. Liu et al. (2022) proposed a deep learning model for stock price prediction in the Chinese stock market using stock price charts as images with deep learning neural networks (DLNNs) for image modeling. The results showed that price trends are more important than stock fundamentals in predicting future price movements.

Finally, Sheth and Shah (2023) propose a literature review considering three methods to predict the stock market: Feedforward Neural Network, Support Vector Machine, and Long Short-Term Memory. The authors found that the ANN outperforms the other two methods, while LSTM gives good results only when a large dataset is available.

News and Stock Prices as Input Data: In Maqsood et al. (2020) the authors presented a Deep Learning approach with Sentiment Analysis to investigate significant events in stock markets using companies from different countries. The data consists of tweets from 2012 to 2016. The Sentiment Analysis task was performed using a word list, which is later used as input to a CNN. The experiments presented that sentiments improve the forecast of the stock market. Nti et al. (2020) presented a Sentiment Analysis-based model to forecast the Ghanaian stock market movement using an MLP neural network. The proposed model uses several time windows to forecast stock prices. The results show that the information available in financial news, tweets, and forums are useful to investors, presenting accuracy that ranges from 49% to 55%.

In Qian et al. (2020) the authors presented a CNN to classify tendencies from online comments, besides the investigation of users’ tendencies on market volatility and returns. The work also presents models to predict returns applied to the Shanghai and Shenzhen indexes. Results showed that negative tendencies from online users correlate with volatility and market returns.

Moreover, Picasso et al. (2019) presented the combination of news and historical stock prices to predict the stock market. In order to perform the Sentiment Analysis of the news, it used a financial dictionary and, to predict the market, employed an MLP-based neural network. Also, experiments showed that the use of news improves the stock market prediction considering both Machine Learning and financial metrics.

Li et al. (2020) presented a stock market forecast strategy which integrates news and technical indicators, used as input to an LSTM neural network applied to the Hong Kong Stock Exchange. The work performs Sentiment Analysis in news using a financial dictionary. The experiments show that the proposed model outperforms the baselines and that the combination between news and technical indicators overcomes the strategies with only news sentiments or technical indicators. Duarte et al. (2020) presented the use of news and stock prices to predict price falls in order to prevent financial losses in the Brazilian stock market. The dataset comprises news ranging from August 2016 to May 2018 and the work uses as prediction techniques 11 classical Machine Learning models. The results show that news correlates to the stock price changes in the Brazilian stock market. Besides, the proposed approach overcomes the baselines Buy & Hold and moving averages when considering an investment simulation.

As presented here, few works combine news and stock prices in a prediction system (Ashtiani and Raahmei, 2023). Thus, it is worth to highlight some differences between our work and the most similar approaches from the literature. Li et al. (2020) and Maqsood et al. (2020) did not design investment simulations to prove the prediction effectiveness in the financial domain, restricting to Machine Learning metrics itself. As previously noticed, this can be a problem in model evaluation, since better performance measures in the Machine Learning metrics don’t imply better performance in investment simulations. Picasso et al. (2019) perform an investment simulation besides ML metrics evaluation. In fact, our experimental methodology was inspired by theirs. Meanwhile, (Picasso et al., 2019) only uses data from approximately 1 year, restricting the period analyzed of the stock market. It is worth highlighting that our work, to the best of our knowledge, is the first that addresses the issue of including news in a prediction system without a word list or dictionary to perform Sentiment Analysis, as presented in (Picasso et al., 2019; Li et al., 2020).

The work presented in Duarte et al. (2020) tries to predict the Brazilian stock market, which differs from ours according to the ML techniques used. While Duarte et al. (2020) used mainly traditional ML models, without deep learning, we focused in deep ANNs. Other factor that differs our work is that we used FastText to extract features from the news, while theirs used term frequency features to feed ML models.

Considering the presented works, our main contributions are:

• 1. The whole process to predict the stock market uses Deep Learning techniques, both applied to news’ Sentiment Analysis and historical stock prices. This brings an advantage mainly in the Sentiment Analysis process; as presented, most of the related works uses lexicons to perform this task.

• 2. The case study is the Brazilian stock market, which lacks studies in the financial field.

• 3. The comparison of the most used ANNs architectures in this area of application.

• 4. The emphasis in the importance of evaluating the proposed models both considering Machine Learning metrics and financial context metrics.

3 Methodology

This paper aims to propose a stock market prediction approach that combines news, technical indicators, and stock prices. It also uses the Brazilian stock market as a case study. The proposed methodology is illustrated in Fig. 5 and summarized as follows.

• Preprocessing Module: first, news were downloaded from the three most relevant online Brazilian newspapers. Then, they were preprocessed and their word embeddings were generated using FastText. Stock prices were also downloaded and their respective technical indicators were generated. This step is described in details in Sect. 3.1.

• Sentiment Analysis Module: the word embeddings generated in the previous step were used as input to a deep neural network to perform Sentiment Analysis of financial news. This step is described in details in Sect. 3.2.

• Data Merge Stage: the resulting data from the previous two tasks (i.e., news sentiments, stock prices and technical indicators) were merged in a vectorial representation in order to prepare the input to the Prediction Module. This step is described in details in Sect. 3.3.

• Prediction Module: the merged data from the previous step was used as input to a second deep neural network, which performs a regression task to predict the return of investment of the next day in the stock market. This step is described in details in Sect. 3.3.

• Evaluation Module: the last step evaluates the prediction results considering two approaches: traditional machine learning metrics and investment metrics from an investment simulation task. This step is described in details in the Sect. 3.4.

Fig. 5

Methodology Representation. Our methodology consists primarily of tree steps: (1) process the input data i.e., news, technical indicators and stock prices; (2) predict the return of investment of the next day in the stock market; and (3) evaluate the results considering metrics of the Machine Learning field and an investment simulation

The following sections detail each one of the proposed modules of our system.

3.1 Input Preprocessing Module

The data used in our work are from three different sources: news, stock prices, and technical indicators. Each one of these sources is explained in the following.

Stock Prices and Technical Indicators: The stock prices time series consists of the following data: open price, high, low, close price, and volume. Once downloaded, they are used to feed the ANNs and to calculate Technical Indicators that feed the ANNs as well.

Technical indicators are mathematical indicators observed by investors in order to study the market behavior. In this work, alongside with news and stock prices, we also use several common technical indicators as input of the ANNs. The indicators were configured by different parameters, normally to reflect market trends or fluctuations considering different aspects. They can also provide rich market signals, and were chosen considering the work of Li et al. (2020), together with volatility and volume indicators (Achelis, 2001). Thus, the following technical indicators are considered in this process:

• Trend Indicators: Moving Average Convergence and Divergence (MACD) with 12 and 26 days period and a signal considering 9 days; and Moving Averages (MA) with 10, 20, and 30 days. While the MACD shows the relationship between two moving averages of prices in order to identify trends, the MA shows the average value of an asset over time.

• Momentum Indicators: Relative Strength Index (RSI) with the parameters: 6, 12 and 24 days; Money Flow Index (MFI) with time period 14; and On Balance Volume (OBV). While RSI shows signs of overbought (generally above 80) or oversought (generally below 20) assets, the MFI shows the intensity of the money flow in and out of an asset. Also, the OBV indicator relates volume changes to price changes of an asset.

• Volatility Indicator: Bollinger Bands (BBs), which represent standard deviation levels above and below a stock moving average. This indicator increases when the market is unstable and decreases in a more stable period.

News Preprocessing: Once downloaded, the daily news need to pass through a Sentiment Analysis classifier. To prepare the data for this task, we first clean the data, removing symbols, links, emoticons, special characters, etc. Finally, we generate the word embeddings using FastText (Bojanowski et al., 2017). This step generates as output a vector composed of the concatenation of each word embedding of the news title.

3.2 Sentiment Analysis Module

Considering the Sentiment Analysis in Brazilian Portuguese, we used a deep Convolutional Neural Network (CNN) as a classifier model, as proposed in Zhang and Wallace (2017), in accordance to our previous study (Carosia et al., 2021), in which we developed a SA module for financial news in Portuguese. In Carosia et al. (2021), we evaluated several ANN architectures to SA in Portuguese, and the CNN presented better results both considering the metrics F1-Score (i.e., the harmonic average calculated between precision and recall) and Accuracy (i.e., the amount of correctly classified documents divided by the total amount of documents).

The proposed CNN has the following characteristics:

• 1. Three filter layers, with two filters each;

• 2. Pooling layers with the maximum operation;

• 3. The last layer with a softmax activation function, to classify the sentence.

Finally, we need to account for the daily news sentiments, i.e., create a vector with two dimensions in which the first one is the number of positive news calculated in a day and the second is the number of negative news of the same day. It is worth highlighting that on days without articles, the positive and negative sentiment values in the features that serve as input to the ANNs are determined as zero, indicating that no positive or negative news was published that day.

The outputs of this step will be used as inputs in the Prediction Module, presented in the next section.

3.3 Prediction Module

The proposed Prediction Module consists of a deep ANN that receive as input a vectorized element with the following features combined in the Data Merge Stage: (1) stock prices (i.e., open, low, high, close) and volume, (2) technical indicators, and (3) the daily news sentiments (i.e., the number of positive news and the number of negative news).

The generated vectors pass through a normalization process before being fed to the ANN. As in Picasso et al. (2019), we used a Min-Max normalization technique (Eq. (1)) to limit the data to a boundary [0,1], where $x_{norm}$ represents the results of the normalization, $x_{min}$ is the minimum value of the attribute and $x_{max}$ is the maximum value of the attribute being normalized:

$$\begin{aligned} x_{norm} = \frac{x - x_{min}}{x_{max}-x_{min}} \end{aligned}$$

(1)

The output of the prediction module is the return percentage of the close price of the next day in the stock market:

$$\begin{aligned} Return = \frac{Close_{t+1} - Close_{t}}{Close_{t}} \end{aligned}$$

(2)

Therefore, the ANN performs a regression task in order to predict the return of the next day in the stock market. To develop the training process, we used the Minimum Squared Error (MSE) as ANN’s loss function to calculate prediction errors and optimize the parameters of the ANN with the back propagation algorithm. MSE is one of the metrics commonly used to regression (Maqsood et al., 2020) and is given in Eq. 3, where $y_i$ is the real output and $y'_i$ represents the predicted value.

$$\begin{aligned} MSE = \frac{1}{n} \sum _{i=1}^{n} (y_i - y'_i)^2 \end{aligned}$$

(3)

3.4 Evaluation Module

As mentioned before, we evaluate our work considering an out-of-sample test set from January, 2017 to December, 2019. Due to the difficulties to evaluate a Machine Learning algorithm in the financial area, we divided the evaluation process into two steps, considering the approach presented in (Picasso et al., 2019).

Machine Learning Metrics. In this step, we evaluate our prediction results considering the following common metrics in regression: Mean Squared Error (Eq. 3) and Mean Absolute Error (MAE), given in Eq. 4.

$$\begin{aligned} MAE = \frac{1}{n} \sum _{i=1}^{n} |y_i - y'_i| \end{aligned}$$

(4)

Investment Simulation Metrics. In this step, we perform a investment simulation to measure the quality of the predictions. To evaluate the results, we used the following financial metrics: final Return of Investment (ROI), Sharpe Ratio (SR) and Maximum Drawdown (MDD).

The final return of investment represents the final result of a BRL 100,000.00 investment in the beginning of the considered period. Sharpe ratio is a risk measure that represents the average return earned (R) early beyond of the risk-free rate (r) divided by the volatility or total risk, i.e, the standard deviation ($\sigma _{R}$) of the earned value (Eq. 5). In Brazil, the value of the risk-free rate is generally used as the Interbank Deposit Certificate (known as CDI), the main reference for the profitability of fixed income investments.

$$\begin{aligned} Sr = \frac{R-r}{\sigma _{R}} \end{aligned}$$

(5)

Finally, the maximum drawdown is an indicator of downside risk over a specified time period, calculated considering the maximum loss from a peak to a minimum of a stock price time series, as presented in Eq. 6.

$$\begin{aligned} MDD = \frac{Peak-Minimum}{Peak} \end{aligned}$$

(6)

In order to perform the investment simulation, we designed a simple investment strategy to be applied considering the result of the Prediction Module, as presented in the following.

• 1. The model predicts the next day’s return ($d+1$).

• 2. If the predicted value is positive and the investor does not own the stock, a buy operation is made.

• 3. If the investor owns the stock, he/she must keep the current (bought) position.

• 4. If the predicted value is negative and the investor owns the stock, a sell operation is made.

• 5. If the investor does not own the asset, he/she must keep the current (sold) position.

These approaches show both the power of the proposed model and the real effectiveness of our predictions when applied to finance.

In our experiments, we considered three scenarios to investigate if news in fact improve the results in a stock market prediction system. The first scenario uses as an input only the historical stock prices, without news and technical indicators. The second scenario uses as input the historical stock prices and the technical indicators. And, finally, the third scenario uses as input all the sources available: news, technical indicators, and historical stock prices. We also compared our investment results with the investment strategy Buy & Hold, which consists in buying an asset in the beginning and holding it during the whole period. The next section addresses the results of our experiments.

4 Results

This section presents the results of our experiments and is organized as follows: (i) the data used in our experiments are presented in Sect. 4.1; (ii) the evaluation of the proposed model according to Machine Learning metrics is presented in Sect. 4.2; (ii) the evaluation of the selected models according to investment metrics is presented in Sect. 4.3. Finally, Sect. 4.4 emphasizes the theoretical and practical contributions of this paper.

4.1 Data Source

All the data used in this work were collected considering the period from January, 2010 to December, 2019. In the experiments, we used data from the Brazilian stock market main index, Ibovespa, and the following representative companies of the Brazilian stock market: Banco do Brasil and Itaú, from the banking sector, Gerdau, from the steel and metallurgy sector, and Ambev, from the beverages sector. Table 1 presents each of the mentioned stocks with the respective ticker, valuation, number of up and down movements in the stock market, alongside with the maximum return possible in the period.

Table 1 Stocks used in the experiments.

The news were downloaded from the three most relevant online Brazilian newspapers (ANJ, 2019): G1, Estadão, and Folha de São Paulo. For this, we designed scripts to access the search engines of each one of these websites, which can bring news that contains the name of each one of the presented companies both in the title or in the content of the news. However, in our work, we used only the news titles to predict the stock market, based on the work of (Li et al., 2020). This process resulted in: 79,943 news from Ibovespa, 111,708 news from Banco do Brasil, 53,589 news from Itaú, 1934 news from Gerdau, and 8817 news from Ambev over the considered period.

The stock price time series was collected from the Yahoo Finance² website. Since technical indicators are calculated over the stock prices, they do not need to be downloaded.

4.2 Machine Learning Metrics

In order to execute the proposed methodology, we use a CNN-based neural network to perform Sentiment Analysis, as in Carosia et al. (2021). This ANN was trained using a data set with 5,000 financial news titles of different data sources and different companies. Part of this data, 2,132 news, was presented in (Martins et al., 2015). The remaining news were collected during the development of this work. All news are in Brazilian Portuguese and were labeled by specialists considering the investor’s sentiment according to the market: if the news could affect the market positively (up movement), they were labeled as positive; if the news could affect the market negatively (down movement), they were labeled as negative. The data used in this step is composed by 2,750 (55%) of negative titles, and 2,250 (45%) of positive ones. The CNN was trained using K-Fold Cross Validation (k = 10) with a stochastic gradient descent algorithm with learning rate of 0.001 and 100 epochs. We used as word embeddings vectors with 128 dimensions, generated by the FastText algorithm. The training step was performed with 80% of the data, while 20% was reserved to the test set. The results of the model trained on the Sentiment Analysis Module were: 86.3% of F1-Score and 86.5% Accuracy.

Thus, the first step of our experiments consists in evaluating the Machine Learning metrics. The ANNs were trained with the Root mean square prop (RMSProp) algorithm, which is usually a good choice for neural networks as presented in (Li et al., 2020). The following parameters were fixed: the initial learning rate was defined as 0.0001; the batch size was set to 32; the number of epochs was defined as 500 and training was performed with an early stopping mechanism, which stops the training if the loss function stops decreasing for 10 epochs. The activation function was defined as ReLu, since it is easy to compute and accelerates convegence (Zhang et al., 2018).

In this module, we have also verified the following ANNs, with increasing complexity, to find the architecture that best performs the regression task to our problem: MLP, LSTM, Bi-LSTM. Besides, to find the most suitable hyperparameters to the tested ANNs, we used grid-search alongside an increasing window cross-validation method to find the optimal values on the following parameter grid:

• Number of layers: [3,5].

• Number of nodes per layer: [5, 10,20,50,100].

• Dropout (i.e., a regularization mechanism to prevent overfitting in ANNs (Li et al., 2020)): [0,0.125,0.375,0.25,0.5].

Since the ANNs are designed to predict stock prices time series, the training step follows an increasing window cross-validation scheme, as stated in Ratto et al. (2018) and also used in (Picasso et al., 2019; Li et al., 2020). In this work, we use a 3-fold time split cross-validation to train and validate each proposed model, as presented in Fig. 6. In this approach, the time series uses the first k splits to train and the $k+1$ split to validate the network. Then, we select the model configuration that performs better considering the average of all the validation sets to measure its results in the out-of-sample test set, as presented in the next section.

Fig. 6

Incresing window cross-validation considering 3-fold time series splitting method. Adapted from Li et al. (2020)

To train the ANNs, the datasets were divided into different parts. The first one, from January, 2010 to December, 2016 is considered as training set; the second one, from January, 2017 to December, 2019 is considered the test set, that is, the out-of-sample data to measure the performances of each model. Within the training set, the last fold of data is used as validation set in order to evaluate the training phase and to act as the early-stop mechanism of the network.

Finally, as a baseline to our experiments, we used Support Vector Machines (SVM), as well as in (Li et al., 2020; Picasso et al., 2019; Maqsood et al., 2020). SVM is one of the most popular methods of Machine Learning applied to the financial field. To find the most appropriate configuration to the algorithm, we used Gaussian Kernel (as well as in Picasso et al. (2019)) and performed the search within the following values of parameter C: $10^{-3}, 10^{-2.5},..., 10^{2.5}, 10^{3}$.

To perform these experiments, we considered three scenarios: (1) only the historical stock prices, represented by OHLCV (open, high, low, close and volume); (2) historical stock prices and the technical indicators, represented by Tech. Ind.; and (3) news, technical indicators, and historical stock prices, represented by Sentiment.

The results in Table 2 were obtained from the baseline, Support Vector Machines (SVM). Tables 3, 4, 5, 6 and 7 present the results obtained from deep ANNs and the respective parameters of the ANN architecture which performed better in the experiments. The metrics considered in this case are MAE and MSE.

Table 2 Machine Learning metrics considering the baseline SVM applied to all stocks

Table 3 Machine Learning metrics from deep ANNs: MLP, LSTM, and Bi-LSTM.

Table 4 Machine Learning metrics from deep ANNs: MLP, LSTM, and Bi-LSTM. Banco do Brasil (BBAS3)

Table 5 Machine Learning metrics from deep ANNs: MLP, LSTM, and Bi-LSTM.

Table 6 Machine Learning metrics from deep ANNs: MLP, LSTM, and Bi-LSTM.

Table 7 Machine Learning metrics from deep ANNs: MLP, LSTM, and Bi-LSTM.

The results show that, in the experiments, the use of stock prices (OHLCV) alongside Technical Indicators and news sentiments improves the prediction capability of the deep ANNs considering the test set, overcoming the baseline SVM. In fact, both MAE and MSE are lower when news sentiments are also used (in datasets BVSP, BBAS3, ITUB4, ABEV3), except for the company GGBR4, which the OHLCV input performed better with a Bi-LSTM architecture. It is noteworthy that the architecture that performed better in most of the cases was the Bi-LSTM, in 4 datasets; and LSTM performed better in just 1 dataset. This result is expected since both architectures are designed to deal with time-series data and Bi-LSTMs are an evolution of LSTMs.

4.3 Investment Metrics

Once the appropriate models were selected in the previous section, we designed an investment simulation of BRL 100,000.00 in the period from January, 2017 to December, 2019 in the Brazilian stock market (i.e., the test set), as explained in Sect. 3.4. The first baseline of this experiment is the investment strategy Buy & Hold, which results are presented in Table 8.

The result of the baseline model of this experiment, SVM, is presented in Table 9, considering the metrics final Return of Investment, Maximum Drawdown and Sharpe Ratio. Finally, the results of the deep ANNs are presented in Tables 10, 11, 12, 13 and 14.

Table 8 Investment simulation considering the strategy Buy & Hold

Table 9 Investment simulation metrics using the baseline SVM considering all stocks

The obtained results in this step show that the combination of stock prices, volume, Technical Indicators, and news sentiments also improves the investment results in the Brazilian stock market, overcoming the baselines Buy & Hold and SVM considering ROI, MDD, and SR. In fact, the ROI increases when news sentiments are used (in datasets BVSP, BBAS3, ITUB4 and GGBR4), except ABEV3, which the OHLCV input performed better with a MLP architecture.

Table 10 Investment Simulation using the deep ANNs models, Ibovespa (BVSP)

Table 11 Investment Simulation using the deep ANNs models, Banco do Brasil (BBAS3)

Table 12 Investment Simulation using the deep ANNs models, Itaú (ITUB4)

Table 13 Investment Simulation using the deep ANNs models, Gerdau (GGBR4)

Table 14 Investment Simulation using the deep ANNs models, Ambev (ABEV3)

It is worth highlighting that the architectures with the best performances: the MLP, in 4 datasets; and the Bi-LSTM in just 1 dataset, BVSP. This result shows that even though an architecture performs better considering Machine Learning metrics, this does not mean that the same architecture will obtain better results considering financial metrics. That is an important point to observe, as many works in the literature only analyze Machine Learning metrics, and do not perform the investment simulation step.

Besides, the results show that there is a trade-off between ROI and other financial metrics, like SR and MDD. For example, ITUB4 presents better ROI with news sentiments, but better SR and MDD with Technical Indicators. These values demonstrate that the investor needs to analyze each dataset separately and choose what architecture fits better his investment profile. If the investor is willing to accept higher risks, he could choose MLPs with news sentiments, that will bring better ROI with a possibility of losses in the period; in the case of a conservative investment strategy, one could choose an MLP architecture with Technical Indicators, which could bring inferior ROI but with less risk than the other options.

The following section expands the discussion considering the obtained results.

4.4 Discussion

Ozbayoglu et al. (2020) showed that financial text mining has received increasing attention in the literature, as one of the most promising areas in the last years. However, as presented before, there is an open issue in the literature: how to combine news into a stock market prediction, alongside financial data? In this work, we proposed an approach that incorporates news sentiments, technical indicators, volume, and stock prices in one system. The whole process is performed with deep neural networks, that is, the news sentiments calculation and the prediction of the stock market.

There are recent works that started to address the issue of combining news and stock market prediction (Li et al., 2020; Picasso et al., 2019; Maqsood et al., 2020). They also find similar results to ours regarding the importance of news as part of a prediction system. In fact, as presented in Maqsood et al. (2020), the behavioral economics hypothesis states that there is a correlation between the market behavior and public mood. That is, the probability of larger investments in the market is higher when the public mood is positive. As such, the key task is the measurement of the public sentiment in order to predict changes in the market behavior.

In one of the few works that consider the Brazilian stock market regarding news and stock exchange, Duarte et al. (2020) emphasizes its high risk factor, which can also be linked to higher profits/losses in the operations. Considering both works, it is possible to notice the higher profitability of the emerging Brazilian market, with returns over 200% in some cases. Moreover, as presented in (Pereira, 2021), the Portuguese language lacks Natural Language Processing resources, since the literature frequently focuses on North American and Asian markets (Kumbure et al., 2022; Olorunnimbe & Viktor, 2023; Ashtiani and Raahmei, 2023).

Finally, our work emphasizes the importance of news in a stock market prediction system, both according to Machine Learning and financial metrics. In fact, our work reiterates the findings of (Bustos & Pomares-Quimbaya, 2020), which indicates that the use of textual data normally does not lead to better performances than the use of technical indicators, but shows that the prediction metrics can be improved if they are combined. However, good results in Machine Learning metrics does not imply necessarily the same good results in financial metrics. Thus, we contribute to the existing literature mainly with our proposal of a stock market prediction system with a deep learning approach combining news, technical indicators and stock prices. Also, our case study of the Brazilian stock market contributes to mitigate a lack of papers analyzing this particular market. Finally, it is worth noting that our study considered a significant period for Brazil (2010-2019), which comprises both troubled and positive times for the country.

Our experiments suggest that there is significant information present in published daily news, suggesting that the news sentiments influences the stock market and that it is worth considering these news before doing investment operations. Furthermore, before choosing one of the proposed models to invest, we believe that the investor must ponder between the return of each model as well as the risk presented by each one of them, observed with the Maximum Drawdown and the Sharpe Ratio.

5 Conclusion

This work proposed a stock market prediction approach based on news, technical indicators, and stock prices. The experiments were performed in the Brazilian financial market considering the period between January, 2010 and December, 2019. This study analyzed a decade-long period that comprises several crises and advances in the Brazilian stock market, allowing the exploration of different periods of the economy. It is worth highlighting that our results show the importance of combining financial news alongside Technical Analysis indicators and historical stock prices to support the investors’ decision in the stock market.

Our main contributions of this study are:

• An innovative approach to predict the stock market, incorporating news, technical indicators and stock prices in a process completely performed with deep ANNs.

• A comparison between the most popular ANNs architectures used in the financial field to predict the Brazilian stock market.

• An experiment showing that news improve the prediction in the stock market, both considering Machine Learning metrics and financial metrics.

Future works could explore more companies and indexes from other countries, and strategies that feed the ANNs directly with news embeddings, without sentiments.