Keywords

1 Introduction

Accurate prediction of stock prices in the stock market is a significant challenge for investors. In recent years, the interest of retail investors has grown significantly, especially among individuals under the age of 35, driven by access to investment platforms and the search for lucrative financial opportunities [13]. In order to mitigate the risks associated with these investments, it is crucial to analyze and accurately predict stock prices.

In this paper, we evaluate the impact of volatility on the accuracy of a stock price prediction model based on neural networks. To conduct this study, we select two stocks, one considered non-volatile and the other volatile, in order to analyze how volatility affects the predictive capability of the model. The chosen stocks were carefully selected from the stock market, representing different levels of volatility.

To achieve our objective, we implemented and compared three neural network models: RNN (Recurrent Neural Networks) [18], LSTM (Long Short-Term Memory) [19], and Feedforward Neural Networks [5]. These neural network architectures have proven to be effective in time series prediction and are therefore considered appropriate for addressing the stock price prediction problem.

The main objective of this study is to evaluate how volatility impacts the accuracy of neural network models in stock price prediction. The results obtained from the evaluation and comparison of the three neural network models will help identify which model is more effective at different levels of volatility. Additionally, this study contributes to advancing knowledge in the field of applying neural networks in stock price prediction and provides valuable insights for investors interested in the stock market, as well as the scientific and academic community involved in machine learning applied to finance [2].

The methodology employed in this article is based on the lifecycle architecture of data mining and analysis projects. Specifically, it follows the approach proposed by CRISP-DM (Cross Industry Standard Process for Data Mining), which provides a structured methodology for conducting data mining projects. This methodology starts with the analysis of the business problem and translates it into a technical data mining problem, thereby facilitating the execution of the project [20].

This document follows the following structure. Related works are presented in Sect. 2. The objective and approach of the article are described in Sect. 3. The understanding of the data used is explained in Sect. 4. Subsequently, data organization and preparation are carried out in Sect. 5. The modeling of neural networks is described in Sect. 6, while the obtained results are detailed in Sect. 7. Finally, conclusions are presented in Sect. 8.

2 Related Work

Table 1 presents previous work that uses various neural network architectures for the prediction of stock prices. Most of these networks focus on forecasting the future value of a specific stock within a given time period. As can be observed from the table, a wide range of different methods and neural networks have been used for prediction in the field of individual stocks and the stock market in general. The three most popular techniques are feedforward networks (e.g., , used by Hammad et al.  [8] and Iguarán Cortés et al.  [10]), Long Short-Term Memory networks (LSTM) (e.g., used by Cabezón et al.  [3]), and Recurrent Neural Networks (RNN) (e.g., used by Deyet al.  [4]). Other work, e.g., Pawar et al.  [15] and Khoa et al.  [11], concentrate on predicting the price of a widely recognized global index. In contrast, our paper, aims to predict prices for two stocks with different degrees of volatility, and compare the obtained results.

Table 1. Survey of related work
Full size table

3 Business Understanding

The prediction of stock prices is a crucial task in the financial field, as it allows investors to make better informed decisions and improve their profit opportunities. In this context, the main objective of this article is to analyze and compare the effectiveness of different neural network models in predicting the prices of stocks belonging to two telecommunications companies: Verizon, which is listed on the New York Stock Exchange, and T-Mobile, listed on NASDAQ.

Understanding the results of this study has important implications for investors and financial market professionals. The findings and conclusions presented in this article can be used as a basis for the development of more informed investment strategies and the improvement of financial advisory systems. Additionally, the knowledge gained through this study can help investors better understand and manage the risks associated with market volatility.

The proposed study is relevant and significant due to the importance of accurate stock prediction in the financial market. Understanding how stock volatility can influence the performance of neural networks in price prediction is essential for making informed and strategic investment decisions. The results of this research can contribute to the field of finance and provide valuable information to investors and market professionals to enhance their prediction strategies and decision-making processes.

4 Data Understanding

In this section, a detailed analysis of the datasets used in the study is presented. The objective is to understand the nature of the collected data and how it was used to address the problem of stock price prediction based on neural networks, evaluating the impact of volatility on model accuracy.

4.1 Stock Price Dataset

To obtain the first dataset, the Yahoo Finance serviceFootnote 1 was used, which is a platform that provides financial information and market insights, with a particular focus on the United States. Two data files were downloaded, one for the Verizon stock, which is considered non-volatile due to its low standard deviation, and another for the T-Mobile stock, which is considered volatile. Both selected stocks are part of the telecommunications sector in the United States. The dataset used spans from September 2020 to February 2023, representing a total of 582 business days. These daily data contain relevant information including open prices, close prices, high and low prices, trading volume, and other relevant indicators.

4.2 News Dataset for Sentiment Analysis

The second dataset was generated by extracting news articles from different media sources using APIs. Services such as mediastacksFootnote 2 and gnews.ioFootnote 3 were used to collect news articles from the year 2020 to 2023. The choice of this dataset is based on the premise that the sentiments expressed in the news articles influence (or may reflect) the stock market behavior. To perform sentiment classification analysis, the NLTK.sentimentFootnote 4 library, was used for natural language processing in Python. This library enables each news article to be classified as positive, negative, or neutral.

4.3 Data Exploration

The initial analysis revealed primarily numerical data, such as daily open, high, low, and close prices, as shown in Fig. 1, as well as transaction volume. These data cover every business day from the period from 2020 to 2023 (Table 2).

Fig. 1.
figure 1

Stock prices from 2020 to 2023

Full size image
Table 2. Statistical measures of the stocks
Full size table

Average True Range (ATR). The Average True Range (ATR) is a technical indicator that provides a measure of the volatility of a financial asset [21]. The calculation of ATR involves determining the greatest difference among the following three values for each time period. Once these differences are calculated for each period, they are averaged to obtain the ATR. The higher the value of ATR, the greater the volatility of the stock.

$$\begin{aligned} Percentage Range = \frac{\left( Max - Min \right) }{Mean} \cdot 100 \end{aligned}$$
(1)

In the case of T-Mobile:

$$\begin{aligned} Percentage Range = \frac{\left( 152.41 - 55.58 \right) }{105.39} \cdot 100 \end{aligned}$$
(2)
$$\begin{aligned} Percentage Range = 91.83 \end{aligned}$$
(3)

The percentage range of 91.83 indicates that T-Mobile stock has experienced a relative volatility of 91.83% in relation to its average prices. The higher the value of the percentage range, the higher the perceived volatility.

In the case of Verizon:

$$\begin{aligned} Percentage Range = \frac{\left( 62.07 - 35.34 \right) }{53.60} \cdot 100 \end{aligned}$$
(4)
$$\begin{aligned} Percentage Range = 49.87 \end{aligned}$$
(5)

The percentage range of 49.88 indicates that Verizon stock has experienced a relative volatility of 49.88% in relation to its average prices. Comparing the percentage ranges, we can see that T-Mobile has a higher percentage range (91.83%) compared to Verizon (49.87%). This indicates that T-Mobile is more volatile than Verizon, meaning it has experienced larger price fluctuations relative to its average. Therefore, in terms of volatility, T-Mobile is the more volatile stock compared to Verizon.

5 Data Cleaning and Preparation

Once the two datasets are obtained, they are integrated. Since the news data is extracted daily, it is necessary to combine it with the corresponding stock price data for each day. To do this, the sentiment analysis results of the news are averaged for each day. This approach allowed an overall view of the sentiment associated with each day and its potential influence on the stock price behavior. Thus, a combined dataset was obtained that includes both the financial information and the sentiments associated with each day.

In summary, the datasets used in this study consist of daily financial information for two stocks and sentiment classification analysis of news. This data is obtained from Yahoo Finance and various media sources using APIs. The integration of both datasets allows for a more comprehensive and balanced view of the factors that can influence stock price behavior (Table 3).

Table 3. Dataset attributes used for prediction
Full size table

The data used in the construction of the prediction models was obtained following the previously established requirements, which consist of using only information from events between September 2022 and February 2023 using the Pandas Library. The training data contains only data corresponding to the months of September 2022 to January 2023. The test data set contains data for the month of February 2022. This data is reserved for evaluating the performance and accuracy of the models. By limiting the training data to the specific period mentioned, we ensure that the models are trained and evaluated using up-to-date information relevant to the target prediction period.

6 Modeling

Three different neural networks are created and trained with an architecture consisting of an input layer, a hidden layer and an output layer. The number of neurons in the hidden layer was varied in three different configurations: 5, 10 and 25 neurons. This resulted in a total of nine different networks for each stock. It is important to note that all networks were constructed using the same library and dataset, with the purpose of obtaining comparable results. The hyperbolic tangent (tanh) activation function was used along with the Adam optimizer. The hyperbolic tangent function is a commonly used activation function in neural networks due to its ability to generate values between -1 and 1. This allows the neural network to have a smoother output and less prone to errors. On the other hand, the Adam optimizer is an optimization algorithm commonly used in neural networks due to its ability to adapt the learning rate during training. For each network, 100 epochs were used during training. The number of epochs determines how many times the entire dataset is passed through the network during training.

7 Results

The performance of each model is evaluated using metrics including MSE, RMSE and MPE, to quantify the error in predicting stock prices. By evaluating these metrics over a period of 19 business days in the month of February 2022, we determine the effectiveness of each model in capturing patterns and trends, as well as its ability to generalize for future data.

Feedfoward Network. Figure 2 presents the results of the experiments conducted for the feedforward network. It is observed that as the number of neurons in the hidden layer increases, the prediction error decreases for both Verizon and T-Mobile. This pattern suggests that a higher number of neurons in the hidden layer improves the accuracy of the stock price prediction model.

Table 4. Summary of results for the feedfoward network
Full size table

Table 4 summarises the results obtained for the feedforward network for both stocks. It is observed that the model exhibits better performance and lower error as the number of neurons in the hidden layer increases. However, when comparing the results between the two analyzed stocks, no significant distinction in terms of error is observed. It is interesting to note that both stocks, with a hidden layer of 25 neurons, obtained very similar results, with an MSE of 0.0073 for T-Mobile and 0.0070 for Verizon.

Overall, the Verizon stock, considered a non-volatile stock, shows better performance in terms of all three errors metrics compared to the more volatile T-Mobile stock. This suggests that the volatility of a stock can influence the outcome of neural network prediction, making it more challenging to accurately predict the behavior of volatile stocks compared to non-volatile stocks.

Furthermore, it is important to mention that, despite the favorable results obtained in this study with the feedforward network, there are certain limitations in its ability to accurately predict stock prices. The static nature of this network architecture may hinder the capture of complex patterns and trends present in financial data. As such, the feedforward network lacks the ability to model long-term dependencies and temporal sequences that are inherent in stock prices. Therefore, in scenarios where greater consideration of temporal dynamics and the ability to capture complex relationships is required, it is worth exploring other neural network architectures such as LSTM or RNN, which are known for their ability to model and predict timeseries data.

Fig. 2.
figure 2

Predictions for the feedfoward network

Full size image
Fig. 3.
figure 3

LSTM network predictions

Full size image

LSTM Network. Figure 3 depicts the predictions for February 2022 obtained through experiments conducted with the LSTM network. It is observed that this neural network architecture outperforms the feedforward architecture in predicting stock prices. This can be attributed to the inherent characteristics of the LSTM network, such as its ability to capture long-term dependencies, handle sequential data, and learn complex patterns. These capabilities enable the LSTM network to effectively adapt to the various time scales present in stock prices.

Table 5. Summary of LSTM network results
Full size table

According to the results presented in Table 5, an interesting trait is observed in the case of the volatile T-Mobile stock. Despite increasing the number of neurons in the LSTM network, no improvement in prediction error was obtained. In fact, it can be observed that with 25 neurons, the error increases, with an MSE of 0.0058, compared to 10 neurons that achieved an MSE of 0.0042.

This result suggests that, for this particular stock, a higher number of neurons in the LSTM network does not necessarily result in an improvement in prediction accuracy, possibly due to overfitting. It is possible that T-Mobile stock exhibits patterns and characteristics that do not benefit from an increase in model complexity. Therefore, it is important to consider the specific characteristics of each stock when selecting the appropriate network architecture.

In general, the results support the choice of LSTM network as an effective tool for predicting stock prices due to its ability to capture long-term dependencies, handle sequential data, learn complex patterns, and adapt to different time scales. However, they also highlight the importance of considering the individual characteristics of each stock when tuning the hyperparameters of the neural network. As such, it is necessary to consider the specific features of each stock, such as volatility, when selecting the optimal number of neurons to achieve the best predictive performance.

Fig. 4.
figure 4

Predictions for the RNN

Full size image

Figure 4 shows that the recurrent neural network (RNN) achieves better results compared to the feedforward network. This indicates that the RNN’s ability to process sequential data and capture contextual information over time is beneficial for stock price prediction.

Table 6. Summary of results for the RNN
Full size table

Based on the results provided in Table 6, we observe a similar scenario as previously seen in the LSTM model, where increasing the number of neurons in the recurrent network does not lead to an improvement in the prediction accuracy for the T-Mobile stock. Despite efforts to increase the complexity of the model, it is found that the error either remains the same or even increases. This suggests the possibility of overfitting, where the model becomes too specialized to the training data and fails to generalize well to new data. It highlights the importance of finding the right balance between model complexity and generalization ability to avoid overfitting in neural networks.

This result may be attributed to the specific characteristics of the T-Mobile stock and its behavior in the market. It is possible that factors not captured by a higher number of neurons in the recurrent network influence the volatility and patterns of this particular stock.

7.1 Cross-Validation

Cross-validation is a technique that allows us to evaluate the performance of a model using multiple divisions of the dataset. In this study, cross-validation is applied to ensure that the models were evaluated in an unbiased manner and that the obtained results are statistically robust.

By using cross-validation in this study, more reliable measures of the performance of neural network models were obtained. This technique allows us to reduce the bias introduced by a single data partition. As a result, we can have greater confidence in model ability to generalize for future data and capacity to handle the inherent uncertainty in financial markets. The dataset was partitioned into 10 subsets, with each subset used as the test set once and the remaining 9 subsets used for model training in each iteration. Additionally, the folds were created by randomly selecting samples.

Fig. 5.
figure 5

Cross-validation

Full size image

Feedforward Network. In cross-validation, the architecture previously found to exhibit best performance is used (i.e., with a hidden layer 25 neurons). According to the results presented in Fig. 5, some splits showed a relatively low mean squared error (MSE), indicating good prediction capability of the model in those cases. However, other splits exhibited higher MSE, suggesting lower prediction accuracy in those particular divisions.

These results indicate that model performance can vary depending on the input data used and their distribution at different time points. It is important to consider this variability when interpreting the results and assessing model ability to generalize to future data. Furthermore, the presence of splits with significantly higher MSE, such as split 9 and split 10 for Verizon stock, suggests that there are certain scenarios or patterns in the data that pose challenges for the feedforward neural network model. This may be due to the presence of nonlinear features or complex temporal dependencies in those specific datasets.

LSTM Network. Based on the findings shown in Fig. 5, the LSTM network demonstrates promising performance in predicting stock prices, as most of the splits exhibit low MSE values.

For Verizon stock, the LSTM network shows consistency in its performance, with relatively low MSE values in the majority of splits. This indicates that the LSTM network is capable of effectively capturing patterns and trends in Verizon’s data. However, for T-Mobile stock, there is greater variability in the results. Some splits exhibit outstanding performance with very low MSE values, while others show poorer performance with higher MSE values.

It is important to note that Split 3 for T-Mobile stock displays a significantly higher MSE compared to other splits. This suggests that there are specific features or patterns in T-Mobile’s data that are more challenging to capture for the LSTM network, resulting in lower prediction accuracy in that particular split.

Recurrent Neural Network. In Fig. 5 it can be observed that for Verizon stock, the MSE values range from 0.002673 (split 6) to 0.282497 (split 9). These results indicate some variability in the predictions of the network in each case, with significant variations in performance. It is noteworthy that the lowest MSE values were achieved in split 4 and split 10, suggesting higher prediction accuracy for those specific scenarios.

On the other hand, in the case of T-Mobile stock, the MSE values obtained in each split range from 0.0015 to 0.1744. As with Verizon, there is considerable variation in the network’s performance across different scenarios. However, it is worth noting that the lowest MSE was achieved in split 2, indicating higher prediction accuracy for that particular case.

In general, it can be concluded that the recurrent network exhibits mixed results in predicting both stocks. While accurate results are achieved in some splits, inferior performance is observed in others.

In summary, the results of the cross-validation highlight the importance of evaluating model performance on multiple data splits and considering the variability in performance. They also indicate the need to further explore the specific characteristics and patterns that can affect the accuracy of the networks in predicting stock prices.

8 Conclusions

In this study, the use of machine learning techniques, specifically neural networks, is explored to predict stock prices in the financial market. Using datasets provided by Yahoo Finance, three neural network models are developed and compared: Feedforward, LSTM, and RNN. We find that neural networks, particularly the LSTM model, demonstrate promising ability to predict stock prices. The results of these models were able to capture trends and patterns in historical price data, allowing them to make predictions with acceptable accuracy. Additionally, it is observed that the volatility of a stock can impact the prediction error. More volatile stocks exhibit a higher degree of uncertainty in predictions, indicating that volatility may be an important factor to consider when using neural network models for stock price prediction. However, it is important to note that this conclusion was based on one volatile and one non-volatile stock and experiments with more stocks would be needed to reach a more general conclusion. Overall, our findings suggest that the use of machine learning techniques, such as neural networks, can be a valuable tool for investors interested in the stock market. These techniques can provide additional information and analysis to support more informed investment decision-making. In conclusion, this study demonstrates the potential of neural networks for predicting stock prices and highlights the importance of considering volatility as an influential factor in prediction accuracy. These results may be of interest to both investors in the stock market and the scientific and academic community interested in the application of machine learning in finance.