1 Introduction

A stock market is a group of exchanges in a place where a publicly listed company’s shares can be sold or purchased. To invest in any company’s stock, and as there is no alternative way to determine the company’s performance on the stock market, everyone accepts data analysis before investing. There are two widely known stock market analysis techniques: fundamental and technical analysis [1]. The analysis technique identifies a company’s worth by examining related economic and financial (earnings per share, price to earnings, price to book value, debt per equity, return on equity) factors [1]. On the other hand, technical analysis uses statistics to find the pattern in the treading activity [1]. According to technical analysis, any asset’s previous trading data and price variation help predict future price fluctuations. Based on the previous price patterns, prediction can be completed. Two standard technical analysis techniques are simple moving average (SMA) and exponential moving average (EMA). The main goal of the research is to compare the two most popular technical analysis techniques with a deep learning model in predicting stock prices [2].

A dynamic feedback neural network is needed to predict the stock price; RNN-based architecture long short-term memory (LSTM) has been used in this analysis [3]. Throughout the analysis for predicting stock price, LSTM has few errors compared to SMA and EMA. A method of identifying a security’s fundamental worth by examining related economic and financial factors is known as fundamental analysis. Fundamental analysts look at everything that might influence the value of a security, from macroeconomic indicators like the health of the economy and industrial circumstances to microeconomic elements like the company’s management performance.

The aim is to obtain a value that can be compared to the present price of an asset to determine whether it is overvalued or undervalued, which anticipates the price patterns by analyzing previous market information, such as price and volume, compared to this stock analysis approach. The rest of the manuscript is organized as follows: Sect. 2 describes the related works, whereas Sect. 3 provides concise information about the dataset. Section 4 presents the method analysis for each model; specifically, Sect. 5 shows the main results and discusses the relevance of our findings and the advantages of the model to compare other models. Finally, the main conclusion drawn from this study is presented in Sect. 6.

2 Related works

Minimal work has been done on technical analysis and deep learning in stock price prediction [4]. In recent years, with the help of new technologies and tools, researchers have always tried to explore new techniques to predict stock prices [5]. Though it is quite impossible to predict the accurate future price of the stock market, to make a secure investment, it is necessary to have the technical approaches for a better outcome. The classification algorithm supports vector machines [6] with the help of feature selection PCA (principal component analysis), which will succeed in making a better prediction [7, 8]. Some work has been done in the previous research; there, LSTM model was compared with other machine learning models like SVM, and it is found that LSTM works better than SVM. The error rate of LSTM is less compared to SVM, and as a result, this is more acceptable when deciding on investment [4, 9, 10]. Some work has been done on support vector regression (SVR) to predict stock prices, which have a significant outcome in stock market price prediction [11,12,13]. In most research, LSTM performs better than other algorithms in terms of error analysis [14,15,16,17]. Feature engineering techniques played a significant role in some research work, where they achieved much higher accuracy [15, 17]. Reinforcement learning with LSTM can make successful predictions of stock prices, as proposed by some researchers [16, 18]. The previous research also compares LSTM and multilayer perceptron (MLP) [17]. In most research work, LSTM is used as a typical deep learning method for most predictions [12, 13, 19,20,21].

In [22]. The ant lion optimize (ALO) is used in the proposed LSTM model to maximize the number of hidden layer neurons. The first investigation into the use of ALO in adjusting parameters of the LSTM model. Evolution may interact with the LSTM during training and automatically process the LSTM model’s parameters; the LSTM-ALO not only inherits the LSTM’s benefits but also uses its superiority of optimization issues [23].

The weighted mean of vectors (INFO) optimizer is a memory cell with a particular neuron structure that is part of the LSTM model [24]. The LSTM-INFO model has the expertise to supply accuracy in the information output at a random time [25].

Bayesian model averaging (BMA) with gradient boosted regression trees (GBRT) is a predictive model. The BMA model performs slightly better than the GBRT, a statistical technique used to combine the predictions of multiple models, evaluate predictive performance accuracy, and model generalization [26, 27].

Reham et al. [28] random vector functional link (RVFL) and relevance vector machine (RVM), enhanced metaheuristic algorithms, and quantum-based avian navigation optimizer algorithm (QANA) for a significant problem for water resources planning and control modeling in evapotranspiration \((ET_0)\) to analyze the efficiency of different machine-learning methods. Adaptive neuro-fuzzy inference system (ANFIS) findings outperformed computing observed models. The artificial neural network (ANN) provided more satisfactory effects than the gene expression programming (GEP) among the delicate computing prototypes. Moth flame optimization (MFO) and water cycle optimization (WCA) integrated with the heuristic algorithm Hybrid Adaptive Neuro-Fuzzy Inferencing (ANFIS-WCAMFO) utilizing different climatic input data. The ANFIS-WCAMFO executed is superior to the state-of-the-art approaches to evaluate all input combinations of \(ET_0\) [29].

3 Datasets

In this research, reduce the dimensionality of the preprocess data to capture the underlying structure or patterns to represent all variance in the data. This can result in accurate results. Given the volatility of stock prices, they are not just numbers generated at random and, as a result, analyzed as a collection of discrete-time data or time-series measurements recorded at various points in time. Time series prediction can help in stock forecasting (forecasting outcomes based on prior values). The sequential model of time-series data includes adopting a strategy to gather this data sequence. Moving average (MA) is the most basic approach among all other choices for smoothing out short-term volatility. Six different stock price datasets from six other companies have been used for the experiment. The datasets are from Apple Inc., Amazon Inc., Google LLC, Tesla Inc., Netflix Inc., and BEXIMCO Pharmaceutical [30,31,32,33,34,35]. The standard features of all six company’s stock price datasets are date, low, high, open, and close. These features represent the daily historical data of any company’s stock price. High and low represent the highest and lowest transacted amount at a given time. Open and close represent the opening and closing prices at a given time. Our experiment focuses on the closing price of any company’s stock data, which is formatted in CSV file format. Figure 1 shows the Apple Inc. stock price sample dataset.

Fig. 1
figure 1

Samples of Apple Inc stock price dataset

Full size image

4 Method analysis

The two most often used techniques for predicting stock prices are simple moving average (SMA) and exponential moving average (EMA). These average moving techniques are widely used to smooth out the random movement in the stock price. SMA calculates the mean value of a given range of data. In the stock price, SMA calculates the closing price of a given time, especially to forecast the next closing price, and continues the process.

$$\begin{aligned} \text {SMA} = \frac{P_1 + P_2 +\cdots + P_n}{N} \end{aligned}$$

Here, N is the number of ranges and \(P_n\) is the price at time n [1].

EMA is slightly different from SMA. In EMA, the recent prices of the timeframe get the highest priority. EMA is also called the weighted moving average because of the variation of importance. The weighting factor is determined by the periods.

$$\begin{aligned} \text {EMA} = P_t * k + \text {EMA}_{t-1} * (1-k) \end{aligned}$$

Here, \(P_t\) is the price at time t. \(\text {EMA}_{t-1}\) is the previously calculated EMA and k is the weighting factor [1]. Calculate SMA and EMA, and there are two standard periods, which are 50 days and 200 days. Our research has found that a 50-day moving average is used as it is more accurate than a 200-day moving average.

4.1 LSTM in stock price prediction

LSTM is a deep learning architecture based on RNN. LSTM has feedback connections, unlike standard feedforward networks. With the help of previous research, it is found that LSTM shows better results than other existing machine learning and deep learning algorithms. LSTM is a very powerful algorithm that can predict future value with high accuracy by capturing historical patterns in the data. Figure 2 represents the cell state of LSTM.

Fig. 2
figure 2

LSTM cell state

Full size image

4.2 Error calculation methods

The root-mean-square error, or RMSE, formula, often used to calculate errors, calculates the predictions’ standard deviation.

4.2.1 Root-mean-square error (RMSE)

The commonly used error calculation formula known as root-mean-square error (RMSE) determines the standard deviation of the predictions.

$$\begin{aligned} \text {RMSE} = \sqrt{\frac{1}{N} * \sum _{n=1}^{N} (At -Ft)^\wedge {2}} \end{aligned}$$

Here, the total number of data points is N, the actual price is represented by \(A_t\), and the forecast price is \(F_t\) [36].

4.3 Mean absolute percentage error (MAPE)

The error calculation function, known as mean absolute percentage error (MAPE), calculates the absolute difference between the actual and anticipated values and delivers the error percentage.

$$\begin{aligned} \text {MAPE} = \sqrt{\frac{1}{N} * \sum _{n=1}^{N} \left| \frac{At -Ft}{At}\right| } \end{aligned}$$

Here, the total number of data points is N, the actual price is represented by \(A_t\), and the forecast price is \(F_t\) [37] (Table 1).

Table 1 Advantages of LSTM compare to other models
Full size table

5 Result analysis

Significant results have been found after running the experiment on the six companies’ datasets, highlighted below. The experiment was done in two steps. In the first step, the models are tested for short-term stock price prediction, where the models predict the recent day’s stock closing price. In the second step of the experiment, the models are tested for long-term prediction, where the models predict the stock price for a more extended period.

Specifying hyperparameters for the LSTM model to predict the Jakarta Composite Index JKSE stock price are C = 100, \(\epsilon =0.01\), and 3.

The MAPE and RMSE obtained with the RBF kernel are 0.30 and 0.34, respectively. Because stock prices are constantly changing and there are no consistent min and max values in the stock market, StandardScaler was chosen over MinMaxScaler.

In the feed-forward network, since signals can only flow one way from input to output and there is no feedback, the output of one layer does not influence the output of another layer. A recurrent neural network (RNN) allows signals to travel in both directions because there is feedback and feedback networks are incredibly powerful, dynamic, and complex. Their state is continuously changing until they reach an equilibrium point. To predict stock prices, a dynamic feedback network is needed to follow the stock price’s unpredictable trajectory dynamically. For these reasons, LSTM has been decided to use in this study, which is an RNN architecture.

5.1 Short-term stock price movement

The trend charts show Apple’s stock price prediction on short-term movement. By analyzing the graph shown in Figs. 3 and 4, it can be seen that LSTM shows more accurate results than SMA and EMA in the short-term stock price movement, respectively. Figure 5 represents the LSTM trend for Apple’s stock. Figures 6 and 7 define Apple’s stock price changes for short-term movement in an easily understandable way. These scatter diagrams have been used to make the visualization of the price changes clearer of SMA and EMA.

Fig. 3
figure 3

SMA trend chart for Apple’s stock price (short-term movement)

Full size image
Fig. 4
figure 4

EMA trend chart for Apple’s stock price (short-term movement)

Full size image
Fig. 5
figure 5

LSTM trend chart for Apple’s stock price (short-term movement)

Full size image
Table 2 Errors of SMA, EMA, and LSTM on different stock price datasets for short-term
Full size table

The same techniques have been used on five more datasets; the findings are listed below. Table 2 shows LSTM has less error in all six datasets. It can be seen that LSTM performs better than SMA and EMA in terms of short-term stock price predictions.

Fig. 6
figure 6

SMA scatter diagram for Apple’s stock price (short-term movement)

Full size image
Fig. 7
figure 7

EMA scatter diagram for Apple’s stock price (short-term movement)

Full size image

5.2 Long-term stock price movement

From the above trend charts, it can be seen that LSTM is not performing well on Apple’s stock price prediction on long-term movement. SMA and EMA are performing better than LSTM in this case. In Fig. 10, it can be seen that LSTM predictions are gradually falling over time. In contrast, SMA and EMA in Figs. 8 and 9 are continuously predicted with minimum error (Fig. 10).

Fig. 8
figure 8

SMA trend chart for Apple’s stock (long-term movement)

Full size image
Fig. 9
figure 9

EMA trend chart for Apple’s stock (long-term movement)

Full size image
Fig. 10
figure 10

LSTM trend chart for Apple’s stock (long-term movement)

Full size image
Table 3 Errors of SMA, EMA, and LSTM on different stock price datasets for long term
Full size table

Table 3 shows that LSTM has a higher error than SMA and EMA in all six datasets. From the findings in Table 3, it is clear that moving average techniques perform better than LSTM regarding continuous stock price prediction. In Table 4 shows in contrast, LSTM performs better in immediate stock price prediction.

Table 4 Compare different error calculations methods
Full size table

Figure 11 represents the comparison of the three different techniques of lower error predictions and higher accuracy.

Fig. 11
figure 11

Comparison of the three different techniques of lower error predictions and higher accuracy

Full size image
Fig. 12
figure 12

LSTM scatter diagram for Apple’s stock price

Full size image

Figure 12 This scatter diagram has made a clear visualization as well as an understanding of the LSTM for Apple stock price, where, along with the scatter plot, a linear regression line has been developed.

Fig. 13
figure 13

Apple stock price using scatter diagram

Full size image

Figure 13 Scatter diagram of Apple stock price has been developed from 1984 to 2019, and based on this estimation, forecasting has also been done for the year 2024.

6 Conclusion

As the stock market price prediction is quite impossible, different techniques are applied for a better outcome. A comparison has been shown between three types of stock price prediction techniques. Two of them are technical stock market analysis models known as SMA and EMA, and the third method is a deep learning model known as LSTM. Six different companies’ stock price datasets have been used for the experiment of these three methods. The experiment results show that SMA and EMA techniques perform better in continuous stock price prediction, but LSTM performs better than SMA and EMA in short-term stock price prediction. The RMSE is 12.312, and MAPE is 2.06% using the LSTM model, which is the lowest among the other techniques. Based on the EMA model, the RMSE value is 36.676, and MAPE is 10.71%, the second-lowest error among the three models. The SMA model shows the highest errors among these three models, with an RMSE value of 43.768 and MAPE value of 12.53%.

In future work, cross-validation and feature engineering can be used with LSTM to improve the model’s performance-a new RNN-based deep learning model can extend short-term memory to LSTM for stock price prediction.