Keywords

1 Introduction

In recent years, people’s economic conditions improved significantly, and speed up the pace of urban life, more and more people choose to drive or take a taxi, thus significantly increased traffic in big cities. According to the statistics of relevant departments, by 2021, the national motor vehicle population is up to 395 million. While private cars bring convenience to people’s travel, urban traffic problems are becoming more and more serious. Therefore, the construction of smart city has attracted more and more attention [1].

Since 1980s, there have been researches on short-time traffic flow prediction technology for Intelligent Transportation System (ITS) [2]. Up to now, researchers have put forward about dozens of different short-time traffic flow prediction methods. The famous short-time traffic flow prediction methods can be divided into non-parametric technology and parametric technology. Among non-parametric techniques, the well-known Neural Network methods include Back Propagation (BP) Neural Network [3], Generalized Regression Neural Network, GRNN) [4] and Wavelet Neural Network (WNN) [5]. Technology with parameters includes linear regression, nonlinear regression, historical average algorithm, smoothing technology, etc. According to the prediction method, it can be divided into traditional traffic flow prediction and traffic flow prediction based on deep learning.

In the early stage, scholars established traffic flow prediction models by using mathematical thinking or physical thinking and their combination, including ARMA [6], ARIMA [7] and HAM [8]. Shekhar S and Williams BM et al. proposed the application of Kalman filter model to short-term traffic flow prediction for the first time in the 19th century [9], and traffic flow prediction made a big step forward. This model can deal with stable and fluctuating data, but it cannot deal with randomness and nonlinear situations, so it has certain limitations in practical production and life.

Huang W et al. first proposed the introduction of Deep learning technology into traffic flow prediction in 2014 [10]. In 2015, Koesdwiady et al. used DBN to predict the traffic data set PeMS [11], which was a major breakthrough in the field of traffic flow prediction. In 2016, Ma et al. used long short-term memory neural network (LSTM) and microwave detector data to predict Beijing traffic speed [12]. LSTM was used to overcome error attenuation caused by back propagation through memory blocks, and showed superior performance in long-term dependent time series prediction. In a 2018 research paper, Zhuo et al. proposed a new traffic prediction model based on LSTM [13] and verified that the model had better performance.

In this paper, by comparing various intelligent optimization algorithms and combining the characteristics of each optimization algorithm, the improved cuckoo optimization algorithm is selected to optimize the random initialization parameters of the traditional neural network. Finally, the optimized network model is applied to short-term traffic flow prediction, and the original data set is preprocessed accordingly. The improved cuckoo search algorithm is compared with the standard cuckoo search algorithm to judge the optimization performance of each algorithm to the benchmark test function. The main research contents of this paper are as follows:

  1. (1)

    Aiming at the problem that the step size factor of the standard cuckoo optimization algorithm is artificially set, which leads to low optimization efficiency, this paper proposes an adaptive step size factor based on logarithm function to improve it. In order to improve the accuracy of the algorithm, gaussian perturbation is used to guide the local search of bird’s nest position. The weight traction component is also proposed to increase the relationship between individuals in the algorithm and strengthen population diversity.

  2. (2)

    Explore the methods of data preprocessing, analyze in detail the methods of outlier processing, data dimension reduction, feature extraction and other preprocessing, and finally preprocess PeMS04 data set through the above methods.

  3. (3)

    In view of the defect that LSTM model parameters need to be set manually, this paper uses the improved CS algorithm to optimize LSTM by taking advantage of cuckoo optimization algorithm’s strong global search characteristics, and then applies the optimized LSTM model to short-term traffic flow prediction. The original LSTM model and other common time series prediction models are compared to predict the same data set, to verify the prediction accuracy of the optimized LSTM.

2 Short-Term Traffic Flow Prediction Based on AWCS-LSTM Model

This chapter mainly introduces the optimization of traditional LSTM hyper-parameters based on improved CS algorithm, and applies the optimized model to traffic flow prediction. First introduced the circulation principle of neural network and its derivative network LSTM, then using improved CS algorithm (AWCS) to optimize the LSTM model, the number of iterations and the number of hidden layer neuron network, and train the prediction model. Finally, validate the improved model in PeMS04 data set; compared with the model accuracy and other evaluation indexes before and after the improvement, the experimental verification shows that the improved model has more advantages in traffic flow prediction.

2.1 Construction of Neural Network Model Based on AWCS-LSTM

Principles and Steps of LSTM Optimization by Cuckoo Optimization Algorithm

Artificial neural network model with animal neural network is simulated for inspiration, make it has the ability of learning and memory, including LSTM improved the long-term dependence of RNN defects, therefore has been widely applied in the field, the time-series data of forecasting and prediction accuracy of the model is closely related to neural network initialization parameter setting, super parameter selection directly influences the performance of the model. In general, the initial weights and thresholds of neural networks in traditional models are set manually, so they are random and not conducive to model training. Therefore, cuckoo search algorithm is introduced and improved to optimize the initial threshold in LSTM.

The flow chart of traditional LSTM model for data prediction is shown in Fig. 1:

Fig. 1.
figure 1

LSTM prediction flow chart

Full size image

Cuckoo optimization algorithm in the search for the optimal parameters showed better performance, thus introducing the LSTM, optimize the traditional LSTM model, the main idea is to the LSTM vector network, the number of iterations and the number of nodes in the hidden layer as a cuckoo search algorithm of the bird’s nest, to verify that the collection of mean square error (MSE) as the fitness value, The optimization principle of CS algorithm is used to find the optimal initial value of the model, so as to improve the fitting accuracy of LSTM model in the direction of traffic flow prediction and obtain the optimal LSTM model.

When optimizing the model, the fitness value is calculated by Eq. (1):

$${\text{fitness}}={\text{MSE}}=\frac{1}{N}{\sum }_{i=1}^{N}({f}_{i}-{y}_{i}{)}^{2}$$
(1)

fi is the predicted value, yi is the actual value, N is the number of samples, MSE is the mean square error, the greater the variance, the greater the dispersion, that is, the greater the error.

The steps of the improved cuckoo search algorithm to improve LSTM and predict traffic flow data are as follows:

Input: preprocessed PeMS04 training set and test set.

Output: the value of traffic flow at a certain time.

Step 1: Parameter setting and group initialization: Randomly generate N initial nests as required and determine their positions. Set the initialization parameters: population size N, number of parameters to be searched corresponding to dimension D, discovery probability Pa, boundary values a and B of each optimization parameter, maximum iteration times itermax and current iteration times i, etc. In this paper, dimension D = 4 is set, optimization parameters are learning rate α, network iteration times epoch, hidden layer node number N1 and n2.

Step 2: The nests generated by the cuckoo search algorithm are decoded as parameters of each layer of the neural network, and the number of parameters corresponds to the dimension D of the nest. The number of parameters is substituted into the improved AWCS algorithm process to obtain the optimal nest of the current iteration number, namely, the parameter value of the neural network.

Step 3: Take parameter values as LSTM initialization parameters to predict traffic flow data;

Step 4: Substitute the predicted value into formula (4–1) to calculate the fitness value;

Step 5: Judge whether the fitness value meets the threshold requirements or reaches the maximum number of iterations. If I ≥ itermax, stop the algorithm; Otherwise, update the nest position and get the parameter value of the next round;

Step 6: The updated optimal nest position of each dimension corresponds to the initial value of LSTM network one by one, that is, a new round of training parameters are obtained and the network is trained until the predicted results are output.

AWCS algorithm optimization LSTM traffic flow prediction process is shown in Fig. 2:

Fig. 2.
figure 2

Improved CS optimized LSTM

Full size image

Model parameter setting.

Parameter Settings of the above training process are shown in Table 1:

Table 1. AGCS/AWCS-LSTM parameter Settings
Full size table

The activation function of LSTM forgetting gate and output gate selects sigmoid function, whose value range is [0,1]. 0 means discarding information, and 1 means reserving all information, as shown in Formula (2):

$$\sigma (z)=\frac{1}{1+{e}^{-z}}$$
(2)

Tanh function is selected to activate the input gate, and the value range is [−1,1], as shown in Formula (3):

$$\mathit{tan}h(x)=\frac{{e}^{x}-{e}^{-x}}{{e}^{x}+{e}^{-x}}$$
(3)

2.2 Experimental Results and Analysis

Evaluation Indicators

After the prediction results are obtained, the pros and cons of the model and the accuracy of the prediction can be determined by evaluating the generalization performance of the prediction model. Common evaluation indexes are mean absolute error (MAE), root mean square error (RMSE), Average absolute percentage error (MAPE) and determinant coefficient (R2).

Comparison of Models Before and After Optimization

In order to verify the practical application of AWCS-LSTM model in short-term traffic flow prediction, this paper selects the public traffic flow data set PeMS04 for verification, and the data set only contains the traffic flow of working days in the selected period.

The training set contains 8778 records as samples from 2016.01.01 to 2016.02.29, and the verification set contains 2033 records as samples from 2016.03.01 to 2016.03.16. The test set selected 2480 records as test samples recorded in six days from 2016.03.17 to 2016.03.31 to verify the accuracy of the model.

In this paper, the traditional LSTM model was firstly verified, and the model training iteration times epoch was set as 10, 20, 30, 40, 50, 60, 70 and 80 respectively for comparative experiments. The evaluation index values predicted by LSTM for PeMS04 data set under different epochs are shown in Table 2:

Table 2. LSTM prediction effects under different epochs
Full size table

It can be seen from the table above, with the increase of the number of iterations, all the indicators tend to be better, but when the number of iterations is set to 80, when the back phenomenon, predicted results as the number of iterations instead of 70, so when the number of iterations is too large, not only the extension of training time, can also cause a trained fitting, resulting in a loss prediction effect. In order to display the prediction results more intuitively, the prediction curves under different iterations are visualized. The prediction results are shown in Fig. 3:

Fig. 3.
figure 3

LSTM prediction effect under different epochs

Full size image

The improved CS algorithm is used to optimize the hyper-parameters of the traditional LSTM model. The AWCS with improved adaptive step size and weight traction component are introduced into LSTM respectively. The CS-LSTM model optimized by traditional LSTM and standard CS and the improved AWCS-LSTM model are used to predict the obtained evaluation indexes on the selected data set, as shown in Table 3:

Table 3. Prediction effects of LSTM, CS-LSTM and AWCS-LSTM
Full size table

It can be seen from Table 3 that the model built by using the improved CS algorithm to optimize the hyper-parameters of the traditional LSTM network has better accuracy in predicting PeMS04. Compared with the model optimized by the traditional LSTM model and the standard CS algorithm, AWCS has better performance in optimizing the hyper-parameters of LSTM. The visualization results of prediction are shown in Fig. 4:

Fig. 4.
figure 4

Visualization of LSTM, CS-LSTM and AWCS-LSTM prediction

Full size image

The parameters of traditional LSTM, CS-LSTM and AWCS-LSTM are shown in Table 4:

Table 4. Parameter Settings of each model
Full size table

The learning rate, number of hidden layer neurons and iteration times of LSTM model are set manually, while the learning rate, number of hidden layer neurons and iteration times of CS-LSTM and AWCS-LSTM are optimized by corresponding algorithms.

3 Conclusion

Aiming at the problems of traditional traffic flow forecasting method, this paper proposes using the intelligent optimization algorithm of neural network parameters optimization of ideas, and set up a neural network based on improved cuckoo search algorithm and short - and long-term memory of traffic flow prediction model, using the improved algorithm used to optimize the LSTM AWCS random initial parameters of the model, LSTM, CS-LSTM and AWCS-LSTM were applied to the preprocessed PeMS04 data set respectively, and the effectiveness of the optimized model was verified by experiments.