Recurrent/Seasonal Mechanism to Improve the Accurate Level of Forecasting

Abstract

As demonstrated in Chap. 4, different hybrid chaotic evolutionary algorithms, including chaotic genetic algorithm (CGA), chaotic simulated annealing (CSA) algorithm, chaotic cloud simulated annealing (CCSA) algorithm, chaotic GASA (CGASA) algorithm, chaotic particle swarm optimization (CPSO) algorithm, chaotic ant swarm (CAS) optimization algorithm, chaotic artificial bee colony (CABC) algorithm, and chaotic immune algorithm (CIA), are employed to determine suitable parameter combination of an SVR-based electric load forecasting model. These forecasting results indicate that almost all SVR-based models with different hybrid chaotic evolutionary algorithms are superior to other competitive forecasting models (including ARIMA, GRNN, and TF-ε-SVR-SA models). However, these hybrid chaotic evolutionary algorithms still do not provide satisfactory forecasting performance (well fitting the actual fluctuation tendency) even their forecasting accuracy receives significant level. To improve the fitting effects for each SVR-chaotic-/cloud-evolutionary algorithm-based model, this chapter introduces two combined mechanisms (recurrent mechanism or seasonal mechanism) to significantly improve the fitting effects with the actual fluctuation tendency.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

As demonstrated in Chap. 4, different hybrid chaotic evolutionary algorithms, including chaotic genetic algorithm (CGA), chaotic simulated annealing (CSA) algorithm, chaotic cloud simulated annealing (CCSA) algorithm, chaotic GASA (CGASA) algorithm, chaotic particle swarm optimization (CPSO) algorithm, chaotic ant swarm (CAS) optimization algorithm, chaotic artificial bee colony (CABC) algorithm, and chaotic immune algorithm (CIA), are employed to determine suitable parameter combination of an SVR-based electric load forecasting model. These forecasting results indicate that almost all SVR-based models with different hybrid chaotic evolutionary algorithms are superior to other competitive forecasting models (including ARIMA, GRNN, and TF-ε-SVR-SA models). However, these hybrid chaotic evolutionary algorithms still do not provide satisfactory forecasting performance (well fitting the actual fluctuation tendency) even their forecasting accuracy receives significant level. To improve the fitting effects for each SVR-chaotic-/cloud-evolutionary algorithm-based model, this chapter introduces two combined mechanisms (recurrent mechanism or seasonal mechanism) to significantly improve the fitting effects with the actual fluctuation tendency.

5.1 Combined Mechanisms

5.1.1 Recurrent Mechanism

For a feed-forward neural network, links may be established within layers of a neural network; these types of networks are so-called recurrent neural networks (RNNs). The main concept on which RNNs are based is that every unit is considered as an output of the network and the provision of adjusted information as input in a training process [1]. RNNs are extensively applied in time series forecasting, such as Jordan recurrent neural network model [2] (Fig. 1.1), Elman recurrent neural network model [3] (Fig. 1.2), and Williams and Zipser recurrent neural network model [4] (Fig. 1.3). These three models mentioned all consist of multilayer perceptron (MLP) with a hidden layer. Jordan networks have a feedback loop from the output layer with past values to an additional input, namely, “context layer.” Then, output values from the context layer are fed back into the hidden layer. Elman networks have a feedback loop from the hidden layer to the context layer. In Williams and Zipser networks, nodes in the hidden layer are fully connected to each other. Both Jordan and Elman networks include an additional information source from the output layer or the hidden layer. Hence, these models use mainly past information to capture detailed information. Williams and Zipser network takes much more information from the hidden layer and back into themselves. Therefore, Williams and Zipser networks are sensitive when models are implemented [5]. Jordan networks and Elman networks are suited to time series forecasting [6, 7]. In this book, the Jordan network is employed as a base to construct the recurrent SVR models.

In a Jordan recurrent neural network, all neurons in a layer except those in the context layer are connected with all neurons in the next layer. A context layer is a special hidden layer. Interactions only occur between neurons in the hidden layer and those in the context layer. For a Jordan network with p inputs, q hidden, and r output neurons, the output of the nth neuron, ${f_n}(t)$, is shown as Eq. (5.1) [8–11]:

$$ {f_n}(t)=\sum\limits_{i=1}^q {{W_i}{\varphi_i}(t)+{b_i}} (t) $$

(5.1)

where W _i are weights between the hidden and output layers and ϕ _i (t) is the output function of the hidden neurons, which is as Eq. (5.2),

$$ {\varphi_i}(t)=g\left( {\sum\limits_{j=1}^P {{v_{ij }}} {x_j}(t)+\sum\limits_{k=1}^s {\sum\limits_{v=1}^r {{w_{ikv }}{f_v}(t-k)+{b_i}} } (t)} \right), $$

(5.2)

where v _ij are weights between the input and the hidden layer, w _ikv are weights between the context and the hidden layer with k delay periods, and s is the total number of context layers in past output data.

Back-propagation yields gradients for adapting weights of a neural network. The back-propagation algorithm is presented as follows. First, the output of the nth neuron in Eq. (5.2) is rewritten as

$$ {f_n}(t)=h({x^T}(t)\phi (t)), $$

(5.3)

where h(·) is the nonlinearity function of x ^T(t) and f _n (t); x ^T(t) = [x ₁(t), …, x _p (t)]^T is the input vector; and $\phi (t)={{[{\phi_1}(t),\ldots,{\phi_P}(t)]}^T}$ is the weight vector. A cost function is then presented to be the instantaneous performance index, as shown in Eq. (5.4):

$$ J(\phi (t))=\frac{1}{2}{{\left[ {d(t)-{f_n}(t)} \right]}^2}=\frac{1}{2}{{\left[ {d(t)-h({x^T}(t)\phi (t))} \right]}^2} $$

(5.4)

where d(t) = [d ₁(t), …, d _p (t)]^T is the desired output.

Second, the instantaneous output error at the output neuron and the revised weight vector in the next moment are given by Eqs. (5.5) and (5.6), respectively:

$$ e(t)=d(t)-{f_n}(t)=d(t)-h({x^T}(t)\phi (t)), $$

(5.5)

$$ \phi (t+1)=\phi (t)-\eta {\nabla_{\phi }}J(\phi (t)), $$

(5.6)

where η is the learning rate.

Third, the gradient ${\nabla_{\phi }}J(\phi (t))$ can be calculated as Eq. (5.7):

$$ {\nabla_{\phi }}J(\phi (t))=\frac{{\partial J\phi (t)}}{{\partial \phi (t)}}=e(t)\times \frac{{\partial e(t)}}{{\partial \phi (t)}}=-e(t){h}^{\prime}({x^T}(t)\phi (t))x(t), $$

(5.7)

where ${h}^{\prime}(\cdot )$ is the first derivative of the nonlinearity $h(\cdot )$. Finally, the weight is revised as Eq. (5.8):

$$ \phi (t+1)=\phi (t)+\eta e(t){h}^{\prime}({x^T}(t)\phi (t))x(t) $$

(5.8)

Figure 5.1 shows the architecture of the general recurrent SVR-based (RSVR-based) model. The output of RSVR-based model (${{\tilde{f}}_n}(t)$) is as Eq. (5.9):

Fig. 5.1

The architecture of RSVR-based model

$$ {{\tilde{f}}_{n}}(t)=\sum\limits_{i=1}^P {{W^T}} \psi ({x^T}(t))+b(t) $$

(5.9)

Then, Eq. (5.9) replaces Eq. (2.47) in the SVR-based model, to run the loop of SVR-based model in the search for values of three parameters. Finally, the forecast values ${{\tilde{f}}_n}(t)$ are calculated using Eq. (5.9).

5.1.2 Seasonal Mechanism

As mentioned, the electric load often demonstrates a cyclic tendency due to economic activities or climate cyclic nature. Lots of researchers in financial fields have explored how to identify the seasonal index to adjust the seasonal biases, such as Martens et al. [12], Taylor and Xu [13], and Andersen and Bollerslev [14] apply flexible Fourier form to estimate the variation of daily stock exchange, then receive the seasonal variation estimator; Deo et al. [15] proposed a revised model to further identify the seasonal variation estimator that is composed of two linear combinations in a cyclic period. Based on the data series-type consideration and inspired from previous papers, this investigation firstly applies ARIMA methodology to identify the seasonal length, then proposes the seasonal index to easily adjust cyclic effects, as shown in Eq. (5.10):

$$ \mathrm{ Seaso}{{\mathrm{ n}}_t}=\ln {{\left( {\frac{{{a_t}}}{{{f_t}}}} \right)}^2}=2\left( {\ln {a_t}-\ln \sum\limits_{i=1}^n {\left( {\beta_i^{*}-{\beta_i}} \right)K(x,{x_i})} +b} \right) $$

(5.10)

where t = j, l + j, 2l + j, …, (m − 1)l + j only for the same time point in each period. Then, the seasonal index (SI) for each time point j is computed as Eq. (5.11):

$$ \mathrm{S}{{\mathrm{I}}_j}={{{\exp \left( {\frac{1}{m}(\mathrm{seaso}{{\mathrm{n}}_j}+\mathrm{seaso}{{\mathrm{n}}_{l+j }}+\cdots +\mathrm{seaso}{{\mathrm{n}}_{(m-1)l+j }})} \right)}} \left/ {2} \right.} $$

(5.11)

where j = 1, 2, …, l. The seasonal mechanism is shown in Fig. 5.2.

Fig. 5.2

The process of seasonal mechanism

Eventually, the forecasting value of the SSVRCCSA is obtained by Eq. (5.12):

$$ {f_{N+k }}=\left( {\sum\limits_{i=1}^N {(\beta_i^{*}-{\beta_i})K({x_i},{x_{N+k }})} +b} \right)\times \mathrm{ S}{{\mathrm{ I}}_k}, $$

(5.12)

where k = 1, 2, …, l implies the time point in another period (for forecasting period).

5.2 Seasonal ARIMA Model and Seasonal HW (SHW) Model

5.2.1 SARIMA Model

For the Seasonal ARIMA (SARIMA) model, by Minitab 14 statistic software, the parameters are determined by taking the first-order regular difference and first seasonal difference to remove nonstationary and seasonality characteristics. Using statistical packages, with no residuals autocorrelated and approximately white noise residuals, the most suitable models for the employed electric load data is SARIMA(4,2,2) × (1,2,1)₁₂ with constant item. The equation used for the SARIMA model is presented as Eq. (5.13):

$$ (1+1.067B+0.6578{B^2}+0.4569{B^3}+0.1819{B^4})(1+0.3012{B^5}){W_{\mathrm{t}}}=-0.7758+(1-0.8055B-0.1857{B^2})(1-0.5054{B^5}){\varepsilon_{\mathrm{t}}}, $$

(5.13)

where W _t  = (1 − B)²(1 − B ¹²)² X _t .

After determining the suitable parameters of the SARIMA model, it is important to examine how closely the proposed model fits a given time series. The autocorrelation function (ACF) is calculated to verify the parameters. Figure 5.3 plots the estimated residual ACF and indicates that the residuals are not autocorrelated. PACF, the partial autocorrelation function, displayed in Fig. 5.4, is also used to check the residuals and indicates that the residuals are not correlated. The forecasting results are shown in the third column of Table 5.1.

Fig. 5.3

Estimated residual ACF

Fig. 5.4

Estimated residual PACF

Table 5.1 Forecasting results of SARIMA, SHW, GRNN, and BPNN models (unit: hundred million kWh)

5.2.2 SHW Model

For the seasonal Holt–Winters (SHW) model, by Minitab 14 statistic software, the α-value and β-value are determined as 0.5618 and 0.0472, respectively.

For the seasonal Holt–Winters (SHW) method, by Minitab 14 statistic software, the appropriate parameters (L, α, β, and γ) are determined 12, 0.95, 0.20, and 0.20, correspondingly. The forecasting results are shown in the fourth column of Table 5.1.

Figure 5.5 is provided to illustrate the forecasting accuracy among different models. Obviously, these four models, excepting GRNN and BPNN models, are fitting much better than ARIMA and HW models. Furthermore, to verify the significance of accuracy improvement of SARIMA(4,2,2) × (1,2,1)₁₂ and SHW(0.12, 0.95, 0.2, 0.2) models comparing with ARIMA(1,1,1) and HW(0.5618, 0.0472) models, respectively, the Wilcoxon signed-rank test and asymptotic test are conducted and shown in Tables 5.2 and 5.3. It is clear to receive that SARIMA and SHW models significantly outperform ARIMA and HW models, respectively. Therefore, SARIMA(4,2,2) × (1,2,1)₁₂ and SHW(0.12, 0.95, 0.2, 0.2) are potential to compare with seasonal-SVR-chaotic-evolutionary algorithm-based models in the following sections.

Fig. 5.5

Forecasting results of SARIMA, SHW, GRNN, and BPNN models

Table 5.2 Wilcoxon signed-rank test

Table 5.3 Asymptotic test

5.3 Seasonal Mechanism in SVRCGA Model and Forecasting Results

Based on the total employed electric load, each fixed point (month) has its electric load status (specific data pattern); therefore, the seasonal (cyclic) length can be estimated as 12 [16]. The 12 seasonal indexes can be estimated by the 46 in-sample forecasting loads of the SVRCGA model mentioned in Sect. 4.2.3, including 32 and 14 in-sample forecasting loads in training stage and validation stage, respectively, as shown in Table 5.4. The actual values and the out-of-sample forecasting loads obtained by different forecasting models, including SARIMA(4,2,2) × (1,2,1)₁₂, TF-ε-SVR-SA, SHW(0.12, 0.95, 0.2, 0.2), SVRCGA, and SSVRCGA models, are illustrated in Table 5.5. The proposed SSVRCGA model with smaller MAPE values is superior to SARIMA(4,2,2) × (1,2,1)₁₂, SHW(0.12, 0.95, 0.2, 0.2), TF-ε-SVR-SA, and SVRCGA models, due to its capability to excellently learn about the monthly load changing tendency. The seasonal mechanism further revises the forecasting results from the SVRCGA model (MAPE = 3.382 %), based on the seasonal indexes (per month) obtained from training and validation stages, to achieve more acceptable forecasting accuracy (2.695 %).

Table 5.4 The seasonal indexes for each time point (month) for the SVRCGA model

Table 5.5 Forecasting results of SARIMA, SHW, TF-ε-SVR-SA, SVRCGA, and SSVRCGA models (unit: hundred million kWh)

Furthermore, for forecasting accuracy improvement significant test, the Wilcoxon signed-rank test and asymptotic test, as mentioned, are also conducted. The test results are shown in Tables 5.6 and 5.7, respectively. Clearly, the SSVRCGA model receives complete significant forecasting accuracy improvement than SARIMA(4,2,2) × (1,2,1)₁₂ model, but incomplete significant forecasting accuracy improvement than SHW(0.12, 0.95, 0.2, 0.2), TF-ε-SVR-SA, and SVRCGA models (only receives significance with α = 0.05 level in Wilcoxon test, and all pass with both levels in asymptotic test). Particularly for comparing with TF-ε-SVR-SA model (also with seasonal adjustment mechanism but without hybrid evolutionary algorithm and chaotic sequence), the comparison results recognize that chaotic sequence could significantly improve the performance in terms of premature convergence. By comparing SVRCGA with SSVRCGA models, it also indicates the significant superiority from seasonal mechanism, even it is a little time-consuming; however, it deserves to pay some attention on those cyclic information while modeling. Figure 5.6 is provided to illustrate the forecasting accuracy among different models.

Table 5.6 Wilcoxon signed-rank test

Table 5.7 Asymptotic test

Fig. 5.6

Forecasting results of SARIMA, SHW, TF-ε-SVR-SA, SVRCGA, and SSVRCGA models

5.4 Seasonal Mechanism in SVRCSA Model and Forecasting Results

Similarly, the seasonal (cyclic) length of the total employed electric load is also set as 12. Thus, the 12 seasonal indexes are estimated by the 46 in-sample forecasting loads of the SVRCSA model mentioned in Sect. 4.3.3, including 32 and 14 in-sample forecasting loads in training stage and validation stage, respectively, as shown in Table 5.8. The actual values and the out-of-sample forecasting loads obtained by different forecasting models, including SARIMA(4,2,2) × (1,2,1)₁₂, TF-ε-SVR-SA, SHW(0.12, 0.95, 0.2, 0.2), SVRCSA, and SSVRCSA models, are illustrated in Table 5.9. The proposed SSVRCSA model with smaller MAPE values is superior to SARIMA(4,2,2) × (1,2,1)₁₂, SHW(0.12, 0.95, 0.2, 0.2), TF-ε-SVR-SA, and SVRCSA models, due to its capability to excellently learn about the monthly load changing tendency. The seasonal mechanism further revises the forecasting results from the SVRCSA model (MAPE = 3.633 %), based on the seasonal indexes (per month) obtained from training and validation stages, to achieve more acceptable forecasting accuracy (2.844 %).

Table 5.8 The seasonal indexes for each time point (month) for the SVRCSA model

Table 5.9 Forecasting results of SARIMA, SHW, TF-ε-SVR-SA, SVRCSA, and SSVRCSA models (unit: hundred million kWh)

For forecasting accuracy improvement significant test, the Wilcoxon signed-rank test and asymptotic test are employed. The test results are shown in Tables 5.10 and 5.11, respectively. Clearly, the SSVRCSA model receives complete significant forecasting accuracy improvement than SARIMA(4,2,2) × (1,2,1)₁₂ and SHW(0.12, 0.95, 0.2, 0.2) models, but incomplete significant forecasting accuracy improvement than TF-ε-SVR-SA and SVRCSA models (only receives significance with α = 0.05 level in Wilcoxon test, and all pass with both levels in asymptotic test). Particularly for comparing with TF-ε-SVR-SA model (also with seasonal adjustment mechanism but without hybrid evolutionary algorithm and chaotic sequence), the comparison results also recognize that chaotic sequence could significantly improve the performance in terms of premature convergence. By comparing SVRCSA with SSVRCSA models, it also indicates the significant superiority from seasonal mechanism. It deserves to pay some attention on those cyclic information while modeling. Figure 5.7 is provided to illustrate the forecasting accuracy among different models.

Table 5.10 Wilcoxon signed-rank test

Table 5.11 Asymptotic test

Fig. 5.7

Forecasting results of SARIMA, SHW, TF-ε-SVR-SA, SVRCSA, and SSVRCSA models

5.5 Seasonal Mechanism in SVRCCSA Model and Forecasting Results

The seasonal (cyclic) length of the total employed electric load is also set as 12. Thus, the 12 seasonal indexes are estimated by the 46 in-sample forecasting loads of the SVRCCSA model mentioned in Sect. 4.4.3, including 32 and 14 in-sample forecasting loads in training stage and validation stage, respectively, as shown in Table 5.12. The actual values and the out-of-sample forecasting loads obtained by different forecasting models, including SARIMA(4,2,2) × (1,2,1)₁₂, TF-ε-SVR-SA, SHW(0.12, 0.95, 0.2, 0.2), SVRCSA, and SSVRCSA models, are illustrated in Table 5.13. The proposed SSVRCCSA model with smaller MAPE values is superior to SARIMA(4,2,2) × (1,2,1)₁₂, SHW(0.12, 0.95, 0.2, 0.2), TF-ε-SVR-SA, and SVRCCSA models, due to its capability to excellently learn about the monthly load changing tendency. The seasonal mechanism further revises the forecasting results from the SVRCCSA model (MAPE = 3.406 %), based on the seasonal indexes (per month) obtained from training and validation stages, to achieve more acceptable forecasting accuracy (1.973 %).

Table 5.12 The seasonal indexes for each month for the SVRCCSA model

Table 5.13 Forecasting results of SARIMA, SHW, TF-ε-SVR-SA, SVRCCSA, and SSVRCCSA models (unit: hundred million kWh)

For forecasting accuracy improvement significant test, the Wilcoxon signed-rank test and asymptotic test are employed. The test results are shown in Tables 5.14 and 5.15, respectively. Clearly, the SSVRCCSA model receives complete significant forecasting accuracy improvement than SARIMA(4,2,2) × (1,2,1)₁₂, SHW(0.12, 0.95, 0.2, 0.2), TF-ε-SVR-SA, and SVRCCSA models. Particularly for comparing with TF-ε-SVR-SA model (also with seasonal adjustment mechanism but without hybrid evolutionary algorithm and chaotic sequence), the comparison results also recognize that chaotic sequence could significantly improve the performance in terms of premature convergence. By comparing SVRCCSA with SSVRCCSA models, it also indicates the significant superiority from seasonal mechanism. It deserves to pay some attention on those cyclic information while modeling. Figure 5.8 is provided to illustrate the forecasting accuracy among different models.

Table 5.14 Wilcoxon signed-rank test

Table 5.15 Asymptotic test

Fig. 5.8

Forecasting results of SARIMA, SHW, TF-ε-SVR-SA, SVRCCSA, and SSVRCCSA models

The significant superiority of the proposed SSVRCCSA model in load forecasting can be summarized as follows. Firstly, the Y condition cloud generator can obviously ensure temperature reducing continuously and to overcome the dilemma of the original SA, easily to accept worsened solution, and lead to converge to local minimum while decreasing to low temperature; that is, it can help the original SA to well simulate the actual physical annealing processes, to avoid premature convergence. Secondly, the seasonal mechanism can successfully determine cyclic length and well calculate suitable seasonal indexes for each cycle point.

5.6 Seasonal Mechanism in SVRCGASA Model and Forecasting Results

The seasonal (cyclic) length of the total employed electric load is also set as 12. Thus, the 12 seasonal indexes are estimated by the 46 in-sample forecasting loads of the SVRCGASA model mentioned in Sect. 4.5.3, including 32 and 14 in-sample forecasting loads in training stage and validation stage, respectively, as shown in Table 5.16. The actual values and the out-of-sample forecasting loads obtained by different forecasting models, including SARIMA(4,2,2) × (1,2,1)₁₂, TF-ε-SVR-SA, SHW(0.12, 0.95, 0.2, 0.2), SVRCGASA, and SSVRCGASA models, are illustrated in Table 5.17. The proposed SSVRCGASA model with smaller MAPE values is superior to SARIMA(4,2,2) × (1,2,1)₁₂, SHW(0.12, 0.95, 0.2, 0.2), TF-ε-SVR-SA, and SVRCGASA models, due to its capability to excellently learn about the monthly load changing tendency. The seasonal mechanism further revises the forecasting results from the SVRCGASA model (MAPE = 3.731 %), based on the seasonal indexes (per month) obtained from training and validation stages, to achieve more acceptable forecasting accuracy (1.901 %).

Table 5.16 The seasonal indexes for each month for the SVRCGASA model

Table 5.17 Forecasting results of SARIMA, SHW, TF-ε-SVR-SA, SVRCGASA, and SSVRCGASA models (unit: hundred million kWh)

For forecasting accuracy improvement significant test, the Wilcoxon signed-rank test and asymptotic test are employed. The test results are shown in Tables 5.18 and 5.19, respectively. Clearly, the SSVRCGASA model receives complete significant forecasting accuracy improvement than SARIMA(4,2,2) × (1,2,1)₁₂, SHW(0.12, 0.95, 0.2, 0.2), TF-ε-SVR-SA, and SVRCGASA models. Particularly for comparing with TF-ε-SVR-SA model (also with seasonal adjustment mechanism but without hybrid evolutionary algorithm and chaotic sequence), the comparison results also recognize that chaotic sequence could significantly improve the performance in terms of premature convergence. By comparing SVRCGASA with SSVRCGASA models, it also indicates the significant superiority from seasonal mechanism, which can successfully determine cyclic length and well calculate suitable seasonal indexes for each cycle point. By the way, it should be noticed that the proposed SSVRCGASA model will implement three processes, including SVR modeling, CGASA conducting, and seasonal mechanism; of course, it will cost some reasonable processing time. However, it deserves to pay some attention on those cyclic information analyses while modeling. Figure 5.9 is provided to illustrate the forecasting accuracy among different models.

Table 5.18 Wilcoxon signed-rank test

Table 5.19 Asymptotic test

Fig. 5.9

Forecasting results of SARIMA, SHW, TF-ε-SVR-SA, SVRCGASA, and SSVRCGASA models

5.7 Seasonal Mechanism in SVRCPSO Model and Forecasting Results

The seasonal (cyclic) length of the total employed electric load is also set as 12. Thus, the 12 seasonal indexes are estimated by the 46 in-sample forecasting loads of the SVRCPSO model mentioned in Sect. 4.6.3, including 32 and 14 in-sample forecasting loads in training stage and validation stage, respectively, as shown in Table 5.20. The actual values and the out-of-sample forecasting loads obtained by different forecasting models, including SARIMA(4,2,2) × (1,2,1)₁₂, TF-ε-SVR-SA, SHW(0.12, 0.95, 0.2, 0.2), SVRCPSO, and SSVRCPSO models, are illustrated in Table 5.21. The proposed SSVRCPSO model with smaller MAPE values is superior to SARIMA(4,2,2) × (1,2,1)₁₂, SHW(0.12, 0.95, 0.2, 0.2), TF-ε-SVR-SA, and SVRCPSO models, due to its capability to excellently learn about the monthly load changing tendency. The seasonal mechanism further revises the forecasting results from the SVRCPSO model (MAPE = 3.231 %), based on the seasonal indexes (per month) obtained from training and validation stages, to achieve more acceptable forecasting accuracy (2.699 %).

Table 5.20 The seasonal indexes for each month for the SVRCPSO model

Table 5.21 Forecasting results of SARIMA, SHW, TF-ε-SVR-SA, SVRCPSO, and SSVRCPSO models (unit: hundred million kWh)

For forecasting accuracy improvement significant test, the Wilcoxon signed-rank test and asymptotic test are employed. The test results are shown in Tables 5.22 and 5.23, respectively. Clearly, the SSVRCPSO model only receives complete significant forecasting accuracy improvement than SHW(0.12, 0.95, 0.2, 0.2) model, but incomplete significant forecasting accuracy improvement than SARIMA(4,2,2) × (1,2,1)₁₂ and TF-ε-SVR-SA models (only receives significance with α = 0.05 level in Wilcoxon test, and all pass with both levels in asymptotic test), and SVRCPSO model (receives significance with both levels in Wilcoxon test, but only receives significance with both α = 0.10 level in asymptotic test). Particularly for comparing with TF-ε-SVR-SA model (also with seasonal adjustment mechanism but without hybrid evolutionary algorithm and chaotic sequence), the comparison results also recognize that chaotic sequence could significantly improve the performance in terms of premature convergence. By comparing SVRCPSO with SSVRCPSO models, it also indicates the significant superiority from seasonal mechanism. By the way, it should be noticed that the proposed SSVRCPSO model will implement three processes, including SVR modeling, CPSO conducting, and seasonal mechanism; of course, it will cost some reasonable processing time. However, it deserves to pay some attention on those cyclic information analyses while modeling. Figure 5.10 is provided to illustrate the forecasting accuracy among different models.

Table 5.22 Wilcoxon signed-rank test

Table 5.23 Asymptotic test

Fig. 5.10

Forecasting results of SARIMA, SHW, TF-ε-SVR-SA, SVRCPSO, and SSVRCPSO models

5.8 Seasonal Mechanism in SVRCAS Model and Forecasting Results

The seasonal (cyclic) length of the total employed electric load is also set as 12. Thus, the 12 seasonal indexes are estimated by the 46 in-sample forecasting loads of the SVRCAS model mentioned in Sect. 4.7.3, including 32 and 14 in-sample forecasting loads in training stage and validation stage, respectively, as shown in Table 5.24. The actual values and the out-of-sample forecasting loads obtained by different forecasting models, including SARIMA(4,2,2) × (1,2,1)₁₂, TF-ε-SVR-SA, SHW(0.12, 0.95, 0.2, 0.2), SVRCAS, and SSVRCAS models, are illustrated in Table 5.25. The proposed SSVRCAS model with smaller MAPE values is superior to SARIMA(4,2,2) × (1,2,1)₁₂, SHW(0.12, 0.95, 0.2, 0.2), TF-ε-SVR-SA, and SVRCAS models, due to its capability to excellently learn about the monthly load changing tendency. The seasonal mechanism further revises the forecasting results from the SVRCAS model (MAPE = 2.881 %), based on the seasonal indexes (per month) obtained from training and validation stages, to achieve more acceptable forecasting accuracy (2.341 %).

Table 5.24 The seasonal indexes for each month for the SVRCAS model

Table 5.25 Forecasting results of SARIMA, SHW, TF-ε-SVR-SA, SVRCAS, and SSVRCAS models (unit: hundred million kWh)

For forecasting accuracy improvement significant test, the Wilcoxon signed-rank test and asymptotic test are also used. The test results are shown in Tables 5.26 and 5.27, respectively. Clearly, the SSVRCAS model only receives complete significant forecasting accuracy improvement than TF-ε-SVR-SA and SVRCAS models, but incomplete significant forecasting accuracy improvement than SARIMA(4,2,2) × (1,2,1)₁₂ and SHW(0.12, 0.95, 0.2, 0.2) models (only receives significance with α = 0.05 level in Wilcoxon test, and all pass with both levels in asymptotic test). Particularly for comparing with TF-ε-SVR-SA model (also with seasonal adjustment mechanism but without hybrid evolutionary algorithm and chaotic sequence), the comparison results also recognize that chaotic sequence could significantly improve the performance in terms of premature convergence. By comparing SVRCAS with SSVRCAS models, it also indicates the significant superiority from seasonal mechanism. By the way, it should be noticed that the proposed SSVRCAS model will implement three processes, including SVR modeling, CAS conducting, and seasonal mechanism; of course, it will cost some reasonable processing time. However, it deserves to pay some attention on those cyclic information analyses while modeling. Figure 5.11 is provided to illustrate the forecasting accuracy among different models.

Table 5.26 Wilcoxon signed-rank test

Table 5.27 Asymptotic test

Fig. 5.11

Forecasting results of SARIMA, SHW, TF-ε-SVR-SA, SVRCAS, and SSVRCAS models

The proposed SSVRCAS model has obtained significant smaller MAPE values than other alternative models (SARIMA(4,2,2) × (1,2,1)₁₂, SHW(0.12, 0.95, 0.2, 0.2), TF-ε-SVR-SA, and SVRCAS models). It is caused by (1) nonlinear mapping capabilities and structural risk minimization of SVR model itself; (2) the CAS algorithm employed the organization variable to perform self-organization foraging process of ant colony to determine proper parameters combination, and applies ergodicity property of chaotic sequences to enrich the searching behavior to avoid premature convergence; (3) the seasonal adjustment with well seasonal/cyclic analytical ability of load demanding tendency.

It is interesting to address the SVRCAS model focuses on the interactions among individual ant’s chaotic behavior and ant colony organization foraging activities, instead of “expert rules,” to negotiate and to coordinate to look for much better solutions. Therefore, the better solution is evolved with “learning by doing” activities among ants and their colony to approximately reach the global optimum or near-optimum. It is sometimes like a “practical (numerical) rule” to guide colony organization variable that enhances its effects on individual ant’s chaotic behaviors. Furthermore, for example, along with the unexpected climate change pattern, the tendencies of the electric load data may present more fluctuant nonhistorically, and the future changes of the electric load data may be more cyclic with more short cycle. The proposed SSVRCAS model is potential to approximately reach the near-optimum by “learning by doing” activities among ants and their colony (organization variable), to adjust any length of seasonal load demand (weekly, monthly, bimonthly, quarterly, etc.) by seasonal adjustment; thus, it is much more potential alternative forecasting model in nonhistorical climate change age.

5.9 Seasonal Mechanism in SVRCABC Model and Forecasting Results

The seasonal (cyclic) length of the total employed electric load is also set as 12. Thus, the 12 seasonal indexes are estimated by the 46 in-sample forecasting loads of the SVRCABC model mentioned in Sect. 4.8.3, including 32 and 14 in-sample forecasting loads in training stage and validation stage, respectively, as shown in Table 5.28. The actual values and the out-of-sample forecasting loads obtained by different forecasting models, including SARIMA(4,2,2) × (1,2,1)₁₂, TF-ε-SVR-SA, SHW(0.12, 0.95, 0.2, 0.2), SVRCABC, and SSVRCABC models, are illustrated in Table 5.29. The proposed SSVRCABC model with smaller MAPE values is superior to SARIMA(4,2,2) × (1,2,1)₁₂, SHW(0.12, 0.95, 0.2, 0.2), TF-ε-SVR-SA, and SVRCABC models, due to its capability to excellently learn about the monthly load changing tendency. The seasonal mechanism further revises the forecasting results from the SVRCABC model (MAPE = 3.164 %), based on the seasonal indexes (per month) obtained from training and validation stages, to achieve more acceptable forecasting accuracy (3.056 %).

Table 5.28 The seasonal indexes for each month for the SVRCABC model

Table 5.29 Forecasting results of SARIMA, SHW, TF-ε-SVR-SA, SVRCABC, and SSVRCABC models (unit: hundred million kWh)

For forecasting accuracy improvement significant test, the Wilcoxon signed-rank test and asymptotic test are also used. The test results are shown in Tables 5.30 and 5.31, respectively. Clearly, the SSVRCABC model only receives incomplete significant forecasting accuracy improvement than SARIMA(4,2,2) × (1,2,1)₁₂, TF-ε-SVR-SA models (only receives significance with α = 0.05 level in Wilcoxon test, and all pass with both levels in asymptotic test), SHW(0.12, 0.95, 0.2, 0.2) and SVRCABC models (receives significance with both levels in Wilcoxon test, and all fails with both levels in asymptotic test). Particularly for comparing with TF-ε-SVR-SA model (also with seasonal adjustment mechanism but without hybrid evolutionary algorithm and chaotic sequence), the comparison results also recognize that chaotic sequence could significantly improve the performance in terms of premature convergence. By comparing SVRCABC with SSVRCABC models, it also indicates the superiority from seasonal mechanism; it deserves to pay some attention on those cyclic information analyses while modeling. To look for more capability to receive complete significant forecasting accuracy improvement, it is necessary to use the final hybridization tool, recurrent mechanism, as shown in the following subsection. Figure 5.12 is provided to illustrate the forecasting accuracy among different models.

Table 5.30 Wilcoxon signed-rank test

Table 5.31 Asymptotic test

Fig. 5.12

Forecasting results of SARIMA, SHW, TF-ε-SVR-SA, SVRCABC, and SSVRCABC models

The proposed SSVRCABC model has obtained smaller MAPE values than other alternative models (SARIMA(4,2,2) × (1,2,1)₁₂, SHW(0.12, 0.95, 0.2, 0.2), TF-ε-SVR-SA, and SVRCABC models). It is caused by (1) nonlinear mapping capabilities and structural risk minimization of SVR model itself, (2) the CABC algorithm that employs both global search and local search in each iteration to receive better performance and applies ergodicity property of chaotic sequences to enrich the searching behavior to avoid premature convergence, and (3) the seasonal mechanism with well seasonal/cyclic analytical ability of load demanding tendency.

5.10 Recurrent and Seasonal Mechanisms in SVRCABC Model and Forecasting Results

This subsection will firstly demonstrate the three parameters determination of the proposed hybrid model (recurrent SVR with CABC), namely, RSVRCABC model. Secondly, conduct the combined model (RSVRCABC with seasonal mechanism), namely, SRSVRCABC model.

For RSVRCABC modeling process, based on the parameter determination results in Sect. 4.8.3, the SVRCABC model with the smallest testing MAPE value is used further to implement the RSVRCABC model. After performing the RSVRCABC model, the final forecasting values are gained, and the kernel parameters, σ, C, and ε, are used as the most suitable model in this example. The forecasting results and the suitable parameters for the RSVRCABC model are illustrated in Table 5.32, in which it is indicated that these three models all perform the best when 25 fed-in data are used.

Table 5.32 Parameters determination of RSVRCABC model

Now the seasonal mechanism is considered. The seasonal (cyclic) length of the total employed electric load is also set as 12. Thus, the 12 seasonal indexes are estimated by the 46 in-sample forecasting loads of the SVRCABC model mentioned in Sect. 4.8.3, including 32 and 14 in-sample forecasting loads in training stage and validation stage, respectively, as shown in Table 5.33. The actual values and the out-of-sample forecasting loads obtained by different forecasting models, including SARIMA(4,2,2) × (1,2,1)₁₂, TF-ε-SVR-SA, SHW(0.12, 0.95, 0.2, 0.2), SSVRCABC, RSVRCABC, and SRSVRCABC models, are illustrated in Table 5.34. The proposed SRSVRCABC model with smaller MAPE values is superior to SARIMA(4,2,2) × (1,2,1)₁₂, SHW(0.12, 0.95, 0.2, 0.2), TF-ε-SVR-SA, SSVRCABC, and RSVRCABC models, due to its capability to excellently learn about the monthly load changing tendency. The seasonal mechanism further revises the forecasting results from the RSVRCABC model (MAPE = 2.960 %), based on the seasonal indexes (per month) obtained from training and validation stages, to achieve more acceptable forecasting accuracy (2.387 %).

Table 5.33 The seasonal indexes for each month for the SVRCABC model

Table 5.34 Forecasting results of SARIMA, SHW, TF-ε-SVR-SA, SSVRCABC, RSVRCABC, and SRSVRCABC models (unit: hundred million kWh)

For forecasting accuracy improvement significant test, the Wilcoxon signed-rank test and asymptotic test are also used. The test results are shown in Tables 5.35 and 5.36, respectively. Clearly, the SRSVRCABC model receives complete significant forecasting accuracy improvement than SARIMA(4,2,2) × (1,2,1)₁₂, SHW(0.12, 0.95, 0.2, 0.2), TF-ε-SVR-SA, SSVRCABC, and RSVRCABC models. Particularly for comparing with TF-ε-SVR-SA model (also with seasonal adjustment mechanism but without hybrid evolutionary algorithm and chaotic sequence), the comparison results also recognize that chaotic sequence could significantly improve the performance in terms of premature convergence. By comparing RSVRCABC with SRSVRCABC models, it also indicates the superiority from seasonal mechanism; it deserves to pay some attention on those cyclic information analyses while modeling. Figure 5.13 is provided to illustrate the forecasting accuracy among different models.

Table 5.35 Wilcoxon signed-rank test

Table 5.36 Asymptotic test

Fig. 5.13

Forecasting results of SARIMA, SHW, TF-ε-SVR-SA, SSVRCABC, RSVRCABC, and SRSVRCABC models

The proposed SRSVRCABC model has obtained smaller MAPE values than other alternative models (SARIMA(4,2,2) × (1,2,1)12, SHW(0.12, 0.95, 0.2, 0.2), TF-ε-SVR-SA, SSVRCABC, and RSVRCABC models). It is caused by (1) nonlinear mapping capabilities and structural risk minimization of SVR model itself, (2) the CABC algorithm that employs both global search and local search in each iteration to receive better performance and applies ergodicity property of chaotic sequences to enrich the searching behavior to avoid premature convergence, (3) the recurrent mechanism with superior capability to capture more data pattern information from past electric load data, and (4) the seasonal adjustment with well cyclic (seasonal) analytical ability of load demanding tendency. For example, recurrent mechanism, hybridized into the SVRCABC model, also plays a contributive role to further improve the better solution of SVRCABC model to another solution (σ, C, ε) = (50.27, 7681.30, 19.3750) of RSVRCABC model to be the more appropriate optimal solution with forecasting error in terms of MAPE (2.960) (refer to Tables 4.27 and 5.38). Finally, the seasonal mechanism further revises the forecasting results from RSVRCABC model, based on their seasonal indexes (per month) obtained from training and validation stages, to achieve more acceptable forecasting accuracies (2.387 %).

It is interesting to address that via recurrent mechanism and seasonal mechanism, the proposed SRSVRCABC model is able to deal with any data pattern no matter data tendencies may present fluctuation or sustained increasing or decreasing types. Furthermore, for example, along with the unexpected climate change pattern, the tendencies of the electric load data or energy-consuming data may present more fluctuant nonhistorically; or along with the large penetration of renewable energies for electricity production, the future changes of the electric load data may be more cyclic with more short cycle. The proposed SRSVRCABC model is potential to approximately reach the global optimum or near-optimum by “communicating in searching” activities among bees and their colony, to learn more fluctuant changed load demand by recurrent mechanism, to adjust any length of seasonal load demand (weekly, monthly, bimonthly, quarterly, etc.) by seasonal mechanism; thus, it is much more potential alternative forecasting model in nonhistorical climate change age.

5.11 Seasonal Mechanism in SVRCIA Model and Forecasting Results

The seasonal (cyclic) length of the total employed electric load is also set as 12. Thus, the 12 seasonal indexes are estimated by the 46 in-sample forecasting loads of the SVRCIA model mentioned in Sect. 4.9.3, including 32 and 14 in-sample forecasting loads in training stage and validation stage, respectively, as shown in Table 5.37. The actual values and the out-of-sample forecasting loads obtained by different forecasting models, including SARIMA(4,2,2) × (1,2,1)₁₂, TF-ε-SVR-SA, SHW(0.12, 0.95, 0.2, 0.2), SVRCIA, and SSVRCIA models, are illustrated in Table 5.38. The proposed SSVRCIA model with smaller MAPE values is superior to SARIMA(4,2,2) × (1,2,1)₁₂, SHW(0.12, 0.95, 0.2, 0.2), TF-ε-SVR-SA, and SVRCIA models, due to its capability to excellently learn about the monthly load changing tendency. The seasonal mechanism further revises the forecasting results from the SVRCIA model (MAPE = 3.041 %), based on the seasonal indexes (per month) obtained from training and validation stages, to achieve more acceptable forecasting accuracy (1.766 %).

Table 5.37 The seasonal indexes for each month for the SVRCIA model

Table 5.38 Forecasting results of SARIMA, SHW, TF-ε-SVR-SA, SVRCIA, and SSVRCIA models (unit: hundred million kWh)

For forecasting accuracy improvement significant test, the Wilcoxon signed-rank test and asymptotic test are also used. The test results are shown in Tables 5.39 and 5.40, respectively. Clearly, the SSVRCIA model almost receives complete significant forecasting accuracy improvement than other alternative models except the SVRCIA model (only receives significance with α = 0.05 level in Wilcoxon test and pass both levels in asymptotic test). Particularly for comparing with TF-ε-SVR-SA model (also with seasonal adjustment mechanism but without hybrid evolutionary algorithm and chaotic sequence), the comparison results also recognize that chaotic sequence could significantly improve the performance in terms of premature convergence due to the superior searching capability of CIA to determine proper parameters in an SVR model and the use of a seasonal mechanism to adjust the seasonal/cyclic effects of electric loads. By comparing SVRCIA with SSVRCIA models, it also indicates the superiority from seasonal mechanism employed here is proficient in dealing with such cyclic data types; thus, it deserves to pay some attention on those cyclic information analyses while modeling. Figure 5.14 is provided to illustrate the forecasting accuracy among different models.

Table 5.39 Wilcoxon signed-rank test

Table 5.40 Asymptotic test

Fig. 5.14

Forecasting results of SARIMA, SHW, TF-ε-SVR-SA, SVRCIA, and SSVRCIA models

The proposed SSVRCIA model has obtained smaller MAPE values than other alternative models (SARIMA(4,2,2) × (1,2,1)₁₂, SHW(0.12, 0.95, 0.2, 0.2), TF-ε-SVR-SA, and SVRCIA models). The superior performance of the SSVRCIA model is not only because of its theoretical assumptions of a convex set while SVR modeling but also because of the superior searching capability of CIA to determine the proper parameters in SVR (this is why it outperforms the TF-ε-SVR-SA model) and effective seasonal mechanism (this is why it outperforms the SVRCIA model). By contrast, SARIMA model employs the parametric technique which is based on specific assumptions, such as linear relationships between the current value of the underlying variables and previous values of the variable and error terms, and these assumptions are not completely in line with real-world problems.