Keywords

1 Introduction

Time series are usually analyzed to understand the past and to predict the future, enabling managers or policy makers to make properly informed decisions. Time series analysis quantifies the main features in data, such as the random variation.

These facts, combined with improved computing power, have made time series methods widely applicable in government, industry, and commerce. In most branches of science, engineering, and commerce, there are variables measured sequentially in time [1, 2]. For example, reserve banks record interest rates and exchange rates each day.

This paper reports the results of the simulations, in which the Mackey-Glass [3, 4] time series prediction using ensembles of ANFIS models with particle swarm optimization of the fuzzy integrators is achieved, where the results for each ANFIS are evaluated by the method of the root mean square error (RMSE).

The selection of the time series for the simulations was based on the fact that these time series are widely quoted in the literature by different researchers [5–8], which allows comparing results with other approaches, such as neural networks and linear regression.

This paper is organized as follows: the next section briefly describes ensemble learning. Section 3 presents the ANFIS models architecture. Section 4 presents the Mackey-Glass time series that we used for the experiments. Section 5 presents a brief review of interval type-2 fuzzy systems. Section 6 presents the particle swarm optimization that we used to optimize the fuzzy integrators. Section 7 presents the proposed architecture for time series prediction. Section 8 presents the simulation results obtained from the experiments. The last section presents the conclusions of this work.

2 Ensemble Learning

The Ensemble consists of a learning paradigm where multiple component learners are trained for a same task, and the prediction of the component learners are combined for dealing with future instances [9]. Since an Ensemble is often more accurate than its component learners, such a paradigm has become a hot topic in recent years and has already been successfully applied to optical character recognition, face recognition, scientific image analysis, medical diagnosis and time series [10].

3 ANFIS Models

There have been proposed systems that have fully achieved the combination of fuzzy systems with neural networks, one of the most mentioned in the literature is the ANFIS (Adaptive Neuro Fuzzy Inference System) as referred to by Jang [5] (Fig. 1), which is a method for creating the rule base of a fuzzy system, using the backpropagation training algorithm from the data collection process. Its architecture is functionally equivalent to a fuzzy inference system of Takagi-Sugeno-Kang form.

Fig. 1.
figure 1

ANFIS architecture.

Full size image

The basic learning rule of ANFIS is the gradient descent backpropagation, which calculates the error rates (defined as the derivative of the squared error for each output node) recursively from the output to the input nodes.

4 Mackey-Glass Time Series

The problem of predicting future values of a time series has been a point of reference for many researchers. The aim is to use the values of the time series known at a point x = t to predict the value of the series at some future point x = t + P. The standard method for this type of prediction is to create a mapping from D points of a Δ spaced time series, is (x (t − (D − 1) Δ)… x (t − Δ), x (t)), to a predicted future value x (t + P). To allow a comparison with previous results in this work [5–8] the values D = 4 and Δ = P = 6 were used.

The chaotic time series data that was used is defined by the Mackey-Glass [3, 4] time series, whose differential equation is given by Eq. (1):

$$ x\left( t \right) = \frac{{0.2x\left( {t - \tau } \right)}}{{1 - x^{10} \left( {t - \tau } \right)}} - 0.1x\left( {t - \tau } \right) $$
(1)

For obtaining the values of the time series at each point, we can apply the Runge-Kutta method for the solution of Eq. (1). The integration step was set at 0.1, with initial condition x (0) = 1.2, τ = 17, x (t) is then obtained for 0 ≤ t ≤ 1200, (Fig. 2) (We assume x (t) = 0 for t < 0 in the integration).

Fig. 2.
figure 2

The Mackey-Glass time series.

Full size image

5 Interval Type-2 Fuzzy Logic Systems

Type-2 fuzzy sets are used to model uncertainty and imprecision; originally they were proposed by Zadeh [11, 12] and they are essentially ‘‘fuzzy–fuzzy’’ sets in which the membership degrees are type-1 fuzzy sets.

The basic structure of a type-2 fuzzy system (Fig. 3) implements a nonlinear mapping of input to output space. This mapping is achieved through a set of type-2 fuzzy if-then rules, each of which describes the local behavior of the mapping.

Fig. 3.
figure 3

Structure of the interval type-2 fuzzy logic system.

Full size image

The uncertainty is represented by a region called footprint of uncertainty (FOU). When \( \mu_{{\tilde{A}}} \left( {x,u} \right) = 1, \forall u \in l_{x} \subseteq \left[ {0,1} \right] \) we have an interval type-2 membership function.

The uniform shading for the FOU (Fig. 4) represents the entire interval type-2 fuzzy set and it can be described in terms of an upper membership function \( \bar{\mu }_{{\tilde{A}}} \left( x \right) \) and a lower membership function \( \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{\mu }_{{\tilde{A}}} \left( x \right) \) [13].

Fig. 4.
figure 4

Interval type-2 membership function.

Full size image

A fuzzy logic systems (FLS) described using at least one type-2 fuzzy set is called a type-2 FLS. Type-1 FLSs are unable to directly handle rule uncertainties, because they use type-1 fuzzy sets that are certain [14]. On the other hand, type-2 FLSs are very useful in circumstances where it is difficult to determine an exact certainty value, and there are measurement uncertainties [15–17].

6 Particle Swarm Optimization (PSO)

Particle Swarm Optimization (PSO) is a metaheuristic search technique based on a population of particles (Fig. 5). The main idea of PSO comes from the social behavior of schools of fish and flocks of birds [18]. In PSO each particle moves in a D-dimensional space based on its own past experience and those of other particles. Each particle has a position and a velocity represented by the vectors \( x_{i} = \left( {x_{i1} , x_{i2} , \ldots , x_{iD} } \right) \) and \( V_{i} = \left( {v_{i1} , v_{i2} , \ldots , v_{iD} } \right) \) for the i-th particle. At each iteration, particles are compared with each other to find the best particle [19, 20]. Each particle records its best position as \( P_{i} = \left( {p_{i1} , p_{i2} , \ldots , p_{iD} } \right) \). The best position of all particles in the swarm is called the global best, and is represented as \( G = \left( {G_{1} , G_{2} , \ldots , G_{D} } \right) \). The velocity of each particle is given by Eq. (2).

Fig. 5.
figure 5

Flowchart of the PSO algorithm.

Full size image
$$ V_{id} = wv_{id} + C_{i} \cdot rand_{1} \left( {} \right) \cdot \left( {pbest_{id} - x_{id} } \right) + C_{2} \cdot rand_{2} \left( {} \right) \cdot \left( {gbest - x_{id} } \right) $$
(2)

In this equation \( i = 1,2, \cdots ,M,d = 1,2, \cdots ,D \), \( C_{1} \) and \( C_{2} \) are positive constants (known as acceleration constants), \( rand_{1} \left( {} \right) \) and \( rand_{2} \left( {} \right) \) are random numbers in [0,1], and \( w \), introduced by Shi and Eberhart [21] is the inertia weight. The new position of the particle is determined by Eq. (3):

$$ x_{id} = x_{id} + v_{id} $$
(3)

7 Problem Statement and Proposed Method

The proposed general architecture combines the ensemble of ANFIS models and the use of interval type-2 and type-1 fuzzy systems as response integrators (Fig. 6) for time series prediction.

Fig. 6.
figure 6

Theproposed general architecture.

Full size image

This architecture is divided into 5 sections, where the first phase represents the data base to simulate in the Ensemble of ANFIS, which in our case is the dataset of the Mackey-Glass time series. From the Mackey-Glass time series we extracted the first 800 pairs of data points (Fig. 1), similar to [6–8].

We predict x (t) from three past (delays) values of the time series, that is, x (t − 18), x (t − 12), and x (t − 6). Therefore the format of the training data is:

$$ \left[ {x\left( {t - 18} \right), x\left( {t - 12} \right), x\left( {t - 6} \right); x\left( t \right)} \right] $$
(4)

Where t = 19 to 818 and x(t) is the desired prediction of the time series.

In the second phase, training (the first 400 pairs of data are used to train the ANFIS) and validation (the second 400 pairs of data are used to validate the ANFIS models) is performed sequentially in each ANFIS model, where the number of ANFIS models to be used can be from 1 to n depending on what the user wants to test, but in our case we are dealing with a set of 3 ANFIS in the Ensemble. Therefore each ANFIS model used in the Ensemble training is formed differently, for example each of these ANFIS model uses different type of (Gaussian, triangular and Generalized Bell) MFs and different type of error goal [0.001,0.000001].

In the third phase we have to generate the results of each ANFIS used in the previous section and in the fourth phase we integrate the overall results of each ANFIS, such integration will be done by type-1 and interval type-2 fuzzy integrators of Mamdani type, but each fuzzy integrator is optimized with PSO in the membership functions. Finally the outcome or the final prediction of the Ensemble ANFIS learning is obtained.

8 Simulation Results

This section presents the results obtained through experiments on the architecture of genetic optimization of type-2 fuzzy integrators in ensembles of ANFIS models fortime series prediction, which show the performance that was obtained from each experiment to simulate the Mackey-Glass time series.

8.1 Design of the Fuzzy Inference Systems Integrators

The design of the type-1 and interval type-2 fuzzy inference systems integrators are of Mamdani type and have 3 inputs (ANFIS1, ANFIS2 and ANFIS3 predictions) and 1 output (Forecast), so each input is assigned two MFs with linguistic labels “Small and Large” and the output will be assigned 3 MFs with linguistic labels “OutANFIS1, OutANFIS2 and OutANFIS3” (Fig. 7) and have 8 rules if-then. The design of the if-then rules for the fuzzy inference system depends on the number of membership functions used in each input variable using the system (e.g. our fuzzy inference system uses 3 input variables which each entry contains two membership functions, therefore the total number of possible combinations for the fuzzy-rules is 8 (e.g. 2*2*2 = 8)), therefore we used 8 fuzzy-rules for the experiments (Fig. 8) because the performance is better and minimized the prediction error of the Mackey-Glass time series.

Fig. 7.
figure 7

Structure of the type-1 FIS (a) and interval tyep-2 FIS (b) integrators.

Full size image
Fig. 8.
figure 8

If-then rules for the fuzzy integrators.

Full size image

In the type-1 FIS integrators we used different MFs (Gaussian, Generalized Bell, and Triangular) (Fig. 9(a)) and for the interval type-2 FIS integrators we used different MFs (igaussmtype2, igbelltype2 and itritype2) (Fig. 9(b)) [13] to observe the behavior of each of them and determine which one provides better forecast of the time series.

Fig. 9.
figure 9

Type-1 MFs (a) and interval type-2 MFs (b) for the fuzzy integrators.

Full size image

8.2 Design the Representation of the Particle Swarm Optimization

The PSO is used to optimize the parameters values of the MFs in each type-1 and interval type-2 fuzzy integrator. The representation in PSO is of Real-Values and the particle size will depend of the number of MFs that are used in each design of the fuzzy integrators.

The objective function is defined to minimize the prediction error as follows in Eq. (5):

$$ f\left( t \right) = \sqrt {\frac{{\mathop \sum \nolimits_{t = 1}^{n} \left( {a_{t} - p_{t} } \right)^{2} }}{n}} $$
(5)

Where a, corresponds to the real data of the time series, p corresponds to the output of each fuzzy integrators, t is de sequence time series, and n is the number of data points of time series.

The general representation of the particles represents the utilized membership functions. The number of parameters varies according to the kind of membership function of the type-1 fuzzy system (e.g. two parameter are needed to represent a Gaussian MF’s are “sigma and mean”) (Fig. 10(a)) and interval type-2 fuzzy system (e.g. three parameter are needed to represent “igaussmtype2” MF’s are “sigma, mean1 and mean2”) (Fig. 10(b)).

Fig. 10.
figure 10

Representation of the particles structure of the type-1 (a) and interval type (b) fuzzy integrators.

Full size image

Therefore the number of parameters that each fuzzy inference system integrator has depends of the MFs type assigned to each input and output variables.

The parameters of particle swarm optimization used for optimizing the type-1 and interval type-2 fuzzy inference systems integrators are shown on Table 1.

Table 1. Parameters of the particle swarm optimization
Full size table

We performed experiments in time-series prediction, specifically for the Mackey-Glass time series in ensembles of ANFIS models using fuzzy integrators optimized with PSO.

The obtained results with the optimized type-1 fuzzy integrator (using Gaussian MFs) with particle swarm optimization are shown on Table 2. The best error is of 0.03154 and the average error is of 0.03848. The forecasts, evolution errors (for iteration), the membership functions and generated output surface were obtained for the optimized type-1 FIS integrator with PSO are shown (Fig. 11).

Table 2. Resultsof the ensembles of ANFIS models using fuzzy integrators optimized with particle swarm optimization for the Mackey-Glass time series.
Full size table
Fig. 11.
figure 11

The forecast time series (a), evolution error of “65 iterations” (b), inputs/output MFs (c) and surface output (d) are generated for the optimized of the type-1 FIS integrator with PSO.

Full size image

The obtained results with an optimized the interval type-2 fuzzy integrator (using igaussmtype2 MFs) and particle swarm optimization are shown on Table 2. The best error is of 0.01891 and the average error is of 0.01999. The forecasts, evolution errors (for iteration), the MFs and generated output surface were obtained for the optimized the interval type-2FIS integrator with PSO and are shown in Fig. 12.

Fig. 12.
figure 12

The forecast time series (a), evolution error of “65 iterations” (b), inputs/output (c) and surface output (d) are generated for the optimized of the interval type-2 FIS integrator with PSO.

Full size image

In this proposal we can notice the accurate results in the prediction error of the time series Mackey-Glass. In Table 3 you can find the best results obtained by the proposed method and we also made a comparison with the proposed method by Gaxiola et al. [8].

Table 3. Comparison of results with the method proposed by GaxiolaF. for the Mackey-Glass time series.
Full size table

Most of the results obtained by the proposed method are better than the results obtained in the work of Gaxiola et al. [8].

9 Conclusions

Time series prediction using ensembles of ANFIS models with particle swarm optimization of the fuzzy integrators was proposed. The best result with the optimized interval type-2 fuzzy inference systems (using igaussmtype2 MFs) integrator with particle swarm optimization for the Mackey-Glass time series is a prediction error of 0.01891 (as shown in Fig. 12 and Table 2). The type-1 fuzzy inference systems (using Gaussian MFs) were optimized with particles warm optimization obtaining a prediction error of 0.03154 (as shown in Fig. 11 and Table 2).

The proposed method presents better accuracy in almost all the experiments than the proposal by Gaxiola et al. [8] (as shown in Table 3).

These results showed efficient results in the prediction error of the time series Mackey-Glass generated by proposal method.