Keywords

1 Introduction

The analysis of the time series consists of a (usually mathematical) description of the movements that compose it, then build models using movements to explain the structure and predict the evolution of a variable over time [1, 2]. The fundamental procedure for the analysis of a series of time as described below:

  1. 1.

    Collecting data time series, trying to ensure that these data are reliable.

  2. 2.

    Representing the time series qualitatively noting the presence of long-term trends, cyclical variations and seasonal variations.

  3. 3.

    Plot a graph or trend line length and obtain the appropriate trend values using either method of least squares.

  4. 4.

    When are present seasonal variations, getting seasonal and adjust the data rate to these seasonal variations (i.e. data seasonally).

  5. 5.

    Adjust the seasonally adjusted trend.

  6. 6.

    Represent cyclical variations obtained in step 5.

  7. 7.

    Combining the results of steps 1–6 and any other useful information to make a prediction (if desired) and if possible discuss the sources of error and their magnitude.

Therefore the above ideas can assist in the important problem of prediction in time series. Along with common sense, experience, skill and judgment of the researcher, such mathematical analysis can, however, be of value for predicting the short, medium and long term [34, 37, 38, 41, 42].

The genetic algorithms are adaptive methods which may be used to solve search and optimization problems. They are based on the genetic process of living organisms. Over generations, the populations evolve in line with the nature of the principles of natural selection and survival of the fittest, postulated by Darwin, in imitation of this process; genetic algorithms are capable of creating solutions to real world problems. The evolution of these solutions to optimal values of the problem depends largely on proper coding them. The basic principles of genetic algorithms were established by Holland [18, 19] and are well described in texts Goldberg [15], Davis [17] and Michalewicz [24]. The evolutionary modeling of fuzzy logic system can be considered as an optimization process where the part or all a fuzzy system parameters constitute a search spaces of model operational (our case), cognitive and structural [8–10].

This chapter reports the results of the simulations, in which the genetic optimization of interval type-2 fuzzy integrators in ensembles of ANFIS models for the Mackey-Glass time series [25, 26], where the results for each ANFIS were evaluated by the method of the root mean square error (RMSE). For the integration of the results of each modular in the ensemble of ANFIS we used the following integration methods: interval type-2 fuzzy systems of kind Mamdani and integrator by interval type-2 FIS optimized [3, 5, 7].

The selection of the time series for the simulations was based on the fact that these time series are widely quoted in the literature by different researchers [1, 2, 20, 26, 28, 39, 40], which allows to compare results with other approaches such as neural networks and linear regression [12, 13, 23, 32].

In the next section we describe the background and basic concepts of ANFIS, Ensemble learning, Interval type-2 fuzzy, Genetic Algorithms and time series. Section 3 presents the proposed architecture of genetic optimization of interval type-2 fuzzy integrators in ensembles of ANFIS models for the time series prediction. Section 4 presents the simulations and the results obtained with different methods of integration that are used in this work. Section 5 presents the conclusions.

2 Background and Basic Concepts

This section presents the basic concepts of ANFIS, Ensemble learning, Interval type-2 fuzzy logic, and Genetic Algorithms.

2.1 ANFIS Models

There are have been proposed systems that have achieved fully the combination of fuzzy systems with neural networks, one of the most intelligent hybrid systems is the ANFIS (Adaptive Neuro Fuzzy Inference System method) as referred to by R. Jang [20, 22] (Fig. 1), which is a method for creating the rule base of a fuzzy system, using the algorithm of backpropagation training from the data collection process. Its architecture is functionally equivalent to a fuzzy inference system of Takagi-Sugeno-Kang.

Fig. 1
figure 1

ANFIS architecture

Full size image

The basic learning rule of ANFIS is the gradient descent backpropagation [40], which calculates the error rates (defined as the derivative of the squared error for each output node) recursively from the output to the input nodes.

As a result we have a hybrid learning algorithm [14], which combines the gradient descent and least-squares estimation. More specifically in, the forward step of the hybrid learning algorithm, functional signals (output nodes) are processed towards layer 4 and the parameters of consequence are identified by least squares. In the backward step the premise parameters are updated by gradient descent.

2.2 Ensemble Learning

The Ensemble consists of a learning paradigm where multiple component learners are trained for a same task, and the predictions of the component learners are combined for dealing with future instances [35, 36]. Since an Ensemble is often more accurate than its component learners, such a paradigm has become a hot topic in recent years and has already been successfully applied to optical character recognition, face recognition, scientific image analysis, medical diagnosis [45].

2.3 Interval Type-2 Fuzzy

Type-2 fuzzy sets are used to model uncertainty and imprecision; originally they were proposed by Zadeh [43, 44] and they are essentially “fuzzy–fuzzy” sets in which the membership degrees are type-1 fuzzy sets [6, 21, 27] (Fig. 2).

Fig. 2
figure 2

Basic structure of the interval type-2 fuzzy logic system

Full size image

The basic structure of a type-2 fuzzy system implements a nonlinear mapping of input to output space. This mapping is achieved through a set of type-2 if–then fuzzy rules, each of which describes the local behavior of the mapping.

The uncertainty is represented by a region called footprint of uncertainty (FOU). When \( \mu_{{\tilde{A}}} \left( {x,u} \right) = 1, \forall u \in l_{x} \subseteq [0,1] \); we have an interval type-2 membership function (Fig. 3).

Fig. 3
figure 3

Interval type-2 membership function

Full size image

The uniform shading for the FOU represents the entire interval type-2 fuzzy set and it can be described in terms of an upper membership function \( \overline{\mu }_{{\tilde{A}}} (x) \) and a lower membership function \( \underline{\mu }_{{\tilde{A}}} (x) \).

A fuzzy logic systems (FLS) described using at least one type-2 fuzzy set is called a type-2 FLS. Type-1 FLSs are unable to directly handle rule uncertainties, because they use type-1 fuzzy sets that are certain [29–31]. On the other hand, type-2 FLSs are very useful in circumstances where it is difficult to determine an exact certainty value, and there are measurement uncertainties.

2.4 Genetic Algorithms

Genetic Algorithms (GAs) are adaptive heuristic search algorithms based on the evolutionary ideas of natural selection and the genetic process [16]. The basic principles of GAs were first proposed by John Holland in 1975, inspired by the mechanism of natural selection, where stronger individuals are likely the winners in a competing environment [4]. GAs assumes that the potential solution of any problems an individual and can represented by a set of parameters [11]. These parameters are recharged as the genes of a chromosome and can be structured by string of values in binary form. A positive value, generally known as a fitness value, is used to reflect the degree of “goodness” of the chromosome for the problem which would be highly related with its objective value. The pseudocode of a GAs is as follows:

  1. 1.

    Start with a randomly generated population of n chromosome (candidate a solutions to problem).

  2. 2.

    Calculate the fitness of each chromosome in the problem.

  3. 3.

    Repeat the following steps until n offspring have been created:

    1. a.

      Select a pair of parent chromosome from the concurrent population, the probability of selection being an increasing function of fitness. Selection is done with replacement, meaning that the same chromosome can be selected more than once t that the same chromosome can be selected more than once to become a parent.

    2. b.

      With the probability (crossover rate), perform crossover to the pair at a randomly chosen point to form two offspring.

    3. c.

      Mutate the two offspring at each locus with probability (mutate rate), and place the resulting chromosome in the new population.

  4. 4.

    Replace the current population with the new population.

  5. 5.

    Go to step 2.

The simple procedure just describe above is the basic one for most applications of GAs found in the literature.

2.5 Time Series

The problem of predicting future values of a time series has been a point of reference for many researchers. The aim is to use the values of the time series known at a point x = t to predict the value of the series at some future point x = t + P. The standard method for this type of prediction is to create a mapping from D points of a Δ spaced time series, is (x (t − (D − 1) Δ), … , x (t − Δ), x (t)), to a predicted future value x (t + P). To allow a comparison with previous results in this work [6] the values D = 4 and Δ = P = 6 were used.

One of the chaotic time series data used is defined by the Mackey-Glass [25, 26] time series, whose differential equation is given by:

$$ x\left( t \right) = \frac{{0.2x\left( {t - \tau } \right)}}{{1 + x^{10} \left( {t - \tau } \right)}} - 0.1x\left( {t - \tau } \right) $$
(1)

For obtaining the values of the time series at each point, we applied the Runge-Kutta method for the solution of Eq. (1). The integration step was set at 0.1, with initial condition x (0) = 1.2, τ = 17, x (t) is then obtained for 0 ≤ t ≤ 1200, (Fig. 4) (We assume x (t) = 0 for t < 0 in the integration).

Fig. 4
figure 4

The Mackey-Glass time series

Full size image

From the Mackey-Glass time series we extracted 800 pairs of data points (Fig. 4) [20, 33], similar to [28].

We predict x (t) from the three past values of the time series, that is, x (t − 18), x (t − 12), and x (t − 6). Therefore the format of the training data is:

$$ \left[ {x\left( {t - 18} \right),x\left( {t - 12} \right),x\left( {t - 6} \right) \, ;x\left( t \right)} \right] $$
(2)

where t = 19 to 818 and x(t) is the desired prediction of the time series. The first 400 pairs of data are used to train the ANFIS, while the other 400 pairs of data are used to validate the model identification.

3 General Architecture of the Proposed Method

The proposed method combines the ensemble of ANFIS models and the use of interval type-2 fuzzy systems as response integrators (Fig. 5).

Fig. 5
figure 5

The architecture of the proposed model

Full size image

This architecture is divided into 5 sections, where the first phase represents the data base to simulate in the Ensemble of ANFIS, which in our case is the data base of the Mackey-Glass [7, 22] time series (used 800 pairs of data points). In the second phase, training (the first 400 pairs of data are used to train the ANFIS) and validation (the second 400 pairs of data are used to value the ANFIS) is performed sequentially in each ANFIS model, where the number of ANFIS to be used can be from 1 to n depending on what the user wants to test, but in our case we are dealing with a set of 3 ANFIS in the Ensemble.

In the third phase we have to generate the results of each ANFIS used in the previous section and in the fourth phase we integrate the overall results of each ANFIS, such integration will be done by interval type-2 fuzzy integrators of Mamdani type, but each fuzzy integrators is optimized (GAs) in the membership functions (Gaussians “iguassmtype2”, Generalized Bell “iguassmtype2” and Triangular “iguassmtype2”) [6, 7]. Finally the outcome or the final prediction of the Ensemble ANFIS learning is obtained.

4 Simulations Results

This section presents the results obtained through experiments on the architecture of genetic optimization of type-2 fuzzy integrators in ensembles of ANFIS models for the time series prediction, which show the performance that was obtained from each experiment to simulate the Mackey-Glass time series.

The design of the fuzzy systems is of Mamdani type and will have 3 inputs (ANFIS1, ANFIS2 and ANFIS3 predictions) and 1 output (forecast), so each input will be assigned two MFs “Small and Large” and the output will be assigned 3 MFs “OutANFIS1, OutANFIS2 and OutANFIS3”. The MFs of each interval type-2 fuzzy integrators will be changing to membership functions MFs (Gaussians, Generalized Bell, and Triangular) to observe the behavior of each of them and determine which one provides better forecast of the time series.

The GAs used to optimize the parameters values of the MFs in each interval type-2 fuzzy integrators. The representation of GAs is of Real-Values and the chromosome size will depend of the MFs that are used in each design of the interval type-2 fuzzy integrators.

The objective function is defined to minimize the prediction error as follows:

$$ f_{(t)} = \sqrt {\frac{{\sum\limits_{t = 1}^{n} {(a_{t} - p_{t} )^{2} } }}{n}} $$
(3)

where a, corresponds to the real data of the time series, p corresponds to the output of each fuzzy integrator, t is de sequence time series, and n is the numbers of data points of time series.

In the Table 1 it is illustrated the general representation of the chromosome and represent the utilized fuzzy membership functions. In this Table 1, the first row represented each input and output of the fuzzy inference system, the second row represents the membership function containing each input and output of the fuzzy inference system, the third row represents the MFs parameter “PL = Lower Parameter” where PL 1  … PL N are the size parameter of the MFs, the fourth row represent the MFs parameter “PL = Upper Parameter” PU 1  … PU N are the size parameter of the MFs that corresponds to each input and output. The number of parameters varies according to the kind of membership function of the interval type-2 fuzzy (e.g. three parameter are needed to represent a iguassmtype2 membership function of type-2 fuzzy “variance, mean1 and mean2”), therefore the numbers of parameters of each variable of input and output depend of the type of membership functions (Gaussians “iguassmtype2”, Generalized Bell “iguassmtype2”, and Triangular “iguassmtype2”) that will contains each the variables of the fuzzy integrators (our case will use two MFs to each variable of input and using three MFs to the variable output).

Table 1 General representation of the chromosome to the optimization of the interval type-2 fuzzy membership functions
Full size table

The GAs used has the following parameters: 75 individuals or genes, 50 generations and 30 iterations (running the GAs), the selection method are the stochastic universal sampling, the percentage of crossover or recombine is 0.8 and the mutation is 0.1. There are fundamentals parameters for test the performances of the GAs.

4.1 Genetic Optimization of Interval Type-2 Fuzzy Integration Using Gaussian MFs

In the design of the interval type-2 fuzzy integrator will have three input variables and one output variable, so each variable input will have two MFs and the variable output will have three MFs. Therefore the numbers of parameters that one used in the representation of the chromosome is 27, because iguassmtype2 MFs used three parameters (Variance, Mean1 and Mean2) to their representation in the interval type-2 fuzzy systems. The results obtained for the optimization of the iguassmtype2 MFs with GAs are the following: the type-2 fuzzy systems (Fig. 6) generated by the GAs, the structure of interval type-2 fuzzy integrator used three input variable (ANFIS1, ANFIS2 AND ANFIS3) and one output variable (forecast), and have 8 rules. The parameters obtained with the GAs for the type-2 fuzzy MFs iguassmtype2 (Fig. 7). The forecast data (Fig. 8) is generated by optimization of the interval type-2 fuzzy integrators. Therefore the evolution error (Fig. 9) obtained with the GAs for this integration is 0.01843.

Fig. 6
figure 6

Plot of the fuzzy inference system integrators generated with the GAs

Full size image
Fig. 7
figure 7

Plot of the optimization of the Memberships functions (input and output) parameters with the GAs

Full size image
Fig. 8
figure 8

Plot of the forecast generated by the genetic optimization of type-2 fuzzy integrators

Full size image
Fig. 9
figure 9

Plot of the evolution error generated by GAs

Full size image

4.2 Genetic Optimization of Interval Type-2 Fuzzy Integration Using Generalized Bell MFs

In the design of the interval type-2 fuzzy integrator will have three input variables and one output variable, so each input variable will have two MFs and the output variable will have three MFs. Therefore the number of parameters of the MFs that are used in the representation of the chromosome is 54, because igbelltype2 MFs used six’s parameters (a1, b1, c1, a2, b2 and c2) to their representation in the interval type-2 fuzzy systems. The results obtained for the optimization of the igbelltype2 MFs with GAs are the following: the type-2 fuzzy systems (Fig. 10) generated by the GAs, the structure of interval type-2 fuzzy integrator systems used three input variables (ANFIS1, ANFIS2 AND ANFIS3) and one output variable (forecast), and have 9 rules. The parameters obtained with the GAs for the type-2 fuzzy MFs igbelltype2 (Fig. 11). The forecast data (Fig. 12) is generated by optimization of the interval type-2 fuzzy integrators. Therefore the evolution error (Fig. 13) obtained with the GAs for this integration is 0.018154.

Fig. 10
figure 10

Interval type-2 fuzzy systems integrator generated with the GAs

Full size image
Fig. 11
figure 11

Optimization of the memberships functions (input and output) parameters with the GAs

Full size image
Fig. 12
figure 12

Forecast generated by the genetic optimization of the type-2 fuzzy integrators

Full size image
Fig. 13
figure 13

Plot of the evolution error generated by GAs

Full size image

4.3 Genetic Optimization of Interval Type-2 Fuzzy Integration Using Triangular MFs

In the design of the interval type-2 fuzzy integrator will have three input variables and one output variable, so each input variable will have two MFs and the output variable will have three MFs. Therefore the number of parameters that are used in the representation of the chromosome is 54, because itritype2 MFs used six’s parameters (a1, b1, c1, a2, b2 and c2) to their representation in the interval type-2 fuzzy systems. The results obtained for the optimization of the itritype2 MFs with GAs are the following: the type-2 fuzzy systems (Fig. 14) generated by the GAs, the structure of interval type-2 fuzzy integrator systems used three input variables (ANFIS1, ANFIS2 AND ANFIS3) and one output variable (forecast), and have 9 rules. The parameters obtained with the GAs for the type-2 fuzzy MFs itritype2 (Fig. 15). The forecast data (Fig. 16) is generated by optimization of the interval type-2 fuzzy integrators. Therefore the evolution error (Fig. 17) obtained with the GAs for this integration is 0.01838.

Fig. 14
figure 14

Interval type-2 fuzzy systems integrator generated with the GAs

Full size image
Fig. 15
figure 15

Optimization of the memberships functions (input and output variables) parameters with the GAs

Full size image
Fig. 16
figure 16

Forecast generated by the genetic optimization of type-2 fuzzy integrators

Full size image
Fig. 17
figure 17

Plot of the evolution error generated by GAs

Full size image

4.4 Results and Comparison

Table 2 shows the results of 30 experiments that were tested with the genetic optimization of interval type-2 fuzzy integrators in ensembles of ANFIS models for the time series prediction. This table shows the best results and mean results of the 30 runs. The Best and Averages results for the Type-1 fuzzy integrator [28] are using Gaussians MFs, which obtained a prediction error of 0.01854 for the best and average prediction error of 0.01954. The Best and Mean results for the interval Type-2 fuzzy integrators using Generalized Bell (igbelltype2) MFs, which obtained a prediction error of 0.01811 for the best and average prediction error of 0.01822.

Table 2 Best and Mean results of the prediction error of Mackey-Glass
Full size table

5 Conclusion

In conclusion we can say that the results obtained with the proposed architecture of genetic optimization of interval type-2 fuzzy integrators in ensembles of ANFIS models for the time series prediction have been good and positive in predicting time series (like the Mackey-Glass), as it has managed to minimize the prediction error of the time series against the results obtained by other researchers.

We conclude that the results obtained with the architectures of genetic optimization of interval type-2 fuzzy integrators in ensembles of ANFIS models for the time series prediction are good, since we achieved [98.5–99] percent of accuracy in the forecast with the Mackey-Glass time series and on the opposite side it was obtained prediction error of 1.5 %. Therefore, we can conclude that interval type-2 fuzzy integrator is better than the type-1 fuzzy integrator, because in most of the experiments that were performed with the proposed architecture of ensembles of ANFIS, the interval type-2 fuzzy gave better prediction errors than the type-1 fuzzy. Therefore the proposal offers efficient results in the prediction of such time series, which can help us, make decisions and avoid unexpected events in the future.