Abstract
In this paper the development of a method for the design of the general architecture of ensemble neural networks for time series prediction is presented. This method consists on the optimization of the ensemble neural network with response integration based on type-1 and type-2 fuzzy logic. The optimization algorithms that are used to illustrate the proposed method are the genetic algorithm and the particle swarm optimization algorithm and the purpose of applying these methods is to find the best possible architecture of the neural network ensemble with the lowest prediction error. Simulation results with several complex times series are presented to show the advantage of the proposed method, also comparisons are made with other time series, such as Mexican Stock Exchange, and Dow Jones Stock Exchange.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
P.D. Brockwell, A.D. Richard, Introduction to Time Series and Forecasting (Springer, New York, 2002), pp. 1–219
O. Castillo, P. Melin, Hybrid intelligent systems for time series prediction using neural networks, fuzzy logic, and fractal theory. Neural Netw. IEEE Trans. 13(6), 1395–1408 (2002)
O. Castillo, P. Melin, Simulation and forecasting complex economic time series using neural networks and fuzzy logic. Proc. Int. Neural Netw. 3, 1805–1810 (2001)
O. Castillo, P. Melin, Simulation and forecasting complex financial time series sing neural networks and fuzzy logic. Proc. IEEE Int. Conf. Syst. Man Cybern. 4, 2664–2669 (2001)
Cherkauer K.J., Human expert level performance on a scientific image analysis task by a system using combined artificial neural networks, in Proceedings of AAAI-96Workshop on Integrating Multiple Learned Models for Improving and Scaling Machine Learning Algorithms ed by P. Chan, S. Stolfo, D. Wolpert (AAAI Press, Portland, OR, Menlo Park, CA, 1996), pp. 15–21
P. Cunningham, J. Carney, S. Jacob, Stability problems with artificial neural networks and the ensemble solution. Artif. Intell. Med. 20(3), 217–225 (2000)
N. Davey, S. Hunt, R. Frank, Time Series Prediction and Neural Networks (University of Hertfordshire, Hatfield, UK, 1999)
M. Dorigo, T. Stutzle, Ant Colony Optimization “Ant Colony Optimization Theory” (A Bradford Book The Milt Press, London England, Massachusetts Institute of Technology, 2004) pp. 121–151
H. Drucker, R. Schapire, P. Simard, Improving performance in neural networks using a boosting algorithm, in Advances in Neural Information Processing Systems, vol. 5, ed. by S.J. Hanson, J.D. Cowan, C.L. Giles (Morgan Kaufmann, Denver, CO, San Mateo, CA, 1993), pp. 42–49
Dow Jones Company. http://www.dowjones.com. 10 Jan 2010
Dow Jones Indexes. http://www.djindexes.com. 5 Sep 2010
Engelbrech. P., in Fundamentals of Computational of Swarm Intelligence, Basic Particle Swarm Optimization, (Wiley y Sons, 2005), pp. 93–129
R.R. Yager et al., in Fuzzy Sets and Applications: Selected Papers, ed. by L.A. Zadeh (John Wiley, New York, 1987)
D. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning. Ed. (Addison Wesley, 1989)
S. Gutta, H. Wechsler, Face recognition using hybrid classifier systems, in Proceedings of ICNN-96 (IEEE Computer Society Press, Washington, DC, Los Alamitos, CA, 1996), pp. 1017–1022
J. Hampshire, A. Waibel, A novel objective function for improved phoneme recognition using time- delay neural networks. IEEE Trans. Neural Netw. 1(2), 216–228 (1990)
L.K. Hansen, P. Salomon, Neural network ensembles. IEEE Trans. Pattern Anal. Mach. Intell. 12(10), 993–1001 (1990)
L.K. Hansen, P. Salomon, Ensemble methods for hand written digit recognition, in Proceedings of IEEE Workshop on Neural Networks for Signal Processing (IEEE Press, Helsingoer, Denmark, Piscataway, NJ, 199)2, pp. 333–342
J. Holland, Adaptation in Natural and Artificial Systems (University of Michigan Press, 1975)
F.J. Huang, Z. Huang. H.-J. Zhang, T.H. Chen, Pose invariant face recognition, in Proceedings of 4th IEEE International Conference on Automatic Face and Gesture Recognition (IEEE Computer Society Press, Grenoble, France, Los Alamitos, CA, 2000), pp. 245–250
N. Karnik, J.M. Mendel, Applications of type-2 fuzzy logic systems to forecasting of time-series. 120(1–4), 89–111 (1999)
A. Kehagias, V. Petridis, Predictive modular neural networks for time series classification. Neural Netw. 10(1), 31–49 (1997)
A. Krogh, J. Vedels, Neural network ensembles, cross validation, and active learning, in Advances in Neural Information Processing Systems 7, ed. by G. Tesauro, D. Touretzky, T. Leen (MIT Press, CO, Denver, Cambridge, MA, 1995), pp. 231–238
M.C. Mackey, Adventures in Poland: having fun and doing research with Andrzej Lasota. Mat. Stosow, 5–32 (2007)
M.C. Mackey, L. Glass, Oscillation and chaos in physiological control systems. Science 197, 287–289 (1997)
Maclin R., Shavlik J.W., Combining the predictions of multiple classifiers: using competitive learning to initialize neural networks, in Proceedings of IJCAI-95 (Montreal, Canada, Morgan Kaufmann, San Mateo, CA, 1995), pp. 524–530
K. Man, K. Tang, S. Kwong, in Genetic Algorithms and Designs, “Introduction, Background and Biological Background Ed. (Springer, London Limited, 1998), pp. 1–62
L.P. Maguire, B. Roche, T.M. McGinnity, J. McDaidL, Predicting a chaotic time series using a fuzzy neural network. Inf. Sci. 112(1–4), 125–136 (1998)
J. Mao, A case study on bagging, boosting and basic ensembles of neural networks for OCR, in Proceedings of IJCNN-98, vol. 3. (IEEE Computer Society Press, Anchorage, AK, Los Alamitos, CA, 1998), pp. 1828–1833
MATLAB 2007, Historical Mackey Glass data
Mexico Bank Database, http://www.banxico.org.mx. 30 Aug 2011
P. Melin, O. Castillo, S. Gonzalez, J. Cota, W. Trujillo, P. Osuna, Design of Modular Neural Networks with Fuzzy Integration Applied to Time Series Prediction, vol. 41 (Springer, Berlin/Heidelberg, 2007), pp. 265–273
I.M. Multaba, M.A. Hussain, Application of Neural Networks and Other Learning. Technologies in Process Engineering (Imperial Collage Press, London, 2001)
J.S.R. Jang, C.T. Sun, E. Mizutani, Neuro-Fuzzy and Soft Computing (Prentice Hall, Englewood Cliffs NJ, 1996)
D.W. Opitz, J.W. Shavlik, Generating accurate and diverse members of a neural network ensemble, in Advances in Neural Information Processing Systems 8, ed. by D.S. Touretzky, M.C. Mozer, M.E. Hasselmo (MIT Press, Cambridge, MA, Denver, CO, 1996), pp. 535–541
E.A. Plummer, in Time Series Forecasting With Feed-Forward Neural Networks: Guidelines and Limitations. University of Wyoming (2000)
C. Reeves, J. Row, Genetic Algorithms: Principles and Perspectives, “Optimization” (Kluwer Academic Publishers, New York, Boston, Dordrecht, London, 2002), pp. 4–8
A. Sharkey, One combining Artificial of Neural Nets (Department of Computer Science University of Sheffield, U.K., 1996)
A. Sharkey, Combining Artificial Neural Nets: Ensemble and Modular Multi-Net Systems (Springer, London, 1999)
P. Sollich, A. Krogh, Learning with ensembles: how over-fitting can be useful, in Advances in Neural Information Processing Systems, vol. 8, ed. by D.S. Touretzky, M.C. Mozer, M.E. Hasselmo (MIT Press, Cambridge, MA, Denver, CO, 1996), pp. 190–196
S. Soltani, On the use of the wavelet decomposition for time series prediction. Neurocomputing 48(1–4), 267–277 (2002)
M. Štěpnička, P. Cortez, J.P. Donate, L. Štěpničková, Forecasting seasonal time series with computational intelligence: On recent methods and the potential of their combinations. Expert Syst. Appl. 40(6), 1981–1992 (2013)
W.W.S. Wei, Time Series Analysis: Univariate and Multivariate Methods, vol. 1 (Addison-Wesley, Reading, MA, 1994), pp. 40–100
R.N. Yadav, P.K. Kalra, J. John, Time series prediction with single multiplicative neuron model. Soft Comput. Time Ser. Prediction Appl. Soft Comput. 7(4), 1157–1163 (2007)
X. Yao, Y. Liu, Making use of population in formation in evolutionary artificial neural networks. IEEE Trans. Syst. Man Cybern. Part B Cybern. 28(3), 417–425 (1998)
J. Yen, R. Langari, Fuzzy Logic: Intelligence, Control and Information Ed. (Prentice Hall, Englewood Cliffs NJ, 1999)
L. Zhao, Y. Yang, PSO-based single multiplicative neuron model for time series prediction. Expert Syst. Appl. 36(2), 2805–2812 (2009). (Part 2)
Z.-H. Zhou, Y. Jiang, Y.-B. Yang, S.-F. Chen, Lung cancer cell identification based on artificial neural network ensembles. Artif. Intell. Med. 24(1), 25–36 (2002)
M.A. Sanchez, O. Castillo, J.R. Castro, P. Melin, Fuzzy granular gravitational clustering algorithm for multivariate data. Inf. Sci. 279, 498–511 (2014)
D. Sánchez, P. Melin, Optimization of modular granular neural networks using hierarchical genetic algorithms for human recognition using the ear biometric measure. Eng. Appl. Artif. Intell. 27, 41–56 (2014)
M.A. Sanchez, J.R. Castro, O. Castillo, O. Mendoza, A. Rodriguez-Diaz, P. Melin, Fuzzy higher type information granules from an uncertainty measurement. Granular Comput. 2(2), 95–103 (2017)
F. Gaxiola, P. Melin, F. Valdez, J.R. Castro, O. Castillo, Optimization of type-2 fuzzy weights in backpropagation learning for neural networks using GAs and PSO. Appl. Soft Comput. 38, 860–871 (2016)
Acknowledgements
We would like to express our gratitude to the CONACYT, Tijuana Institute of Technology for the facilities and resources granted for the development of this research.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Pulido, M., Melin, P. (2021). Comparison of Genetic Algorithm and Particle Swarm Optimization of Ensemble Neural Networks for Complex Time Series Prediction. In: Melin, P., Castillo, O., Kacprzyk, J. (eds) Recent Advances of Hybrid Intelligent Systems Based on Soft Computing. Studies in Computational Intelligence, vol 915. Springer, Cham. https://doi.org/10.1007/978-3-030-58728-4_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-58728-4_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-58727-7
Online ISBN: 978-3-030-58728-4
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)