Abstract
Although there have been a few approaches to achieve the goal of fault tolerance by diversifying redundancy of the individual networks that make up a neural network ensemble, some of which include ensembles of neural networks of different sizes, and ensembles of different models of neural networks such as Radial Basis Function Networks and Multilayer Perceptron, there is yet to be an empirical study on hybrid neural networks that makes use of a diverse set of transfer functions, which we would expect to be able to exhibit diverse network architectures, and thus possibly more diverse error patterns. In this paper, we present an approach that uses transfer function diversity to achieve significant results on ensembles. The results show that even with relatively small networks having 5 hidden nodes, and a relatively small ensemble size of just 10 members, the ensemble is able to get competitive results on the Iris data set. It also capable of obtaining competitive results with 20 ensemble members of relatively small networks on other popular data sets such as the Diabetes, Sonar, Hepatitis, and Australian Credit Card problems. In addition to that, it is shown that these results can be achieved with a simple sorting and selection of the Top N solutions of the population, in contrast to other methods of selecting ensemble members that can be computationally expensive, such as selection of the Pareto-front, or hill climbing methods of selection.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
Keywords
References
Lower Your Risks: Age, Race, Gender & Family History (2013), http://www.diabetes.org/are-you-at-risk/lower-your-risk/nonmodifiables.html
Abbass, H.: Pareto neuro-evolution: constructing ensemble of neural networks using multi-objective optimization. In: The 2003 Congress on Evolutionary Computation, CEC 2003, vol. 3, pp. 2074–2080. IEEE, Cancun (2003)
Bache, K., Lichman, M.: UCI Machine Learning Repository (2013), http://archive.ics.uci.edu/ml
Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press (1995)
Briggman, K., Kristan, W.: Multifunctional pattern-generating circuits. Annual Review of Neuroscience 31, 2710294 (2008)
Brown, G., Wyatt, J., Harris, R., Yao, X.: Diversity creation methods: a survey and categorisation. Information Fusion 6(1), 5–20 (2005)
Chandra, A., Yao, X.: Ensemble Learning Using Multi-Objective Evolutionary Algorithms. Journal of Mathematical Modelling and Algorithms 5(4), 417–445 (2006)
Gutierrez, P., Hervas, C., Carbonero, M., Fernandez, J.: Combined projection and kernel basis functions for classification in evolutionary neural networks. Neurocomputing 72(13-15), 2731–2742 (2009)
Gutiérrez, P.A., Hervás-Martínez, C.: Hybrid Artificial Neural Networks: Models, Algorithms and Data. In: Cabestany, J., Rojas, I., Joya, G. (eds.) IWANN 2011, Part II. LNCS, vol. 6692, pp. 177–184. Springer, Heidelberg (2011)
Hansen, L., Salamon, P.: Neural network ensembles. IEEE Transactions on Pattern Analysis and Machine Intelligence 12(10), 993–1001 (1990)
Higuchi, T.: Evolutionary ensembles with negative correlation learning. IEEE Transactions on Evolutionary Computation 4(4), 380–387 (2000)
Islam, M.M., Yao, X., Murase, K.: A constructive algorithm for training cooperative neural network ensembles. IEEE Transactions on Neural Networks 14(4), 820–834 (2003)
Jankowski, N., Duch, W.: Optimal transfer function neural networks. In: 9th European Symposium on Artificial Neural Networks, ESANN 2001, Bruges, vol. (I), pp. 101–106 (2001)
Krogh, A., Vedelsby, J.: Neural network ensembles, cross validation, and active learning. In: Advances in Neural Information Processing Systems, pp. 231–238 (1995)
Liu, Y., Yao, X.: Ensemble learning via negative correlation. Neural Networks: The Official Journal of the International Neural Network Society 12(10), 1399–1404 (1999)
Maul, T.: Early experiments with neural diversity machines. Neurocomputing 113, 36–48 (2013)
Opitz, D., Shavlik, J.: Generating Accurate and Diverse Members of a Neural-Network Ensemble. In: Advances in Neural Information Processing Systems, vol. 8, pp. 535–541 (1996)
Perrone, M., Cooper, L.: When networks disagree: Ensemble methods for hybrid neural networks. In: Neural Networks for Speech and Image Processing, pp. 126–142 (1993)
Sharkey, A., Sharkey, N.: Combining diverse neural nets. The Knowledge Engineering Review 12(3), 231–247 (1997)
Wu, Z., Chen, Y.: Genetic algorithm based selective neural network ensemble. In: Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence, IJCAI 2001, vol. 1, pp. 797–802 (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Adamu, A., Maul, T., Bargiela, A., Roadknight, C. (2015). Preliminary Experiments with Ensembles of Neurally Diverse Artificial Neural Networks for Pattern Recognition. In: Unger, H., Meesad, P., Boonkrong, S. (eds) Recent Advances in Information and Communication Technology 2015. Advances in Intelligent Systems and Computing, vol 361. Springer, Cham. https://doi.org/10.1007/978-3-319-19024-2_9
Download citation
DOI: https://doi.org/10.1007/978-3-319-19024-2_9
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-19023-5
Online ISBN: 978-3-319-19024-2
eBook Packages: EngineeringEngineering (R0)