Abstract
The ensemble classification paradigm is an effective way to improve the performance and stability of individual predictors. Many ways to build ensembles have been proposed so far, most notably bagging and boosting based techniques. Evolutionary algorithms (EAs) also have been widely used to generate ensembles. In the context of heterogeneous ensembles EAs have been successfully used to adjust weights of base classifiers or to select ensemble members. Usually, a weighted sum is used for combining classifiers outputs in both classical and evolutionary approaches. This study proposes a novel genetic program that learns a fusion function for combining heterogeneous-classifiers outputs. It evolves a population of fusion functions in order to maximize the classification accuracy. Highly non-linear functions are obtained with the proposed method, subsuming the existing weighted-sum formulations. Experimental results show the effectiveness of the proposed approach, which can be used not only with heterogeneous classifiers but also with homogeneous-classifiers and under bagging/boosting based formulations.
Chapter PDF
Similar content being viewed by others
References
Dietterich, T.: Ensemble methods in machine learning. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857, pp. 1–15. Springer, Heidelberg (2000)
Kuncheva, L., Whitaker, C.: Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Mach. Learn. 51(2), 181–207 (2003)
Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 55, 119–139 (1997)
Breiman, L.: Random forest. Mach. Learn. 24(2), 123–140 (2001)
Bian, S., Wang, W.: On diversity and accuracy of homogeneous and heterogeneous ensembles. International Journal of Hybrid Intelligent Systems 4, 103–128 (2007)
de Oliveira, D., Canuto, A., De Souto, M.C.P.: Use of multi-objective genetic algorithms to investigate the diversity/accuracy dilemma in heterogeneous ensembles. In: Proc. of IJCNN, pp. 2339–2346 (2010)
Park, C., Cho, S.: Evolutionary computation for optimal ensemble classifier in lymphoma cancer classification. In: Zhong, N., Raś, Z.W., Tsumoto, S., Suzuki, E. (eds.) ISMIS 2003. LNCS (LNAI), vol. 2871, pp. 521–530. Springer, Heidelberg (2003)
Macaš, M., Gabrys, B., Ruta, D., Lhotská, L.: Particle swarm optimisation of multiple classifier systems. In: Sandoval, F., Prieto, A.G., Cabestany, J., Graña, M. (eds.) IWANN 2007. LNCS, vol. 4507, pp. 333–340. Springer, Heidelberg (2007)
Yang, L., Qin, Z.: Combining classifiers with particle swarms. In: Wang, L., Chen, K., S. Ong, Y. (eds.) ICNC 2005. LNCS, vol. 3611, pp. 756–763. Springer, Heidelberg (2005)
Langdon, W.B., Barrett, S.J., Buxton, B.F.: Combining decision trees and neural networks for drug discovery. In: Foster, J.A., Lutton, E., Miller, J., Ryan, C., Tettamanzi, A.G.B. (eds.) EuroGP 2002. LNCS, vol. 2278, pp. 60–70. Springer, Heidelberg (2002)
Espejo, P., Ventura, S., Herrera, F.: A survey on the application of genetic programming to classification. IEEE T. Syst. Man. Cyb. C 40(2), 121–144 (2010)
Escalante, H.J., Montes, M., Sucar, L.E.: Ensemble particle swarm model selection. In: Proc. of IJCNN, pp. 1–10 (2010)
Bhowan, U., Johnston, M., Zhang, M., Yao, X.: Evolving diverse ensembles using genetic programming for classification with unbalanced data. IEEE Transactions on Evolutionary Computation 17(3), 368–386 (2013)
Langdon, W.B., Poli, R.: Foundations of Genetic Programming. Springer (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Escalante, H.J., Acosta-Mendoza, N., Morales-Reyes, A., Gago-Alonso, A. (2013). Genetic Programming of Heterogeneous Ensembles for Classification. In: Ruiz-Shulcloper, J., Sanniti di Baja, G. (eds) Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications. CIARP 2013. Lecture Notes in Computer Science, vol 8258. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-41822-8_2
Download citation
DOI: https://doi.org/10.1007/978-3-642-41822-8_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-41821-1
Online ISBN: 978-3-642-41822-8
eBook Packages: Computer ScienceComputer Science (R0)