Abstract
A new method for ensemble generation is presented. It is based on grouping the attributes in different subgroups, and to apply, for each group, an axis rotation, using Principal Component Analysis. If the used method for the induction of the classifiers is not invariant to rotations in the data set, the generated classifier can be very different. Hence, once of the objectives aimed when generating ensembles is achieved, that the different classifiers were rather diverse. The majority of ensemble methods eliminate some information (e.g., instances or attributes) of the data set for obtaining this diversity. The proposed ensemble method transforms the data set in a way such as all the information is preserved. The experimental validation, using decision trees as base classifiers, is favorable to rotation based ensembles when comparing to Bagging, Random Forests and the most well-known version of Boosting.
This work has been supported by the Spanish MCyT project DPI2001-4404-E and the “Junta de Castilla y León” project VA101/01.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Bauer, E., Kohavi, R.: An empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Machine Learning 36(1-2), 105–139 (1999)
Blake, C.L., Merz, C.J.: UC1 repository of machine learning databases (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html
Breiman, L.: Bagging predictors. Machine Learning 24(2), 123–140 (1996)
Breiman, L.: Random forests. Machine Learning 45(l), 5–32 (2001)
Dietterich, T.G.: Ensemble methods in machine learning. Multiple Classifier Systems 2000, 1–15 (2000)
Han, J., Kamber, M.: Data Mining: Concepts and Techniques. Morgan Kanfmann, San Francisco (2001)
Ho, T.K.: The random subspace method for constructing decision forests. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(8), 832–844 (1998)
Ho, T.K.: A data complexity analysis of comparative advantages of decision forest constructors. Pattern Analysis and Application 5, 102–112 (2002)
Ross Quinlan, J.: C4.5: programs for machine learning. In: Machine Learning, Morgan Kaufuiann, San Mateo (1993)
Schapire, R.E.: The boosting approach to machine learning: An overview. In: MSRI Workshop on Nonlinear Estimation and Classification (2002), http://www.cs.princeton.edu/~schapire/papers/msri.ps.gz
Tumer, K., Oza, N.C.: Input decimated ensembles. Pattern Anal Applic. 6, 65–77 (2003)
Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann, San Francisco (1999), http://www.cs.waikato.ac.nz/ml/weka
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Rodríguez, J.J., Alonso, C.J. (2004). Rotation-Based Ensembles. In: Conejo, R., Urretavizcaya, M., Pérez-de-la-Cruz, JL. (eds) Current Topics in Artificial Intelligence. TTIA 2003. Lecture Notes in Computer Science(), vol 3040. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-25945-9_49
Download citation
DOI: https://doi.org/10.1007/978-3-540-25945-9_49
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-22218-7
Online ISBN: 978-3-540-25945-9
eBook Packages: Springer Book Archive