Abstract
Ensemble Methods represent an important research area within machine learning. Here, we argue that the use of such methods can be generalized and applied in many more situations than they have been previously. Instead of using them only to combine the output of an algorithm, we can apply them to the decisions made inside the learning algorithm, itself. We call this approach Inner Ensembles. The main contribution of this work is to demonstrate how broadly this idea can applied. Specifically, we show that the idea can be applied to different classes of learner such as Bayesian networks and K-means clustering.
Chapter PDF
Similar content being viewed by others
References
Geurts, P., Wehenkel, L.: Investigation and Reduction of Discretization Variance in Decision Tree Induction. In: Lopez de Mantaras, R., Plaza, E. (eds.) ECML 2000. LNCS (LNAI), vol. 1810, pp. 162–170. Springer, Heidelberg (2000)
Prez, J.M., Muguerza, J., Arbelaitz, O., Gurrutxaga, I.: A New Algorithm to Build Consolidated Trees: Study of the Error Rate and Steadiness. In: Proceedings of the International Intelligent Information Processing and Web Mining Conference. Advances in Soft Computing, pp. 79–88 (2004)
Ferri, C., Hernández-Orallo, J., Ramírez-Quintana, M.J.: From Ensemble Methods to Comprehensible Models. In: Proceedings of the 5th International Conference on Discovery Science, pp. 165–177 (2002)
Wolberg, W.H., Street, W.N., Mangasarian, O.L.: Machine Learning Techniques to Diagnose Breast Cancer from Image-Processed Nuclear Features of Fine Needle Aspirates. Cancer Letters 77, 163–171 (1994)
Flann, N.S., Dietterich, T.G.: Selecting Appropriate Representations for Learning from Examples. In: Proceedings of the 5th National Conference on Artificial Intelligence, pp. 460–466 (1986)
Craven, M.W., Shavlik, J.W.: Extracting Comprehensible Concept Representations from Trained Neural Networks. In: Working Notes on the International Joint Conference on AI Workshop on Comprehensibility in Machine Learning, pp. 61–75 (1995)
Dwyer, K., Holte, R.: Decision Tree Instability and Active Learning. In: Kok, J.N., Koronacki, J., Lopez de Mantaras, R., Matwin, S., Mladenič, D., Skowron, A. (eds.) ECML 2007. LNCS (LNAI), vol. 4701, pp. 128–139. Springer, Heidelberg (2007)
Blumer, A., Ehrenfeucht, A., Haussler, D., Warmuth, M.K.: Occam’s Razor. Information Processing Letters 24(6), 377–380 (1987)
Domingos, P.: Occam’s Two Razors: The Sharp and the Blunt. In: Knowledge Discovery and Data Mining, pp. 37–43 (1998)
Williams, N., Zander, S., Armitage, G.: A Preliminary Performance Comparison of Five Machine Learning Algorithms for Practical IP Traffic Flow Classification. Special Interest Group on Data Communication 36(5), 5–16 (2006)
Zimmermann, A.: Ensemble-Trees: Leveraging Ensemble Power Inside Decision Trees. In: Proceedings of the 11th International Conference on Discovery Science, pp. 76–87 (2008)
Lou, Y., Caruana, R., Gehrke, J.: Intelligible Models for Classification and Regression. In: Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 150–158 (2012)
Van Assche, A., Blockeel, H.: Seeing the Forest through the Trees: Learning a Comprehensible Model from a First Order Ensemble. In: Kok, J.N., Koronacki, J., Lopez de Mantaras, R., Matwin, S., Mladenič, D., Skowron, A. (eds.) ECML 2007. LNCS (LNAI), vol. 4701, pp. 418–429. Springer, Heidelberg (2007)
Zhou, Z.H., Jiang, Y., Chen, S.F.: Extracting Symbolic Rules from Trained Neural Network Ensembles. AI Communications 16(1), 3–15 (2003)
Domingos, P.: Knowledge Discovery Via Multiple Models. Intelligent Data Analysis 2, 187–202 (1998)
Pourret, O., Marcot, B., Naim, P.: Bayesian Networks: A Practical Guide to Applications. Statistics in Practice. Wiley, Chichester (2008)
Desmedt, J.: Computer-Aided Electromyography and Expert Systems. Clinical Neurophysiology Updates. Elsevier, Amsterdam (1989)
Heckerman, D.E., Nathwani, B.N.: Towards Normative Expert Systems: Part ii, Probability-Based Representations for Efficient Knowledge Acquisition and Inference Methods of Information in Medicine. In: Methods of Information in Medicine, pp. 106–116 (1992)
Vomlel, J.: Two Applications of Bayesian Networks. In: Proceedings of Conference Znalosti, pp. 73–82 (2002)
Mitchell, T.M.: Machine Learning, 1st edn. McGraw-Hill, Inc., New York (1997)
Cooper, G.F., Herskovits, E.: A Bayesian Method for the Induction of Probabilistic Networks from Data. Machine Learning 9(4), 309–347 (1992)
Bouckaert, R.: Bayesian Network Classifiers in Weka. Working paper series. Department of Computer Science, University of Waikato (2004)
Lakoff, G.: Women, Fire and Dangerous Things. University of Chicago Press, Chicago (1987)
Tan, P.N., Steinbach, M., Kumar, V.: Introduction to Data Mining, 1st edn. Addison-Wesley Longman Publishing Co., Inc., Boston (2005)
Duda, R.O., Hart, P.E.: Pattern Classification and Scene Analysis. John Willey & Sons, New York (1973)
Breiman, L.: Bias, Variance, and Arcing Classifiers. Technical report, Statistics Department, University of California at Berkeley (1996)
Asuncion, A., Newman, D.J.: UCI Machine Learning Repository (2007)
Strehl, A., Ghosh, J., Cardie, C.: Cluster Ensembles - A Knowledge Reuse Framework for Combining Multiple Partitions. Journal of Machine Learning Research 3, 583–617 (2002)
Rendón, E., Abundez, I.M., Gutierrez, C., Zagal, S.D., Arizmendi, A., Quiroz, E.M., Arzate, H.E.: A Comparison of Internal and External Cluster Validation Indexes. In: Proceedings of the American Conference on Applied Mathematics and the 5th International Conference on Computer Engineering and Applications, pp. 158–163 (2011)
Nguyen, N., Caruana, R.: Consensus Clusterings. In: Proceedings of the 7th International Conference on Data Mining, pp. 607–612 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Abbasian, H., Drummond, C., Japkowicz, N., Matwin, S. (2013). Inner Ensembles: Using Ensemble Methods Inside the Learning Algorithm. In: Blockeel, H., Kersting, K., Nijssen, S., Železný, F. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2013. Lecture Notes in Computer Science(), vol 8190. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-40994-3_3
Download citation
DOI: https://doi.org/10.1007/978-3-642-40994-3_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-40993-6
Online ISBN: 978-3-642-40994-3
eBook Packages: Computer ScienceComputer Science (R0)