Abstract
Rule based classification is a popular approach for decision making. It is also achievable that multiple rule based classifiers work together for group decision making by using ensemble learning approach. This kind of expert system is referred to as ensemble rule based classification system by means of a system of systems. In machine learning, an ensemble learning approach is usually adopted in order to improve overall predictive accuracy, which means to provide highly trusted decisions. This chapter introduces basic concepts of ensemble learning and reviews Random Prism to analyze its performance. This chapter also introduces an extended framework of ensemble learning, which is referred to as Collaborative and Competitive Random Decision Rules (CCRDR) and includes Information Entropy Based Rule Generation (IEBRG) and original Prism in addition to PrismTCS as base classifiers. This is in order to overcome the identified limitations of Random Prism. Each of the base classifiers mentioned above is also introduced with respects to its essence and applications. An experimental study is undertaken towards comparative validation between the CCRDR and Random Prism. Contributions and Ongoing and future works are also highlighted.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
Keywords
References
Knoneko, I., Kukar, M.: Machine Learning and Data Mining: Introduction to Principles and Algorithms. Horwood Publishing, Chichester (2007)
Breiman, L.: Bagging Predictors. Machine Learning 2(24), 123–140 (1996)
Tan, P.N., Steinbach, M., Kumar, V.: Introduction to Data Mining. Pearson Education, Inc., New Jersey (2006)
Cendrowska, J.: PRISM: an algorithm for inducing modular rules. International Journal of Man-Machine Studies 27, 349–370 (1987)
Bramer, M.A.: Automatic induction of classification rules from examples using N-Prism. In: Research and Development in Intelligent Systems, vol. XVI, pp. 99–121. Springer, Cambridge (2000)
Liu, H., Gegov, A.: Induction of modular classification rules by Information Entropy Based Rule Generation. In: Sgurev, V., Yager, R., Kacprzyk, J. (eds.) Innovative Issues in Intelligent Systems. Springer (in press)
Michalski, R.S.: On the Quasi-Minimal solution of the general covering problem. In: Proceedings of the Fifth International Symposium on Information Processing, Bled, Yugoslavia, pp. 125–128 (1969)
Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufman (1993)
Bramer, M.A.: Using J-Pruning to Reduce Overfitting of Classification Rules in Noisy Domains. In: Hameurlain, A., Cicchetti, R., Traunmüller, R. (eds.) DEXA 2002. LNCS, vol. 2453, p. 433. Springer, Heidelberg (2002)
Stahl, F., Bramer, M.A.: Random Prism: a noise-tolerant alternative to Random Forests. Expert Systems, Wiley Online Library (2013)
Stahl, F., Bramer, M.A.: Random Prism: an alternative to Random Forests. In: Research and Development in Intelligent Systems XXVIII, pp. 5–18. Springer (2011)
Bramer, M.A.: Principles of Data Mining. Springer, London (2007)
Stahl, F., Bramer, M.A.: Jmax-pruning: A facility for the information theoretic pruning of modular classification rules. Knowledge-Based Systems 29, 12–19 (2012)
Stahl, F., Bramer, M.A.: Induction of modular classification rules: using Jmax-pruning. In: Thirtieth SGAI International Conference on Innovative Techniques and Applications of Artificial Intelligence, Cambridge, December 14-16 (2011)
Bramer, M.A.: Inducer: a public domain workbench for data mining. International Journal of Systems Science 36(14), 909–919 (2005)
Liu, H., Gegov, A., Stahl, F.: J-measure based hybrid pruning for complexity reduction in classification rules. WSEAS Transaction on Systems 12(9), 433–446 (2013)
Liu, H., Gegov, A., Stahl, F.: Unified Framework for Construction of Rule Based Classification Systems. In: Pedrycz, W., Chen, S.M. (eds.) Information Granularity, Big Data and Computational Intelligence. SBD, vol. 8, pp. 209–230. Springer, Heidelberg (2015)
Shannon, C.: A mathematical theory of communication. Bell System Technical Journal 27(3), 379–423 (1948)
Deng, X.: A Covering-based Algorithm for Classification: PRISM. CS831: Knowledge Discover in Databases (2012)
Breiman, L.: Random Forests. Machine Learning 45(1), 5–32 (2001)
Bache, K., Lichman, M.: UCI Machine Learning Repository. University of California, School of Information and Computer Science, Irvine (2013), http://archive.ics.uci.edu/ml
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The WEKA Data Mining Software: An Update; SIGKDD Explorations 11(1) (2009)
Machine Learning Open Source Software, http://jmlr.org/mloss/ (accessed on June 15, 2014)
Gegov, A.: Fuzzy Networks for Complex Systems: A Modular Rule Base Approach. Springer, Berlin (2010)
Gegov, A., Petrov, N., Vatchova, B.: Advanced Modelling of Complex Processes by Rule Based Networks. In: 5th IEEE International Conference on Intelligent Systems, London, July 7-9, pp. 197–202 (2010)
Gegov, A., Petrov, N., Vatchova, B., Sanders, D.: Advanced Modelling of Complex Processes by Fuzzy Networks. WSEAS Transactions on Circuits and Systems 10(10), 319–330 (2011)
Gegov, A.: Rule Based Networks: Theory and Applications. Plenary Lecture in IEEE International Conference on Intelligent Systems, London, UK (2010)
Gegov, A.: Rule Based Network Models for Complex Systems. Tutorial in ENNS International Conference on Neural Networks, Sofia, Bulgaria (2013)
Wooldridge, M.: An Introduction to Multi-Agent Systems. John Wiley & Sons (2002)
Gegov, A.: Complexity Management in Fuzzy Systems. Springer, Berlin (2007)
Schlager, J.: Systems engineering: key to modern development. IRE Transactions EM-3(3), 64–66 (1956)
Sage, A.P.: Systems Engineering. Wiley IEEE (1992)
Hall, A.D.: A Methodology for Systems Engineering. Van Nostrand Reinhold (1962)
Goode, H.H., Machol, R.E.: System Engineering: An Introduction to the Design of Large-scale Systems. McGraw-Hill (1957)
Gaber, M.M., Stahl, F., Gomes, J.B.: Pocket Data Mining. SBD, vol. 2. Springer, Heidelberg (2014)
Stahl, F., Jordanov, I.: An Overview of Use of Neural Networks for Data Mining Tasks. WIREs: Data Mining and Knowledge Discovery 3(2), 193–208 (2012)
Press, W.H., Teukolsky, S.A., Vetterling, W.T., Flannery, B.P.: Section 16.5. Support Vector Machines. In: Numerical Recipes: The Art of Scientific Computing, 3rd edn. Cambridge University Press, New York (2007)
Kerber, R.: ChiMerge: Discretization of Numeric Attributes. In: Proceedings of the 10th National Conference on Artificial Intelligence, pp. 123–128. AAAI Press (1992)
Houle, M.E., Kriegel, H.-P., Kröger, P., Schubert, E., Zimek, A.: Can Shared-Neighbor Distances Defeat the Curse of Dimensionality? In: Gertz, M., Ludäscher, B. (eds.) SSDBM 2010. LNCS, vol. 6187, pp. 482–500. Springer, Heidelberg (2010)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Liu, H., Gegov, A. (2015). Collaborative Decision Making by Ensemble Rule Based Classification Systems. In: Pedrycz, W., Chen, SM. (eds) Granular Computing and Decision-Making. Studies in Big Data, vol 10. Springer, Cham. https://doi.org/10.1007/978-3-319-16829-6_10
Download citation
DOI: https://doi.org/10.1007/978-3-319-16829-6_10
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-16828-9
Online ISBN: 978-3-319-16829-6
eBook Packages: EngineeringEngineering (R0)