Abstract
Fuzzy rule-based systems have shown a high capability of knowledge extraction and representation when modeling complex, non-linear classification problems. However, they suffer from the so-called curse of dimensionality when applied to high dimensional datasets, which consist of a large number of variables and/or examples. Multiclassification systems have shown to be a good approach to deal with this kind of problems. In this contribution, we propose an multiclassification system-based global framework allowing fuzzy rule-based systems to deal with high dimensional datasets avoiding the curse of dimensionality. Having this goal in mind, the proposed framework will incorporate several multiclassification system methodologies as well as evolutionary algorithms to design fuzzy rule-based multiclassification systems. The proposed framework follows a two-stage structure: 1) fuzzy rule-based multiclassification system design from classical and advanced multiclassification system design approaches, and 2) novel designs of evolutionary component classifier combination. By using our methodology, different fuzzy rule-based multiclassification systems can be designed dealing with several aspects such as improvement of the performance in terms of accuracy, and obtaining a good accuracy-complexity trade-off.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Kuncheva, L.I.: Combining Pattern Classifiers: Methods and Algorithms. Wiley (2004)
Ho, T.: The random subspace method for constructing decision forests. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(8), 832–844 (1998)
Optiz, D., Maclin, R.: Popular ensemble methods: An empirical study. Journal of Artificial Intelligence Research 11, 169–198 (1999)
Canul-Reich, J., Shoemaker, L., Hall, L.O.: Ensembles of fuzzy classifiers. In: IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), London, pp. 1–6 (2007)
Pedrycz, W., Kwak, K.C.: Boosting of granular models. Fuzzy Sets and Systems 157(22), 2934–2953 (2006)
Cordón, O., Quirin, A.: Comparing two genetic overproduce-and-choose strategies for fuzzy rule-based multiclassification systems generated by bagging and mutual information-based feature selection. International Journal of Hybrid Intelligent Systems 7(1), 45–64 (2010)
Ishibuchi, H., Nakashima, T., Nii, M.: Classification and Modeling With Linguistic Information Granules. Springer (2005)
Casillas, J., Cordon, O., Herrera, F., Magdalena, L.: Interpretability Issues in Fuzzy Modeling. Springer, Heidelberg (2003)
Alonso, J.M., Magdalena, L., González-Rodríguez, G.: Looking for a good fuzzy system interpretability index: An experimental approach. International Journal of Approximate Reasoning 51, 115–134 (2009)
Dasarathy, B.V., Sheela, B.V.: A composite classifier system design: Concepts and methodology. Proceedings of IEEE 67(5), 708–713 (1979)
Breiman, L.: Bagging predictors. Machine Learning 24(2), 123–140 (1996)
Schapire, R.: The strength of weak learnability. Machine Learning 5(2), 197–227 (1990)
Tsymbal, A., Pechenizkiy, M., Cunningham, P.: Diversity in search strategies for ensemble feature selection. Information Fusion 6(1), 83–98 (2005)
Breiman, L.: Random forests. Machine Learning 45(1), 5–32 (2001)
Xu, L., Krzyzak, A., Suen, C.Y.: Methods of combining multiple classifiers and their application to handwriting recognition. IEEE Transactions on Systems, Man, and Cybernetics 22(3), 418–435 (1992)
Woods, K., Kegelmeyer, W.P., Bowyer, K.: Combination of multiple classifiers using local accuracy estimates. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(4), 405–410 (1997)
Giacinto, G., Roli, F.: Dynamic classifier selection based on multiple classifier behaviour. Pattern Recognition 34(9), 1879–1881 (2001)
Partridge, D., Yates, W.B.: Engineering multiversion neural-net systems. Neural Computation 8(4), 869–893 (1996)
Hernández-Lobato, D., Martínez-Muñoz, G., Suárez, A.: How large should ensembles of classi ers be? Pattern Recognition 46(5), 1323–1336 (2013)
Hühn, J.C., Hüllermeier, E.: FURIA: an algorithm for unordered fuzzy rule induction. Data Mining and Knowledge Discovery 19(3), 293–319 (2009)
Cohen, W.W.: Fast effective rule induction. In: Proceedings of the Twelfth International Conference on Machine Learning, pp. 115–123. Morgan Kaufmann (1995)
Ishibuchi, H., Nakashima, T., Morisawa, T.: Voting in fuzzy rule-based systems for pattern classification problems. Fuzzy Sets and Systems 103(2), 223–238 (1999)
Cordón, O., del Jesus, M.J., Herrera, F.: A proposal on reasoning methods in fuzzy rule-based classification systems. International Journal of Approximate Reasoning 20, 21–45 (1999)
Takahashi, H., Honda, H.: Lymphoma prognostication from expression profiling using a combination method of boosting and projective adaptive resonance theory. Journal of Chemical Engineering of Japan 39(7), 767–771 (2006)
Bonissone, P.P., Cadenas, J.M., Garrido, M.C., Díaz-Valladares, R.A.: A fuzzy random forest. International Journal of Approximate Reasoning 51(7), 729–747 (2010)
Marsala, C.: Data mining with ensembles of fuzzy decision trees. In: IEEE Symposium on Computational Intelligence and Data Mining, Nashville, USA, pp. 348–354 (2009)
Janikow, C.Z.: Fuzzy decision trees: issues and methods. IEEE Transactions on Systems, Man, and Cybernetics, Part B 28(1), 1–14 (1998)
Aguilera, J.J., Chica, M., del Jesus, M.J., Herrera, F.: Niching genetic feature selection algorithms applied to the design of fuzzy rule based classification systems. In: IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), London, pp. 1794–1799 (2007)
Goldberg, D.E.: Genetic Algorithms in Search Optimization and Machine Learning. Addison-Wesley (1989)
Nojima, Y., Ishibuchi, H.: Designing fuzzy ensemble classifiers by evolutionary multiobjective optimization with an entropy-based diversity criterion. In: International Conference on Hybrid Intelligent Systems and Conference on Neuro-Computing and Evolving Intelligence, CD-ROM, 4 pages (2006)
Nojima, Y., Ishibuchi, H.: Genetic rule selection with a multi-classifier coding scheme for ensemble classifier design. International Journal of Hybrid Intelligent Systems 4(3), 157–169 (2007)
Ishibuchi, H., Nojima, Y.: Evolutionary multiobjective optimization for the design of fuzzy rule-based ensemble classifiers. International Journal of Hybrid Intelligent Systems 3(3), 129–145 (2006)
Yager, R.R., Filev, D.P.: Essentials of fuzzy modeling and control. Wiley-Interscience, New York (1994)
Cordón, O., Herrera, F., Hoffmann, F., Magdalena, L.: Genetic Fuzzy Systems. Evolutionary Tuning and Learning of Fuzzy Knowledge Bases. World Scientific (2001)
Cordón, O., Gomide, F., Herrera, F., Hoffmann, F., Magdalena, L.: Ten years of genetic fuzzy systems: Current framework and new trends. Fuzzy Sets and Systems 141(1), 5–31 (2004)
Herrera, F.: Genetic fuzzy systems: taxonomy, current research trends and prospects. Evolutionary Intelligence 1, 27–46 (2008)
Cordón, O.: A historical review of evolutionary learning methods for mamdani-type fuzzy rule-based systems: Designing interpretable genetic fuzzy systems. International Journal of Approximate Reasoning 52(6), 894–913 (2011)
Kuncheva, L.I., Rodríguez, J.J.: Classifier ensembles with a random linear oracle. IEEE Transactions on Knowledge and Data Engineering 19(4), 500–508 (2007)
Rodríguez, J.J., Kuncheva, L.I.: Naïve bayes ensembles with a random oracle. In: Haindl, M., Kittler, J., Roli, F. (eds.) MCS 2007. LNCS, vol. 4472, pp. 450–458. Springer, Heidelberg (2007)
Sharkey, A.J.C., Sharkey, N.E.: The test and select approach to ensemble combination. In: International Workshop on Multiclassifier Systems, Cagliari, pp. 30–44 (2000)
Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation 6, 182–197 (2002)
Panov, P., Džeroski, S.: Combining bagging and random subspaces to create better ensembles. In: Berthold, M., Shawe-Taylor, J., Lavrač, N. (eds.) IDA 2007. LNCS, vol. 4723, pp. 118–129. Springer, Heidelberg (2007)
Stefanowski, J.: An experimental study of methods combining multiple classifiers - diversified both by feature selection and bootstrap sampling. In: Atanassov, K.T., Kacprzyk, J., Krawczak, M., Szmidt, E. (eds.) Issues in the Representation and Processing of Uncertain and Imprecise Information, pp. 337–354. Akademicka Oficyna Wydawnicza EXIT, Warsaw (2005)
Trawiński, K., Cordón, O., Quirin, A.: On designing fuzzy rule-based multiclassification systems by combining furia with bagging and feature selection. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 19(4), 589–633 (2011)
Battiti, R.: Using mutual information for selecting features in supervised neural net learning. IEEE Transactions on Neural Networks 5(4), 537–550 (1994)
Feo, T.A., Resende, M.G.C.: Greedy randomized adaptive search procedures. Journal of Global Optimization 6, 109–133 (1995)
Shannon, C.E., Weaver, W.: The Mathematical Theory of Communication. University of Illlinois Press (1949)
Blake, C.L., Merz, C.J.: UCI repository of machine learning databases (1998), http://archive.ics.uci.edu/ml
Dietterich, T.G.: Approximate statistical test for comparing supervised classification learning algorithms. Neural Computation 10(7), 1895–1923 (1998)
Quinlan, J.R.: C4.5: programs for machine learning. Morgan Kaufmann Publishers Inc., San Francisco (1993)
Dietterich, T.G.: An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization. Machine Learning 40(2), 139–157 (2000)
Trawiński, K., Cordón, O., Sánchez, L., Quirin, A.: Multiobjective genetic classifier selection for random oracles fuzzy rule-based multiclassifiers: How benefical is the additional diversity? Technical Report AFE 2012-17, European Centre for Soft Computing, Mieres, Spain (2012)
Dos Santos, E.M., Sabourin, R., Maupin, P.: A dynamic overproduce-and-choose strategy for the selection of classifier ensembles. Pattern Recognition 41(10), 2993–3009 (2008)
Kuncheva, L.I., Whitaker, C.J.: Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Machine Learning 51(2), 181–207 (2003)
Domingos, P., Pazzani, M.J.: On the optimality of the simple bayesian classifier under zero-one loss. Machine Learning 29(2-3), 103–130 (1997)
Alcalá-Fdez, J., Fernández, A., Luengo, J., Derrac, J., García, S.: Keel data-mining software tool: Data set repository, integration of algorithms and experimental analysis framework. Journal of Multiple-Valued Logic and Soft Computing 17(2-3), 255–287 (2011)
Margineantu, D.D., Dietterich, T.G.: Pruning adaptive boosting. In: Proceedings of the Fourteenth International Conference on Machine Learning, ICML 19897, pp. 211–218. Morgan Kaufmann Publishers Inc., San Francisco (1997)
Trawiński, K., Quirin, A., Cordón, O.: A study on the use of multi-objective genetic algorithms for classifier selection in furia-based fuzzy multiclassifers. International Journal of Computational Intelligence Systems 5(2), 231–253 (2012)
Coello, C.A., Lamont, G.B., van Veldhuizen, D.A.: Evolutionary Algorithms for Solving Multi-Objective Problems, 2nd edn. Springer (2007)
Zitzler, E., Thiele, L.: Multiobjective evolutionary algorithms: a comparative case study and the strength pareto approach. IEEE Transactions on Evolutionary Computation 3, 257–271 (1999)
Trawiński, K., Cordón, O., Sánchez, L., Quirin, A.: A genetic fuzzy linguistic combination method for fuzzy rule-based multiclassifiers. IEEE Transactions on Fuzzy Systems (in press, 2013), doi:10.1109/TFUZZ.2012.2236844.
Torra, V.: A review of the construction of hierarchical fuzzy systems. International Journal of Intelligent Systems 17(5), 531–543 (2002)
Gegov, A.E., Frank, P.M.: Hierarchical fuzzy control of multivariable systems. Fuzzy Sets and Systems 72, 299–310 (1995)
Yager, R.R.: On the construction of hierarchical fuzzy systems model. IEEE Transactions on Systems, Man, and Cybernetics - Part B 28(1), 55–66 (1998)
Cordón, O., Herrera, F., Zwir, I.: A hierarchical knowledge-based environment for linguistic modeling: Models and iterative methodology. Fuzzy Sets and Systems 138(2), 307–341 (2003)
Wolpert, D.: Stacked generalization. Neural Networks 5(2), 241–259 (1992)
Dimililer, N., Varoglu, E., Altincay, H.: Classifier subset selection for biomedical named entity recognition. Applied Intelligence 31, 267–282 (2009)
Kuncheva, L.I., Bezdek, J.C., Duin, R.P.W.: Decision templates for multiple classifier fusion: An experimental comparison. Pattern Recognition 34(2), 299–314 (2001)
Kuncheva, L.I.: “Fuzzy” versus “nonfuzzy” in combining classifiers designed by boosting. IEEE Transactions on Fuzzy Systems 11(6), 729–741 (2003)
Ruta, D., Gabrys, B.: Classifier selection for majority voting. Information Fusion 6(1), 63–81 (2005)
Trawiński, K., Alonso, J.M., Hernández, N.: A multiclassifier approach for topology-based wifi indoor localization. Soft Computing (in press, 2013)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Cordón, O., Trawiński, K. (2013). A Novel Framework to Design Fuzzy Rule-Based Ensembles Using Diversity Induction and Evolutionary Algorithms-Based Classifier Selection and Fusion. In: Rojas, I., Joya, G., Gabestany, J. (eds) Advances in Computational Intelligence. IWANN 2013. Lecture Notes in Computer Science, vol 7902. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-38679-4_3
Download citation
DOI: https://doi.org/10.1007/978-3-642-38679-4_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-38678-7
Online ISBN: 978-3-642-38679-4
eBook Packages: Computer ScienceComputer Science (R0)