Abstract
In the paper a new method is proposed which improves the classification accuracy of decision trees for samples with missing values. This aim was achieved by adding new nodes to the decision tree. The proposed procedure applies structures and functions of well-known C4.5 algorithm. However, it can be easily adapted to other methods, for forming decision trees. The efficiency of the new algorithm has been confirmed by tests using eleven databases from UCI Repository. The research has been concerned classification but the method is not limited to classification tasks.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
References
Bartczuk, Ł., Rutkowska, D.: Type-2 fuzzy decision trees. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2008. LNCS (LNAI), vol. 5097, pp. 197–206. Springer, Heidelberg (2008)
Bonissone, P., Cadenas, J.M., Carmen Garrido, M., Andrés Díaz-Valladares, R.: A fuzzy random forest (2010)
Breiman, L., Friedman, J., Olshen, R., Stone, C.: Classification and Regression Trees. Wadsworth Int. Group (1984)
Brodley, C.E., Utgoff, P.E.: Multivariate decision trees (1995)
Cpalka, K.: A method for designing flexible neuro-fuzzy systems. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Żurada, J.M. (eds.) ICAISC 2006. LNCS (LNAI), vol. 4029, pp. 212–219. Springer, Heidelberg (2006)
Cpałka, K.: On evolutionary designing and learning of flexible neuro-fuzzy structures for nonlinear classification. Nonlinear Analysis: Theory, Methods & Applications 71(12), 1659–1672 (2009)
Gabryel, M., Scherer, R.: Determining fuzzy relation by evolutionary learning in neuro-fuzzy systems. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Zurada, J. (eds.) Computational Intelligence: Methods and Applications, pp. 176–182. Academic Publishing House EXIT (2008)
Greblicki, W., Rutkowski, L.: Density-free bayes risk consistency of nonparametric pattern recognition procedures. Proceedings of the IEEE 69(4), 482–483 (1981)
Hashemi, S., Yang, Y.: Flexible decision tree for data stream classification in the presence of concept change, noise and missing values. Data Mining and Knowledge Discovery 19, 95–131 (2009)
Haykin, S., Network, N.: A comprehensive foundation. Neural Networks 2 (2004)
Horzyk, A., Tadeusiewicz, R.: Self-optimizing neural networks. In: Yin, F.-L., Wang, J., Guo, C. (eds.) ISNN 2004. LNCS, vol. 3173, pp. 150–155. Springer, Heidelberg (2004)
Korytkowski, M., Rutkowski, L., Scherer, R.: From ensemble of fuzzy classifiers to single fuzzy rule base classifier. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2008. LNCS (LNAI), vol. 5097, pp. 265–272. Springer, Heidelberg (2008)
Korytkowski, M., Scherer, R., Rutkowski, L.: On combining backpropagation with boosting. In: 2006 International Joint Conference on Neural Networks, IEEE World Congress on Computational Intelligence, Vancouver, BC, Canada, pp. 1274–1277 (2006)
Mertz, C.J., Murphy, P.M.: UCI machine learning repository, http://archive.ics.uci.edu/ml/datasets.html
Nowicki, R.: On combining neuro–fuzzy architectures with the rough set theory to solve classification problems with incomplete data. IEEE Trans. on Knowledge and Data Engineering 20(9), 1239–1253 (2008)
Nowicki, R.: Rough–neuro–fuzzy structures for classification with missing data. IEEE Trans. on Systems, Man, and Cybernetics—Part B: Cybernetics 39(6), 1334–1347 (2009)
Pawlak, Z.: Rough sets. International Journal of Computer and Information Sciences 11(5), 341–356 (1982)
Przybył, A., Cpałka, K.: A new method to construct of interpretable models of dynamic systems. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2012, Part II. LNCS, vol. 7268, pp. 697–705. Springer, Heidelberg (2012)
Qian, Y., Dang, C., Liang, J., Zhang, H., Ma, J.: On the evaluation of the decision performance of an incomplete decision table. Data & Knowledge Engineering 65(3), 373–400 (2008)
Quinlan, J.: C4.5: Programs for Machine Learning. Morgan Kaufmann (1993)
Quinlan, J.R.: Improved use of continuous attributes in c4.5. Journal of Artificial Intelligence Research 4, 77–90 (1996)
Rutkowska, D., Nowicki, R.: Implication-based neuro–fuzzy architectures. International Journal of Applied Mathematics and Computer Science 10(4), 675–701 (2000)
Rutkowska, D., Rutkowski, L., Nowicki, R.: On processing of noisy data by fuzzy inference neural networks. In: Proceedings of the IASTED International Conference, Signal and Image Processing, Nassau, Bahamas, pp. 314–318 (October 1999)
Rutkowski, L.: Sequential estimates of probability densities by orthogonal series and their application in pattern classification. IEEE Transactions on Systems, Man and Cybernetics SMC-10(12), 918–920 (1980)
Rutkowski, L.: Adaptive probabilistic neural networks for pattern classification in time-varying environment. IEEE Transactions on Neural Networks 15(4), 811–827 (2004)
Rutkowski, L.: Generalized regression neural networks in time-varying environment. IEEE Transactions on Neural Networks 15(3), 576–596 (2004)
Rutkowski, L., Pietruczuk, L., Duda, P., Jaworski, M.: Decision trees for mining data streams based on the McDiarmid’s bound. IEEE Transactions on Knowledge and Data Engineering 25 (2013)
Scherer, R.: Boosting ensemble of relational neuro-fuzzy systems. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Żurada, J.M. (eds.) ICAISC 2006. LNCS (LNAI), vol. 4029, pp. 306–313. Springer, Heidelberg (2006)
Scherer, R., Korytkowski, M., Nowicki, R., Rutkowski, L.: Modular rough neuro-fuzzy systems for classification. In: Wyrzykowski, R., Dongarra, J., Karczewski, K., Wasniewski, J. (eds.) PPAM 2007. LNCS, vol. 4967, pp. 540–548. Springer, Heidelberg (2008)
Scherer, R., Rutkowski, L.: A fuzzy relational system with linguistic antecedent certainty factors. In: Rutkowski, L., Kacprzyk, J. (eds.) Proceedings of the Sixth International Conference on Neural Network and Soft Computing. Advances in Soft Computing, pp. 563–569. Springer, Heidelberg (2003)
Scherer, R., Rutkowski, L.: Neuro-fuzzy relational classifiers. In: Rutkowski, L., Siekmann, J.H., Tadeusiewicz, R., Zadeh, L.A. (eds.) ICAISC 2004. LNCS (LNAI), vol. 3070, pp. 376–380. Springer, Heidelberg (2004)
Scherer, R., Rutkowski, L.: Connectionist fuzzy relational systems. In: Hagamuge, S.K., Wang, L. (eds.) Computational Intelligence for Modelling and Prediction. SCI, vol. 2, pp. 35–47. Springer, Heidelberg (2005)
Starczewski, J.T.: On defuzzification of interval type-2 fuzzy sets. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2008. LNCS (LNAI), vol. 5097, pp. 333–340. Springer, Heidelberg (2008)
Starczewski, J.T.: A type-1 approximation of interval type-2 FLS. In: Di Gesù, V., Pal, S.K., Petrosino, A. (eds.) WILF 2009. LNCS, vol. 5571, pp. 287–294. Springer, Heidelberg (2009)
Starczewski, J.T.: General type-2 fls with uncertainty generated by fuzzy rough sets. In: FUZZ-IEEE, pp. 1–6 (2010)
Zhang, S.: Decision tree classifiers sensitive to heterogeneous costs. Journal of Systems and Software 85(4), 771–779 (2012)
Zhang, S., Qin, Z., Ling, C., Sheng, S.: “Missing is useful”: missing values in cost-sensitive decision trees. IEEE Transactions on Knowledge and Data Engineering 17(12), 1689–1693 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Nowak, B.A., Nowicki, R.K., Mleczko, W.K. (2013). A New Method of Improving Classification Accuracy of Decision Tree in Case of Incomplete Samples. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds) Artificial Intelligence and Soft Computing. ICAISC 2013. Lecture Notes in Computer Science(), vol 7894. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-38658-9_40
Download citation
DOI: https://doi.org/10.1007/978-3-642-38658-9_40
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-38657-2
Online ISBN: 978-3-642-38658-9
eBook Packages: Computer ScienceComputer Science (R0)