Abstract
Cost-sensitive decision tree and cost-sensitive naïve Bayes are both new cost-sensitive learning models proposed recently to minimize the total cost of test and misclassifications. Each of them has its advantages and disadvantages. In this paper, we propose a novel cost-sensitive learning model, a hybrid cost-sensitive decision tree, called DTNB, to reduce the minimum total cost, which integrates the advantages of cost-sensitive decision tree and of the cost-sensitive naïve Bayes together. We empirically evaluate it over various test strategies, and our experiments show that our DTNB outperforms cost-sensitive decision and the cost-sensitive naïve Bayes significantly in minimizing the total cost of tests and misclassification based on the same sequential test strategies, and single batch strategies.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Blake, C.L., Merz, C.J.: UCI Repository of machine learning databases (website). Irvine, CA: University of California, Department of Information and Computer Science (1998)
Chai, X., Deng, L., Yang, Q., Ling, C.X.: Test-Cost Sensitive Naïve Bayesian Classification. In: Proceedings of the Fourth IEEE International Conference on Data Mining, Brighton, UK. IEEE Computer Society Press, Los Alamitos (2004)
Chawla, N.V., Japkowicz, N., Kolcz, A. (eds.): Special Issue on Learning from Imbalanced Datasets. SIGKDD 6(1) (2004)
Domingos, P.: MetaCost: A General Method for Making Classifiers Cost-Sensitive. In: Proceedings of the Fifth International Conference on Knowledge Discovery and Data Mining, San Diego, CA, pp. 155–164. ACM Press, New York (1999)
Elkan, C.: The Foundations of Cost-Sensitive Learning. In: In Proceedings of the Seventeenth International Joint Conference of Artificial Intelligence, Seattle, Washington, pp. 973–978. Morgan Kaufmann, San Francisco (2001)
Fayyad, U.M., Irani, K.B.: Multi-interval discretization of continuous-valued attributes for classification learning. In: Proceedings of the 13th International Joint Conference on Artificial Intelligence, France, pp. 1022–1027. Morgan Kaufmann, San Francisco (1993)
Ting, K.M.: Inducing Cost-Sensitive Trees via Instance Weighting. In: Proceedings of the Second European Symposium on Principles of Data Mining and Knowledge Discovery, pp. 23–26. Springer, Heidelberg (1998)
Kohavi, R.: Scaling up the accuracy of Naïve-Bayes Classifier: a Decision-Tree Hybrid. In: Proceeding of the Second International Conference on Knowledge Discovery and Data Mining (KDD 1996), pp. 202–207. AAAI Press, Menlo Park (1996)
Ling, C.X., Yang, Q., Wang, J., Zhang, S.: Decision Trees with Minimal Costs. In: Proceedings of the Twenty-First International Conference on Machine Learning, Banff, Alberta. Morgan Kaufmann, San Francisco (2004)
Nunez, M.: The use of background knowledge in decision tree induction. Machine learning 6, 231–250 (1991)
Quinlan, J.R. (ed.): C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1993)
Tan, M.: Cost-sensitive learning of classification knowledge and its applications in robotics. Machine Learning Journal 13, 7–33 (1993)
Turney, P.D.: Cost-Sensitive Classification: Empirical Evaluation of a Hybrid Genetic Decision Tree Induction Algorithm. Journal of Artificial Intelligence Research 2, 369–409 (1995)
Turney, P.D.: Types of cost in inductive concept learning. In: Proceedings of the Workshop on Cost-Sensitive Learning at the Seventeenth International Conference on Machine Learning, Stanford University, California (2000)
Zubek, V.B., Dietterich, T.: Pruning improves heuristic search for cost-sensitive learning. In: Proceedings of the Nineteenth International Conference of Machine Learning, Sydney, Australia, pp. 27–35. Morgan Kaufmann, San Francisco (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Sheng, S., Ling, C.X. (2005). Hybrid Cost-Sensitive Decision Tree. In: Jorge, A.M., Torgo, L., Brazdil, P., Camacho, R., Gama, J. (eds) Knowledge Discovery in Databases: PKDD 2005. PKDD 2005. Lecture Notes in Computer Science(), vol 3721. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11564126_29
Download citation
DOI: https://doi.org/10.1007/11564126_29
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29244-9
Online ISBN: 978-3-540-31665-7
eBook Packages: Computer ScienceComputer Science (R0)