Abstract
This paper examines the use of bootstrap aggregating (bagging) with classifier learning methods based upon hold-out pruning (or growing) for misclassification cost reduction. Both decision tree and rule set classifiers are used. The paper introduces a “repechange” variation of bagging, that uses, as the hold-out data for cost reduction, the “out of bag” items, which would be unused in standard bagging. The paper presents experimental evidence that, when used with the hold-out cost reduction methods, the repechage, method can achieve better misclassification cost results than the straightforward use of standard bagging used with the same hold-out cost reduction method. Superior results for the repechange method on some problems with previously defined cost matrices are shown for a cost reduction decision tree method and two cost reduction rule set methods.
Preview
Unable to display preview. Download preview PDF.
References
K.M. Ali and M. Pazzani. Error reduction through learning multiple descriptions. Machine Learning, 24:173–202, 1996.
L. Breiman. Bagging predictors. Machine Learning, 24:123–140, 1996.
L. Breiman. Out-of-bag Estimation. available from ftp.stat.berkeley.edu as /users/pub/breiman/OOBestimation.ps, 1996.
L. Breiman, J.H. Friedman, R.A. Olshen, and C.J. Stone. Classification and Regression Trees. Wadsworth, 1984.
C.A. Brunk and M.J. Pazzani. An investigation of noise-tolerant relational concept learning algorithms. In Proceedings of the Eighth International Workshop of Machine Learning, pages 389–393. Morgan Kaufmann, 1991.
R.M. Cameron-Jones. The complexity of batch approaches to reduced error rule set induction. In Proceedings of the Fourth Pacific Rim International Conference on Artificial Intelligence, pages 348–359. Springer Verlag, 1996.
R.M. Cameron-Jones and J.R. Quinlan. Efficient top-down induction of logic programs. SIGART, 5:33–42, 1994.
P. Clark and R. Boswell. Rule induction with CN2: Some recent improvements. In Machine Learning-EWSL91: Proceedings of the Fifth European Working Session on Machine Learning. Springer-Verlag, 1991. Lecture Notes in Artificial Intelligence, 482.
W.W. Cohen. Efficient pruning methods for separate-and-conquer rule learning systems. In Proceedings of the Thirteenth International Joint Conference on Artificial Intelligence, pages 988–994. Morgan Kaufmann, 1993.
W.W. Cohen. Fast effective rule induction. In Proceedings of the Twelfth International Conference on Machine Learning (ML95), pages 115–123. Morgan Kaufmann, 1995.
C.J. Merz and P.M. Murphy. UCI repository of machine learning databases, 1996. http://www.ics.uci.edu/~mlearn/MLRepository.html.
D. Michie, D.J. Spiegelhalter, and C.C. (eds.) Taylor. Machine learning, neural and statistical classification. Ellis Horwood, 1994.
M. Pazzani, C. Merz, P. Murphy, K. Ali, T. Hume, and C. Brunk. Reducing misclassification costs. In Proceedings of the Eleventh International Conference on Machine Learning (ML94), pages 217–225. Morgan Kaufmann, 1994.
J.R. Quinlan. Simplifying decision trees. International Journal of Man-Machine Studies, 27:221–234, 1987.
J.R. Quinlan. C4.5: Programs for Machine Learning. Morgan Kaufmann, 1993. The Morgan Kaufmann Series in Machine Learning.
J.R. Quinlan. Bagging, boosting and C4.5. In Proceedings of the Thirteenth American Association for Artificial Intelligence National Conference on Artificial Intelligence, pages 725–730. AAAI Press, 1996.
L.M. Richards. Reduced cost pruning and classifier combination, 1997. Honours thesis, School of Computing, University of Tasmania.
K.M. Ting and Z. Zheng. Boosting trees for cost-sensitive classifications. In Proceedings of the Tenth European Conference on Machine Learning, pages 190–195. Springer-Verlag, 1998. Lecture Notes in Artificial Intelligence, 1398.
P.D. Turney. Cost-sensitive classification: Empirical evaluation of a hybrid genetic decision tree induction algorithm. Journal of Artificial Intelligence Research, 2:369–409, 1995.
G.I. Webb. Cost-sensitive specialization. In Proceedings of the Fourth Pacific Rim International Conference on Artificial Intelligence, pages 23–34. Springer Verlag, 1996.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1998 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Cameron-Jones, M., Richards, L. (1998). Repechage bootstrap aggregating for misclassification cost reduction. In: Lee, HY., Motoda, H. (eds) PRICAI’98: Topics in Artificial Intelligence. PRICAI 1998. Lecture Notes in Computer Science, vol 1531. Springer, Berlin, Heidelberg . https://doi.org/10.1007/BFb0095253
Download citation
DOI: https://doi.org/10.1007/BFb0095253
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-65271-7
Online ISBN: 978-3-540-49461-4
eBook Packages: Springer Book Archive