Abstract
This paper consists of two parts, one theoretical, and one experimental. And while its primary focus is the development of a mathematically rigorous, theoretical foundation for the field of supervised learning, including a discussion of what constitutes a “solvable pattern recognition problem”, it will also provide some algorithmic detail for implementing the general classification method derived from the theory, a method based on classifier combination, and will discuss experimental results comparing its performance to other well-known methods on standard benchmark problems from the U.C. Irvine, and Statlog, collections. The practical consequences of this work are consistent with the mathematical predictions. Comparing our experimental results on 24 standard benchmark problems taken from the U.C. Irvine, and Statlog, collections, with those reported in the literature for other well-known methods, our method placed 1st on 19 problems, 2nd on 2 others, 4th on another, and 5th on the remaining 2.
This work uses software copyrighted by K Square Inc
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
L. Breiman, Bagging Predictors, Machine Learning, 24, 1996, pp. 123–140.
D. Chen, Statistical Estimates for Kleinberg’s Method of Stochastic Discrimination, Ph.D. Thesis, SUNY/Buffalo, 1998.
Y. Freund, R. E. Schapire, Experiments with a New Boosting Algorithm, Proceedings of the Thirteenth International Conference on Machine Learning, Bari, Italy, July 3–6, 1996, pp. 148–156.
T. K. Ho, Random Decision Forests, Proc. of the 3rd Int’l Conference on Document Analysis and Recognition, Montreal, Canada, 1995, pp. 278–282.
E. M. Kleinberg, Stochastic Discrimination, Annals of Mathematics and Artificial Intelligence, 1990, pp. 207–239.
E. M. Kleinberg, An Overtraining-Resistant Stochastic Modeling Method for Pattern Recognition, Annals of Statistics, 1996, pp. 2319–2349.
E. M. Kleinberg, On the Algorithmic Implementation of Stochastic Discrimination, IEEE Transactions on Pattern Analysis and Machine Intelligence, to appear.
E. M. Kleinberg, A Note on the Mathematics Underlying Boosting, preprint, to appear.
D. Michie, D. Spiegelhalter, C. C. Taylor, Machine Learning, Neural and Statistical Classification, Ellis Horwood, 1994.
R. Quinlan, C4.5: Programs for Machine Learning, Morgan Kaufmann, Oct 1993.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kleinberg, E.M. (2000). A Mathematically Rigorous Foundation for Supervised Learning. In: Multiple Classifier Systems. MCS 2000. Lecture Notes in Computer Science, vol 1857. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45014-9_6
Download citation
DOI: https://doi.org/10.1007/3-540-45014-9_6
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-67704-8
Online ISBN: 978-3-540-45014-6
eBook Packages: Springer Book Archive