Abstract
Support vector machines (SVMs) are primarily designed for 2-class classification problems. Although in several papers it is mentioned that the combination of K SVMs can be used to solve a K-class classification problem, such a procedure requires some care. In this paper, the scaling problem of different SVMs is highlighted. Various normalization methods are proposed to cope with this problem and their efficiencies are measured empirically. This simple way of ssing SVMs to learn a K-class classification problem consists in choosing the maximum applied to the outputs of K SVMs solving a one-per-class decomposition of the general problem. In the second part of this paper, more sophisticated techniques are suggested. On the one hand, a stacking of the K SVMs with other classification techniques is proposed. On the other end, the one-per-class decomposition scheme is replaced by more elaborated schemes based on error-correcting codes. An incremental algorithm for the elaboration of pertinent decomposition schemes is mentioned, which exploits the properties of SVMs for an efficient computation.
Preview
Unable to display preview. Download preview PDF.
References
B. E. Boser, I. M. Guyon, and V. N. Vapnik. A training algorithm for optimal margin classifiers. In Proceedings of the Conference on Learning Theory, COLT'92, pages 144–152, 1992.
L. Breiman, J. Olshen, and C. Stone. Classification and Regression Trees. Wadsworth International Group, 1984.
C. Burges. A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, to appear. available at http://svm.research.bell-labs.com/SVMdoc.html.
C. Cortes and V. Vapnik. Support vector network. Machine Learning, 20:273–297, 1995.
Thomas G. Dietterich and Ghulum Bakiri. Solving multiclass learning problems via error-correcting output codes. Journal of Artificial Intelligence Research, 2:263–286, 1995.
T. G. Dietterich and G. Bakiri. Error-correcting output codes: A general method for improving multiclass inductive learning programs. In Proceedings of AAAI-91, pages 572–577. AAAI Press/MIT Press, 1991.
R. O. Duda and P. E. Hart. Pattern Classification and Scene Analysis. John Wiley & Sons, New York, 1973.
Eddy Mayoraz and Miguel Moreira. On the decomposition of polychotomies into dichotomies. In Douglas H. Fisher, editor, The Fourteenth International Conference on Machine Learning, pages 219–226, 1997.
Ana Merchan and Eddy Mayoraz. Combination of binary classifiers for multi-class classification. IDIAP-Com 02, IDIAP, 1998. paper 22 in the Proceedings of Learning'98, Madrid, September 98, http://learn98.tsc.uc3m.es/~learn98/papers/abstracts.
C. J. Merz and P. M. Murphy. UCI repository of machine learning databases. Machine-readable data repository http://www.ics.uci.edu/~mlearn/mlrepository.html, Irvine, CA: University of California, Department of Information and Computer Science, 1998.
Miguel Moreira and Eddy Mayoraz. Improved pairwise coupling classification with correcting classifiers. IDIAP-RR 9, IDIAP, 1997. To appear in the Proceedings of the European Conference on Machine Learning, ECML'98.
J. R. Quinlan. Induction of decision trees. Machine Learning, 1:81–106, 1986.
B. Schölkopf, C. Burges, and V. Vapnik. Extracting support data for a given task. In U. M. Fayyad and R. Uthurusamy, editors. Proceedings of the First International Conference on Knowledge Discovery and Data Mining, pages 252–257. AAAI Press. 1995
V. N. Vapnik. The Nature of Statistical Learning Theory. Springer, New York, 1995.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1999 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Mayoraz, E., Alpaydin, E. (1999). Support vector machines for multi-class classification. In: Mira, J., Sánchez-Andrés, J.V. (eds) Engineering Applications of Bio-Inspired Artificial Neural Networks. IWANN 1999. Lecture Notes in Computer Science, vol 1607. Springer, Berlin, Heidelberg . https://doi.org/10.1007/BFb0100551
Download citation
DOI: https://doi.org/10.1007/BFb0100551
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-66068-2
Online ISBN: 978-3-540-48772-2
eBook Packages: Springer Book Archive