Abstract
Discriminative methods for learning structured output classifiers have been gaining popularity in recent years due to their successful applications in fields like computer vision, natural language processing, etc. Learning of the structured output classifiers leads to solving a convex minimization problem, still hard to solve by standard algorithms in real-life settings. A significant effort has been put to development of specialized solvers among which the Bundle Method for Risk Minimization (BMRM) [1] is one of the most successful. The BMRM is a simplified variant of bundle methods well known in the filed of non-smooth optimization. In this paper, we propose two speed-up improvements of the BMRM: i) using the adaptive prox-term known from the original bundle methods, ii) starting optimization from a non-trivial initial solution. We combine both improvements with the multiple cutting plane model approximation [2]. Experiments on real-life data show consistently faster convergence achieving speedup up to factor of 9.7.
Chapter PDF
Similar content being viewed by others
References
Teo, C., Vishwanathan, S., Smola, A., Quoc, V.: Bundle Methods for Regularized Risk Minimization. Journal of Machine Learning Research 11, 311–365 (2010)
Uřičář, M., Franc, V.: Efficient Algorithm for Regularized Risk Minimization. In: CVWW 2012: Proceedings of the 17th Computer Vision Winter Workshop, pp. 57–64 (February 2012)
Tsochantaridis, I., Joachims, T., Hofmann, T., Altun, Y.: Large Margin Methods for Structured and Interdependent Output Variables. Journal of Machine Learning Research 6, 1453–1484 (2005)
Bertsekas, D.P.: Nonlinear Programming. Athena Scientific, Belmont (1999)
Bordes, A., Bottou, L., Gallinari, P.: SGD-QN: Careful Quasi-Newton Stochastic Gradient Descent. Journal of Machine Learning Research 10, 1737–1754 (2009)
Shwartz, S., Singer, Y., Srebro, N.: Pegasos: Primal Estimated sub-GrAdient SOlver for SVM. In: Proceedings of International Conference on Machine Learning (ICML), pp. 807–814. ACM Press (2007)
Joachims, T., Finley, T., Yu, C.N.: Cutting-Plane Training of Structural SVMs. Machine Learning 77(1), 27–59 (2009)
Lemaréchal, C., Nemirovskii, A., Nesterov, Y.: New variants of bundle methods. Mathematical Programming 69, 111–147 (1995)
Cheney, E., Goldstain, A.: Newton’s method for convex programming and Tchebytcheff approximation. Numerische Mathematick 1, 253–268 (1959)
Lemaréchal, C.: Nonsmooth optimization and descend methods. Technical report, IIASA, Laxenburg, Austria (1978)
Uřičář, M., Franc, V., Hlaváč, V.: Detector of facial landmarks learned by the structured output svm. In: VISAPP, vol. (1), pp. 547–556. SciTe Press (2012)
Sonnenburg, S., Rätsch, G., Henschel, S., Widmer, C., Behr, J., Zien, A., Bona, F.D., Binder, A., Gehl, C., Franc, V.: The shogun machine learning toolbox. J. Mach. Learn. Res. 99, 1799–1802 (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Uřičář, M., Franc, V., Hlaváč, V. (2013). Bundle Methods for Structured Output Learning — Back to the Roots. In: Kämäräinen, JK., Koskela, M. (eds) Image Analysis. SCIA 2013. Lecture Notes in Computer Science, vol 7944. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-38886-6_16
Download citation
DOI: https://doi.org/10.1007/978-3-642-38886-6_16
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-38885-9
Online ISBN: 978-3-642-38886-6
eBook Packages: Computer ScienceComputer Science (R0)