Abstract
Maximum margin clustering (MMC) approaches extend the large margin principle of SVM to unsupervised learning with considerable success. In this work, we utilize the ratio between the margin and the intra-cluster variance, to explicitly consider both the separation and the compactness of the clusters in the objective. Moreover, we employ multiple kernel learning (MKL) to jointly learn the kernel and a partitioning of the instances, thus overcoming the kernel selection problem of MMC. Importantly, the margin alone cannot reliably reflect the quality of the learned kernel, as it can be enlarged by a simple scaling of the kernel. In contrast, our ratio-based objective is scale invariant and also invariant to the type of norm constraints on the kernel parameters. Optimization of the objective is performed using an iterative gradient-based algorithm. Comparative clustering experiments on various datasets demonstrate the effectiveness of the proposed formulation.
Chapter PDF
Similar content being viewed by others
References
Arthur, D., Vassilvitskii, S.: k-means++: the advantages of careful seeding. In: ACM-SIAM Symposium on Discrete Algorithms (SODA), pp. 1027–1035 (2007)
Chang, C.C., Lin, C.J.: LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology 2(3), 1–27 (2011), http://www.csie.ntu.edu.tw/~cjlin/libsvm
Cortes, C., Mohri, M., Rostamizadeh, A.: Learning non-linear combinations of kernels. In: Advances in Neural Information Processing Systems (NIPS), pp. 396–404 (2009)
Danskin, J.M.: The theory of max-min, with applications. SIAM Journal on Applied Mathematics 14(4), 641–664 (1966)
Dhillon, I.S., Guan, Y., Kulis, B.: Weighted graph cuts without eigenvectors a multilevel approach. IEEE Transactions on Pattern Analysis and Machine Intelligence 29(11), 1944–1957 (2007)
Duchi, J.C., Shalev-Shwartz, S., Singer, Y., Chandra, T.: Efficient projections onto the l 1-ball for learning in high dimensions. In: International Conference on Machine Learning (ICML), pp. 272–279 (2008)
Gai, K., Chen, G., Zhang, C.: Learning kernels with radiuses of minimum enclosing balls. In: Advances in Neural Information Processing Systems (NIPS), pp. 649–657 (2010)
Gönen, M., Alpaydin, E.: Localized multiple kernel learning. In: International Conference on Machine Learning (ICML), pp. 352–359 (2008)
Gönen, M., Alpaydin, E.: Multiple kernel learning algorithms. Journal of Machine Learning Research 12, 2211–2268 (2011)
Kernighan, B.W., Lin, S.: An efficient heuristic procedure for partitioning graphs. The Bell System Technical Journal 49(2), 291–308 (1970)
Kloft, M., Brefeld, U., Sonnenburg, S., Laskov, P., Müller, K.R., Zien, A.: Efficient and accurate l p-norm multiple kernel learning. In: Advances in Neural Information Processing Systems (NIPS), pp. 997–1005 (2009)
Kloft, M., Brefeld, U., Sonnenburg, S., Zien, A.: lp-norm multiple kernel learning. Journal of Machine Learning Research 12, 953–997 (2011)
Nene, S.A., Nayar, S.K., Murase, H.: Columbia Object Image Library (COIL-20). Tech. Rep. CUCS-005-96, Department of Computer Science, Columbia University (1996), http://www.cs.columbia.edu/CAVE/software/softlib/coil-20.php
Rakotomamonjy, A., Bach, F.R., Canu, S., Grandvalet, Y.: SimpleMKL. Journal of Machine Learning Research 9, 2491–2521 (2008)
Songsiri, J.: Projection onto an l1-norm ball with application to identification of sparse autoregressive models. In: Asean Symposium on Automatic Control, ASAC (2011)
Sonnenburg, S., Rätsch, G., Schäfer, C., Schölkopf, B.: Large scale multiple kernel learning. Journal of Machine Learning Research 7, 1531–1565 (2006)
Tzortzis, G., Likas, A.: Kernel-based weighted multi-view clustering. In: International Conference on Data Mining (ICDM), pp. 675–684 (2012)
Valizadegan, H., Jin, R.: Generalized maximum margin clustering and unsupervised kernel learning. In: Advances in Neural Information Processing Systems (NIPS), pp. 1417–1424 (2006)
Varma, M., Babu, B.R.: More generality in efficient multiple kernel learning. In: International Conference on Machine Learning (ICML), pp. 1065–1072 (2009)
Wang, F., Zhao, B., Zhang, C.: Linear time maximum margin clustering. IEEE Transactions on Neural Networks 21(2), 319–332 (2010)
Xu, L., Neufeld, J., Larson, B., Schuurmans, D.: Maximum margin clustering. In: Advances in Neural Information Processing Systems (NIPS), pp. 1537–1544 (2004)
Xu, L., Schuurmans, D.: Unsupervised and semi-supervised multi-class support vector machines. In: AAAI Conference on Artificial Intelligence (AAAI). pp. 904–910 (2005)
Xu, Z., Jin, R., Yang, H., King, I., Lyu, M.R.: Simple and efficient multiple kernel learning by group lasso. In: International Conference on Machine Learning (ICML), pp. 1175–1182 (2010)
Zhang, K., Tsang, I.W., Kwok, J.T.: Maximum margin clustering made practical. In: International Conference on Machine Learning (ICML), pp. 1119–1126 (2007)
Zhao, B., Kwok, J.T., Zhang, C.: Multiple kernel clustering. In: SIAM International Conference on Data Mining (SDM), pp. 638–649 (2009)
Zien, A., Ong, C.S.: Multiclass multiple kernel learning. In: International Conference on Machine Learning (ICML), pp. 1191–1198 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Tzortzis, G., Likas, A. (2014). Ratio-Based Multiple Kernel Clustering. In: Calders, T., Esposito, F., Hüllermeier, E., Meo, R. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2014. Lecture Notes in Computer Science(), vol 8726. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-44845-8_16
Download citation
DOI: https://doi.org/10.1007/978-3-662-44845-8_16
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-44844-1
Online ISBN: 978-3-662-44845-8
eBook Packages: Computer ScienceComputer Science (R0)