Abstract
Domain adaptation learning (DAL) methods have shown promising results by utilizing labeled samples from the source (or auxiliary) domain(s) to learn a robust classifier for the target domain which has a few or even no labeled samples. However, there exist several key issues which need to be addressed in the state-of-theart DAL methods such as sufficient and effective distribution discrepancy metric learning, effective kernel space learning, and multiple source domains transfer learning, etc. Aiming at the mentioned-above issues, in this paper, we propose a unified kernel learning framework for domain adaptation learning and its effective extension based on multiple kernel learning (MKL) schema, regularized by the proposed new minimum distribution distance metric criterion which minimizes both the distribution mean discrepancy and the distribution scatter discrepancy between source and target domains, into which many existing kernel methods (like support vector machine (SVM), v-SVM, and least-square SVM) can be readily incorporated. Our framework, referred to as kernel learning for domain adaptation learning (KLDAL), simultaneously learns an optimal kernel space and a robust classifier by minimizing both the structural risk functional and the distribution discrepancy between different domains. Moreover, we extend the framework KLDAL to multiple kernel learning framework referred to as MKLDAL. Under the KLDAL or MKLDAL framework, we also propose three effective formulations called KLDAL-SVM or MKLDAL-SVM with respect to SVM and its variant µ-KLDALSVM or µ-MKLDALSVM with respect to v-SVM, and KLDAL-LSSVM or MKLDAL-LSSVM with respect to the least-square SVM, respectively. Comprehensive experiments on real-world data sets verify the outperformed or comparable effectiveness of the proposed frameworks.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Quanz B, Huan J. Large margin transductive transfer learning. In: Proceedings of the 18th ACM Conference on Information and Knowledge Management (CIKM). New York: ACM, 2009. 1327–1336
Pan S J, Tsang I W, Kwok J T, et al. Domain adaptation via transfer component analysis. IEEE Trans Neural Netw, 2011, 22: 199–210
Xiang E W, Cao B, Hu D H, et al. Bridging domains using world wide knowledge for transfer learning. IEEE Trans Knowl Data Eng, 2010, 22: 770–783
Bruzzone L, Marconcini M. Domain adaptation problems: A DASVM classification technique and a circular validation strategy. IEEE Trans Pattern Anal Mach Intell, 2010, 32: 770–787
Blitzer J, Crammer K, Kulesza A, et al. Learning bounds for domain adaptation. In: Proceedings of The Neural Information Processing Systems (NIPS). Cambridge: MIT Press, 2007. 129–136
Gao J, Fan W, Jiang J, et al. Knowledge transfer via multiple model local structure mapping. In: Proceedings of the 14th ACM SIGKDD conference on Knowledge Discovery and Data Mining. New York: ACM, 2008
Ling X, Dai W, Xue G, et al. Spectral domain transfer learning. In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2008
Pan S J, Yang Q. A survey on transfer learning. IEEE Trans Knowl Data Eng, 2010, 22: 1345–1359
Blitzer J, McDonald R, Pereira F. Domain adaptation with structural correspondence learning. In: Proceedings of 2006 Conference Empirical Methods Natural Lang, Sydney, 2006. 120–128
Blitzer J, Dredze M, Pereira F. Boom-Boxes and Blenders: domain adaptation for sentiment classification. In: Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics (ACL’ 07), Pereira, 2007. 440–447
Mansour Y, Mohri M, Rostamizadeh A. Domain adaptation: learning bounds and algorithms. In: The 22nd Annual Conference on Learning Theory (COLT 2009), Montreal, 2009
Daume’ III H. Frustratingly easy domain adaptation. In: Proceedings of Annual Meeting Association for Computational Linguistics, Prague, 2007. 256–263
Duan L X, Tsang I W, Xu D, et al. Domain transfer SVM for video concept detection. In: Proc IEEE Int’l Conf Computer Vision and Pattern Recognition, Miami, 2009. 1375–1381
Gretton A, Harchaoui Z, Fukumizu K, et al. A fast, consistent kernel two-sample test. In: Lafferty J, Williams C K I, Shawe-Taylor J, et al, eds. Advances in Neural Information Processing Systems, Vancouver, 2010. 673–681
Huang J, Smola A, Gretton A, et al. Correcting sample selection bias by unlabeled data. In: Proceedings of Twentieth Annual Conference on Neural Information Processing Systems, Vancouver, 2006
Duan L X, Xu D, Tsang I W. Domain adaptation from multiple sources: a domain-dependent regularization approach. IEEE Trans Neural Netw Learn Syst, 2012, 23: 504–518
Duan L X, Tsang I W, Xu D, et al. Domain adaptation from multiple sources via auxiliary classifiers. In: Proceedings of the 26th International Conference on Machine Learning (ICML 2009), Montreal, 2009
Duan L X, Tsang I W, Xu D. Domain transfer multiple kernel learning. IEEE Trans Pattern Anal Mach Intell, 2012, 34: 465–479
Bach F R, lanckriet G R G, Jordan M. Multiple kernel learning, conic duality, and the SMO algorithm. In: Proc Int’l Conf Machine Learning. Banff: IEEE Press, 2004
Rakotomamonjy A, Bach F R, Canu S, et al. SimpleMKL. Mach Learn Res, 2008, 9: 2491–2521
Sonnenburg S, Ra tsch G, Scha fer C, et al. Large scale multiple kernel learning. J Mach Learn Res, 2006, 7: 1531–1565
Hu M Q, Chen Y Q, Kwok J T Y. Building sparse multiple-kernel SVM classifiers. IEEE Trans Neural Netw, 2009, 20
Rosenstein M T, Marx Z, Kaelbling L P. To transfer or not to transfer. In: Advances in Neural Information Processing Systems. Cambridge: MIT Press, 2005
Seah C W, Tsang I W, Ong Y S, et al. Predictive distribution matching SVM for multi-domain learning. In: Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD 2010), Barcelona, 2010
Luo P, Zhuang F, Xiong H, et al. Transfer learning from multiple source domains via consensus regularization. In: Proc. ACM Conf. Inf. Knowledge. Management, Napa Valley, 2008. 103–112
Mansour Y, Mohri M, Rostamizadeh A. Domain adaptation with multiple sources. In: Advances in Neural Information Processing Systems 21. Cambridge: MIT Press, 2009. 1041–1048
Crammer K, Kearns M, Wortman J. Learning from multiple sources. J Mach Learn Res, 2008, 9: 1757–1774
Scholkopf B, Herbrich R, Smola A J. A generalized representer theorem. In: Proc. COLT’2001. Amsterdam: Springer Press, 2001. 416–426
Vapnik V. Statistical Learning Theory. New York: John Wiley and Sons, 1998
Schölkopf B, Smola A J, Williamson R, et al. New support vector algorithms. Neural Comput, 2000, 12: 1207–1245
Joachims T. Transductive inference for text classification using support vector machines. In: Bratko I, Dzeroski S, eds. Proceedings of ICML-99, 16th International Conference on Machine Learning. San Francisco: Morgan Kaufmann Publishers, 1999. 200–209
Suykens J A K, Vandewalle J. Least squares support vector machine classifiers. Neural Process Lett, 1999, 9: 293–300
Jiang W, Zavesky E, Chang S F, et al. Cross-domain learning methods for high-level visual concept classification. In: Proc IEEE Int’l Conf Image Processing, San Diego, 2008. 161–164
Yang J, Yan R, Hauptmann A G. Cross-domain video concept detection using adaptive SVMs. In: Proc ACM Int’l Conf Multimedia, Augsburg, 2007. 188–197
Ben-David S, Blitzer J, Crammer K, et al. Analysis of representations for domain adaptation. In: The Neural Information Processing Systems, Cambridge: MIT Press, 2007
Ben-David S, Luu T, Lu T, et al. Impossibility theorems for domain adaptation. J Mach Learn Res, 2010, 9: 129–136
Ben-David S, Blitzer J, Crammer K, et al. A theory of learning from different domains. Mach Learn, 2010, 79: 151–175
Zhu X. Semi-supervised learning literature survey. Madison: Department of Computer Science, University of Wisconsin. Technical Report. 2008
Hofmann T, Schölkopf B, Smola A J. Kernel methods in machine learning. Annal Stat, 2007, 36: 1171–1220
Sriperumbudur B K, Gretton A, Fukumizu K, et al. Hilbert space embeddings and metrics on probability measures. J Mach Learn Res, 2010, 11: 1517–1561
Sriperumbadur B K, Fukumizu K, Gretton A, et al. Kernel choice and classifiability for RKHS embeddings of probability distributions. In: Advances in Neural Information Processing Systems 21. Cambridge: MIT Press, 2010. 1750–1758
Wu Y, Liu Y. Robust truncated hinge loss support vector machines. J Am Stat Assoc, 2007, 102: 974–983
Tao J W, Wang S T. Locality-preserved maximum information variance v-support vector machine. Acta Autom Sin, 2012, 38: 97–108
Szedmak S, Shawe-Taylor J. Muticlass learning at one-class complexity. Southampton: School of Electronics and Computer Science. Technical Report No: 1508. 2005
Chang C C, Lin C J. Training v-support vector classifiers: Theory and algorithms. Neural Comput, 2001, 13: 2119–2147
Dai Q Y W, Xue G R, Yu Y. Co-clustering based classification for out-of-domain documents. In: Proceedings of the Thirteenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Jose, 2007. 210–219
Cai D, He X F, Han J W. Orthogonal Laplacianfaces for face recognition. IEEE Trans Image Process, 2006, 15: 3608–3614
Schweikert G, Widmer C, Schölkopf B, et al. An empirical analysis of domain adaptation algorithms for genomic sequence analysis. In: Advances in Neural Information Processing Systems 21. Cambridge: MIT Press, 2009. 1433–1440
Naphade M R, Smith J, Tesic J, et al. Large-scale concept ontology for multimedia. IEEE Multimed Mag, 2006,13: 86–91
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Tao, J., Chung, F. & Wang, S. A kernel learning framework for domain adaptation learning. Sci. China Inf. Sci. 55, 1983–2007 (2012). https://doi.org/10.1007/s11432-012-4611-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11432-012-4611-x