Abstract
Multi-source domain adaptation utilizes multiple source domains to learn the knowledge and transfers it to an unlabeled target domain. To address the problem, most of the existing methods aim to minimize the domain shift by auxiliary distribution alignment objectives, which reduces the effect of domain-specific features. However, without explicitly modeling the domain-specific features, it is not easy to guarantee that the domain-invariant representation extracted from input domains contains domain-specific information as few as possible. In this work, we present a different perspective on MSDA, which employs the idea of feature elimination to reduce the influence of domain-specific features. We design two different ways to extract domain-specific features and total features and construct the domain-invariant representations by eliminating the domain-specific features from total features. The experimental results on different domain adaptation datasets demonstrate the effectiveness of our method and the generalization ability of our model.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016, 770–778
Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C Y, Berg A C. SSD: single shot MultiBox detector. In: Proceedings of the 14th European Conference on Computer Vision. 2016, 21–37
He K, Gkioxari G, Dollár P, Girshick R. Mask R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision. 2017, 2980–2988
Fang T, Lu N, Niu G, Sugiyama M. Rethinking importance weighting for deep learning under distribution shift. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. 2020, 11996–12007
Sun Y P, Zhang M L. Compositional metric learning for multi-label classification. Frontiers of Computer Science, 2021, 15(5): 155320
Li Y F, Liang D M. Safe semi-supervised learning: a brief introduction. Frontiers of Computer Science, 2019, 13(4): 669
Fan J, Wang Y, Guan H, Song C, Zhang Z. Toward few-shot domain adaptation with perturbation-invariant representation and transferable prototypes. Frontiers of Computer Science, 2022, 16(3): 163347
Gao Z, Guo L, Guan W, Liu A A, Ren T, Chen S. A pairwise attentive adversarial spatiotemporal network for cross-domain few-shot action recognition-R2. IEEE Transactions on Image Processing, 2021, 30: 767
Quan R, Wu Y, Yu X, Yang Y. Progressive transfer learning for face anti-spoofing. IEEE Transactions on Image Processing, 2021, 30: 3946
Yang Y, Zhuang Y, Pan Y. Multiple knowledge representation for big data artificial intelligence: framework, applications, and case studies. Frontiers of Information Technology & Electronic Engineering, 2021, 22(12): 1551
Zhao S, Li B, Reed C, Xu P, Keutzer K. Multi-source domain adaptation in the deep learning era: a systematic survey. 2020, arXiv preprint arXiv: 2002.12169
Mansour Y, Mohri M, Rostamizadeh A. Multiple source adaptation and the rényi divergence. In: Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence. 2009, 367–374
Cui S, Jin X, Wang S, He Y, Huang Q. Heuristic domain adaptation. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. 2020, 7571–7583
Long M, Cao Y, Wang J, Jordan M I. Learning transferable features with deep adaptation networks. In: Proceedings of the 32nd International Conference on Machine Learning. 2015, 97–105
Sun B, Feng J, Saenko K. Correlation alignment for unsupervised domain adaptation. In: Csurka G, ed. Domain Adaptation in Computer Vision Applications. Cham: Springer, 2017, 153–171
Li S, Xie B, Wu J, Zhao Y, Liu C H, Ding Z. Simultaneous semantic alignment network for heterogeneous domain adaptation. In: Proceedings of the 28th ACM International Conference on Multimedia. 2020, 3866–3874
Jing T, Xia H, Ding Z. Adaptively-accumulated knowledge transfer for partial domain adaptation. In: Proceedings of the 28th ACM International Conference on Multimedia. 2020, 1606–1614
Kang G, Jiang L, Yang Y, Hauptmann A G. Contrastive adaptation network for unsupervised domain adaptation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, 4888–4897
Hoffman J, Tzeng E, Park T, Zhu J Y, Isola P, Saenko K, Efros A, Darrell T. CyCADA: cycle-consistent adversarial domain adaptation. In: Proceedings of the 35th International Conference on Machine Learning. 2018, 1989–1998
Sankaranarayanan S, Balaji Y, Castillo C D, Chellappa R. Generate to adapt: aligning domains using generative adversarial networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018, 8503–8512
Luo Y, Huang Z, Wang Z, Zhang Z, Baktashmotlagh M. Adversarial bipartite graph learning for video domain adaptation. In: Proceedings of the 28th ACM International Conference on Multimedia. 2020, 19–27
Hu H, Yang Y, Yin Y, Wu J. Metric learning for domain adversarial network. Frontiers of Computer Science, 2022, 16(5): 165341
Wang Y, Wang C, Xue H, Chen S. Self-corrected unsupervised domain adaptation. Frontiers of Computer Science, 2022, 16(5): 165323
Tian Q, Sun H, Peng S, Ma T. Self-adaptive label filtering learning for unsupervised domain adaptation. Frontiers of Computer Science, 2023, 17(1): 171308
Shao H. Query by diverse committee in transfer active learning. Frontiers of Computer Science, 2019, 13(2): 280
Hoffman J, Mohri M, Zhang N. Algorithms and theory for multiple-source adaptation. In: Proceedings of the 32nd International Conference on Information Processing Systems. 2018, 8256–8266
Zhao H, Zhang S, Wu G, Costeira J P, Moura J M F, Gordon G J. Adversarial multiple source domain adaptation. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems. 2018, 8568–8579
Xu R, Chen Z, Zuo W, Yan J, Lin L. Deep cocktail network: multi-source unsupervised domain adaptation with category shift. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018, 3964–3973
Zhao S, Wang G, Zhang S, Gu Y, Li Y, Song Z, Xu P, Hu R, Chai H, Keutzer K. Multi-source distilling domain adaptation. In: Proceedings of the 34th the AAAI Conference on Artificial Intelligence. 2020, 12975–12983
Wang H, Xu M, Ni B, Zhang W. Learning to combine: knowledge aggregation for multi-source domain adaptation. In: Proceedings of the 16th European Conference on Computer Vision. 2020, 727–744
Cui S, Wang S, Zhuo J, Su C, Huang Q, Tian Q. Gradually vanishing bridge for adversarial domain adaptation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020, 12452–12461
Saito K, Watanabe K, Ushiku Y, Harada T. Maximum classifier discrepancy for unsupervised domain adaptation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018, 3723–3732
Peng X, Bai Q, Xia X, Huang Z, Saenko K, Wang B. Moment matching for multi-source domain adaptation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019, 1406–1415
Zhu Y, Zhuang F, Wang D. Aligning domain-specific distribution and classifier for cross-domain classification from multiple sources. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 2019, 5989–5996
Yang L, Balaji Y, Lim S N, Shrivastava A. Curriculum manager for source selection in multi-source domain adaptation. In: Proceedings of the 16th European Conference on Computer Vision. 2020, 608–624
Venkat N, Kundu J N, Singh D K, Revanur A, Babu R V. Your classifier can secretly suffice multi-source domain adaptation. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. 2020, 390
Sun B, Saenko K. Deep CORAL: correlation alignment for deep domain adaptation. In: Proceedings of the European Conference on Computer Vision. 2016, 443–450
Ganin Y, Lempitsky V. Unsupervised domain adaptation by backpropagation. In: Proceedings of the 32nd International Conference on Machine Learning. 2015, 1180–1189
Tzeng E, Hoffman J, Saenko K, Darrell T. Adversarial discriminative domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017, 2962–2971
Venkateswara H, Eusebio J, Chakraborty S, Panchanathan S. Deep hashing network for unsupervised domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017, 5385–5394
Saenko K, Kulis B, Fritz M, Darrell T. Adapting visual category models to new domains. In: Proceedings of the 11th European Conference on Computer Vision. 2010, 213–226
van der Maaten L, Hinton G. Visualizing data using t-SNE. Journal of Machine Learning Research, 2008, 9(86): 2579
Selvaraju R R, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-CAM: visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE International Conference on Computer Vision. 2017, 618–626
Salzmann M, Ek C H, Urtasun R, Darrell T. Factorized orthogonal latent spaces. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics. 2010, 701–708
Bousmalis K, Trigeorgis G, Silberman N, Krishnan D, Erhan D. Domain separation networks. In: Proceedings of the 30th International Conference on Neural Information Processing Systems. 2016, 343–351
Acknowledgements
This work was supported by the National Natural Science Foundation of China (NSFC) (Grant Nos. 61876130 and 61932009).
Author information
Authors and Affiliations
Corresponding author
Additional information
Kunhong Wu received the BS degree from Tianjin University, China in 2020. He is currently pursuing the MS degree with the College of Intelligence and Computing, Tianjin University, China. His research interests include computer vision and domain adaptation.
Fan Jia received the BS degree from Tianjin University, China in 2019. He also received the MS degree with the College of Intelligence and Computing, Tianjin University, China. He is interested in computer vision and adversarial machine learning.
Yahong Han received the PhD degree from Zhejiang University, China in 2012. He is currently a professor with the College of Intelligence and Computing, Tianjin University, China. From Nov. 2014 to Nov. 2015, he visited Prof. Bin Yu’s group at UC Berkeley, USA as a Visiting Scholar. His current research interests include multimedia analysis, computer vision, and machine learning.
Electronic supplementary material
Rights and permissions
About this article
Cite this article
Wu, K., Jia, F. & Han, Y. Domain-specific feature elimination: multi-source domain adaptation for image classification. Front. Comput. Sci. 17, 174705 (2023). https://doi.org/10.1007/s11704-022-2146-x
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11704-022-2146-x