Abstract
Humongous amounts of data bring various challenges to face image retrieval. This paper proposes an efficient method to solve those problems. Firstly, we use accurate facial landmark locations as shape features. Secondly, we utilise shape priors to provide discriminative texture features for convolutional neural networks. These shape and texture features are fused to make the learned representation more robust. Finally, in order to increase efficiency, a coarse-tofine search mechanism is exploited to efficiently find similar objects. Extensive experiments on the CASIAWebFace, MSRA-CFW, and LFW datasets illustrate the superiority of our method.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Chan, C. H.; Tahir, M. A.; Kittler, J.; Pietikainen, M. Multiscale local phase quantization for robust component-based face recognition using kernel fusion of multiple descriptors. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 35, No. 5, 1164–1177, 2013.
Wu, Z.; Ke, Q.; Sun, J.; Shum, H.-Y. Scalable face image retrieval with identity-based quantization and multireference reranking. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 33, No. 10, 1991–2001, 2011.
Bach, J. R.; Paul, S.; Jain, R. A visual information management system for the interactive retrieval of faces. IEEE Transactions on Knowledge and Data Engineering Vol. 5, No. 4, 619–628, 1993.
Eickeler, S. Face database retrieval using pseudo 2D hidden Markov models. In: Proceedings of the 5th IEEE International Conference on Automatic Face Gesture Recognition, 58–63, 2002.
Gudivada, V. N.; Raghavan, V. V. Modeling and retrieving images by content. Information Processing & Management Vol. 33, No. 4, 427–452, 1997.
Wang, X.; Zhang, C.; Zhang, Z. Boosted multitask learning for face verification with applications to web image and video search. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 142–149, 2009.
Ojala, T.; Pietikainen, M.; Maenpaa, T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 24, No. 7, 971–987, 2002.
Parkhi, O. M.; Vedaldi, A.; Zisserman, A. Deep face recognition. In: Proceedings of the British Machine Vision Conference, 2015. Available at http://www.bmva.org/bmvc/2015/papers/paper041/abstract041.pdf.
Schroff, F.; Kalenichenko, D.; Philbin, J. FaceNET: A unified embedding for face recognition and clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 815–823, 2015.
Sun, Y.; Chen, Y.; Wang, X.; Tang, X. Deep learning face representation by joint identification-verification. In: Proceedings of the Advances in Neural Information Processing Systems 27, 1988–1996, 2014.
Sun, Y.; Wang, X.; Tang, X. Hybrid deep learning for face verification. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 38, No. 10, 1997–2009, 2016.
Taigman, Y.; Yang, M.; Ranzato, M.; Wolf, L. DeepFace: Closing the gap to human-level performance in face verification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1701–1708, 2014.
Wen, Y.; Li, Z.; Qiao, Y. Latent factor guided convolutional neural networks for age-invariant face recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4893–4901, 2016.
Chopra, S.; Hadsell, R.; Lecun, Y. Learning a similarity metric discriminatively, with application to face verification. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1, 539–546, 2005.
Sun, Y.; Wang, X.; Tang, X. Deep learning face representation from predicting 10,000 classes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1891–1898, 2014.
Sun, Y.; Wang, X.; Tang, X. Deeply learned face representations are sparse, selective, and robust. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2892–2900, 2015.
Liu, J.; Deng, Y.; Bai, T.; Wei, Z.; Huang, C. Targeting ultimate accuracy: Face recognition via deep embedding. arXiv preprint arXiv:1506.07310, 2015.
Xiong, X.; la Torre, F. D. Supervised descent method and its applications to face alignment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 532–539, 2013.
Szegedy, C.; Liu, W.; Jia, Y.; Sermanet, P.; Reed, S.; Anguelov, D.; Erhan, D.; Vanhoucke, V.; Rabinovich, A. Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1–9, 2015.
Szegedy, C.; Vanhoucke, V.; Ioffe, S.; Shlens, J.; Wojna, Z. Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2818–2826, 2016.
Lin, M.; Chen, Q.; Yan, S. Network in network. arXiv preprint arXiv:1312.4400, 2013.
Yi, D.; Lei, Z.; Liao, S.; Li, S. Z. Learning face representation from scratch. arXiv preprint arXiv:1411.7923, 2014.
MSRA-CFW: Data set of celebrity faces on the web. Available at https://www.microsoft.com/en-us/research/project/msra-cfw-data-set-of-celebrity-faceson- the-web/.
Huang, G. B.; Mattar, M.; Berg, T.; Learned-Miller, E. Labeled faces in the wild: A database for studying face recognition in unconstrained environments. In: Proceedings of the Workshop on Faces in “Real-Life” Images: Detection, Alignment, and Recognition, 2008.
Cao, Z.; Yin, Q.; Tang, X.; Sun, J. Face recognition with learning-based descriptor. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2707–2714, 2010.
Kumar, N.; Berg, A. C.; Belhumeur, P. N.; Nayar, S. K. Attribute and simile classifiers for face verification. In: Proceedings of the IEEE 12th International Conference on Computer Vision, 365–372, 2009.
Yin, Q.; Tang, X.; Sun, J. An associate-predict model for face recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 497–504, 2011.
Taigman, Y.; Wolf, L. Leveraging billions of faces to overcome performance barriers in unconstrained face recognition. arXiv preprint arXiv:1108.1122, 2011.
Chen, D.; Cao, X.; Wang, L.; Wen, F.; Sun, J. Bayesian face revisited: A joint formulation. In: Computer Vision–ECCV 2012. Fitzgibbon, A.; Lazebnik, S.; Perona, P.; Sato, Y.; Schmid, C. Eds. Springer Berlin Heidelberg, 566–579, 2012.
Berg, T.; Belhumeur, P. N. POOF: Part-based onevs.- one features for fine-grained categorization, face verification, and attribute estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 955–962, 2013.
Berg, T.; Belhumeur, P. N. Tom-vs-Pete classifiers and identity-preserving alignment for face verification. In: Proceedings of the British Machine Vision Conference, 2012. Available at http://www.bmva.org/bmvc/2012/BMVC/paper129/paper129.pdf.
Chen, D.; Cao, X.; Wen, F.; Sun, J. Blessing of dimensionality: High-dimensional feature and its efficient compression for face verification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3025–3032, 2013.
Cao, X.; Wipf, D.; Wen, F.; Duan, G.; Sun, J. A practical transfer learning algorithm for face verification. In: Proceedings of the IEEE International Conference on Computer Vision, 3208–3215, 2013.
Zhu, Z.; Luo, P.; Wang, X.; Tang, X. Recover canonical-view faces in the wild with deep neural networks. arXiv preprint arXiv:1404.3543, 2014.
TCIT. Available at http://www.tcit-us.com/.
INSKY. Available at http://www.insky.so/.
DaHua-FaceImage. Available at http://www. dahuatech.com/.
Author information
Authors and Affiliations
Corresponding author
Additional information
This article is published with open access at Springerlink.com
Zongguang Lu received his B.E. degree in information engineering (system engineering) from Nanjing University of Information Science and Technology, Nanjing, China, in 2015. Since 2015, he has been a master student in the School of Information and Control Engineering at Nanjing University of Information Science and Technology, Nanjing, China. His research interests include pattern recognition, face analysis, and computer vision.
Jing Yang has been a master student in the School of Information and Control Engineering at Nanjing University of Information Science and Technology since September 2014. She received her bachelor degree in system engineering from Nanjing University of Information Science and Technology in June 2014. Her research interests include machine learning and computer vision.
Qingshan Liu is a professor in the School of Information and Control Engineering at Nanjing University of Information Science and Technology, Nanjing, China. He received his Ph.D. degree from the National Laboratory of Pattern Recognition, Chinese Academy of Sciences, Beijing, China, in 2003, and his M.S. degree from the Department of Auto Control at Southeast University, Nanjing, China, in 2000. He was an assistant research professor in the Department of Computer Science, Computational Biomedicine Imaging and Modeling Center, Rutgers, the State University of New Jersey, from 2010 to 2011. Before that, he was an associate professor in the National Laboratory of Pattern Recognition, Chinese Academy of Sciences, and an associate researcher in the Multi-media Laboratory, Chinese University of Hong Kong, in 2004–2005. He was a recipient of the President’s Scholarship of the Chinese Academy of Sciences in 2003. His current research interests are image and vision analysis, including face image analysis, graph and hypergraph-based image and video understanding, medical image analysis, and event-based video analysis.
Rights and permissions
Open Access The articles published in this journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.
About this article
Cite this article
Lu, Z., Yang, J. & Liu, Q. Face image retrieval based on shape and texture feature fusion. Comp. Visual Media 3, 359–368 (2017). https://doi.org/10.1007/s41095-017-0091-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41095-017-0091-7