Abstract
A method of choosing a pre-trained convolutional neural network (CNN) for transfer learning on the new image classification problem is proposed. The method can be used for quick estimation of which of the CNNs trained on the ImageNet dataset images (AlexNet, VGG16, VGG19, GoogLeNet, etc.) will be the most accurate after its fine tuning on the new sample of images. It is shown that there is high correlation (ρ ≈ 0.74, p < 0.01) between the characteristics of the features obtained at the output of the pre-trained CNN’s convolutional part and its accuracy on the test sample after fine tuning. The proposed method can be used to make recommendations for researchers who want to apply the pre-trained CNN and transfer learning approach to solve their own classification problems and don’t have sufficient computational resources and time for multiple fine tunings of available free CNNs with consequent choosing the best one.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ma, N., et al.: Shufflenet v2: practical guidelines for efficient CNN architecture design. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 116–131 (2018)
Baker, B., et al.: Designing neural network architectures using reinforcement learning. arXiv preprint. arXiv:1611.02167 (2016)
Mortazi, A., Bagci, U.: Automatically designing CNN architectures for medical image segmentation. In: International Workshop on Machine Learning in Medical Imaging, pp. 98–106. Springer, Cham (2018)
Sun, Y., et al.: Automatically designing CNN architectures using genetic algorithm for image classification. arXiv preprint. arXiv:1808.03818 (2018)
Oquab, M., Bottou, L., Laptev, I., Sivic, J.: Learning and transferring mid-level image representations using convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1717–1724 (2014)
Huang, Z., Pan, Z., Lei, B.: Transfer learning with deep convolutional neural network for SAR target classification with limited labeled data. Remote Sens. 9(9), 907 (2017)
Weiss, K., Khoshgoftaar, T.M., Wang, D.: A survey of transfer learning. J. Big data 3(1), 9 (2016)
Kulik, S.: Neural network model of artificial intelligence for handwriting recognition. J. Theor. Appl. Inf. Technol. 73(2), 202–211 (2015)
Larsen-Freeman, D.: Transfer of learning transformed. Lang. Learn. 63, 107–129 (2013)
Ghafoorian, M., et al.: Transfer learning for domain adaptation in MRI: application in brain lesion segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 516–524. Springer, Cham (2017)
Tang, Y., Peng, L., Xu, Q., Wang, Y., Furuhata, A.: CNN based transfer learning for historical Chinese character recognition. In: 2016 12th IAPR Workshop on Document Analysis Systems (DAS), pp. 25–29 (2016)
Akcay, S., et al.: Transfer learning using convolutional neural networks for object classification within x-ray baggage security imagery. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 1057–1061 (2016)
ImageNet. http://www.image-net.org
Yosinski, J., et al.: How transferable are features in deep neural networks? In: Advances in Neural Information Processing Systems, pp. 3320–3328 (2014)
Zhang, W., et al.: Deep model based transfer and multi-task learning for biological image analysis. IEEE Trans. Big Data 99, 1 (2016)
Trofimov, A.G., Velichkovskiy, B.M., Shishkin, S.L.: An approach to use convolutional neural network features in eye-brain-computer-interface. In: International Conference on Neuroinformatics, pp. 132–137. Springer, Cham (2017)
He, K., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Reyes, A.K., Caicedo, J.C., Camargo, J.E.: Fine-tuning deep convolutional networks for plant recognition. CLEF (Working Notes), p. 1391 (2015)
Desgraupes, B.: Clustering Indices. University of Paris Ouest-Lab Modal’X, Paris (2013)
Tsoumakas, G., Vlahavas, I.: Random k-labelsets: an ensemble method for multilabel classification. In: European Conference on Machine Learning, pp. 406–417. Springer, Berlin, Heidelberg (2007)
The MNIST database. http://yann.lecun.com/exdb/mnist/
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Trofimov, A.G., Bogatyreva, A.A. (2020). A Method of Choosing a Pre-trained Convolutional Neural Network for Transfer Learning in Image Classification Problems. In: Kryzhanovsky, B., Dunin-Barkowski, W., Redko, V., Tiumentsev, Y. (eds) Advances in Neural Computation, Machine Learning, and Cognitive Research III. NEUROINFORMATICS 2019. Studies in Computational Intelligence, vol 856. Springer, Cham. https://doi.org/10.1007/978-3-030-30425-6_31
Download citation
DOI: https://doi.org/10.1007/978-3-030-30425-6_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-30424-9
Online ISBN: 978-3-030-30425-6
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)