Abstract
One of the most common techniques for approaching image classification problems are Deep Neural Networks. These systems are capable of classifying images with different levels of detail at different levels of detail, with an accuracy that sometimes can surpass even manual classification by humans. Most common architectures for Deep Neural Networks are based on convolutional layers, which perform at the same time a convolution on each input channel and a linear aggregation on the convoluted channels. In this work, we develop a new method for augmenting the information of a layer inside a Deep Neural Network using channel-wise ordered aggregations. We develop a new layer that can be placed at different points inside a Deep Neural Network. This layer takes the feature maps of the previous layer and adds new feature maps by applying several channel-wise ordered aggregations based on learned weighting vectors. We perform several experiments introducing this layer in a VGG neural network and study the impact of the new layer, obtaining better accuracy scores over a sample dataset based on ImageNet. We also study the convergence and evolution of the weighting vectors of the new layers over the learning process, which gives a better understanding of the way the system is exploiting the additional information to gain new knowledge.
This work was partially supported by the Public University of Navarre under the projects PJUPNA13 and PJUPNA1926.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Anderson, D.T., Scott, G.J., Islam, M.A., Murray, B., Marcum, R.: Fuzzy choquet integration of deep convolutional neural networks for remote sensing. In: Computational Intelligence for Pattern Recognition, pp. 1–28. Springer, Cham (2018)
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a Large-Scale Hierarchical Image Database. In: CVPR 2009 (2009)
Dias, C.A., Bueno, J.C.S., Borges, E.N., Botelho, S.S.C., Dimuro, G.P., Lucca, G., Fernandéz, J., Bustince, H., Drews Junior, P.L.J.: Using the choquet integral in the pooling layer in deep learning networks. In: Barreto, G.A., Coelho, R. (eds.) Fuzzy Information Processing, pp. 144–154. Springer, Cham (2018)
Dias, C.A., Bueno, J.C.S., Borges, E.N., Lucca, G., Santos, H., Dimuro, G.P., Bustince, H., Junior, P.L.J.D., Botelho, S.S.C., Palmeira, E.: Simulating the behaviour of choquet-like (pre) aggregation functions for image resizing in the pooling layer of deep learning networks. In: International Fuzzy Systems Association World Congress, pp. 224–236. Springer, Cham (2019)
Du, X., Zare, A.: Multiple instance choquet integral classifier fusion and regression for remote sensing applications. IEEE Trans. Geosci. Remote Sens. 57(5), 2741–2753 (2019)
Mann, H.B., Whitney, D.R.: On a test of whether one of two random variables is stochastically larger than the other. In: The Annals of Mathematical Statistics, pp. 50–60 (1947)
Price, S.R., Price, S.R., Anderson, D.T.: Introducing fuzzy layers for deep learning. In: 2019 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), pp. 1–6 (2019)
Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60(1), 259–268 (1992)
Scott, G.J., Hagan, K.C., Marcum, R.A., Hurt, J.A., Anderson, D.T., Davis, C.H.: Enhanced fusion of deep neural networks for classification of benchmark high-resolution image data sets. IEEE Geosci. Remote Sens. Lett. 15(9), 1451–1455 (2018)
Scott, G.J., Marcum, R.A., Davis, C.H., Nivin, T.W.: Fusion of deep convolutional neural networks for land cover classification of high-resolution imagery. IEEE Geosci. Remote Sens. Lett. 14(9), 1638–1642 (2017)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv 1409.1556 (2014)
Smith, L.N.: A disciplined approach to neural network hyper-parameters: part 1 – learning rate, batch size, momentum, and weight decay. arXiv 1803.09820 (2018)
Veal, C., Yang, A., Hurt, A., Islam, M.A., Anderson, D.T., Scott, G., Keller, J.M., Havens, T.C., Tang, B.: Linear order statistic neuron. In: 2019 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), pp. 1–6 (2019)
Yager, R.R.: On ordered weighted averaging aggregation operators in multicriteria decisionmaking. IEEE Trans. Syst. Man Cybern. 18(1), 183–190 (1988)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Dominguez-Catena, I., Paternain, D., Galar, M. (2021). Learning Channel-Wise Ordered Aggregations in Deep Neural Networks. In: Kahraman, C., Cevik Onar, S., Oztaysi, B., Sari, I., Cebi, S., Tolga, A. (eds) Intelligent and Fuzzy Techniques: Smart and Innovative Solutions. INFUS 2020. Advances in Intelligent Systems and Computing, vol 1197. Springer, Cham. https://doi.org/10.1007/978-3-030-51156-2_119
Download citation
DOI: https://doi.org/10.1007/978-3-030-51156-2_119
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-51155-5
Online ISBN: 978-3-030-51156-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)