A Study of Architecture Optimization Techniques for Convolutional Neural Networks

Abstract

Edge devices such as smartphones or embedded computing platforms require a resource-aware approach. Therefore, it is often necessary to modify CNN models to make them compatible with the limited infrastructure. The diversity of solutions raises the question of how specific techniques may affect model performance. We address this question by empirically evaluating many techniques proposed in the literature on ResNet-101 and VGG-19 architectures. Our main contribution is the ablation study of how different approaches affect the final results in terms of the reduced number of model parameters, FLOPS, and unwanted accuracy drops. Thus, we also presented the possibility of implementing architecture compression methods that interfere with the low- or high-level model structure. We achieved a reduced ResNet-101 model with about 280 times fewer parameters with only 3.51 pp. accuracy drop compared to the baseline. We also performed post-training methods: pruning and quantization at different model sparsity levels. (All results are fully reproducible, the source code is available at https://github.com/artur-sobolewski/CNN-compression)