Abstract
We apply augmentations to our dataset to enhance the quality of our predictions and make our final models more resilient to noisy data and domain drifts. Yet the question remains, how are these augmentations going to perform with different hyper-parameters? In this study we evaluate the sensitivity of augmentations with regards to the model’s hyper parameters along with their consistency and influence by performing a Local Surrogate (LIME) interpretation on the impact of hyper-parameters when different augmentations are applied to a machine learning model. The methodology consists of creating lime vectors, training machine learning models with different combinations of hyper-parameters and augmentation, training a linear regression model on the accuracy of the models, and utilizing the Linear regression coefficients for weighing each augmentation. Our research has proved that there are some augmentations which are highly sensitive to hyper-parameters and others which are more resilient and reliable.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Khattak, A.M., Khan, A.M., Tufai, A.: Activity recognition on smartphones via sensor-fusion and KDA-based SVMs (2014)
Siddiqi, M.H., Khan, A.M.: Promoting a healthier life-style using activity-aware smart phones (2012)
Lee, S.-Y., Khan, A.M, Lee, Y.-K.: Accelerometer’s position free human activity recognition using a hierarchical recognition model (2010)
Lee, S.-Y., Khan, A.M., Lee, Y.-K.: Human activity recognition via an accelerometer-enabled-smartphone using kernel discriminant analysis (2010)
Rivera, A.R., Khusainova, A., Khan, A.: SART - similarity, analogies, and relatedness for Tatar language: new benchmark datasets for word embeddings evaluation (2019)
Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.: Inception and ResNet (2016)
Hussain, Z., Ratner, A.J., Ehrenberg, H.R.: Learning to compose domain-specific transformations for data augmentation (2017)
Lee, S., Khan, A.M., Khan, F.I.: Prospects identification scheme for supermarkets by classification of customer behavior using time based analysis of transactional data (2007)
Khan, A., Dobrenkii, A., Kuleev, R.: Large residual multiple view 3D CNN for false positive reduction in pulmonary nodule detection (2017)
Creswell, A., White, T., Dumoulin, V., Arulkumaran, K.: Generative adversarial networks: an overview (2018)
Batanina, E., Bekkouch, I.E.I., Youssry, Y., Khan, A., Khattak, A.M., Bortnikov, M.: Domain adaptation for car accident detection in videos. In: 2019 Ninth International Conference on Image Processing Theory, Tools and Applications (IPTA), pp. 1–6 (2019)
Bekkouch, I.E.I., Aidinovich, T., Vrtovec, T., Kuleev, R., Ibragimov, B.: Multi-agent shape models for hip landmark detection in MR scans. In: Medical Imaging 2021: Image Processing, vol. 11596. SPIE, February 2021
Bekkouch, I.E.I., Constantin, N.D., Eyharabide, V., Billiet, F.: Adversarial domain adaptation for medieval instrument recognition. In: Arai, K. (ed.) IntelliSys 2021. LNNS, vol. 295, pp. 674–687. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-82196-8_50
Bekkouch, I.E.I., Nicolae, D.C., Khan, A., Kazmi, S.A., Khattak, A.M., Ibragimov, B.: Adversarial reconstruction loss for domain generalization. IEEE Access 9, 42424–42437 (2021)
Ng, A.Y., Sapp, B., Saxena, A.: A fast data collection and augmentation procedure for object recognition (2008)
Buslaev, A., Iglovikov, V.I., Khvedchenya, E., Parinov, A., Druzhinin, M., Kalinin, A.A.: Albumentations: fast and flexible image augmentations. Information 11(2), 125 (2020)
Bharath, A.A., Charalambous, C.C.: A data augmentation methodology for training machine/deep learning gait recognition algorithms (2016)
Cubuk, E.D., Zoph, B., Shlens, J., Randaugment, Q.L.: Practical automated data augmentation with a reduced search space. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2020
Fergus, R., Eigen, D., Puhrsch, C.: Depth map prediction from a single image using a multi-scale deep network (2014)
Cubuk, E.D., Zoph, B., Mane, D., Vasudevan, V.: AutoAugment: learning augmentation policies from data (2018)
Eyharabide, V., Bekkouch, I.E.I., Constantin, N.D.: Knowledge graph embedding-based domain adaptation for musical instrument recognition. Computers 10(8), 94 (2021)
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database (2009)
Gong, C., Wang, D., Li, M., Chandra, V., Liu, Q.: Keepaugment: a simple information-preserving data augmentation approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1055–1064, June 2021
Vollgraf, R., Xiao, H., Rasul, K.: Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms (2017)
Hauberg, S.: Dreaming more data: class-dependent distributions over diffeomorphisms for learned data augmentation (2016)
Lodha, V., Kekre, H.B., Thepade, S.D.: Augmentation of block truncation coding based image retrieval by using even and odd images with sundry colour spaces (2010)
Ibrahim, B.I., Nicolae, D.C., Khan, A., Ali, S.I., Khattak, A.: VAE-GAN based zero-shot outlier detection. In: Proceedings of the 2020 4th International Symposium on Computer Science and Intelligent Control, ISCSIC 2020. Association for Computing Machinery (2020)
Martin, J.M., El Naqa, I.: What Is Machine Learning? (2015)
Khan, A., Panchenko, I.: On expert-defined versus learned hierarchies for image classification (2019)
Gould, S., Guo, J.: Deep CNN ensemble with data augmentation for object detection (2015)
Perez, L., Wang, J.: The effectiveness of data augmentation in image classification using deep learning (2017)
Corcoran, P., Lemley, J., Bazrafkan, S.: Smart augmentation learning an optimal data augmentation strategy (2017)
Laine, T., Seo, J., Chiang, Y.: Step counting on smartphones using advanced zero-crossing and linear regression (2015)
Ren, S., He, K., Zhang, X., Sun, J.: Deep residual learning for image recognition (2016)
Yakovlev, K., Bekkouch, I.E.I., Khan, A.M., Khattak, A.M.: Abstraction-based outlier detection for image data. In: Arai, K., Kapoor, S., Bhatia, R. (eds.) IntelliSys 2020. AISC, vol. 1250, pp. 540–552. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-55180-3_40
Khan, A.M., Sozykin, K., Protasov, S.: Multi-label class-imbalanced action recognition in hockey videos via 3D convolutional neural networks (2018)
Shao, L., Zhu, F., Li, X.: Transfer learning for visual categorization: a survey (2015)
Guestrin, C., Ribeiro, M.T., Singh, S.: “Why should i trust you?”: explaining the predictions of any classifier (2016)
Khan, A., Gusarev, M., Kuleev, R.: Deep learning models for bone suppression in chest radiographs (2017)
Bortnikov, M., Khan, A., Khattak, A.M., Ahmad, M.: Accident recognition via 3D CNNs for automated traffic monitoring in smart cities. In: Arai, K., Kapoor, S. (eds.) CVC 2019. AISC, vol. 944, pp. 256–264. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-17798-0_22
Mazzara, M., Ahmad, M., Khan, A.M.: Multi-layer extreme learning machine-based autoencoder for hyperspectral image classification (2019)
Siddiqi, M.H., Ali, R., Khan, A.M., Kim, E.S., Kim, G.J., Lee, S.: Facial expression recognition using active contour-based face detection, facial movement-based feature extraction, and non-linear feature selection. Multimed. Syst. 21(6), 541–555 (2014). https://doi.org/10.1007/s00530-014-0400-2
Khan, A.M., Siddiqi, M.H., Ali, R.: Human facial expression recognition using stepwise linear discriminant analysis and hidden conditional random fields (2015)
Hong, C.S., Siddiqi, M.H., Alam, M.G.R.: A novel maximum entropy Markov model for human facial expression recognition (2016)
Chung, T.C., Siddiqi, M.H., Khan, A.M.: A precise recognition model for human facial expressions recognition systems (2013)
Lee, Y.-K., Siddiqi, M.H., Lee, S.: Hierarchical recognition scheme for human facial expression recognition systems (2013)
Molchanov, P., Gupta, S., Kim, K., Kautz, J.: Hand gesture recognition with 3D convolutional neural networks (2015)
Probst, P., Bischl, B., Boulesteix, A.-L.: Tunability: importance of hyperparameters of machine learning algorithms (2018)
Liu, X., Chi, M., Zhang, Y., Qin, Y.: Classifying high resolution remote sensing images by fine-tuned VGG deep networks (2018)
Tran, T., Pham, T., Carneiro, G., Palmer, L., Reid, I.: A Bayesian data augmentation approach for learning deep models (2017)
Wu, R., Yan, S., Shan, Y., Dang, Q., Sun, G.: Deep image: scaling up image recognition (2015)
Rivera, A.R., Khan, A., Bekkouch, I.E.I., Sheikh, T.S.: Anomaly detection based on zero-shot outlier synthesis and hierarchical feature distillation. IEEE Trans. Neural Netw. Learn. Syst. 1–11 (2020)
Han, D., Liu, Q., Fan, W.: A new image classification method using CNN transfer learning and web data augmentation (2018)
Yang, L., Shami, A.: On hyperparameter optimization of machine learning algorithms: theory and practice. CoRR, abs/2007.15745 (2020)
Bengio, Y., LeCun, Y., Hinton, G.: Deep learning (2015)
Zagoruyko, S., Komodakis, N.: Wide residual networks (2017)
Li, Y., Allen-Zhu, Z.: What can ResNet learn efficiently, going beyond kernels? (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Awais, C.M., Bekkouch, I.E.I., Khan, A.M. (2022). What Augmentations are Sensitive to Hyper-Parameters and Why?. In: Arai, K. (eds) Intelligent Computing. SAI 2022. Lecture Notes in Networks and Systems, vol 506. Springer, Cham. https://doi.org/10.1007/978-3-031-10461-9_31
Download citation
DOI: https://doi.org/10.1007/978-3-031-10461-9_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-10460-2
Online ISBN: 978-3-031-10461-9
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)