Abstract
Neural networks have proven to be a good alternative in application fields such as healthcare, time-series forecasting and artificial vision, among others, for tasks like regression or classification. Their potential has been particularly remarkable in unstructured data, but recently developed architectures or their ensemble with other classical methods have produced competitive results in structured data. Feature selection has several beneficial properties: improve efficacy, performance, problem understanding and data recollection time. However, as new data sources become available and new features are generated using feature engineering techniques, more computational resources are required for feature selection methods. Feature selection takes an exorbitant amount of time in datasets with numerous features, making it impossible to use or achieving suboptimal selections that do not reflect the underlying behavior of the problem. We propose a nonparametric neural network layer which provides all the benefits of feature selection while requiring few changes to the architecture. Our method adds a novel layer at the beginning of the neural network, which removes the influence of features during training, adding inherent interpretability to the model without extra parameterization. In contrast to other feature selection methods, we propose an efficient and model-aware method to select the features with no need to train the model several times. We compared our method with a variety of popular feature selection strategies and datasets, showing remarkable results
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Borisov, V., Leemann, T., Seßler, K., Haug, J., Pawelczyk, M., Kasneci, G.: Deep Neural Networks and Tabular Data: A Survey (2021)
Cancela, B., Bolón-Canedo, V., Alonso-Betanzos, A.: E2E-FS: An End-to-End Feature Selection Method for Neural Networks. CoRR (2020)
Nepal Earthquake Open Data (2015). http://eq2015.npc.gov.np/#/
E-Shannon, C.: A mathematical theory of communication. ACM SIGMOBILE Mob. Comput. Commun. Rev. 5(1), 3–55 (2001)
Fiorini, S.: UCI Gene Expression Cancer RNA-Seq (2016)
Gómez-Losada, A., Asencio-Cortés, G., Martínez-Àlvarez, F., Riquelme, J.C.: A novel approach to forecast urban surface-level ozone considering heterogeneous locations and limited information. Environ. Model. Softw. 110, 52–61 (2018)
Khemphila, A., Boonjing, V.: Heart disease classification using neural network and feature selection. In: Proceedings - ICSEng 2011: International Conference on Systems Engineering (2011)
Liu, B., Cui, Q., Jiang, T., Ma, S.: A combinational feature selection and ensemble neural network method for classification of gene expression data. BMC Bioinform. 136, 10 (2004)
Marcano-Cedeño, A., Quintanilla-Domínguez, J., Cortina-Januchs, M.G., Andina, D.: Feature selection using sequential forward selection and classification applying artificial metaplasticity neural network. In: IECON Proceedings (Industrial Electronics Conference) (2010)
Miao, J., Niu, L.: A survey on feature selection. Proc. Comput. Sci. 91, 12 (2016)
Monirul Kabir, Md., Monirul Islam, Md., Murase, K.: A new wrapper feature selection approach using neural network. Neurocomputing. 73(16), 3273–3283 (2010)
Pytlak, K.: Personal Key Indicators of Heart Disease (2020). www.kaggle.com/datasets/kamilpytlak/personal-key-indicators-of-heart-disease
Taarifa: Water pump (2022). https://taarifa.org/
Tong, D., Mintram, R.: Genetic algorithm-neural network (GANN): a study of neural network activation functions and depth of genetic algorithm search applied to feature selection. Int. J. Mach. Learn. Cybern. 1, 75–87 (2010)
Verikas, A., Bacauskiene, M.: Feature selection with neural networks. Pattern Recogn. Lett. 23(11), 1323–1335 (2002)
Wang, J., Zhang, H., Wang, J., Pu, Y., Pal, N.R.: Feature selection using a neural network with group lasso regularization and controlled redundancy. IEEE Trans. Neural Netw. Learn. Syst. 32(3), 1110–1123 (2021)
Wolberg, W.H., Nick Street, W., Mangasarian, O.L.: UCI Breast Cancer Wisconsin (Diagnostic) (1995)
Acknowledgements
The authors would like to thank the Spanish Ministry of Science and Innovation for the support under the project PID2020-117954RB and the European Regional Development Fund and Junta de Andalucía for projects PY20-00870 and UPO-138516.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Jiménez-Navarro, M.J., Martínez-Ballesteros, M., Sousa Brito, I.S., Martínez-Álvarez, F., Asencio-Cortés, G. (2023). Feature-Aware Drop Layer (FADL): A Nonparametric Neural Network Layer for Feature Selection. In: García Bringas, P., et al. 17th International Conference on Soft Computing Models in Industrial and Environmental Applications (SOCO 2022). SOCO 2022. Lecture Notes in Networks and Systems, vol 531. Springer, Cham. https://doi.org/10.1007/978-3-031-18050-7_54
Download citation
DOI: https://doi.org/10.1007/978-3-031-18050-7_54
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-18049-1
Online ISBN: 978-3-031-18050-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)