Skip to main content
SpringerLink
Log in

Feature Selection: Traditional and Wrapping Techniques with Tabu Search

  • Chapter
  • First Online:
Innovations in Machine and Deep Learning

Part of the book series: Studies in Big Data ((SBD,volume 134))

  • 350 Accesses

Abstract

Feature selection is an important step in improving the performance of machine learning algorithms. This paper describes a comparative study of traditional feature selection techniques and a wrapping technique with tabu search. To validate our wrapper with tabu search approach, we implemented three feature selection techniques: correlation, entropy, and principal component analysis. Nevertheless, to evaluate their performance, we used three classification algorithms: a J48 decision tree, a random forest, and an artificial neural network. We selected five datasets with a large number of features from public repositories. The experimental results showed that the subsets provided by tabu search have high performance with a J48 decision tree. Additionally, tabu search was better ranked than the other feature selection techniques. Finally, we consider that tabu search is a good alternative for feature selection paired with a simple classification algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Adeel, A., Khan, M.A., Akram, T., Sharif, A., Yasmin, M., Saba, T., Javed, K.: Entropy-controlled deep features selection framework for grape leaf diseases recognition. Expert Syst. 39 (2022). https://doi.org/10.1111/exsy.12569

  2. Agrawal, U., Rohatgi, V., Katarya, R.: Normalized mutual information-based equilibrium optimizer with chaotic maps for wrapper-filter feature selection. Expert Syst. Appl. 207, 118107 (2022). https://doi.org/10.1016/j.eswa.2022.118107

  3. Ahmed, Z.E., Saeed, R.A., Mukherjee, A., Ghorpade, S.N.: Energy optimization in low-power wide area networks by using heuristic techniques. In: LPWAN Technologies for IoT and M2M Applications, pp. 199–223 (2020). https://doi.org/10.1016/B978-0-12-818880-4.00011-9

  4. Ahmed, Z.H., Yousefikhoshbakht, M.: An improved tabu search algorithm for solving heterogeneous fixed fleet open vehicle routing problem with time windows. Alex. Eng. J. 64, 349–363 (2023). https://doi.org/10.1016/j.aej.2022.09.008

  5. Al-Batah, M., Zaqaibeh, B., Alomari, S.A., Alzboon, M.S.: Gene microarray cancer classification using correlation based feature selection algorithm and rules classifiers. Int. J. Online Biomed. Eng. 15, 62–73 (2019). https://doi.org/10.3991/ijoe.v15i08.10617

  6. Alazzam, H., Sharieh, A., Sabri, K.E.: A feature selection algorithm for intrusion detection system based on pigeon inspired optimizer. Expert Syst. Appl. 148 (2020). https://doi.org/10.1016/j.eswa.2020.113249

  7. Alidaee, B., Wang, H.: Uncapacitated (facility) location problem: A hybrid genetic-tabu search approach. IFAC-PapersOnLine 55(10), 1619–1624 (2022). https://doi.org/10.1016/j.ifacol.2022.09.622. 10th IFAC Conference on Manufacturing Modelling, Management and Control MIM 2022

  8. Alotaibi, Y.: A new meta-heuristics data clustering algorithm based on tabu search and adaptive search memory. Symmetry 14 (2022). https://doi.org/10.3390/sym14030623

  9. Bentsen, H., Hoff, A., Hvattum, L.M.: Exponential extrapolation memory for tabu search. EURO J. Comput. Optim. 10, 100028 (2022). https://doi.org/10.1016/j.ejco.2022.100028

  10. Bolívar, A., García, V., Florencia, R., Alejo, R., Rivera, G., Sánchez-Solís, J.P.: A preliminary study of smote on imbalanced big datasets when dealing with sparse and dense high dimensionality. In: Pattern Recognition: 14th Mexican Conference, MCPR 2022, Ciudad Juárez, Mexico, Proceedings, pp. 46–55. Springer, Berlin (2022). https://doi.org/10.1007/978-3-031-07750-0_5

  11. Bommert, A., Sun, X., Bischl, B., Rahnenführer, J., Lang, M.: Benchmark for filter methods for feature selection in high-dimensional classification data. Comput. Stat. Data Anal. 143 (2020). https://doi.org/10.1016/j.csda.2019.106839

  12. Chen, P., Li, F., Wu, C.: Research on intrusion detection method based on pearson correlation coefficient feature selection algorithm. J. Phys. Conf. Ser. 1757 (2021). IOP Publishing Ltd. https://doi.org/10.1088/1742-6596/1757/1/012054

  13. Ghane, M., Ang, M.C., Nilashi, M., Sorooshian, S.: Enhanced decision tree induction using evolutionary techniques for parkinson’s disease classification. Biocybern. Biomed. Eng. 42(3), 902–920 (2022). https://doi.org/10.1016/j.bbe.2022.07.002, www.sciencedirect.com/science/article/pii/S0208521622000663

  14. Ghosh, M., Guha, R., Sarkar, R., Abraham, A.: A wrapper-filter feature selection technique based on ant colony optimization. Neural Comput. Appl. 32, 7839–7857 (2020). https://doi.org/10.1007/s00521-019-04171-3

  15. Got, A., Moussaoui, A., Zouache, D.: Hybrid filter-wrapper feature selection using whale optimization algorithm: A multi-objective approach. Expert Syst. Appl. 183 (2021). https://doi.org/10.1016/j.eswa.2021.115312

  16. Gárate-Escamila, A.K., Hassani, A.H.E., Andrès, E.: Classification models for heart disease prediction using feature selection and pca. Inform. Med. Unlocked 19 (2020). https://doi.org/10.1016/j.imu.2020.100330

  17. Hanafi, S., Wang, Y., Glover, F., Yang, W., Hennig, R.: Tabu search exploiting local optimality in binary optimization. Eur. J. Oper. Res. (2023). https://doi.org/10.1016/j.ejor.2023.01.001

  18. He, Y., Jia, T., Zheng, W.: Tabu search for dedicated resource-constrained multiproject scheduling to minimise the maximal cash flow gap under uncertainty. Eur. J. Oper. Res. (2023). https://doi.org/10.1016/j.ejor.2023.02.029

  19. Khaire, U.M., Dhanalakshmi, R.: Stability of feature selection algorithm: A review. J. King Saud Univ. Comput. Inf. Sci. 34, 1060–1073 (2022). https://doi.org/10.1016/j.jksuci.2019.06.012

  20. de Lima, M.D., de Oliveira Roque e Lima, J., Barbosa, R.M.: Medical data set classification using a new feature selection algorithm combined with twin-bounded support vector machine. Med. Biol. Eng. Comput. 58, 519–528 (2020). https://doi.org/10.1007/s11517-019-02100-z

  21. Liu, H., Zhou, M., Liu, Q.: An embedded feature selection method for imbalanced data classification. IEEE/CAA J. Automatica Sinica 6, 703–715 (2019). https://doi.org/10.1109/JAS.2019.1911447

  22. Liu, Z., Chang, B., Cheng, F.: An interactive filter-wrapper multi-objective evolutionary algorithm for feature selection. Swarm Evol. Comput. 65, 100925 (2021). https://doi.org/10.1016/j.swevo.2021.100925

  23. Lucay, F.A., Gálvez, E.D., Cisternas, L.A.: Design of flotation circuits using tabu-search algorithms: Multispecies, equipment design, and profitability parameters. Minerals 9 (2019). https://doi.org/10.3390/min9030181

  24. Ma, J., Xia, D., Guo, H., Wang, Y., Niu, X., Liu, Z., Jiang, S.: Metaheuristic-based support vector regression for landslide displacement prediction: a comparative study. Landslides 19, 2489–2511 (2022). https://doi.org/10.1007/s10346-022-01923-6

  25. Momenzadeh, M., Sehhati, M., Rabbani, H.: A novel feature selection method for microarray data classification based on hidden markov model. J. Biomed. Inf. 95 (2019). https://doi.org/10.1016/j.jbi.2019.103213

  26. Omuya, E.O., Okeyo, G.O., Kimwele, M.W.: Feature selection for classification using principal component analysis and information gain. Expert Syst. Appl. 174 (2021). https://doi.org/10.1016/j.eswa.2021.114765

  27. Panigrahi, R., Borah, S.: Rank allocation to j48 group of decision tree classifiers using binary and multiclass intrusion detection datasets. Procedia Comput. Sci. 132, 323–332 (2018). https://doi.org/10.1016/j.procs.2018.05.186. International Conference on Computational Intelligence and Data Science

  28. Prajapati, V.K., Jain, M., Chouhan, L.: Tabu search algorithm (tsa): A comprehensive survey. In: Proceedings of 3rd International Conference on Emerging Technologies in Computer Engineering: Machine Learning and Internet of Things, ICETCE 2020, pp. 222–229. Institute of Electrical and Electronics Engineers Inc. (2020). https://doi.org/10.1109/ICETCE48199.2020.9091743

  29. Pudjihartono, N., Fadason, T., Kempa-Liehr, A.W., O’Sullivan, J.M.: A review of feature selection methods for machine learning-based disease risk prediction. Frontiers Bioinf. 2 (2022). https://doi.org/10.3389/fbinf.2022.927312

  30. Rivera, G., Florencia, R., García, V., Ruiz, A., Sánchez-Solís, J.P.: News classification for identifying traffic incident points in a spanish-speaking country: A real-world case study of class imbalance learning. Appl. Sci. 10(18), 6253 (2020). https://doi.org/10.3390/app10186253

  31. Shanthi, S., Akshaya, V.S., Smitha, J.A., Bommy, M.: Hybrid tabu search with sds based feature selection for lung cancer prediction. Int. J. Intell. Netw. 3, 143–149 (2022). https://doi.org/10.1016/j.ijin.2022.09.002

  32. Singh, N., Singh, P.: A hybrid ensemble-filter wrapper feature selection approach for medical data classification. Chemom. Intell. Lab. Syst. 217, 104396 (2021). https://doi.org/10.1016/j.chemolab.2021.104396

  33. Solorio-Fernández, S., Carrasco-Ochoa, J.A., Martínez-Trinidad, J.F.: A review of unsupervised feature selection methods. Artif. Intell. Rev. 53, 907–948 (2020). https://doi.org/10.1007/s10462-019-09682-y

  34. Spencer, R., Thabtah, F., Abdelhamid, N., Thompson, M.: Exploring feature selection and classification methods for predicting heart disease. Digit. Health 6 (2020). https://doi.org/10.1177/2055207620914777

  35. Tambake, N., Deshmukh, B., Patange, A.: Development of a low cost data acquisition system and training of j48 algorithm for classifying faults in cutting tool. Mater. Today: Proc. 72, 1061–1067 (2023). https://doi.org/10.1016/j.matpr.2022.09.163. 2nd International Conference and Exposition on Advances in Mechanical Engineering (ICoAME 2022)

  36. Venkateswarlu, C.: A metaheuristic tabu search optimization algorithm: Applications to chemical and environmental processes. In: Tsuzuki, M.S., Takimoto R.Y., Sato, A.K., Saka, T., Barari, A., Rahman, R.O.A., Hung, Y.T. (eds.) Engineering Problems, Chap. 10. IntechOpen, Rijeka (2021). https://doi.org/10.5772/intechopen.98240

  37. Venkatraman, P., Levin, M.W.: A congestion-aware tabu search heuristic to solve the shared autonomous vehicle routing problem. J. Intell. Transp. Syst.: Technol., Plan., Oper. 25, 343–355 (2021). https://doi.org/10.1080/15472450.2019.1665521

  38. Vommi, A.M., Battula, T.K.: A hybrid filter-wrapper feature selection using fuzzy knn based on bonferroni mean for medical datasets classification: A covid-19 case study. Expert Syst. Appl. 218, 119612 (2023). https://doi.org/10.1016/j.eswa.2023.119612

  39. Weinberg, A.I., Last, M.: Enhat—synergy of a tree-based ensemble with hoeffding adaptive tree for dynamic data streams mining. Inf. Fusion 89, 397–404 (2023). https://doi.org/10.1016/j.inffus.2022.08.026

  40. Yu, C., Lahrichi, N., Matta, A.: Optimal budget allocation policy for tabu search in stochastic simulation optimization. Comput. Oper. Res. 150, 106046 (2023). https://doi.org/10.1016/j.cor.2022.106046

  41. Zhang, J., Xiong, Y., Min, S.: A new hybrid filter/wrapper algorithm for feature selection in classification. Anal. Chim. Acta 1080, 43–54 (2019). https://doi.org/10.1016/j.aca.2019.06.054

  42. Zhou, H.F., Zhang, J.W., Zhou, Y.Q., Guo, X.J., Ma, Y.M.: A feature selection algorithm of decision tree based on feature weight. Expert Syst. Appl. 164 (2021). https://doi.org/10.1016/j.eswa.2020.113842

Download references

Acknowledgements

Thanks to CONACYT for support with number 848089.

Author information

Authors and Affiliations

  1. Departamento de Posgrado e Investigación, Facultad de Ingeniería, Universidad Autónoma de Tamaulipas (UAT), Tampico Tamaulipas, Mexico

    Laurentino Benito-Epigmenio, Salvador Ibarra-Martínez, Mirna Ponce-Flores & José Antonio Castán-Rocha

Authors
  1. Laurentino Benito-Epigmenio
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Salvador Ibarra-Martínez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mirna Ponce-Flores
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. José Antonio Castán-Rocha
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Salvador Ibarra-Martínez .

Editor information

Editors and Affiliations

  1. División Multidisciplinaria de Ciudad Universitaria, Universidad Autónoma de Ciudad Juárez, Chihuahua, Mexico

    Gilberto Rivera

  2. Universidad Tecnológica de La Habana, La Habana, Cuba

    Alejandro Rosete

  3. School of Engineering, University of Cadiz, Cadiz, Spain

    Bernabé Dorronsoro

  4. Instituto Tecnológico de Ciudad Madero, Tecnológico Nacional de México, Tamaulipas, Mexico

    Nelson Rangel-Valdez

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Benito-Epigmenio, L., Ibarra-Martínez, S., Ponce-Flores, M., Castán-Rocha, J.A. (2023). Feature Selection: Traditional and Wrapping Techniques with Tabu Search. In: Rivera, G., Rosete, A., Dorronsoro, B., Rangel-Valdez, N. (eds) Innovations in Machine and Deep Learning. Studies in Big Data, vol 134. Springer, Cham. https://doi.org/10.1007/978-3-031-40688-1_2

Download citation

Publish with us

Policies and ethics

Search

Navigation