Abstract
Feature selection has two main objectives of maximising the classification performance and minimising the number of features. However, most existing feature selection algorithms are single objective wrapper approaches. In this work, we propose a multi-objective filter feature selection algorithm based on binary particle swarm optimisation (PSO) and probabilistic rough set theory. The proposed algorithm is compared with other five feature selection methods, including three PSO based single objective methods and two traditional methods. Three classification algorithms (naïve bayes, decision trees and k-nearest neighbours) are used to test the generality of the proposed filter algorithm. Experiments have been conducted on six datasets of varying difficulty. Experimental results show that the proposed algorithm can automatically evolve a set of non-dominated feature subsets. In almost all cases, the proposed algorithm outperforms the other five algorithms in terms of both the number of features and the classification performance (evaluated by all the three classification algorithms). This paper presents the first study on using PSO and rough set theory for multi-objective feature selection.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
References
Almuallim, H., Dietterich, T.G.: Learning boolean concepts in the presence of many irrelevant features. Artificial Intelligence 69, 279–305 (1994)
Cervante, L., Xue, B., Shang, L., Zhang, M.: A Dimension Reduction Approach to Classification Based on Particle Swarm Optimisation and Rough Set Theory. In: Thielscher, M., Zhang, D. (eds.) AI 2012. LNCS, vol. 7691, pp. 313–325. Springer, Heidelberg (2012)
Chakraborty, B.: Genetic algorithm with fuzzy fitness function for feature selection. In: International Symposium on Industrial Electronics, vol. 1, pp. 315–319 (2002)
Dash, M., Liu, H.: Feature selection for classification. Intelligent Data Analysis 1(4), 131–156 (1997)
Frank, A., Asuncion, A.: UCI machine learning repository (2010)
Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. The Journal of Machine Learning Research 3, 1157–1182 (2003)
Hall, M.A.: Correlation-based Feature Subset Selection for Machine Learning. Ph.D. thesis, The University of Waikato, Hamilton, New Zealand (1999)
Jensen, R.: Performing Feature Selection with ACO. In: Abraham, A., Grosan, C., Ramos, V. (eds.) Swarm Intelligence in Data Mining. SCI, vol. 34, pp. 45–73. Springer, Heidelberg (2006)
Kennedy, J., Eberhart, R.: A discrete binary version of the particle swarm algorithm. In: IEEE International Conference on Systems, Man, and Cybernetics, vol. 5, pp. 4104–4108 (1997)
Marill, T., Green, D.: On the effectiveness of receptors in recognition systems. IEEE Transactions on Information Theory 9(1), 11–17 (1963)
Mohemmed, A., Zhang, M., Johnston, M.: Particle swarm optimization based adaboost for face detection. In: IEEE Congress on Evolutionary Computation (CEC 2009), pp. 2494–2501 (2009)
Neshatian, K., Zhang, M.: Pareto front feature selection: using genetic programming to explore feature space. In: Proceedings of the 11th Annual Conference on Genetic and Evolutionary Computation, New York, NY, USA, pp. 1027–1034 (2009)
Pawlak, Z.: Rough sets. International Journal of Parallel Programming 11, 341–356 (1982)
Shi, Y., Eberhart, R.: A modified particle swarm optimizer. In: IEEE International Conference on Evolutionary Computation (CEC 1998), pp. 69–73 (1998)
Sierra, M.R., Coello Coello, C.A.: Improving PSO-Based Multi-objective Optimization Using Crowding, Mutation and ε-Dominance. In: Coello Coello, C.A., Hernández Aguirre, A., Zitzler, E. (eds.) EMO 2005. LNCS, vol. 3410, pp. 505–519. Springer, Heidelberg (2005)
Stearns, S.: On selecting features for pattern classifier. In: Proceedings of the 3rd International Conference on Pattern Recognition, pp. 71–75 (1976)
Unler, A., Murat, A.: A discrete particle swarm optimization method for feature selection in binary classification problems. European Journal of Operational Research 206(3), 528–539 (2010)
Wang, X., Yang, J., Teng, X., Xia, W., Jensen, R.: Feature selection based on rough sets and particle swarm optimization. Pattern Recognition Letters 28(4), 459–471 (2007)
Whitney, A.: A direct method of nonparametric measurement selection. IEEE Transactions on Computers C-20(9), 1100–1103 (1971)
Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques, 2nd edn. Morgan Kaufmann (2005)
Yao, Y., Zhao, Y.: Attribute reduction in decision-theoretic rough set models. Information Sciences 178(17), 3356–3373 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Cervante, L., Xue, B., Shang, L., Zhang, M. (2013). A Multi-objective Feature Selection Approach Based on Binary PSO and Rough Set Theory. In: Middendorf, M., Blum, C. (eds) Evolutionary Computation in Combinatorial Optimization. EvoCOP 2013. Lecture Notes in Computer Science, vol 7832. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-37198-1_3
Download citation
DOI: https://doi.org/10.1007/978-3-642-37198-1_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-37197-4
Online ISBN: 978-3-642-37198-1
eBook Packages: Computer ScienceComputer Science (R0)