Summary
Feature subset selection is an important pre-processing step for classification. A more general framework of feature selection is feature ranking. A feature ranking provides an ordered list of the features, sorted according to their relevance. Using such a ranking provides a better overview of the feature elimination process, and allows the human expert to gain more insight into the processes underlying the data. In this chapter, we describe a technique to derive a feature ranking directly from the estimated distribution of an EDA. As an example, we apply the method to the biological problem of acceptor splice site prediction, demonstrating the advantages for knowledge discovery in biological datasets with many features.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
A.I. Blum and P. Langley. Selection of relevant features and examples in machine learning. Artificial Intelligence, 97:245–271, 1997.
S. Degroeve, B. De Baets, Y. Van de Peer, and P. Rouzé. Feature subset selection for splice site prediction. Bioinformatics, 18(2):75–83, 2002.
R. Duda and P. Hart. Pattern Classification and Scene Analysis. John Wiley and Sons, New York, 1973.
R. Etxeberria and P. Larrañaga. Global optimization with Bayesian networks. In Proceedings of the Second Symposium on Artificial Intelligence. Special Session on Distributions and Evolutionary Optimization, pp. 332–339, 1999.
I. Guyon and A. Elisseeff. An introduction to variable and feature selection. Journal of Machine Learning Research, 3:1157–1182, 2003.
I. Guyon, J. Weston, S. Barnhill, and V.N. Vapnik. Gene selection for cancer classification using support vector machines. Machine Learning, 46(1–3):389–422, 2000.
M. Henrion. Propagating uncertainty in Bayesian networks by probabilistic logic sampling. In J. F. Lemmer and L. N. Kanal, editors, Uncertainty in Artificial Intelligence, volume 2, pp. 149–163. North-Holland, Amsterdam, 1988.
I. Inza, P. Larrañaga, R. Etxeberria, and B. Sierra. Feature subset selection by Bayesian network-based optimization. Artificial Intelligence, 123(1–2):157–184, 2000.
I. Inza, P. Larrañaga, and B. Sierra. Feature Subset Selection by Estimation of Distribution Algorithms. In P. Larrañaga and J. A. Lozano, editors, Estimation of Distribution Algorithms. A New Tool for Evolutionary Computation, pp. 269–294. Kluwer Academic Publishers, 2001.
A. K. Jain, R. W. Duin, and J. Mao. Statistical Pattern Recognition. A review. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(1):4–37, 2000.
J. Kittler. Feature set search algorithms. In C.H. Chen, editor, Pattern Recognition and Signal Processing, pp. 41–60. Sithoff and Noordhoff, 1978.
R. Kohavi and G. John. Wrappers for feature subset selection. Artificial Intelligence, 97(1–2):273–324, 1997.
M. Kudo and J. Sklansky. Comparison of algorithms that select features for pattern classifiers. Pattern Recognition, 33:25–41, 2000.
P. Larrañnaga and J. A. Lozano. Estimation of Distribution Algorithms. A New Tool for Evolutionary Computation. Kluwer Academic Press, 2001.
S.J. Louis and G.J.E. Rawlins. Predicting convergence time for genetic algorithms. Technical Report TR370, Indiana University, 1993.
C. Mathé, M.F. Sagot, T. Schiex, and P. Rouzé. Current methods of gene prediction, their strengths and weaknesses. Nucleic Acids Research, 30:4103–4117, 2002.
D. Mladenić and M. Grobelnik. Feature selection on hierarchy of web documents. Decision Support Systems, 35:45–87, 2003.
H. Mühlenbein. The equation for response to selection and its use for prediction. Evolutionary Computation, 5:303–346, 1998.
J. Pearl. Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, 1988.
M. Pelikan, D.E. Goldberg, and E. Cantú-Paz. BOA: the Bayesian optimization algorithm. In Proceedings of the Genetic and Evolutionary Computation Conference, pp. 525–532, 1999.
M. Pelikan and H. Mühlenbein. The bivariate marginal distribution algorithm. In Advances in Soft Computing-Engineering Design and Manufacturing, pp. 521–535, 1999.
Y. Saeys, S. Degroeve, D. Aeyels, Y. Van de Peer, and P. Rouzé. Fast feature selection using a simple estimation of distribution algorithm: a case study on splice site prediction. Bioinformatics, 19(2):179–188, 2003.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Saeys, Y., Degroeve, S., Van de Peer, Y. (2006). Feature Ranking Using an EDA-based Wrapper Approach. In: Lozano, J.A., Larrañaga, P., Inza, I., Bengoetxea, E. (eds) Towards a New Evolutionary Computation. Studies in Fuzziness and Soft Computing, vol 192. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-32494-1_10
Download citation
DOI: https://doi.org/10.1007/3-540-32494-1_10
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29006-3
Online ISBN: 978-3-540-32494-2
eBook Packages: EngineeringEngineering (R0)