Abstract
Feature selection has been an important preprocessing step in high-dimensional data analysis and pattern recognition. In this paper, we propose a locality preserving multimodal discriminative learning method called LPMDL for supervised feature selection, which arises by solving two standard eigenvalue problems and seeks to find a pair of optimal transformations for two sets of multivariate data in different classes. This topic can optimally discover the local structure information of the given data hided in the original space and aims at structuring an effective low-dimensional embedding space, under which LPMDL keeps nearby data pairs in the same class close and between-class data pairs apart, and the projections of the original data in different classes can be appropriately separated from each other. LPMDL can be performed either in the input space or the reproducing kernel Hilbert space which gives rise to the kernelized version of LPMDL. We also evaluate the feasibility and efficiency of the LPMDL approach by conducting extensive data visualization and classification tasks. Experimental results on a broad range of data sets show LPMDL tends to capture the intrinsic structure characteristics of the samples data due to the effective representation of the points and achieves similar or even better performance than the conventional PCA, NPE, LPP and LFDA methods.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Belkin M, Niyogi P (2001) Laplacian eigenmaps and spectral techniques for embedding and clustering. Advances in neural information processing system. MIT Press, Cambridge, pp 585–591
Belkin M, Niyogi P (2003) Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput 15(6): 1373–1396
Blake C, Keogh E, Merz CJ (1998) UCI repository of machine learning databases. Department of Information and Computer Science, University of California, Irvine. http://www.ics.uci.edu/~mlearn/MLRepository.html
Chapelle, O, Schölkopf, B, Zien, A (eds) (2006) Semi-supervised learning. MIT Press, Cambridge
Chung FRK (1997) Spectral graph theory. AMS, pp 43–107
Dy JGC, Brodley E (2004) Feature selection for unsupervised learning. J Mach Learn Res 5(August): 845–889
Dy JG, Brodley CE, Kak AC, Broderick LS, Aisen AM (2003) Unsupervised feature selection applied to content-based retrieval of lung images. IEEE Trans Pattern Anal Mach Intell 25(3): 373–378
Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3(March): 1157–1182
Hastie, T, Tibshirani, R, Friedman, J (eds) (2001) The elements of statistical learning: data mining, inference, and prediction. Springer, New York, pp 534–553
He X, Niyogi P (2003) Locality preserving projections. Advances in neural information processing systems. MIT Press, Cambridge, pp 585–591
He X, Deng C, Yan SC, Zhang HJ (2005) Neighborhood preserving embedding. In: Proceeding of international conference on computer vision. IEEE CS Press, Washington, DC, pp 1208–1213
He X, Yan S, Hu Y, Niyogi P, Zhang H (2005) Face recognition using Laplacianfaces. IEEE Trans Pattern Anal Mach Intell 27(3): 328–340
Kim TK, Kittler J (2005) Locally linear discriminant analysis for multimodally distributed classes for face recognition with a single model image. IEEE Trans Pattern Anal Mach Intell 27(3): 318–327
Lin YY, Liu TL, Chen HT (2005) Semantic manifold learning for image retrieval. In: Proceedings of the ACM conference on multimedia. ACM Press, Singapore, pp 249–258
Mika S, Ratsch G, Weston J et al (1999) Fisher discriminant analysis with kernels. In: Hu YH, Larsen J, Wilson E, Douglas S (eds) Proceeding of the IEEE international workshop on neural networks for signal processing. IEEE Press, Madison, pp 41–48
Min W, Lu K, He X (2004) Locality pursuit embedding. Pattern Recognit 37(4): 781–788
Mokbel MF, Aref WG, Grama A (2003) Spectral LPM: an optimal locality-preserving mapping using the spectral (not fractal) order. In: Proceedings of 19th international conference on data engineering. IEEE Press, Bangalore, India, pp 699–701
Ojala T, Pietikäinen M, Mäenpää T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7): 971–987
Saul LK, Roweis ST (2003) Think globally, fit locally: unsupervised learning of low dimensional manifolds. J Mach Learn Res 4(December): 119–155
SchÄolkopf B, Smola AJ (2002) Learning with kernels. MIT Press, Cambridge, pp 25–55
Silven O, Niskanen M, Kauppinen H (2003) Wood inspection with non-supervised clustering. Mach Vis Appl 13(5–6): 275–285
Sommardahl O, Usenius A (1999) Wood samples image database. VTT Building Technology. http://www.ee.oulu.fi/~olli/Projects/Lumber.Grading.html
Song GJ, Cui B, Zheng BH, Xie Kq, Yang DQ (2008) Accelerating sequence searching: dimensionality reduction method. Knowl Inf Syst 20(3): 301–322
Sugiyama M (2007) Dimensionality reduction of multimodal labeled data by local fisher discriminant analysis. J Mach Learn Res 8(May): 1027–1061
Tsang IW, Kwok JT (2003) Distance metric learning with kernels. In: Proceeding of international conference on artificial neural networks, pp 126–129
Vlachos M, Domeniconi C, Gunopulos D (2002) Non-linear dimensionality reduction techniques for classification and visualization. In: Proceedings of international conference on knowledge discovery and data mining. ACM Press, Edmonton, Canada, pp 645–651
Verbeek JJ, Roweis ST, Vlassis N (2003) Non-linear CCA and PCA by alignment of local models. Advances in neural information processing systems. MIT Press, Cambridge, pp 297–304
Xiang SM, Nie FP, Zhang CS, Zhang CX (2006) Spline embedding for nonlinear dimensionality reduction. In: Proceedings of European conference on machine learning. Lecture Notes in Computer Science, Berlin, Germany, pp 825–832
Xiang SM, Nie FP, Song YQ, Zhang CS, Zhang CX (2009) Embedding new data points for manifold learning via coordinate propagation. Knowl Inf Syst 19(2): 159–184
Zelnik-Manor L, Perona P (2005) Self-tuning spectral clustering. In: Saul LK, Weiss Y, Bottou L (eds) Advances in neural information processing systems. MIT Press, Cambridge, pp 1601–1608
Zhang DQ, Chen SC (2003) Clustering incomplete data using kernel-based fuzzy c-means algorithm. Neural Process Lett 18(3): 155–162
Zhao HT, Sun SY, Jing ZL, Yang JY (2006) Local structure based supervised feature extraction. Pattern Recognit 39(8): 1546–1550
Zhou D, Bousquet O, Lal T, Weston J, Schölkopf B (2004) Learning with local and global consistency. Advances in neural information processing systems. MIT Press, Cambridge, pp 321–328
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zhang, Z., Ye, N. Locality preserving multimodal discriminative learning for supervised feature selection. Knowl Inf Syst 27, 473–490 (2011). https://doi.org/10.1007/s10115-010-0306-z
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10115-010-0306-z