Abstract
This paper is an announcement for our longer paper in preparation. Traditional kernel based methods utilize either a fixed kernel or a combination of judiciously chosen kernels from a fixed dictionary. In contrast, we construct a data-dependent kernel utilizing the components of the eigen-decompositions of different kernels constructed using ideas from diffusion geometry, and use a regularization technique with this kernel with adaptively chosen parameters. In this paper, we illustrate our method using the two moons dataset, where we obtain a zero test error using only a minimal number of training samples.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
References
Belkin, M., Matveeva, I., Niyogi P.: Regularization and semi-supervised learning on large graphs. In: Learning theory, pp. 624–638. Springer (2004)
Belkin, M., Niyogi P.: Laplacian eigenmaps for dimensionality reduction and data representation. Neural computation 15, 1373–1396 (2003)
Belkin, M., Niyogi P.: Semi-supervised learning on Riemannian manifolds. Machine learning 56, 209–239 (2004)
Belkin, M., Niyogi P.: Convergence of Laplacian eigenmaps. Adv. neur. inform. process. 19 129, (2007)
Belkin, M., Niyogi P., Sindhwani V.: Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J. Mach. Learn. Res. 7, 2399–2434 (2006)
Bertozzi A.L., Luo Xi., Stuart A.M., Zygalakis K.C.: Uncertainty Quantification in the Classification of High Dimensional Data. In: CoRR (2017) http://arxiv.org/abs/1703.08816
Chui, C.K., Donoho D.L.: Special issue: Diffusion maps and wavelets. Appl. Comput. Harm. Anal. 21 (2006)
Coifman, R. R. and Hirn, M. J.: Diffusion maps for changing data. Appl. Comput. Harmon. Anal. 36, 79–107 (2014)
Ehler, M., Filbir, F., Mhaskar H.N.: Locally Learning Biomedical Data Using Diffusion Frames. J. Comput. Biol. 19, 1251–1264 (2012)
Filbir, F., Mhaskar H.N.: Marcinkiewicz–Zygmund measures on manifolds. J. Complexity. 27, 568–596 (2011)
Lafon, S.S: Diffusion maps and geometric harmonics. Yale University (2004)
Maggioni, M., Mhaskar H.N.: Diffusion polynomial frames on metric measure space. Appl. Comput. Harmon. Anal. 24, 329–353 (2008)
Mhaskar, H.N.: Eignets for function approximation on manifolds. Appl. Comput. Harm. Anal. 29, 63–87 (2010)
Mhaskar, H.N.: A generalized diffusion frame for parsimonious representation of functions on data defined manifolds. Neural Networks 24, 345–359 (2011)
Mhaskar, H.N., Pereverzyev S.V., Semenov, V.Yu., Semenova E.V.: Data based construction of kernels for semi-supervised learning with less labels. RICAM Preprint (2018). https://www.ricam.oeaw.ac.at/files/reports/18/rep18-25.pdf
Micchelli, C.A., Mhaskar H.N.: Learning the kernel function via regularization. J.Mach.Learn.Res. 6, 10127–10134 (2005)
Pereverzyev, S.V., Tkachenko, P.: Regularization by the Linear Functional Strategy with Multiple Kernels. Frontiers Appl. Math. Stat. 3, 1 (2017)
Tikhonov, A.N., Glasko, V.B.: Use of regularization method in non-linear problems. Zh. Vychisl. Mat. Mat. Fiz. 5, 463–473 (1965)
von Luxburg U., Belkin, M., Bousquet O.: Consistency of spectral clustering. Ann. Statist. 36, 555–586 (2008)
Acknowledgements
Sergei V. Pereverzyev and Evgeniya Semenova gratefully acknowledge the support of the consortium AMMODIT funded within EU H2020-MSCA-RICE. The research of Hrushikesh Mhaskar is supported in part by the Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA), via 2018-18032000002. The paper has been finalized while the first two co-authors took part in the workshop “On the frontiers of high-dimensional computation” at the MATRIX Research Institute, Creswick, June 2018. The support of MATRIX is gratefully acknowledged.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Mhaskar, H.N., Pereverzyev, S.V., Semenov, V.Y., Semenova, E.V. (2020). Data Based Construction of Kernels for Classification. In: de Gier, J., Praeger, C., Tao, T. (eds) 2018 MATRIX Annals. MATRIX Book Series, vol 3. Springer, Cham. https://doi.org/10.1007/978-3-030-38230-8_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-38230-8_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-38229-2
Online ISBN: 978-3-030-38230-8
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)