Abstract
We develop a super-fast kernel density estimation algorithm (FastKDE) and based on this a fast kernel independent component analysis algorithm (KDICA). FastKDE calculates the kernel density estimator exactly and its computation only requires sorting n numbers plus roughly 2n evaluations of the exponential function, where n is the sample size. KDICA converges as quickly as parametric ICA algorithms such as FastICA. By comparing with state-of-the-art ICA algorithms, simulation studies show that KDICA is promising for practical usages due to its computational efficiency as well as statistical efficiency. Some statistical properties of KDICA are analyzed.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
References
Bach, F., Jordan, M.: Kernel independent component analysis. Journal of Machine Learning Research 3, 1–48 (2002)
Bickel, P., Klaassen, C., Ritov, Y., Wellner, J.: Efficient and Adaptive Estimation for Semiparametric Models. Springer, New York (1993)
Boscolo, R., Pan, H., Roychowdhury, V.P.: Independent component analysis based on nonparametric density estimation. IEEE Trans. Neural Networks 15(1), 55–65 (2004)
Cardoso, J.F.: Blind signal separation: statistical principles. Proceedings of the IEEE 9(10), 2009–2025 (1998)
Cardoso, J.F.: High-order contrasts for independent component analysis. Neural Computation 11(1), 157–192 (1999)
Chen, A.: Semiparametric inference for independent component analysis. Ph.D Thesis, Advisor: Peter J. Bickel, Department of Statistics, University of California, Berkeley (2004)
Chen, A., Bickel, P.J.: Consistent independent component analysis and prewhitening. IEEE Trans. on Signal Processing 53(10), 3625–3632 (2005)
Comon, P.: Independent component analysis, a new concept? Signal Processing 36(3), 287–314 (1994)
Edelman, A., Arias, T., Smith, S.: The geometry of algorithms with orthogonality constraints. SIAM journal on Matrix Analysis and Applications 20(2), 303–353 (1999)
Eriksson, J., Koivunen, V.: Characteristic-function based independent component analysis. Signal Processing 83, 2195–2208 (2003)
Greengard, L., Strain, J.: The fast Gauss transform. SIAM J. Sci. Stat. Comput. 12, 79–94 (1991)
Gray, A., Moore, A.: Very fast multivariate kernel density estimation via computational geometry. In: Proceedings of the Joint Statistical Meeting, San Francisco, CA (2003)
Gretton, A., Herbrich, R., Smola, A., Bousquet, O., Scholkopf, B.: Kernel methods for testing independence (2005) (submitted)
Hastie, T., Tibshirani, R.: Independent component analysis through product density estimation, Technical report, Department of Statistics, Stanford University (2002)
Hyvarinen, A.: Fast and robust fixed-point algorithms for independent component analysis. IEEE Trans. on Neural Networks 10(3), 626–634 (1999)
Hyvarinen, A., Karhunen, J., Oja, E.: Independent Component Analysis. John Wiley & Sons, New York (2001)
Jutten, C., Babaie-Zadeh, M., Hosseini, S.: Three easy ways for separating nonlinear mixtures? Signal Processing 84, 217–229 (2004)
Lee, T.W., Girolami, M., Bell, A., Sejnowski, T.: A unifying information-theoretic framework for independent component analysis. Computers and Mathematics with Applications 39, 1–21 (2000)
Miller, E., Fisher, J.: ICA using spacings estimates of entropy. Journal of Machine Learning Research 4, 1271–1295 (2003)
Murphy, S., van der Vaart, A.: On profile likelihood. Journal of the American Statistical Association 95, 449–485 (2000)
Pham, D.T.: Fast algorithms for mutual information based independent component analysis. IEEE Trans. on Signal Processing 52(10), 2690–2700 (2004)
Shwartz, S., Zibulevsky, M., Schechner, Y.: Fast kernel entropy estimation and optimization. Signal Processing 85, 1045–1058 (2005)
Silverman, B.W.: Density Estimation for Statistics and Data Analysis. Chapman Hall, London (1986)
Vlassis, N., Motomura, Y.: Efficient source adaptivity in independent component analysis. IEEE Trans. Neural Networks 12(3), 559–565 (2001)
Yang, H.H., Amari, S.: Adaptive on-line learning algorithms for blind separation - maximum entropy and minimum mutual information. Neural Computation 9(7), 1457–1482 (1997)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Chen, A. (2006). Fast Kernel Density Independent Component Analysis. In: Rosca, J., Erdogmus, D., Príncipe, J.C., Haykin, S. (eds) Independent Component Analysis and Blind Signal Separation. ICA 2006. Lecture Notes in Computer Science, vol 3889. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11679363_4
Download citation
DOI: https://doi.org/10.1007/11679363_4
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-32630-4
Online ISBN: 978-3-540-32631-1
eBook Packages: Computer ScienceComputer Science (R0)