Abstract
Dimensionality reduction and data visualization are useful and important processes in pattern recognition. Many techniques have been developed in the recent years. The self-organizing map (SOM) can be an efficient method for this purpose. This paper reviews recent advances in this area and related approaches such as multidimensional scaling (MDS), nonlinear PCA, principal manifolds, as well as the connections of the SOM and its recent variant, the visualization induced SOM (ViSOM), with these approaches. The SOM is shown to produce a quantized, qualitative scaling and while the ViSOM a quantitative or metric scaling and approximates principal curve/surface. The SOM can also be regarded as a generalized MDS to relate two metric spaces by forming a topological mapping between them. The relationships among various recently proposed techniques such as ViSOM, Isomap, LLE, and eigenmap are discussed and compared.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
E. Condon, B. Golden, S. Lele, S. Raghavan, E. Wasil. A Visualization Model Based on Adjacency Data. Decision Support Systems, vol. 33, no. 4, pp. 349–362, 2002.
H. Ni, H. Yin. Recurrent Self-organizing Maps and Local Support Vector Machine Models for Exchange Rate Prediction, In Proceedings of International Symposium on Neural Networks, Chengdu, China, vol. 3, pp. 504–511, 2006.
R. Freeman, H. Yin. Web Content Management by Self-organization. IEEE Transactions on Neural Networks, vol. 16, no. 5, pp. 1256–1268, 2005.
T. Kohonen, S. Kaski, K. Lagus, J. Salojarvi, J. Honkela, V. Paetero, A. Saerelo. Self-organization of Massive Text Document Collection, in Kohonen Maps, E. Oja and S. Kaski (eds.), pp. 171–182, 1999.
P. Toronen, K. Kolehmainen, G. Wong, E. Castren. Analysis of Gene Expression Data Using Self-organizing Maps. FEBS Letters, vol. 451, no. 2, pp. 142–146, 1999.
E. C. Malthouse. Limitations of Nonlinear PCA as Performed with Generic Neural Networks. IEEE Transactions on Neural Networks, vol. 9, no. 1, pp. 165–173, 1998.
M. A. Kramer. Nonlinear Principal Component Analysis Using Autoassociative Neural Networks. AIChE Journal, vol. 37, no. 2, pp. 233–243, 1991.
J. Karhunen, J. Joutsensalo. Generalization of Principal Component Analysis, Optimization Problems, and Neural Networks. Neural Networks, vol. 8, no. 4, pp. 549–562, 1995. [51] B.
B. Schőlkopf, A. Smola, K. R. Müller, Nonlinear Component Analysis as a Kernel Eigenvalue Problem, Neural Computation, vol. 10, no. 5, pp. 1299–1319, 1998.
T. Hastie, W. Stuetzle. Principal Curves. Journal of the American Statistical Association, vol. 84, no. 406, pp. 502–516, 1989.
M. LeBlanc, R. J. Tibshirani. Adaptive Principal Surfaces. Journal of the American Statistical Association, vol. 89, no. 425, pp. 53–64, 1994.
J. B. Tenenbaum, V. de Silva, J. C. Langford. A Global Geometric Framework for Nonlinear Dimensionality Reduction. Science, vol. 290, pp. 2319–2323, 2000.
S. T. Roweis, L.K. Saul. Nonlinear Dimensionality Reduction by Locally Linear Embedding. Science, vol. 290, pp. 2323–2326, 2000.
T. F. Cox, M. A. A. Cox, Multidimensional Scaling, Chapman & Hall, 1994.
B. D. Ripley. Pattern Recognition and Neural Networks, Cambridge University Press, Cambridge, 1996.
J. W. Sammon. A Nonlinear Mapping for Data Structure Analysis. IEEE Transactions on Computer, vol. C-18, no. 5, pp. 401–409, 1969.
J. Mao, A. K. Jain. Artificial Neural Networks for Feature Extraction and Multivariate Data Projection. IEEE Transactions on Neural Networks, vol. 6, no. 2, pp. 296–317, 1995.
D. Lowe, M. E. Tipping. Feed-forward Neural Networks and Topographic Mappings for Exploratory Data Analysis. Neural Computing & Applications, vol. 4, no. 2, pp. 83–95, 1996.
T. Kohonen. Self-organized Formation of Topologically Correct Feature Map. Biological Cybernetics, vol. 43, no. 1, pp. 56–69, 1982.
T. Kohonen. Self-organizing Maps, Springer-Verlag, Berlin, 1997.
H. Yin. ViSOM-A Novel Method for Multivariate Data Projection and Structure Visualization. IEEE Transactions on Neural Networks, vol. 13, no. 1, pp. 237–243, 2002.
C. von der Malsburg, D. J. Willshaw. Self-organization of Orientation Sensitive Cells in the Striate Cortex. Biological Cybernetic, vol. 14, no. 2, pp. 85–100, 1973.
D. J. Willshaw, C. von der Malsburg. How Patterned Neural Connections Can be Set Up by Self-Organization. Proceedings of Royal Society of London, Series B, Biological Sciences, vol. 194, no. 1117, pp. 431–445, 1976.
T. Kohonen. Self-organization and Associative Memory, Springer-Verlag, Berlin, 1984.
T. Kohonen. Representation of Sensory Information in Self-Organizing Feature Maps, and Relation of These Maps to Distributed Memory Networks, Proceedings of SPIE, vol. 634, pp. 248–259, 1986.
H. Yin. Self-Organizing Maps: Statistical Analysis, Treatment and Applications, Ph.D. dissertation, University of York, UK, 1996.
H. Yin, N. M. Allinson. On the Distribution and Convergence of the Feature Space in Self-organizing Maps, Neural Computation, vol. 7, pp. 1178–1187, 1995.
S. P. Luttrell. Derivation of a Class of Training Algorithms. IEEE Transactions on Neural Networks, vol. 1, no. 2, pp. 229–232, 1990.
S. P. Luttrell. A Bayesian Analysis of Self-organizing Maps. Neural Computation, vol. 6, no. 5, pp. 767–794, 1994.
H. Ritter, K. Schulten. Convergence Properties of Kohonen’s Topology Conserving Maps: Fluctuations, Stability, and Dimension Selection. Biological Cybernetics, vol. 60, no. 1, pp. 59–71, 1988.
E. Erwin, K. Obermayer, K. Schulten. Self-organizing Maps: Ordering, Convergence Properties and Energy Functions. Biological Cybernetics, vol. 67, no. 1, pp. 47–55, 1992.
E. Erwin, K. Obermayer, K. Schulten. Self-organizing Maps: Stationary States, Metastability and Convergence Rate. Biological Cybernetics, vol. 67, no. 1, pp. 35–45, 1992.
Z.P. Lo, B. Bavarian. On the Rate of Convergence in Topology Preserving Neural Networks. Biological Cybernetics, vol. 65, no. 1, pp. 55–63, 1991.
S. Lin, J. Si. Weight-value Convergence of the SOM Algorithm for Discrete Input. Neural Computation, vol. 10, no. 4, pp. 807–814, 1998.
E. de Bolt, M. Cottrell, M. Verleysen. Statistical Tools to Assess the Reliability of Self-organising Maps. Neural Networks, vol. 15, no. 8–9, pp. 967–978, 2002.
R. Durbin, G. Mitchison. A Dimension Reduction Framework for Understanding Cortical Maps. Nature, vol. 343, no. 6259, pp. 644–647, 1990.
G. Mitchison. A Type of Duality Between Self-organizing Maps and Minimal Wiring. Neural Computation, vol. 7, no. 1, pp. 25–35, 1995.
T. Kohonen. Self-organizing Maps: Optimization Approaches. Artificial Neural Networks, vol. 2, no. 5, pp. 981–990, 1991.
J. Lampinen, E. Oja. Clustering Properties of Hierarchical Selforganizing Maps. Journal of Mathematical Imaging and Vision, vol. 2, no. 2–3, pp. 261–272, 1992.
T. Heskes. Energy Functions for Self-organizing Maps. Kohonen Maps, E. Oja and S. Kaski (eds.), pp. 303–315, 1999.
G. J. Goodhill, T. Sejnowski. A Unifying Objective Function for Topographic Mappings. Neural Computation, vol. 9, no. 6, pp. 1291–1303, 1997.
H.-U. Bauer, K. R. Pawelzik. Quantifying the Neighborhood Preservation of Self-organizing Feature Maps. IEEE Transactions on Neural Networks, vol. 3, no. 4, pp. 570–579, 1992.
T. Villmann, R. Der, M. Herrmann, T. M. Martinetz. Topology Preservation in Self-organizing Feature Maps: Exact Definition and Measurement. IEEE Transactions on Neural Networks, vol. 8, no. 2, pp. 256–266, 1997.
S. Lawrence, C. L. Giles, A. C. Tsoi, A. D. Back. Face Recognition: A Convolutional Neural-network Approach. IEEE Transactions on Neural Networks, vol. 8, no. 1, pp. 98–113, 1997.
H. Ritter, T. Martinetz, K. Schulten. Neural Computation and Self-organizing Maps: An Introduction, Addison Wesley Publishing Company, Danvers, 1992.
H. Yin, N. M. Allinson. Interpolating Self-organizing Map (iSOM). Electronics Letters, vol. 35, no. 19, pp. 1649–1650, 1999.
H. Yin. Resolution Enhancement for the ViSOM, In Proceedings of the Workshop on Self-organizing Maps, Kitakyushu, Japan, pp. 208–212. 2003.
H. Yin. Data Visualization and Manifold Mapping Using the ViSOM. Neural Networks, vol. 15, no. 8–9, pp. 1005–1016, 2002.
S. Wu, T. W. S. Chow. PRSOM: A New Visualization Method by Hybridizing Multidimensional Scaling and Self-organizing Map. IEEE Transactions on Neural Networks, vol. 16, no. 6, pp. 1362–1380, 2005.
P. A. Estévez, C. J. Figueroa. Online Data Visualization Using the Neural Gas Network. Neural Networks, vol. 19, no. 6–7, pp. 923–934, 2006.
R. C. T. Lee, J. R. Slagle, H. Blum. A Triangulation Method for the Sequential Mapping of Points from N-Space to Two-Space. IEEE Transactions on Computers, vol. C-26, no. 3, pp. 288–292, 1977.
D. de Ridder, R. P. W. Duin. Sammon’s Mapping Using Neural Networks: A Comparison. Pattern Recognition Letters, vol. 18, no. 11–13, pp. 1307–1316, 1997.
M. Balasubramanian, E. L. Schwartz. The Isomap Algorithm and Topological Stability. Science, vol. 295, no. 5552, pp. 7, 2002.
E. Oja. Neural Networks, Principal Components, and Subspaces. International Journal of Neural Systems, vol. 1, no. 1, pp. 61–68, 1989.
T. D. Sanger. Optimal Unsupervised Learning in a Single-layer Linear Feedforward Network. Neural Networks, vol. 2, no. 6, pp. 459–473, 1989.
M. Belkin, P. Niyogi. Laplacian Eigenmaps for Dimensionality Reduction and Data Representation. Neural Computation, vol. 15, no. 6, pp. 1373–1396, 2003.
Y. Weiss. Segmentation Using Eigenvectors: A Unifying View. In Proceedings of the 7th IEEE International Conference on Computer Vision, Kerkyra, Greece, pp. 975–982, 1999.
R. Tibshirani. Principal Curves Revisited. Statistics and Computation, vol. 2, no. 4, pp. 183–190, 1992.
Y. Linde, A. Buzo, R. M. Gray. An Agorithm for Vctor Qantizer Dsign. IEEE Transactions on Communications, vol. 28, no. 1, pp. 84–95, 1980.
J. D. Banfield, A. E. Raftery. Ice Floe Identification in Satellite Images Using Mathematical Morphology and Clustering about Principal Curves. Journal of the American Statistical Association, vol. 87, no. 417, pp. 7–16, 1992.
B. Kegl, A. Krzyzak, T. Linder, K. Zeger. A Polygonal Line Algorithm for Constructing Principal Curves, In Proceeding of Neural Information Processing Systems, Computer Press, Denver Colorado, USA, pp. 501–507, 1998.
C. M. Bishop, M. Svensn, C. K. I. Williams. GTM: The Generative Topographic Mapping. Neural Computation, vol. 10, no. 1, pp. 215–235, 1998.
K. Y. Chang, J. Ghosh. A Unified Model for Probabilistic Principal Surfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 1, pp. 22–41, 2001.
A. Flexer. Limitations of Self-organizing Maps for Vector Quantization and Multidimensional Scaling, In Proceedings of Neural Information Processing Systems, Computer Press, Denver Colorado, USA, pp. 445–451, 1997.
H. Yin. Connection Between Self-organizing Maps and Metric Multidimensional Scaling, In Proceedings of International Joint Conference on Neural Networks, Orlando, Florida, USA, pp. 12–17, 2007.
P. Delicado. Another Look at Principal Curves and Surfaces. Journal of Multivariate Analysis. vol. 77, no. 1, pp. 84–116, 2001.
Author information
Authors and Affiliations
Additional information
Hujun Yin Senior lecturer at the University of Manchester, School of Electrical and Electronic Engineering. He received his B.Eng., M.Sc. from Southeast University and Ph.D. from University of York in 1983, 1986 and 1996 respectively. His research interests include neural networks, self-organizing systems in particular, pattern recognition, image processing and bioinformatics.
Dr. Yin has studied, extended and applied the self-organizing map (SOM) and related topics (principal manifolds and data visualization) extensively in the last ten years and proposed a number of extensions including the Bayesian SOM and ViSOM. He has served on the Programme Committee for more than twenty international conferences. He was the Organizing and Programme Committee Chair and General Chair for a number of conferences, including, International Workshop on Self-Organizing Maps (WSOM’01), International Conference on Intelligent Data Engineering and Automated Learning (IDEAL’03-IDEAL’07), and International Symposium on Neural Networks (ISNN’04-ISNN’06). He sits on the Steering Committee of the WSOM series. He was a guest editor of Neural Networks: 2002 Special Issue on New Developments in Self-Organizing Maps, among two other special issues on two other international journals. He has received research funding from the EPSRC, BBSRC and DTI.
Dr. Yin has published more than 90 peer-reviewed articles. He is a senior member of the IEEE and a member of the EPSRC Peer Review College. He has also been a regular referee for the EPSRC, BBSRC and Royal Society grant proposals, Hong Kong Research Grant Councils, etherlands Organization for Scientific Research, and Slovakia Research and Development Council. He is an associate editor of the IEEE Transactions on Neural Networks and a member of the Editorial Board of the International Journal of Neural Systems.
Rights and permissions
About this article
Cite this article
Yin, H. Nonlinear dimensionality reduction and data visualization: A review. Int J Automat Comput 4, 294–303 (2007). https://doi.org/10.1007/s11633-007-0294-y
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/s11633-007-0294-y