Abstract
In this review I present several representation learning methods, and discuss the latest advancements with emphasis in applications to network science. Representation learning is a set of techniques that has the goal of efficiently mapping data structures into convenient latent spaces. Either for dimensionality reduction or for gaining semantic content, this type of feature embeddings has demonstrated to be useful, for example, for node classification or link prediction tasks, among many other relevant applications to networks. I provide a description of the state-of-the-art of network representation learning as well as a detailed account of the connections with other fields of study such as continuous word embeddings and deep learning architectures. Finally, I provide a broad view of several applications of these techniques to networks in various domains.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
G. Alanis-Lobato, Mining protein interactomes to improve their reliability and support the advancement of network medicine, Front. Genet. 6, 296 (2015)
G. Alanis-Lobato, C.V. Cannistraci, A. Eriksson, A. Manica, T. Ravasi, Highlighting nonlinear patterns in population genetics datasets, Sci. Rep. 5, 8140 (2015)
G. Alanis-Lobato, C.V. Cannistraci, T. Ravasi, Exploitation of genetic interaction network topology for the prediction of epistatic behavior, Genomics 102, 202 (2013)
G. Alanis-Lobato, P. Mier, M.A. Andrade-Navarro, Efficient embedding of complex networks to hyperbolic space via their Laplacian, Sci. Rep. 6, 30108 (2016)
T. Aste, T. Di Matteo, S.T. Hyde, Complex networks on hyperbolic surfaces, Physica A 346, 20 (2005)
T. Aste, R. Gramatica, T.D. Matteo, Exploring complex networks via topological embedding on surfaces, Phys. Rev. E 86, 036109 (2012)
M. Baroni, G. Dinu, G. Kruszewski, Don’t count, predict! a systematic comparison of context-counting vs. context-predicting semantic vectors, in Proceedings of Association for Computational Linguistics (ACL, 2014), p. 238
M. Barthélemy, Spatial networks, Phys. Rep. 499, 1 (2011)
M. Belkin, P. Niyogi, in Laplacian eigenmaps and spectral techniques for embedding and clustering (NIPS, 2014), Vol 14, pp. 585–591
M. Belkin, P. Niyogi, V. Sindhwani, Manifold regularization: A geometric framework for learning from labeled and unlabeled examples, J. Mach. Learn. Res. 7, 2399 (2006)
Y. Bengio, Learning deep architectures for AI, Found. Trends® Mach. Learn. 2, 1 (2009)
Y. Bengio, A. Courville, P. Vincent, Representation learning: A review and new perspectives, IEEE Trans. Pattern Anal. Mach. Intell. 35, 1798 (2013)
D.M. Blei, A.Y. Ng, M.I. Jordan, Latent dirichlet allocation, J. Mach. Learn. Res. 3, 993 (2003)
V.D. Blondel, J.-L. Guillaume, R. Lambiotte, E. Lefebvre, Fast unfolding of communities in large networks, J. Stat. Mech.: Theor. Exp. 2008, P10008 (2008)
M. Boguñá, D. Krioukov, Navigating ultrasmall worlds in ultrashort time, Phys. Rev. Lett. 102, 058701 (2009)
M. Boguñá, D. Krioukov, K.C. Claffy, Navigability of complex networks, Nat. Phys. 5, 74 (2009)
M. Boguñá, F. Papadopoulos, D. Krioukov, Sustaining the internet with hyperbolic mapping, Nat. commun. 1, 62 (2010)
S. Bourigault, C. Lagnier, S. Lamprier, L. Denoyer, P. Gallinari, Learning social network embeddings for predicting information diffusion, in Proceedings of the 7th ACM international conference on Web search and data mining (ACM, 2014), p. 393
C.V. Cannistraci, G. Alanis-Lobato, T. Ravasi, Minimum curvilinearity to enhance topological prediction of protein interactions by network embedding, Bioinformatics, 29, i199 (2013)
S. Cao, W. Lu, Q. Xu, GraRep: Learning graph representations with global structural information, in Proceedings of the 24th ACM International on Conference on Information and Knowledge Management (ACM, 2015), p. 891
S. Cao, W. Lu, Q. Xu, Deep neural networks for learning graph representations, in Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI-16) (AAAI Press, 2016)
P.R. Cavalin, L.G. Moyano, P.P. Miranda, A multiple classifier system for classifying life events on social media, in 2015 IEEE International Conference on Data Mining Workshop (ICDMW) (IEEE, 2015), pp. 1332–1335
S. Chang, W. Han, J. Tang, G.-J. Qi, C.C. Aggarwal, T.S. Huang, Heterogeneous network embedding via deep architectures, in Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (ACM, 2015), p. 119
W. Chen, W. Fang, G. Hu, M.W. Mahoney, On the hyperbolicity of small-world and treelike random graphs, Int. Math. 9, 434 (2013)
R.F. Cohen, P. Eades, T. Lin, F. Ruskey, Three-dimensional graph drawing, Algorithmica 17, 199 (1997)
A. Dallmann, T. Niebler, F. Lemmerich, A. Hotho, Extracting semantics from random walks on wikipedia: Comparing learning and counting methods, in Tenth International AAAI Conference on Web and Social Media (2016)
L. Daqing, K. Kosmidis, A. Bunde, S. Havlin, Dimension of spatially embedded networks, Nat. Phys. 7, 481 (2011)
S. Deerwester, S.T. Dumais, G.W. Furnas, T.K. Landauer, R. Harshman, Indexing by latent semantic analysis, J. Amer. Soc. Inf. Sci. 41, 391 (1990)
R.O. Duda, P.E. Hart, D.G. Stork, Pattern classification (John Wiley & Sons, 2012)
J. Ganesh, S. Ganguly, M. Gupta, V. Varma, V. Pudi, Author2vec: Learning author representations by combining content and link information, in Proceedings of the 25th International Conference Companion on World Wide Web, International World Wide Web Conferences Steering Committee (2016), p. 49
L. Getoor, B. Taskar, Introduction to statistical relational learning (MIT Press, Cambridge, 2007)
A. Grover, J. Leskovec, node2vec: Scalable feature learning for networks, in Proceedings of the 22th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (ACM, 2016)
M.S. Handcock, A.E. Raftery, J.M. Tantrum, Model-based clustering for social networks, J. R. Stat. Soc.: Ser. Stat. Soc. 170, 301 (2007)
X. He, P. Niyogi, Locality preserving projections, in Neural Information Processing Systems (MIT, 2004), Vol 16, p. 153
L. Heck, Deep learning of knowledge graph embeddings for semantic parsing of twitter dialogs, in The 2nd IEEE Global Conference on Signal and Information Processing (IEEE, 2014)
G.E. Hinton, S. Osindero, Y.-W. Teh, A fast learning algorithm for deep belief nets, Neur. Comput. 18, 1527 (2006)
L. Huang, J. May, X. Pan, H. Ji, Building a fine-grained entity typing system overnight for a new x (x= language, domain, genre) arXiv:1603.03112 (2016)
P.-S. Huang, X. He, J. Gao, L. Deng, A. Acero, L. Heck, Learning deep structured semantic models for web search using clickthrough data, in Proceedings of the 22Nd ACM International Conference on Information & Knowledge Management, CIKM ’13, (New York, USA, 2013, ACM), p. 2333
Y. Jacob, L. Denoyer, P. Gallinari, Classification dans les graphes hétérogènes basée sur une représentation latente des nœuds, in CORIA 2013 (2013), pp. 85–100
Y. Jacob, L. Denoyer, P. Gallinari, Learning latent representations of nodes for classifying in heterogeneous social networks, in Proceedings of the 7th ACM international conference on Web search and data mining (ACM, 2014), p. 373
R. Kiros, R. Zemel, R.R. Salakhutdinov, A multiplicative model for learning distributed text-based attribute representations, in Advances in Neural Information Processing Systems (2014), pp. 2348–2356
J.M. Kleinberg, Navigation in a small world, Nature 406, 845 (2000)
R. Kleinberg, Geographic routing using hyperbolic space, in IEEE INFOCOM 2007-26th IEEE International Conference on Computer Communications (IEEE, 2007), pp. 1902–1909
D. Krioukov, F. Papadopoulos, M. Boguñá, A. Vahdat, Greedy forwarding in scale-free networks embedded in hyperbolic metric spaces, ACM SIGMETRICS Perform. Eval. Rev. 37, 15 (2009)
D. Krioukov, F. Papadopoulos, M. Kitsak, A. Vahdat, M. Boguná, Hyperbolic geometry of complex networks, Phys. Rev. E 82, 036106 (2010)
D. Krioukov, F. Papadopoulos, A. Vahdat, M. Boguñá, Curvature and temperature of complex networks, Phys. Rev. E 80, 035101 (2009)
C. Lagnier, S. Bourigault, S. Lamprier, L. Denoyer, P. Gallinari, Learning information spread in content networks, arXiv:1312.6169 (2014)
Y.-Y. Lai, C. Li, D. Goldwasser, J. Neville, Better together: Combining language and social interactions into a shared representation, in Proceedings of TextGraphs-10: the Workshop on Graph-based Methods for Natural Language Processing (San Diego, CA, USA, 2016. Association for Computational Linguistics), p. 29
Q. Le, T. Mikolov, Distributed representations of sentences and documents, in Proceedings of The 31st International Conference on Machine Learning (ICML, 2014), Vol 14, p. 1188
Y. LeCun, Y. Bengio, G. Hinton, Deep learning, Nature 521, 436 (2015)
O. Levy, Y. Goldberg, Neural word embedding as implicit matrix factorization, in Advances in Meural Information Processing Systems (2014), pp. 2177–2185
K. Li, J. Gao, S. Guo, N. Du, X. Li, A. Zhang, Lrbm: A restricted boltzmann machine based approach for representation learning on linked data, in 2014 IEEE International Conference on Data Mining (IEEE, 2014), pp. 300–309
Y. Li, L. Xu, F. Tian, L. Jiang, X. Zhong, E. Chen, Word embedding revisited: A new representation learning and explicit matrix factorization perspective, in Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence (IJCAI, 2015), p. 25
D. Liben-Nowell, J. Kleinberg, The link-prediction problem for social networks, J. Amer. Soc. Inf. Sci. Tech. 58, 1019 (2007)
F. Liu, B. Liu, C. Sun, M. Liu, X. Wang, Deep learning approaches for link prediction in social network services, in International Conference on Neural Information Processing (Springer, 2013), pp. 425–432
F. Liu, B. Liu, C. Sun, M. Liu, X. Wang, Deep belief network-based approaches for link prediction in signed social networks, Entropy 17, 2140 (2015)
Y. Long, Characterizing video diffusion patterns in online social networks, HKU Theses Online (HKUTO), 2015
K. Lu, Z. Ding, S. Ge, Sparse-representation-based graph embedding for traffic sign recognition, IEEE Trans. Intell. Trans. Syst. 13, 1515 (2012)
Y. Luo, Q. Wang, B. Wang, L. Guo, Context-dependent knowledge graph embedding, in EMNLP, editor, Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (ACL, 2015), p. 1656
S.A. Macskassy, F. Provost, Classification in networked data: A toolkit and a univariate case study, J. Mach. Learn. Res. 8, 935 (2007)
T. Mikolov, K. Chen, G. Corrado, J. Dean, Efficient estimation of word representations in vector space, in Workshop paper at International Conference on Learning Representations (ICLR, 2013)
T. Mikolov, I. Sutskever, K. Chen, G.S. Corrado, J. Dean, Distributed representations of words and phrases and their compositionality, in Advances in neural information processing systems, (2013), pp. 3111–3119
L.G. Moyano, J.P. Cárdenas, J. Salcedo, M.L. Mouronte, R.M. Benito, Information transfer dynamics in fixed-pathways networks, Chaos 21, 013126 (2011)
S. Nandanwar, M. Murty, Structural neighborhood based classification of nodes in a network, in Proceedings of the 22th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (ACM, 2016)
O. Narayan, I. Saniee, Large-scale curvature of networks, Phys. Rev. E. 84, 066108 (2011)
L. Niu, X.-Y. Dai, S. Huang, J. Chen, A unified framework for jointly learning distributed representations of word and attributes, in Proceedings of The 7th Asian Conference on Machine Learning (2015), p. 143
D. Nozza, D. Maccagnola, V. Guigue, E. Messina, P. Gallinari, A latent representation model for sentiment analysis in heterogeneous social networks, in International Conference on Software Engineering and Formal Methods (Springer, 2014), pp. 201–213
F. Papadopoulos, M. Kitsak, M.Á. Serrano, M. Boguná, D. Krioukov, Popularity versus similarity in growing networks, Nature 489, 537 (2012)
F. Papadopoulos, D. Krioukov, M. Boguna, A. Vahdat, Greedy forwarding in dynamic scale-free networks embedded in hyperbolic metric spaces, in INFOCOM, 2010 Proceedings IEEE, (IEEE, 2010), p. 1
C.D. Manning, J. Pennington, R. Socher, Glove: Global vectors for word representation, in Proceedings of the 2014 Conference on Empirical Methods in Natural Language (2014)
B. Perozzi, R. Al-Rfou, S. Skiena, Deepwalk: Online learning of social representations, in Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining (ACM, 2014), p. 701
B. Perozzi, V. Kulkarni, S. Skiena, Walklets: Multiscale graph embeddings for interpretable network classification, arXiv:1605.02115 (2016)
X. Ren, W. He, M. Qu, C.R. Voss, H. Ji, J. Han, Label noise reduction in entity typing by heterogeneous partial-label embedding, in Proceedings of the 22th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (ACM, 2016)
S.T. Roweis, L.K. Saul, Nonlinear dimensionality reduction by locally linear embedding, Science 290, 2323 (2000)
P. Sanguansat, Paragraph2vec-based sentiment analysis on social media for business in thailand, in 2016 8th International Conference on Knowledge and Smart Technology (KST) (IEEE, 2016), pp. 175–178
P. Sen, G. Namata, M. Bilgic, L. Getoor, B. Galligher, T. Eliassi-Rad, Collective classification in network data, AI magazine 29, 93 (2008)
M.A. Serrano, D. Krioukov, M. Boguná, Self-similarity of complex networks and hidden metric spaces, Phys. Rev. Lett. 100, 078701 (2008)
Y. Shavitt, T. Tankel, Big-bang simulation for embedding network distances in euclidean space, IEEE/ACM Trans. Networking (TON) 12, 993 (2004)
Y. Shavitt, T. Tankel, On the curvature of the internet and its usage for overlay construction and distance estimation, in INFOCOM 2004. Twenty-third AnnualJoint Conference of the IEEE Computer and Communications Societies (IEEE, 2004), Vol 1
Y. Shavitt, T. Tankel, Hyperbolic embedding of internet graph for distance estimation and overlay construction, IEEE/ACM Trans. Networking (TON) 16, 25 (2008)
T.A.B. Snijders, K. Nowicki, Estimation and prediction for stochastic blockmodels for graphs with latent block structure, J. Classification 14, 75 (1997)
J. Tang, J. Liu, M. Zhang, Q. Mei, Visualizing large-scale and high-dimensional data, in Proceedings of the 25th International Conference on World Wide Web (International World Wide Web Conferences Steering Committee, 2016), p. 287
J. Tang, M. Qu, Q. Mei, Pte: Predictive text embedding through large-scale heterogeneous text networks, in Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (ACM, 2015), p. 1165
J. Tang, M. Qu, M. Wang, M. Zhang, J. Yan, Q. Mei, Line: Large-scale information network embedding, in Proceedings of the 24th International Conference on World Wide Web (ACM, 2015), p. 1067
L. Tang, H. Liu, Relational learning via latent social dimensions, in Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining (ACM, 2009), p. 817
L. Tang, H. Liu, Leveraging social media networks for classification, Data Mining Knowl. Discovery 23, 447 (2011)
J.B. Tenenbaum, V. De Silva, J.C. Langford, A global geometric framework for nonlinear dimensionality reduction, Science 290, 2319 (2000)
F. Tian, B. Gao, Q. Cui, E. Chen, T.-Y. Liu, Learning deep representations for graph clustering, in AAAI (2014), pp. 1293–1299
C. Tu, W. Zhang, Z. Liu, M. Sun, Max-margin deepwalk: Discriminative learning of network representation, in Proceedings of the 25th International Conference on Artificial Intelligence (AAAI Press, 2016)
V. Venkataraman, P. Srinivasan, Graph embedding aided relationship prediction in heterogeneous networks (CS 512 Project Report, 2016)
K. Verbeek, S. Suri, Metric embedding, hyperbolic space, and social networks, in Proceedings of the thirtieth annual symposium on Computational geometry (ACM, 2014), p. 501
O. Vinyals, A. Toshev, S. Bengio, D. Erhan, Show and tell: A neural image caption generator, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015), p. 315
D. Wang, P. Cui, W. Zhu, Structural deep network embedding, in Proceedings of the 22th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (ACM, 2016)
D.R. White, M. Houseman, The navigability of strong ties: Small worlds, tie strength, and network topology, Complexity 8, 72 (2002)
F. Wu, X. Lu, J. Song, S. Yan, Z. M. Zhang, Y. Rui, Y. Zhuang, Learning of multimodal representations with random walks on the click graph, IEEE Trans. Image Proc. 25, 630 (2016)
S. Xiang, F. Nie, C. Zhang, C. Zhang, Nonlinear dimensionality reduction with local spline embedding. IEEE Trans, Knowl. Data Eng. 21, 1285 (2009)
L. Xiaoyi, L.H. Du Nan et al., A deep learning approach to link prediction in dynamic networks, in Proceedings of the 2013 SIAM International Conference on Data Mining (Philadelphia, PA, USA: SIAM, 2013)
P. Yanardag, S. Vishwanathan, Deep graph kernels, in Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (ACM, 2015), pp. 1365–1374
C. Yang, Z. Liu. Comprehend deepwalk as matrix factorization, arXiv:1501.00358 (2015)
C. Yang, Z. Liu, D. Zhao, M. Sun, E.Y. Chang, Network representation learning with rich text information, in Proceedings of the 24th International Joint Conference on Artificial Intelligence, Buenos Aires, Argentina (2015), p. 2111
Z. Yang, W. Cohen, R. Salakhutdinov, Revisiting semi-supervised learning with graph embeddings, in ICML 2016 (2016)
Z. Yang, J. Tang, W. Cohen, Multi-modal bayesian embeddings for learning social knowledge graphs, in Proceedings of International Joint Conference on Artificial Intelligence (IJCAI) (AAAI Press, 2016)
S. Zhai, Z.M. Zhang, Dropout Training of Matrix Factorization and Autoencoder for Link Prediction in Sparse Graphs, (SIAM, 2015), Chap. 51, pp. 451–459
X. Zhao, A. Sala, H. Zheng, B.Y. Zhao, Efficient shortest paths on massive social graphs, in Collaborative Computing: Networking, Applications and Worksharing (CollaborateCom), 2011 7th International Conference on (IEEE, 2011), pp. 77–86
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Moyano, L.G. Learning network representations. Eur. Phys. J. Spec. Top. 226, 499–518 (2017). https://doi.org/10.1140/epjst/e2016-60266-2
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1140/epjst/e2016-60266-2