Abstract
In the last decade, deep learning has reinvigorated the machine learning field. It has solved many problems in computer vision, speech recognition, natural language processing, and other domains with state-of-the-art performances. In these domains, the data is generally represented in the Euclidean space. Various other domains conform to non-Euclidean space, for which a graph is an ideal representation. Graphs are suitable for representing the dependencies and inter-relationships between various entities. Traditionally, handcrafted features for graphs are incapable of providing the necessary inference for various tasks from this complex data representation. Recently, there has been an emergence of employing various advances in deep learning for graph-based tasks (called Graph Neural Networks (GNNs)). This article introduces preliminary knowledge regarding GNNs and comprehensively surveys GNNs in different learning paradigms—supervised, unsupervised, semi-supervised, self-supervised, and few-shot or meta-learning. The taxonomy of each graph-based learning setting is provided with logical divisions of methods falling in the given learning setting. The approaches for each learning task are analyzed from theoretical and empirical standpoints. Further, we provide general architecture design guidelines for building GNN models. Various applications and benchmark datasets are also provided, along with open challenges still plaguing the general applicability of GNNs.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Abadal S, Jain A, Guirado R, López-Alonso J, Alarcón E (2021) Computing graph neural networks: a survey from algorithms to accelerators
Adhikari B, Zhang Y, Ramakrishnan N, Prakash BA (2018) Sub2vec: feature learning for subgraphs. In: Pacific-Asia conference on knowledge discovery and data mining. Springer, Melbourne, Australia. pp 170–182
Afzal MZ, Kölsch A, Ahmed S, Liwicki M (2017) Cutting the error by half: Investigation of very deep CNN and advanced training strategies for document image classification. In: 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), vol 1, pp 883–888. IEEE, Kyoto, Japan. IEEE
Ahmed A, Shervashidze N, Narayanamurthy S, Josifovski V, Smola AJ (2013) Distributed large-scale natural graph factorization. In: Proceedings of the 22nd international conference on World Wide Web, pp 37–48. Association for Computing Machinery, New York, NY, USA
Ahmad W, Zhang Z, Ma X, Hovy E, Chang K-W, Peng N (2019) On difficulties of cross-lingual transfer with order differences: a case study on dependency parsing. In: Proceedings of the 2019 conference of the north american chapter of the association for computational linguistics: human language technologies, volume 1 (Long and Short Papers), pp. 2440–2452. Association for Computational Linguistics, Minneapolis, Minnesota
Albert R, Jeong H, Barabási A-L (1999) Diameter of the world-wide web. Nature 401(6749):130–131
Alibaba: Euler (2021a). https://github.com/alibaba/euler
Alibaba: Graph-learn (2021b) https://github.com/alibaba/graph-learn
Allamanis M, Brockschmidt M, Khademi M (2017) Learning to represent programs with graphs
Audebert N, Herold C, Slimani K, Vidal C (2019) Multimodal deep networks for text and image-based document classification
Bai C, Kumar S, Leskovec J, Metzger M, Nunamaker J, Subrahmanian V (2019) Predicting the visual focus of attention in multi-person discussion videos. In: IJCAI 2019, pp 4504–4510. International Joint Conferences on Artificial Intelligence
Bansal M, Yang J, Karan C, Menden MP, Costello JC, Tang H, Xiao G, Li Y, Allen J, Zhong R et al (2014) A community computational challenge to predict the activity of pairs of compounds. Nat Biotechnol 32(12):1213–1222
Barceló P, Kostylev EV, Monet M, Pérez J, Reutter J, Silva JP (2019) The logical expressiveness of graph neural networks. In: International conference on learning representations, pp 1–21. ICLR, Ethiopia
Bastings J, Titov I, Aziz W, Marcheggiani D, Sima’an K (2017) Graph convolutional encoders for syntax-aware neural machine translation
Battaglia PW, Pascanu R, Lai M, Rezende D, Kavukcuoglu K (2016) Interaction networks for learning about objects, relations and physics
Beck D, Haffari G, Cohn T (2018) Graph-to-sequence learning using gated graph neural networks
Belkin M, Niyogi P (2001) Laplacian eigenmaps and spectral techniques for embedding and clustering. In: Proceedings of the 14th International conference on neural information processing systems: natural and synthetic. NIPS’01, pp 585–591. MIT Press, Cambridge, MA, USA
Berg Rvd, Kipf TN, Welling M (2017) Graph convolutional matrix completion
Bertinetto L, Henriques JF, Valmadre J, Torr P, Vedaldi A (2016) Learning feed-forward one-shot learners. Advances in neural information processing systems 29
Berton L, De Andrade Lopes A (2014) Graph construction based on labeled instances for semi-supervised learning. In: 2014 22nd international conference on pattern recognition, pp 2477–2482. IEEE, Stockholm, Sweden. https://doi.org/10.1109/ICPR.2014.428
Berton L, de Paulo FT, Valejo A, Valverde-Rebaza J, de Andrade LA (2017) Rgcli: Robust graph that considers labeled instances for semi-supervised learning. Neurocomputing 226:238–248
Borgwardt KM, Ong CS, Schönauer S, Vishwanathan S, Smola AJ, Kriegel H-P (2005) Protein function prediction via graph kernels. Bioinformatics 21:47–56
Bruna J, Zaremba W, Szlam A, LeCun Y (2013) Spectral networks and locally connected networks on graphs. arXiv:1312.6203
Bui ND, Yu Y, Jiang L (2021) Infercode: Self-supervised learning of code representations by predicting subtrees. In: 2021 IEEE/ACM 43rd international conference on software engineering (ICSE), pp 1186–1197. IEEE, Madrid, ES. IEEE
Cai D, Lam W (2020) Graph transformer for graph-to-sequence learning. Proc AAAI Conf Artif Intell 34:7464–7471
Cai H, Zheng VW, Chang KC-C (2017) Active learning for graph embedding
Cao S, Lu W, Xu Q (2015) Grarep: Learning graph representations with global structural information. In: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. CIKM ’15, pp. 891–900. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/2806416.2806512
Cao S, Lu W, Xu Q (2016) Deep neural networks for learning graph representations. In: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence. AAAI’16, pp. 1145–1152. AAAI Press, Phoenix, Arizona
Cao Z, Li X, Zhao L (2020) Unsupervised Feature Learning by Autoencoder and Prototypical Contrastive Learning for Hyperspectral Classification
Cao J, Lin X, Guo S, Liu L, Liu T, Wang B (2021) Bipartite graph embedding via mutual information maximization. In: Proceedings of the 14th ACM International Conference on Web Search and Data Mining. WSDM ’21, pp. 635–643. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3437963.3441783
Caron M, Bojanowski P, Joulin A, Douze M (2018) Deep clustering for unsupervised learning of visual features. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y (eds) Computer vision - ECCV 2018. Springer, Cham, pp 139–156
Chami I, Ying Z, Ré C, Leskovec J (2019) Hyperbolic graph convolutional neural networks. Advances in neural information processing systems 32
Che F, Yang G, Zhang D, Tao J, Liu T (2021) Self-supervised graph representation learning via bootstrapping. Neurocomputing 456:88–96
Chen C, Wang H-L, Wu S-H, Huang H, Zou J-L, Chen J, Jiang T-Z, Zhou Y, Wang G-H (2015) Abnormal degree centrality of bilateral putamen and left superior frontal gyrus in schizophrenia with auditory hallucinations: a resting-state functional magnetic resonance imaging study. Chin Med J 128(23):3178
Chen X, Sun Y, Athiwaratkun B, Cardie C, Weinberger K (2018a) Adversarial deep averaging networks for cross-lingual sentiment classification. Trans Assoc Comput Linguist 6:557–570
Chen X, Li L-J, Fei-Fei L, Gupta A (2018b) Iterative visual reasoning beyond convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7239–7248. IEEE, Salt Lake City, UT, USA
Chen B, Sun L, Han X (2018c) Sequence-to-action: End-to-end semantic graph generation for semantic parsing
Chen T, Kornblith S, Norouzi M, Hinton G (2020b) A simple framework for contrastive learning of visual representations. Proceedings of the 37th International Conference on Machine Learning 119:1597–1607
Chen Z, Villar S, Chen L, Bruna J (2019) On the equivalence between graph isomorphism testing and function approximation with gnns
Chen D, Lin Y, Li W, Li P, Zhou J, Sun X (2020a) Measuring and relieving the over-smoothing problem for graph neural networks from the topological view. Proc AAAI Conf Artif Intell 34(04):3438–3445. https://doi.org/10.1609/aaai.v34i04.5747
Cheng S, Zhang L, Jin B, Zhang Q, Lu X (2021) Drug target prediction using graph representation learning via substructures contrast
Choi E, Bahadori MT, Song L, Stewart WF, Sun J (2017) Gram: Graph-based attention model for healthcare representation learning. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. KDD ’17, pp. 787–795. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3097983.3098126
Choi E, Xiao C, Stewart WF, Sun J (2018) Mime: Multilevel medical embedding of electronic health records for predictive healthcare
Choudhary N, Rao N, Katariya S, Subbian K, Reddy CK (2021) Self-supervised hyperboloid representations from logical queries over knowledge graphs. In: Proceedings of the Web Conference 2021. WWW ’21, pp. 1373–1384. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3442381.3449974
Chu Y, Kaushik AC, Wang X, Wang W, Zhang Y, Shan X, Salahub DR, Xiong Y, Wei D-Q (2021) Dti-cdf: a cascade deep forest model towards the prediction of drug-target interactions based on hybrid features. Brief Bioinform 22(1):451–462
Cui G, Zhou J, Yang C, Liu Z (2020) Adaptive graph encoder for attributed graph embedding. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. KDD ’20, pp. 976–985. Association for Computing Machinery, NY, USA
Daiquocnguyen: QGNN. (2021) https://github.com/daiquocnguyen/QGNN
Danielegrattarola: Spektral (2021) https://github.com/danielegrattarola/spektral
Das A, Roy S, Bhattacharya U, Parui SK (2018) Document image classification with intra-domain transfer learning and stacked generalization of deep convolutional neural networks. In: 2018 24th International Conference on Pattern Recognition (ICPR), pp. 3180–3185. IEEE, Beijing, China. IEEE
Data61 C (2018) StellarGraph Machine Learning Library. GitHub
De Cao N, Kipf T (2018) MolGAN: An implicit generative model for small molecular graphs
Debnath AK, Lopez de Compadre RL, Debnath G, Shusterman AJ, Hansch C (1991) Structure–activity relationship of mutagenic aromatic and heteroaromatic nitro compounds correlation with molecular orbital energies and hydrophobicity. J Med Chem 34(2):786–797
DeepGraphLearning: graphvite (2021). https://github.com/DeepGraphLearning/graphvite
Deepmind: Graph_nets (2021). https://github.com/deepmind/graph_nets
Defferrard M, Bresson X, Vandergheynst P (2016) Convolutional neural networks on graphs with fast localized spectral filtering. Adv Neural Inf Process Syst 29:3844–3852
Dehmamy N, Barabási A-L, Yu R (2019) Understanding the representation power of graph neural networks in learning graph topology
Deng W, Zhang B, Zou W, Zhang X, Cheng X, Guan L, Lin Y, Lao G, Ye B, Li X et al (2019) Abnormal degree centrality associated with cognitive dysfunctions in early bipolar disorder. Front Psych 10:140
Dengel A (1993) Initial learning of document structure. In: Proceedings of 2nd International Conference on Document Analysis and Recognition (ICDAR’93), pp. 86–90. IEEE Tsukuba, Japan. IEEE
Denk TI, Reisswig C (2019) Bertgrid: Contextualized embedding for 2d document representation and understanding
Devlin J, Chang M-W, Lee K, Toutanova K (2018) Bert: Pre-training of deep bidirectional transformers for language understanding
Devlin J, Chang M-W, Lee K, Toutanova K (2019) BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171–4186. Association for Computational Linguistics, Minneapolis, Minnesota
Dhillon PS, Talukdar PP, Crammer K (2010) Learning better data representation using inference-driven metric learning. In: Proceedings of the ACL 2010 Conference Short Papers. ACLShort ’10, pp. 377–381. Association for Computational Linguistics, USA
Diligenti M, Frasconi P, Gori M (2003) Hidden tree Markov models for document image classification. IEEE Trans Pattern Anal Mach Intell 25(4):519–523
Dong XL, de Melo G (2019) A robust self-learning framework for cross-lingual text classification. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 6306–6310. Association for Computational Linguistics, Hong Kong, China
Dong Y, Chawla NV, Swami A (2017) Metapath2vec: Scalable representation learning for heterogeneous networks. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 135–144. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3097983.3098036
Du L, Wang Y, Song G, Lu Z, Wang J (2018) Dynamic network embedding: An extended approach for skip-gram based network embedding. In: IJCAI, vol. 2018, pp. 2086–2092. AAAI Press Stockholm, Sweden
Duvenaud D, Maclaurin D, Aguilera-Iparraguirre J, Gómez-Bombarelli R, Hirzel T, Aspuru-Guzik A, Adams RP (2015) Convolutional networks on graphs for learning molecular fingerprints
Dwivedi VP, Joshi CK, Laurent T, Bengio Y (2020) Benchmarking graph neural networks
Edwards H, Storkey A (2016) Towards a neural statistician. arXiv:1606.02185
Fei-Fei L, Fergus R, Perona P (2006) One-shot learning of object categories. IEEE Trans Pattern Anal Mach Intell 28(4):594–611
Feng Y, You H, Zhang Z, Ji R, Gao Y (2019) Hypergraph neural networks. Proc AAAI Conf Artif Intell 33:3558–3565
Feng Y-H, Zhang S-W, Shi J-Y (2020) Dpddi: a deep predictor for drug–drug interactions. BMC Bioinform 21(1):1–15
Fey M, Lenssen JE (2019) Fast graph representation learning with PyTorch Geometric. In: ICLR Workshop on Representation Learning on Graphs and Manifolds, pp. 1–9
Finn C, Abbeel P, Levine S (2017) Model-agnostic meta-learning for fast adaptation of deep networks. In: International Conference on Machine Learning, pp. 1126–1135. PMLR
Fornito A, Zalesky A, Breakspear M (2013) Graph analysis of the human connectome: promise, progress, and pitfalls. Neuroimage 80:426–444
Fouss F, Pirotte A, Renders J-M, Saerens M (2007) Random-walk computation of similarities between nodes of a graph with application to collaborative recommendation. IEEE Trans Knowl Data Eng 19(3):355–369
Fout AM (2017) Protein interface prediction using graph convolutional networks. PhD thesis, Colorado State University
Gao L, Yang H, Zhou C, Wu J, Pan S, Hu Y (2018) Active discriminative network representation learning. In: IJCAI International Joint Conference on Artificial Intelligence, pp. 2142–2148. AAAI Press, Stockholm, Sweden
Garcia V, Bruna J (2017) Few-shot learning with graph neural networks. arXiv:1711.04043
Garg V, Jegelka S, Jaakkola T (2020) Generalization and representational limits of graph neural networks. Proceedings of the 37th International Conference on Machine Learning 119:3419–3430
Giles CL, Bollacker KD, Lawrence S (1998) Citeseer: An automatic citation indexing system. In: Proceedings of the Third ACM Conference on Digital Libraries. DL ’98, pp. 89–98. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/276675.276685
Gilmer J, Schoenholz SS, Riley PF, Vinyals O, Dahl GE (2017) Neural message passing for quantum chemistry. Proceedings of the 34th International Conference on Machine Learning 70:1263–1272
Godwin* J, Keck* T, Battaglia P, Bapst V, Kipf T, Li Y, Stachenfeld K, Veličković P, Sanchez-Gonzalez A (2020) Jraph: A library for graph neural networks in jax. http://github.com/deepmind/jraph
Gori M, Monfardini G, Scarselli F (2005) A new model for learning in graph domains. In: Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005., vol. 2, pp. 729–734. IEEE, Montreal, QC, Canada. IEEE
Grill J-B, Strub F, Altché F, Tallec C, Richemond PH, Buchatskaya E, Doersch C, Pires BA, Guo ZD, Azar MG, et al (2020) Bootstrap your own latent: a new approach to self-supervised learning
Grover A, Leskovec J (2016) node2vec: Scalable feature learning for networks. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 855–864. Association for Computing Machinery, New York, NY, USA
Gu S, Wang X, Shi C, Xiao D Self-supervised graph neural networks for multi-behavior recommendation
Guan C, Zhang Z, Li H, Chang H, Zhang Z, Qin Y, Jiang J, Wang X, Zhu W (2021) AutoGL: A library for automated graph learning. In: ICLR 2021 Workshop on Geometrical and Topological Representation Learning, pp. 1–8. https://openreview.net/forum?id=0yHwpLeInDn
Guo M, Chou E, Huang D-A, Song S, Yeung S, Fei-Fei L (2018) Neural graph matching networks for fewshot 3d action recognition. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 653–669. Springer, Cham
Hamilton WL, Ying R, Leskovec J (2017a) Inductive representation learning on large graphs. Proceedings of the 31st International Conference on Neural Information Processing Systems 30:1025–1035
Hamilton WL, Ying R, Leskovec J (2017b) Representation learning on graphs: Methods and applications
Hammond DK, Vandergheynst P, Gribonval R (2011) Wavelets on graphs via spectral graph theory. Appl Comput Harmon Anal 30(2):129–150
Hassani K, Khasahmadi AH (2020) Contrastive multi-view representation learning on graphs. Proceedings of the 37th International Conference on Machine Learning 119, 4116–4126. PMLR
He R, McAuley J (2016) Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In: Proceedings of the 25th International Conference on World Wide Web, pp. 507–517. International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE
He R, Packer C, McAuley J (2016) Learning compatibility across categories for heterogeneous item recommendation. In: 2016 IEEE 16th International Conference on Data Mining (ICDM), pp. 937–942. IEEE, Barcelona, Spain. IEEE
He K, Fan H, Wu Y, Xie S, Girshick R (2020) Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9729–9738. IEEE Seattle, WA, USA
Henaff M, Bruna J, LeCun Y (2015) Deep convolutional networks on graph-structured data. arXiv:1506.05163
Hjelm RD, Fedorov A, Lavoie-Marchildon S, Grewal K, Bachman P, Trischler A, Bengio Y (2018) Learning deep representations by mutual information estimation and maximization
Hu Z, Fan C, Chen T, Chang K-W, Sun Y (2019a) Pre-training graph neural networks for generic structural feature extraction
Hu W, Liu B, Gomes J, Zitnik M, Liang P, Pande V, Leskovec J (2019b) Strategies for pre-training graph neural networks
Hu Z, Dong Y, Wang K, Chang K-W, Sun Y (2020a) Gpt-gnn: Generative pre-training of graph neural networks. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1857–1867. Association for Computing Machinery, New York, NY, USA
Hu W, Fey M, Zitnik M, Dong Y, Ren H, Liu B, Catasta M, Leskovec J (2020b) Open graph benchmark: Datasets for machine learning on graphs
Hu J, Ruder S, Siddhant A, Neubig G, Firat O, Johnson M (2020c) XTREME: A massively multilingual multi-task benchmark for evaluating cross-lingual generalisation. Proceedings of the 37th International Conference on Machine Learning 119, 4411–4421
Hu S, Xiong Z, Qu M, Yuan X, Côté M-A, Liu Z, Tang J (2020d) Graph policy network for transferable active learning on graphs
Hu Y, Li X, Wang Y, Wu Y, Zhao Y, Yan C, Yin J, Gao Y (2021) Adaptive hypergraph auto-encoder for relational data clustering. IEEE Transactions on Knowledge and Data Engineering
Huang C, Xu H, Xu Y, Dai P, Xia L, Lu M, Bo L, Xing H, Lai X, Ye Y (2021a) Knowledge-aware coupled graph neural network for social recommendation. Proc AAAI Conf Artif Intell 35:4115–4122
Huang H, Shi R, Zhou W, Wang X, Jin H, Fu X (2021b) Temporal heterogeneous information network embedding. In: IJCAI, pp. 1470–1476
Hwang D, Park J, Kwon S, Kim K-M, Ha J-W, Kim HJ (2020) Self-supervised auxiliary learning with meta-paths for heterogeneous graphs
Ioannidis VN, Marques AG, Giannakis GB (2019) A recurrent graph neural network for multi-relational data. In: ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 8157–8161. IEEE, Brighton, UK. IEEE
Jain A, Zamir AR, Savarese S, Saxena A (2016) Structural-rnn: Deep learning on spatio-temporal graphs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5308–5317. IEEE, Las Vegas, NV, USA
Jamasb AR, Lio P, Blundell T (2020) Graphein - a python library for geometric deep learning and network analysis on protein structures. https://doi.org/10.1101/2020.07.15.204701
Jiao Y, Xiong Y, Zhang J, Zhang Y, Zhang T, Zhu Y (2020) Sub-graph contrast for scalable self-supervised graph representation learning. In: 2020 IEEE International Conference on Data Mining (ICDM), pp. 222–231. IEEE, Sorrento, Italy. IEEE
Ji H, Wang X, Shi C, Wang B, Yu P (2021) Heterogeneous graph propagation network. IEEE Transactions on Knowledge and Data Engineering
Jin W, Derr T, Liu H, Wang Y, Wang S, Liu Z, Tang J (2020) Self-supervised learning on graphs: deep insights and new direction
Jin W, Liu X, Zhao X, Ma Y, Shah N, Tang J (2021a) Automated self-supervised learning for graphs
Jin M, Zheng Y, Li Y-F, Gong C, Zhou C, Pan S (2021b) Multi-scale contrastive Siamese networks for self-supervised graph representation learning
Johnson DD (2016) Learning Graphical State Transitions. In: Proceedings of 5th International Conference on Learning Representations, pp. 1–19. ICLR, Palais des Congrès Neptune, Toulon, France
Johnson J, Gupta A, Fei-Fei L (2018) Image generation from scene graphs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1219–1228. IEEE, Salt Lake City, UT, USA
Kang L, Kumar J, Ye P, Li Y, Doermann D (2014) Convolutional neural networks for document image classification. In: 2014 22nd International Conference on Pattern Recognition, pp. 3168–3172. IEEE Stockholm, Sweden. IEEE
Kawamoto T, Tsubaki M, Obuchi T (2019) Mean-field theory of graph neural networks in graph partitioning. J Stat Mech 2019(12):124007
Kearnes S, McCloskey K, Berndl M, Pande V, Riley P (2016) Molecular graph convolutions: moving beyond fingerprints. J Comput Aided Mol Des 30(8):595–608
Keriven N, Peyré G (2019) Universal invariant and equivariant graph neural networks. Adv Neural Inf Process Syst 32:7092–7101
Kim D, Oh A (2020) How to find your friendly neighborhood: Graph attention design with self-supervision. In: International Conference on Learning Representations, pp. 1–25. ICLR, Vienna, Austria
Kim J, Kim T, Kim S, Yoo CD (2019) Edge-labeling graph neural network for few-shot learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11–20
Kingma DP, Welling M (2013) Auto-encoding variational bayes. arXiv:1312.6114
Kipf TN, Welling M (2016a) Semi-supervised classification with graph convolutional networks
Kipf TN, Welling M (2016b) Variational graph auto-encoders
Klicpera J, Weißenberger S, Günnemann S (2019) Diffusion improves graph learning. Adv Neural Inf Process Syst 32:13354–13366
Knyazev B, Taylor GW, Amer MR (2019) Understanding attention and generalization in graph neural networks
Kumar S, Hamilton WL, Leskovec J, Jurafsky D (2018) Community interaction and conflict on the web. In: Proceedings of the 2018 World Wide Web Conference on World Wide Web, pp. 933–943. International World Wide Web Conferences Steering Committee
Kumar S, Zhang X, Leskovec J (2019) Predicting dynamic embedding trajectory in temporal interaction networks. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1269–1278. ACM
Kunal K, Dhar T, Madhusudan M, Poojary J, Sharma A, Xu W, Burns SM, Hu J, Harjani R, Sapatnekar SS (2020) Gana: Graph convolutional network based automated netlist annotation for analog circuits. In: 2020 Design, Automation & Test in Europe Conference & Exhibition (DATE), pp. 55–60. IEEE
Lample G, Conneau A (2019) Cross-lingual language model pretraining
Landrieu L, Simonovsky M (2018) Large-scale point cloud semantic segmentation with superpoint graphs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4558–4567. IEEE Salt Lake City, UT, USA
Leskovec J, Krevl A (2014) SNAP Datasets: Stanford large network dataset collection. http://snap.stanford.edu/data
Leskovec J, Kleinberg J, Faloutsos C (2005) Graphs over time: densification laws, shrinking diameters and possible explanations. In: Proceedings of the Eleventh ACM SIGKDD International Conference on Knowledge Discovery in Data Mining, pp. 177–187
Leskovec J, Adamic LA, Huberman BA (2007) The dynamics of viral marketing. ACM Trans Web (TWEB) 1(1):5
Leskovec J, Lang K, Dasgupta A, Mahoney M (2008) Community structure in large networks: Natural cluster sizes and the absence of large well-defined clusters. arXiv:0810.1355
Leskovec J, Lang KJ, Dasgupta A, Mahoney MW (2009) Community structure in large networks: natural cluster sizes and the absence of large well-defined clusters. Internet Math 6(1):29–123
Leskovec J, Huttenlocher D, Kleinberg J (2010) Signed networks in social media. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp. 1361–1370
Levie R, Huang W, Bucci L, Bronstein M.M, Kutyniok G (2019) Transferability of spectral graph convolutional neural networks
Lewis D, Agam G, Argamon S, Frieder O, Grossman D, Heard J (2006) Building a test collection for complex document information processing. In: Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 665–666. Association for Computing Machinery, New York, NY, USA
Li MM, Zitnik M (2021) Deep contextual learners for protein networks
Li Y, Tarlow D, Brockschmidt M, Zemel R (2015) Gated graph sequence neural networks
Li Y, Yu R, Shahabi C, Liu Y (2017) Diffusion convolutional recurrent neural network: data-driven traffic forecasting
Li Q, Han Z, Wu X-M (2018a) Deeper insights into graph convolutional networks for semi-supervised learning. In: Thirty-second AAAI conference on artificial intelligence, pp. 3538–3545. AAAI Press, New Orleans, USA
Li Y, Vinyals O, Dyer C, Pascanu R, Battaglia P (2018b) Learning deep generative models of graphs
Li Z, Chen Q, Koltun V (2018c) Combinatorial optimization with graph convolutional networks and guided tree search
Li Y, Ouyang W, Zhou B, Shi J, Zhang C, Wang X (2018d) Factorizable net: an efficient subgraph-based framework for scene graph generation. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y (eds) Computer vision—ECCV 2018. Springer, Cham, pp 346–363
Li P, Wang J, Qiao Y, Chen H, Yu Y, Yao X, Gao P, Xie G, Song S (2020a) Learn molecular representations from large-scale unlabeled molecules for drug discovery. arXiv:2012.11175
Li Z, Kumar M, Headden W, Yin B, Wei Y, Zhang Y, Yang Q (2020c) Learn to cross-lingual transfer with meta graph learning across heterogeneous languages. In: Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), pp. 2290–2301. Association for computational linguistics
Li S, Xu F, Wang R, Zhong S (2021a) Self-supervised incremental deep graph learning for ethereum phishing scam detection
Li Y, Jin W, Xu H, Tang J (2020b) Deeprobust: a pytorch library for adversarial attacks and defenses. arXiv:2005.06149
Li I, Yan V, Li T, Qu R, Radev D (2021b) Unsupervised cross-domain prerequisite chain learning using variational graph autoencoders
Li P, Wang J, Qiao Y, Chen H, Yu Y, Yao X, Gao P, Xie G, Song S (2021c) An effective self-supervised framework for learning expressive molecular global representations to drug discovery. Brief Bioinform 22(6):109
Liu Y, Lee J, Park M, Kim S, Yang E, Hwang SJ, Yang Y (2018) Learning to propagate labels: transductive propagation network for few-shot learning. arXiv:1805.10002
Li S, Zhou J, Xu T, Huang L, Wang F, Xiong H, Huang W, Dou D, Xiong H (2021d) Structure-aware interactive graph neural networks for the prediction of protein–ligand binding affinity. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 975–985
Lin J, Cai Q, Lin M (2021a) Multi-label classification of fundus images with graph convolutional network and self-supervised learning. IEEE Signal Process Lett 28:454–458
Lin Q, Zhu F-Y, Shu Y-Q, Zhu P-W, Ye L, Shi W-Q, Min Y-L, Li B, Yuan Q, Shao Y (2021b) Altered brain network centrality in middle-aged patients with retinitis pigmentosa: a resting-state functional magnetic resonance imaging study. Brain Behav 11(2):01983
Linmei H, Yang T, Shi C, Ji H, Li X (2019) Heterogeneous graph attention networks for semi-supervised short text classification. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pp. 4821–4830. Association for Computational Linguistics, Hong Kong, China
Liu Q, Nickel M, Kiela D (2019a) Hyperbolic graph neural networks. Advances in Neural Information Processing Systems 32
Liu L, Zhou T, Long G, Jiang J, Yao L, Zhang C (2019b) Prototype propagation networks (PPN) for weakly-supervised few-shot learning on category graph
Liu N, Tan Q, Li Y, Yang H, Zhou J, Hu X (2019c) Is a single vector enough? exploring node polysemy for network embedding. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 932–940. Association for Computing Machinery, New York, NY, USA
Liu Z, Huang C, Yu Y, Fan B, Dong J (2020a) Fast attributed multiplex heterogeneous network embedding. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pp. 995–1004
Liu M, Zhu K, Gu J, Shen L, Tang X, Sun N, Pan DZ (2020b) Towards decrypting the art of analog layout: Placement quality prediction via transfer learning. In: 2020 Design, Automation & Test in Europe Conference & Exhibition (DATE), pp. 496–501. IEEE
Liu Q, Hu Z, Jiang R, Zhou M (2020c) Deepcdr: a hybrid graph convolutional network for predicting cancer drug response. Bioinformatics 36:911–918
Liu Z, Li X, You Z, Yang T, Fan W, Yu P (2021a) Medical triage chatbot diagnosis improvement via multi-relational hyperbolic graph neural network. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1965–1969
Liu Y, Pan S, Jin M, Zhou C, Xia F, Yu PS (2021b) Graph self-supervised learning: a survey
Liu M, Turner WJ, Kokai GF, Khailany B, Pan DZ, Ren H (2021c) Parasitic-aware analog circuit sizing with graph neural networks and bayesian optimization. In: 2021 Design, Automation & Test in Europe Conference & Exhibition (DATE), pp. 1372–1377. IEEE
Liu X, Zhang F, Hou Z, Mian L, Wang Z, Zhang J, Tang J (2021d) Self-supervised learning: generative or contrastive. IEEE Trans Knowl Data Eng Early Access. https://doi.org/10.1109/TKDE.2021.3090866
Liu Y, Li M, Li X, Giunchiglia F, Feng X, Guan R (2022) Few-shot node classification on attributed networks with graph meta-learning. In: Proceedings of the 45th international ACM SIGIR conference on research and development in information retrieval, pp. 471–481
Long X, Little G, Treit S, Beaulieu C, Gong G, Lebel C (2020) Altered brain white matter connectome in children and adolescents with prenatal alcohol exposure. Brain Struct Funct 225(3):1123–1133
Loukas A (2019) What graph neural networks cannot learn: depth vs width
Lu C, Liu Q, Wang C, Huang Z, Lin P, He L (2019) Molecular property prediction: a multilevel quantum interactions modeling perspective. Proc AAAI Conf Artif Intell 33:1052–1060
Maiya AS (2020) ktrain: a low-code library for augmented machine learning. arXiv:2004.10703
Mallat S (1999) A wavelet tour of signal processing. Elsevier, Amsterdam
Manessi F, Rozza A (2021) Graph-based neural network models with multiple self-supervised auxiliary tasks. Pattern Recogn Lett 148:15–21
Marcheggiani D, Perez-Beltrachini L (2018) Deep graph convolutional encoders for structured data to text generation. arXiv:1810.09995
Marcheggiani D, Bastings J, Titov I (2018) Exploiting semantics in neural machine translation with graph convolutional networks. arXiv:1804.08313
Ma Y, Ren H, Khailany B, Sikka H, Luo L, Natarajan K, Yu B (2019) High performance graph convolutional networks with applications in testability analysis. In: Proceedings of the 56th Annual Design Automation Conference 2019, pp. 1–6
Maron H, Ben-Hamu H, Shamir N, Lipman Y (2018) Invariant and equivariant graph networks
Maron H, Fetaya E, Segol N, Lipman Y (2019) On the universality of invariant networks. In: International Conference on Machine Learning, pp. 4363–4371. PMLR
Mavromatis C, Karypis G (2020) Graph InfoClust: Leveraging cluster-level node information for unsupervised graph representation learning
McAuley JJ, Leskovec J (2012) Learning to discover social circles in ego networks. In: NIPS, vol. 2012, pp. 548–56. Citeseer
McAuley J, Pandey R, Leskovec J (2015a) Inferring networks of substitutable and complementary products. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 785–794. Association for Computing Machinery, New York, NY, USA
McAuley J, Targett C, Shi Q, Van Den Hengel A (2015b) Image-based recommendations on styles and substitutes. In: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 43–52
McCallum AK, Nigam K, Rennie J, Seymore K (2000) Automating the construction of internet portals with machine learning. Inf Retrieval 3(2):127–163
McPherson M, Smith-Lovin L, Cook JM (2001) Birds of a feather: homophily in social networks. Ann Rev Sociol 27(1):415–444
Merkwirth C, Lengauer T (2005) Automatic generation of complementary descriptors with molecular graph networks. J Chem Inf Model 45(5):1159–1168
Micheli A, Sona D, Sperduti A (2004) Contextual processing of structured data by recursive cascade correlation. IEEE Trans Neural Netw 15(6):1396–1410
Mikolov T, Chen K, Corrado G, Dean J (2013a) Efficient estimation of word representations in vector space
Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J (2013b) Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems, pp. 3111–3119
Mirhoseini A, Goldie A, Yazgan M, Jiang JW, Songhori E, Wang S, Lee Y-J, Johnson E, Pathak O, Nazi A et al (2021) A graph placement methodology for fast chip design. Nature 594(7862):207–212
Mislove A, Koppula HS, Gummadi KP, Druschel P, Bhattacharjee B (2008) Growth of the flickr social network. In: Proceedings of the First Workshop on Online Social Networks. WOSN ’08, pp. 25–30. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/1397735.1397742
Monti F, Bronstein MM, Bresson X (2017) Geometric matrix completion with recurrent multi-graph neural networks
Morris C, Ritzert M, Fey M, Hamilton WL, Lenssen JE, Rattan G, Grohe M (2019) Weisfeiler and leman go neural: higher-order graph neural networks. Proc AAAI Conf Artif Intell 33(01):4602–4609
Narasimhan M, Lazebnik S, Schwing AG (2018) Out of the box: Reasoning with graph convolution nets for factual visual question answering
Neudorf J, Ekstrand C, Kress S, Borowsky R (2020) Brain structural connectivity predicts brain functional complexity: diffusion tensor imaging derived centrality accounts for variance in fractal properties of functional magnetic resonance imaging signal. Neuroscience 438:1–8
Newman ME (2005) A measure of betweenness centrality based on random walks. Soc Netw 27(1):39–54
Nguyen TH, Grishman R (2018) Graph convolutional networks with argument-aware pooling for event detection. In: Thirty-second AAAI Conference on Artificial Intelligence, pp. 5900–5907
Nguyen T, Le H, Quinn TP, Nguyen T, Le TD, Venkatesh S (2021) Graphdta: predicting drug–target binding affinity with graph neural networks. Bioinformatics 37(8):1140–1147
Nt H, Maehara T (2019) Revisiting graph neural networks: All we have is low-pass filters
Okuda M, Satoh S, Sato Y, Kidawara Y (2021) Unsupervised common particular object discovery and localization by analyzing a match graph. In: ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1540–1544. IEEE
Oono K, Suzuki T (2019) Graph neural networks exponentially lose expressive power for node classification
Oord A, Li Y, Vinyals O (2018) Representation learning with contrastive predictive coding
Opolka FL, Solomon A, Cangea C, Veličković P, Liò P (2019) Spatio-temporal deep graph infomax
Ouali Y, Hudelot C, Tami M (2020) An overview of deep semi-supervised learning
Ou M, Cui P, Pei J, Zhang Z, Zhu W (2016) Asymmetric transitivity preserving graph embedding. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1105–1114. Association for Computing Machinery, New York, NY, USA
Ozaki K, Shimbo M, Komachi M, Matsumoto Y (2011) Using the mutual k-nearest neighbor graphs for semi-supervised classification on natural language data. In: Proceedings of the Fifteenth Conference on Computational Natural Language Learning, pp. 154–162
PaddlePaddle: PGL (2021) https://github.com/PaddlePaddle/PGL
Page L, Brin S, Motwani R, Winograd T (1999) The pagerank citation ranking: bringing order to the web. Technical report, Stanford InfoLab
Pan S, Hu R, Fung S-F, Long G, Jiang J, Zhang C (2019) Learning graph embedding with adversarial training methods. IEEE Trans Cybern 50(6):2475–2487
Pan S, Hu R, Long G, Jiang J, Yao L, Zhang C (2018) Adversarially regularized graph autoencoder for graph embedding. In: Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18, pp. 2609–2615. International Joint Conferences on Artificial Intelligence Organization
Paranjape A, Benson AR, Leskovec J (2017) Motifs in temporal networks. In: Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, pp. 601–610
Park J, Cho J, Chang HJ, Choi JY (2021) Unsupervised hyperbolic representation learning via message passing auto-encoders. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5516–5526
Park C, Kim D, Han J, Yu H (2020) Unsupervised attributed multiplex network embedding. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 5371–5378
Park J, Lee M, Chang HJ, Lee K, Choi JY (2019) Symmetric graph convolutional autoencoder for unsupervised graph representation learning. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6519–6528
Peng Z, Dong Y, Luo M, Wu X-M, Zheng Q (2020a) Self-supervised graph representation learning via global context prediction
Peng Z, Huang W, Luo M, Zheng Q, Rong Y, Xu T, Huang J (2020b) Graph representation learning via graphical mutual information maximization. In: Proceedings of The Web Conference 2020, pp. 259–270
Perozzi B, Al-Rfou R, Skiena S (2014) Deepwalk: Online learning of social representations. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 701–710. Association for Computing Machinery New York, NY, USA
Pise NN, Kulkarni P (2008) A survey of semi-supervised learning methods. In: 2008 International Conference on Computational Intelligence and Security, vol. 2, pp. 30–34. IEEE
Prakash VJ, Nithya DL (2014) A survey on semi-supervised learning techniques
Qi X, Liao R, Jia J, Fidler S, Urtasun R (2017) 3d graph neural networks for rgbd semantic segmentation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5199–5208
Qiu J, Chen Q, Dong Y, Zhang J, Yang H, Ding M, Wang K, Tang J (2020) Gcc: Graph contrastive coding for graph neural network pre-training. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1150–1160. Association for Computing Machinery, New York, NY, USA
Qiu J, Dong Y, Ma H, Li J, Wang K, Tang J (2018a) Network embedding as matrix factorization: Unifying deepwalk, line, pte, and node2vec. In: Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, pp. 459–467
Qiu J, Tang J, Ma H, Dong Y, Wang K, Tang J (2018b) Deepinf: Social influence prediction with deep learning. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2110–2119. Association for Computing Machinery
Qi S, Wang W, Jia B, Shen J, Zhu S-C (2018) Learning human-object interactions by graph parsing neural networks. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 401–417. Springer Cham
Raizman R, Tavor I, Biegon A, Harnof S, Hoffmann C, Tsarfaty G, Fruchter E, Tatsa-Laur L, Weiser M, Livny A (2020) Traumatic brain injury severity in a network perspective: a diffusion MRI based connectome study. Sci Rep 10(1):1–12
Rakesh V, Wang S, Shu K, Liu H (2019) Linked variational autoencoders for inferring substitutable and supplementary items. In: Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, pp. 438–446
Ravi S, Larochelle H (2016) Optimization as a model for few-shot learning
Ren H, Kokai GF, Turner WJ, Ku T-S (2020a) Paragraph: Layout parasitics and device parameter prediction using graph neural networks. In: 2020 57th ACM/IEEE Design Automation Conference (DAC), pp. 1–6. IEEE
Ren Y, Liu B, Huang C, Dai P, Bo L, Zhang J (2020b) Hdgi: An unsupervised graph neural network for representation learning in heterogeneous graph. In: AAAI Workshop, pp. 1638–1645
Ribeiro LF, Saverese PH, Figueiredo DR (2017) struc2vec: Learning node representations from structural identity. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 385–394. Association for Computing Machinery, New York, NY, USA
Rohani N, Eslahchi C, Katanforoush A (2020) Iscmf: integrated similarity-constrained matrix factorization for drug–drug interaction prediction. Netw Model Anal Health Inform Bioinform 9(1):1–8
Rohban MH, Rabiee HR (2012) Supervised neighborhood graph construction for semi-supervised classification. Pattern Recogn 45(4):1363–1372
Rong Y, Bian Y, Xu T, Xie W, Wei Y, Huang W, Huang J (2020) Self-supervised graph transformer on large-scale molecular data
Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326
Rozemberczki B, Scherer P, He Y, Panagopoulos G, Riedel A, Astefanoaei M, Kiss O, Beres F, Lopez G, Collignon N, Sarkar R (2021) PyTorch geometric temporal: spatiotemporal signal processing with neural machine learning models
Ruiz L, Chamon L, Ribeiro A (2020) Graphon neural networks and the transferability of graph neural networks. Advances in Neural Information Processing Systems 33
Sakhuja A (2021) Unsupervised learning of latent edge types from multi-relational data. PhD thesis, Applied sciences: school of computing science
Salakhutdinov R, Tenenbaum J, Torralba A (2012) One-shot learning with a hierarchical nonparametric bayesian model. In: Proceedings of ICML Workshop on Unsupervised and Transfer Learning, pp. 195–206. JMLR Workshop and Conference Proceedings
Santoro A, Bartunov S, Botvinick M, Wierstra D, Lillicrap T (2016) Meta-learning with memory-augmented neural networks. In: International Conference on Machine Learning, pp. 1842–1850. PMLR
Sato R (2020) A survey on the expressive power of graph neural networks
Scarselli F, Gori M, Tsoi AC, Hagenbuchner M, Monfardini G (2008) The graph neural network model. IEEE Trans Neural Netw 20(1):61–80
Scarselli F, Tsoi AC, Hagenbuchner M (2018) The Vapnik-Chervonenkis dimension of graph and recursive neural networks. Neural Netw 108:248–259
Schlichtkrull M, Kipf TN, Bloem P, Van Den Berg R, Titov I, Welling M (2018) Modeling relational data with graph convolutional networks. In: European Semantic Web Conference, pp. 593–607. Springer
Sen P, Namata G, Bilgic M, Getoor L, Galligher B, Eliassi-Rad T (2008) Collective classification in network data. AI Magn 29(3):93–93
SeongokRyu: Graph-neural-networks (2021). https://github.com/SeongokRyu/Graph-neural-networks
Shchur O, Mumme M, Bojchevski A, Günnemann S (2018) Pitfalls of graph neural network evaluation
Shin C, Doermann D, Rosenfeld A (2001) Classification of document pages using structure-based features. Int J Doc Anal Recogn 3(4):232–247
Song Z, Yang X, Xu Z, King I (2021) Graph-based semi-supervised learning: a comprehensive review
Song L, Zhang Y, Wang Z, Gildea D (2018) A graph-to-sequence model for AMR-to-text generation
Sperduti A, Starita A (1997) Supervised neural networks for the classification of structures. IEEE Trans Neural Netw 8(3):714–735
Spitzer F (2013) Principles of random walk, vol 34. Springer, Cham
Subramonian A (2021) MOTIF-driven contrastive learning of graph representations. AAAI 35(18):15980–15981
Sun F-Y, Hoffmann J, Verma V, Tang J (2019) Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization
Sun Q, Li J, Peng H, Wu J, Ning Y, Yu PS, He L (2021b) Sugar: Subgraph neural network with reinforcement pooling and self-supervised mutual information mechanism. In: Proceedings of the web conference 2021, pp. 2081–2091
Sun K, Lin Z, Zhu Z (2020) Multi-stage self-supervised learning for graph convolutional networks on graphs with few labeled nodes. In: Proceedings of the AAAI conference on artificial intelligence, vol. 34, pp. 5892–5899
Sun Y, Shan Y, Tang C, Hu Y, Dai Y, Yu J, Sun J, Huang F, Si L (2021c) Unsupervised learning of deterministic dialogue structure with edge-enhanced graph auto-encoder. In: Proceedings of the AAAI conference on artificial intelligence, vol. 35, pp. 13869–13877
Sun X, Yin H, Liu B, Chen H, Cao J, Shao Y, Viet Hung NQ (2021d) Heterogeneous hypergraph embedding for graph classification. In: Proceedings of the 14th ACM international conference on web search and data mining, pp. 725–733
Sung F, Yang Y, Zhang L, Xiang T, Torr PH, Hospedales TM (2018) Learning to compare: Relation network for few-shot learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1199–1208
Sun L, Zhang Z, Zhang J, Wang F, Peng H, Su S, Philip SY (2021a) Hyperbolic variational graph neural network for modeling dynamic graphs. Proc AAAI Conf Artif Intell 35:4375–4383
Svjan5: GNNs-for-NLP (2021). https://github.com/svjan5/GNNs-for-NLP
Taheri A, Gimpel K, Berger-Wolf T (2019) Learning to represent the evolution of dynamic graphs with recurrent models. In: Companion Proceedings of The 2019 World Wide Web Conference. WWW ’19, pp. 301–307. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3308560.3316581
Taherkhani F, Kazemi H, Nasrabadi NM (2019) Matrix completion for graph-based deep semi-supervised learning. Proc AAAI Conf Artif Intell 33:5058–5065
Tang J, Liu H (2012) Unsupervised feature selection for linked social media data. In: Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 904–912
Tang J, Qu M, Wang M, Zhang M, Yan J, Mei Q (2015b) Line: Large-scale information network embedding. In: Proceedings of the 24th International Conference on World Wide Web, pp. 1067–1077. International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE
Tang J, Zhang J, Yao L, Li J, Zhang L, Su Z (2008) Arnetminer: Extraction and mining of academic social networks. In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. KDD ’08, pp. 990–998. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/1401890.1402008
Tang J, Qu M, Mei Q (2015a) Pte: Predictive text embedding through large-scale heterogeneous text networks. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1165–1174. Association for Computing Machinery, New York, NY, USA
Tatonetti NP, Patrick PY, Daneshjou R, Altman RB (2012) Data-driven prediction of drug effects and interactions. Sci Transl Med 4(125):125
Te G, Hu W, Zheng A, Guo Z (2018) Rgcnn: Regularized graph cnn for point cloud segmentation. In: Proceedings of the 26th ACM International Conference on Multimedia, pp. 746–754. Association for Computing Machinery, New York, NY, USA
Thudm: Cogdl (2021). https://github.com/THUDM/cogdl
Thunlp: OpenNE (2021). https://github.com/thunlp/OpenNE/tree/pytorch
Tu K, Cui P, Wang X, Yu PS, Zhu W (2018) Deep recursive network embedding with regular equivalence. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2357–2366. Association for Computing Machinery, New York, NY, USA
Turkiewicz J, Bhatt RR, Wang H, Vora P, Krause B, Sauk JS, Jacobs JP, Bernstein CN, Kornelsen J, Labus JS et al (2021) Altered brain structural connectivity in patients with longstanding gut inflammation is correlated with psychological symptoms and disease duration. NeuroImage 30:102613
Van Engelen JE, Hoos HH (2020) A survey on semi-supervised learning. Mach Learn 109(2):373–440
Veličković P, Cucurull G, Casanova A, Romero A, Lio P, Bengio Y (2017) Graph attention networks
Velickovic P, Fedus W, Hamilton WL, Liò P, Bengio Y, Hjelm RD (2019) Deep graph infomax. ICLR 2(3):4
Verma S, Zhang Z-L (2019) Stability and generalization of graph convolutional neural networks. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1539–1548. Association for Computing Machinery, New York, NY, USA
Vincent P, Larochelle H, Bengio Y, Manzagol P-A (2008) Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th International Conference on Machine Learning, pp. 1096–1103
Vinyals O, Blundell C, Lillicrap T, Wierstra D, et al. (2016) Matching networks for one shot learning. Advances in neural information processing systems 29
Wan X (2009) Co-training for cross-lingual sentiment classification. In: Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pp. 235–243
Wang M, Zheng D, Ye Z, Gan Q, Li M, Song X, Zhou J, Ma C, Yu L, Gai Y, Xiao T, He T, Karypis G, Li J, Zhang Z (2021d) DGL. https://github.com/dmlc/dgl
Wang C, Pan S, Long G, Zhu X, Jiang J (2017) Mgae: Marginalized graph autoencoder for graph clustering. In: Proceedings of the 2017 ACM on conference on information and knowledge management, pp. 889–898
Wang H, Wang K, Yang J, Shen L, Sun N, Lee H-S, Han S (2020) Gcn-rl circuit designer: Transferable transistor sizing with graph neural networks and reinforcement learning. In: 2020 57th ACM/IEEE design automation conference (DAC), pp. 1–6. IEEE
Wang Y, Sun Y, Liu Z, Sarma SE, Bronstein MM, Solomon JM (2019a) Dynamic graph CNN for learning on point clouds. Acm Trans Graph 38(5):1–12
Wan S, Pan S, Yang J, Gong C (2020) Contrastive and generative graph convolutional networks for graph-based semi-supervised learning
Wang Z, Jiang Z, Ren Z, Tang J, Yin D (2018) A path-constrained framework for discriminating substitutable and complementary products in e-commerce. In: Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, pp. 619–627
Wang Z, Liu X, Yang P, Liu S, Wang Z (2021b) Cross-lingual text classification with heterogeneous graph neural network
Wang P, Agarwal K, Ham C, Choudhury S, Reddy CK (2021a) Self-supervised learning of contextual embeddings for link prediction in heterogeneous networks. In: Proceedings of the Web Conference 2021, pp. 2946–2957
Wang D, Cui P, Zhu W (2016) Structural deep network embedding. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1225–1234. Association for Computing Machinery, New York, NY, USA
Wang Y, Fass J, Stern C, hodera J (2019b) Luolibrary for graph neural networks in jaxK. Graph nets for partial charge prediction. arXiv:1909.07903
Wang H, Xu T, Liu Q, Lian D, Chen E, Du D, Wu H, Su W (2019c) MCNE: an end-to-end framework for learning multiple conditional network representations of social network. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 1064–1072. Association for computing machinery, New York, NY, USA
Wang Y, Min Y, Chen X, Wu J (2021c) Multi-view graph contrastive representation learning for drug-drug interaction prediction. In: Proceedings of the Web Conference 2021, pp. 2921–2933
Wink AM, Tijms BM, Ten Kate M, Raspor E, de Munck JC, Altena E, Ecay-Torres M, Clerigue M, Estanga A, Garcia-Sebastian M et al (2018) Functional brain network centrality is related to APOE genotype in cognitively normal elderly. Brain Behav 8(9):01080
Wu F, Souza A, Zhang T, Fifty C, Yu T, Weinberger K (2019) Simplifying graph convolutional networks. Proceedings of the 36th international conference on machine learning, vol 97, pp 6861–6871. PMLR
Wu Y, Song Y, Huang H, Ye F, Xie X, Jin H (2021a) Enhancing graph neural networks via auxiliary training for semi-supervised node classification. Knowl-Based Syst 220:106884
Wu Z, Pan S, Chen F, Long G, Zhang C, Yu PS (2021b) A comprehensive survey on graph neural networks. IEEE Trans Neural Netw Learn Syst 32(1):4–24. https://doi.org/10.1109/TNNLS.2020.2978386
Wu X, Cheng Q (2021) Deepened graph auto-encoders help stabilize and enhance link prediction
Wu S, Dredze M (2019) Beto, bentz, becas: The surprising cross-lingual effectiveness of BERT. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 833–844. Association for Computational Linguistics
Wu J, Wang X, Feng F, He X, Chen L, Lian J, Xie X (2021c) Self-supervised graph learning for recommendation. In: Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval, pp. 726–735
Xu X, Pang G, Wu D, Shang M (2022) Joint hyperbolic and Euclidean geometry contrastive graph neural networks. Inf Sci
Xu K, Hu W, Leskovec J, Jegelka S (2018) How powerful are graph neural networks?
Xu B, Shen H, Cao Q, Qiu Y, Cheng X (2019b) Graph wavelet neural network
Xu Q-H, Li Q-Y, Yu K, Ge Q-M, Shi W-Q, Li B, Liang R-B, Lin Q, Zhang Y-Q, Shao Y (2020b) Altered brain network centrality in patients with diabetic optic neuropathy: a resting-state FMRI study. Endocr Pract 26(12):1399–1405
Xu B, Lin Y, Tang X, Li S, Shen L, Sun N, Pan DZ (2019a) Wellgan: Generative-adversarial-network-guided well generation for analog/mixed-signal circuit layout. In: 2019 56th ACM/IEEE Design Automation Conference (DAC), pp. 1–6. IEEE
Xu D, Zhu Y, Choy CB, Fei-Fei L (2017) Scene graph generation by iterative message passing. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5410–5419
Xu Y, Li M, Cui L, Huang S, Wei F, Zhou M (2020a) Layoutlm: Pre-training of text and layout for document image understanding. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1192–1200. Association for Computing Machinery, New York, NY, USA
Yang L, Li L, Zhang Z, Zhou X, Zhou E, Liu Y (2020b) Dpgn: Distribution propagation graph network for few-shot learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 13390–13399
Yan S, Xiong Y, Lin D (2018) Spatial temporal graph convolutional networks for skeleton-based action recognition. In: Thirty-second AAAI conference on artificial intelligence, pp. 744–7452. AAAI Press New Orleans, USA
Yang Z, Cohen W, Salakhudinov R (2016) Revisiting semi-supervised learning with graph embeddings. In: Proceedings of the 33rd international conference on machine learning, vol 48, pp 40–48. PMLR
Yang L, Gu J, Wang C, Cao X, Zhai L, Jin D, Guo Y (2020a) Toward unsupervised graph neural network: interactive clustering and embedding via optimal transport. In: 2020 IEEE international conference on data mining (ICDM), pp. 1358–1363. IEEE
Yang K, Swanson K, Jin W, Coley C, Eiden P, Gao H, Guzman-Perez A, Hopper T, Kelley B, Mathea M et al (2019) Analyzing learned molecular representations for property prediction. J Chem Inf Model 59(8):3370–3388
Yang J, Leskovec J (2015) Defining and evaluating network communities based on ground-truth. Knowl Inf Syst 42(1):181–213
Yang X, Yumer E, Asente P, Kraley M, Kifer D, Lee Giles C (2017) Learning to extract semantic structure from documents using multimodal fully convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5315–5324
Yang J, Lu J, Lee S, Batra D, Parikh D (2018) Graph R-CNN for scene graph generation. In: Proceedings of the European conference on computer vision (ECCV), pp. 670–685. Springer, Cham
Yao L, Mao C, Luo Y (2019) Graph convolutional networks for text classification. In: Proceedings of the AAAI conference on artificial intelligence, vol. 33, pp. 7370–7377
Yao H, Wu F, Ke J, Tang X, Jia Y, Lu S, Gong P, Ye J, Li Z (2018) Deep multi-view spatial-temporal network for taxi demand prediction. In: Proceedings of the AAAI conference on artificial intelligence, vol. 32, pp. 2588–2595
Ying R, He R, Chen K, Eksombatchai P, Hamilton WL, Leskovec J (2018) Graph convolutional neural networks for web-scale recommender systems. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 974–983. Association for Computing Machinery, New York, NY, USA
Yi L, Su H, Guo X, Guibas LJ (2017) SyncSpecCNN: Synchronized spectral CNN for 3D shape segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2282–2290. IEEE Computer Society, Los Alamitos, CA, USA. https://doi.org/10.1109/CVPR.2017.697
You Y, Chen T, Sui Y, Chen T, Wang Z, Shen Y (2020a) Graph contrastive learning with augmentations. Adv Neural Inf Process Syst 33:5812–5823
You Y, Chen T, Wang Z, Shen Y (2020b) When does self-supervision help graph convolutional networks? In: Proceedings of the 37th international conference on machine learning, vol 119, pp 10871–10880. PMLR
You J, Liu B, Ying R, Pande V, Leskovec J (2018) Graph convolutional policy network for goal-directed molecular graph generation
You J, Ying R, Leskovec J (2019) Position-aware graph neural networks. In: International conference on machine learning, pp. 7134–7143. PMLR
You J, Ying Z, Leskovec J (2020c) Design space for graph neural networks. Adv Neural Inform Process Syst 33
Yu Z, Tao L, Qian Z, Wu J, Liu H, Yu Y, Song J, Wang S, Sun J (2016) Altered brain anatomical networks and disturbed connection density in brain tumor patients revealed by diffusion tensor tractography. Int J Comput Assist Radiol Surg 11(11):2007–2019
Yu T, He S, Song Y-Z, Xiang T (2022) Hybrid graph neural networks for few-shot learning. In: Proceedings of the AAAI conference on artificial intelligence, vol. 36, pp. 3179–3187
Yu E-Y, Wang Y-P, Fu Y, Chen D-B, Xie M (2020) Identifying critical nodes in complex networks via graph convolutional networks. Knowl-Based Syst 198:105893
Yu B, Yin H, Zhu Z (2017) Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting
Yu J, Lin Z, Yang J, Shen X, Lu X, Huang TS (2018) Generative image inpainting with contextual attention. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5505–5514. IEEE, Salt Lake City, UT, USA
Zachary WW (1977) An information flow model for conflict and fission in small groups. J Anthropol Res 452–473
Zeng J, Xie P (2020) Contrastive self-supervised learning for graph classification
Zhang M, Chen Y (2018) Link prediction based on graph neural networks. Adv Neural Inf Process Syst 31:5165–5175
Zhang M, Fujinuma Y, Boyd-Graber J (2020b) Exploiting cross-lingual subword similarities in low-resource document classification. Proc AAAI Conf Artif Intell 34:9547–9554
Zhang H, Lin S, Liu W, Zhou P, Tang J, Liang X, Xing EP (2020c) Iterative graph self-distillation
Zhang M, Wu S, Yu X, Liu Q, Wang L (2022) Dynamic graph neural networks for sequential recommendation. IEEE Trans Knowl Data Eng
Zhang G, He H, Katabi D (2019a) Circuit-GNN: graph neural networks for distributed circuit design. In: International conference on machine learning, pp. 7364–7373. PMLR
Zhang Z, Cui P, Zhu W (2020a) Deep learning on graphs: a survey. IEEE Trans Knowl Data Eng
Zhang J, Shi X, Xie J, Ma H, King I, Yeung D-Y (2018b) Gaan: Gated attention networks for learning on large and spatiotemporal graphs
Zhang S, Zhou Z, Huang Z, Wei Z (2018c) Few-shot classification on graphs with structural regularized gcns
Zhang X, Liu H, Li Q, Wu X-M (2019b) Attributed graph clustering via adaptive graph convolution
Zhang L, Wang X, Li H, Zhu G, Shen P, Li P, Lu X, Shah SAA, Bennamoun M (2020d) Structure-feature based graph self-adaptive pooling. In: Proceedings of the web conference 2020, pp. 3098–3104
Zhang B, Yu Z, Zhang W (2020f) Community-centric graph convolutional network for unsupervised community detection. In: IJCAI, pp. 551–556
Zhang J, Zhang H, Xia C, Sun L (2020g) Graph-bert: only attention is needed for learning graph representations
Zhang R, Xu L, Yu Z, Shi Y, Mu C, Xu M (2021) Deep-irtarget: an automatic target detector in infrared imagery using dual-domain feature extraction and allocation. IEEE Trans Multimedia 24:1735–1749
Zhang Y, Lu H, Niu W, Caverlee J (2018a) Quality-aware neural complementary item recommendation. In: Proceedings of the 12th ACM conference on recommender systems, pp. 77–85
Zhang Y, Yu X, Cui Z, Wu S, Wen Z, Wang L (2020e) Every document owns its structure: inductive text classification via graph neural networks. In: Proceedings of the 58th annual meeting of the association for computational linguistics, pp. 334–339. Association for Computational Linguistics
Zhao Q, Zhang Y, Zhang Y, Friedman D (2017) Multi-product utility maximization for economic recommendation. In: Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, pp. 435–443
Zhou Q, Womer FY, Kong L, Wu F, Jiang X, Zhou Y, Wang D, Bai C, Chang M, Fan G (2017) Trait-related cortical-subcortical dissociation in bipolar disorder: analysis of network degree centrality. J Clin Psychiatry 78(5):0–0
Zhou J, Cui G, Hu S, Zhang Z, Yang C, Liu Z, Wang L, Li C, Sun M (2020) Graph neural networks: a review of methods and applications. AI Open 1:57–81
Zhu XJ (2005) Semi-supervised learning literature survey. University of Wisconsin–Madison Department of Computer Sciences
Zhuang L, Zhou Z, Gao S, Yin J, Lin Z, Ma Y (2017) Label information guided graph construction for semi-supervised learning. IEEE Trans Image Process 26(9):4182–4192
Zhu K, Liu M, Lin Y, Xu B, Li S, Tang X, Sun N, Pan DZ (2019) Geniusroute: a new analog routing paradigm using generative neural network guidance. In: 2019 IEEE/ACM international conference on computer-aided design (ICCAD), pp. 1–8. IEEE
Zhu Y, Xu Y, Yu F, Liu Q, Wu S, Wang L (2020a) Deep graph contrastive representation learning
Zhu Y, Xu Y, Yu F, Liu Q, Wu S, Wang L (2021) Graph contrastive learning with adaptive augmentation. In: Proceedings of the web conference 2021, pp. 2069–2080
Zhu Y, Xu Y, Yu F, Wu S, Wang L (2020b) Cagnn: Cluster-aware graph neural networks for unsupervised graph representation learning
Zitnik M, Agrawal M, Leskovec J (2018) Modeling polypharmacy side effects with graph convolutional networks. Bioinformatics 34(13):457–466
Zuo X-N, Ehmke R, Mennes M, Imperati D, Castellanos FX, Sporns O, Milham MP (2012) Network centrality in the human functional connectome. Cereb Cortex 22(8):1862–1875
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Waikhom, L., Patgiri, R. A survey of graph neural networks in various learning paradigms: methods, applications, and challenges. Artif Intell Rev 56, 6295–6364 (2023). https://doi.org/10.1007/s10462-022-10321-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10462-022-10321-2