A comprehensive survey on pretrained foundation models: A history from bert to chatgpt

C Zhou, Q Li, C Li, J Yu, Y Liu, G Wang… - International Journal of …, 2024 - Springer
Abstract Pretrained Foundation Models (PFMs) are regarded as the foundation for various
downstream tasks across different data modalities. A PFM (eg, BERT, ChatGPT, GPT-4) is …

Graph representation learning in biomedicine and healthcare

MM Li, K Huang, M Zitnik - Nature Biomedical Engineering, 2022 - nature.com
Networks—or graphs—are universal descriptors of systems of interacting elements. In
biomedicine and healthcare, they can represent, for example, molecular interactions …

Graphmae: Self-supervised masked graph autoencoders

Z Hou, X Liu, Y Cen, Y Dong, H Yang, C Wang… - Proceedings of the 28th …, 2022 - dl.acm.org
Self-supervised learning (SSL) has been extensively explored in recent years. Particularly,
generative SSL has seen emerging success in natural language processing and other …

Deep learning with graph convolutional networks: An overview and latest applications in computational intelligence

UA Bhatti, H Tang, G Wu, S Marjan… - International Journal of …, 2023 - Wiley Online Library
Convolutional neural networks (CNNs) have received widespread attention due to their
powerful modeling capabilities and have been successfully applied in natural language …

Graph neural networks: foundation, frontiers and applications

L Wu, P Cui, J Pei, L Zhao, X Guo - … of the 28th ACM SIGKDD Conference …, 2022 - dl.acm.org
The field of graph neural networks (GNNs) has seen rapid and incredible strides over the
recent years. Graph neural networks, also known as deep learning on graphs, graph …

A comprehensive survey on deep graph representation learning

W Ju, Z Fang, Y Gu, Z Liu, Q Long, Z Qiao, Y Qin… - Neural Networks, 2024 - Elsevier
Graph representation learning aims to effectively encode high-dimensional sparse graph-
structured data into low-dimensional dense vectors, which is a fundamental task that has …

Data augmentation for deep graph learning: A survey

K Ding, Z Xu, H Tong, H Liu - ACM SIGKDD Explorations Newsletter, 2022 - dl.acm.org
Graph neural networks, a powerful deep learning tool to model graph-structured data, have
demonstrated remarkable performance on numerous graph learning tasks. To address the …

Graph contrastive learning automated

Y You, T Chen, Y Shen, Z Wang - … Conference on Machine …, 2021 - proceedings.mlr.press
Self-supervised learning on graph-structured data has drawn recent interest for learning
generalizable, transferable and robust representations from unlabeled graphs. Among …

Graph self-supervised learning: A survey

Y Liu, M Jin, S Pan, C Zhou, Y Zheng… - IEEE transactions on …, 2022 - ieeexplore.ieee.org
Deep learning on graphs has attracted significant interests recently. However, most of the
works have focused on (semi-) supervised learning, resulting in shortcomings including …

Motif-based graph self-supervised learning for molecular property prediction

Z Zhang, Q Liu, H Wang, C Lu… - Advances in Neural …, 2021 - proceedings.neurips.cc
Predicting molecular properties with data-driven methods has drawn much attention in
recent years. Particularly, Graph Neural Networks (GNNs) have demonstrated remarkable …