A comprehensive survey on pretrained foundation models: A history from bert to chatgpt

C Zhou, Q Li, C Li, J Yu, Y Liu, G Wang… - International Journal of …, 2024 - Springer
Abstract Pretrained Foundation Models (PFMs) are regarded as the foundation for various
downstream tasks across different data modalities. A PFM (eg, BERT, ChatGPT, GPT-4) is …

A comprehensive survey on test-time adaptation under distribution shifts

J Liang, R He, T Tan - International Journal of Computer Vision, 2024 - Springer
Abstract Machine learning methods strive to acquire a robust model during the training
process that can effectively generalize to test samples, even in the presence of distribution …

Graphgpt: Graph instruction tuning for large language models

J Tang, Y Yang, W Wei, L Shi, L Su, S Cheng… - Proceedings of the 47th …, 2024 - dl.acm.org
Graph Neural Networks (GNNs) have evolved to understand graph structures through
recursive exchanges and aggregations among nodes. To enhance robustness, self …

Graphmae: Self-supervised masked graph autoencoders

Z Hou, X Liu, Y Cen, Y Dong, H Yang, C Wang… - Proceedings of the 28th …, 2022 - dl.acm.org
Self-supervised learning (SSL) has been extensively explored in recent years. Particularly,
generative SSL has seen emerging success in natural language processing and other …

Spatio-temporal graph neural networks for predictive learning in urban computing: A survey

G Jin, Y Liang, Y Fang, Z Shao, J Huang… - … on Knowledge and …, 2023 - ieeexplore.ieee.org
With recent advances in sensing technologies, a myriad of spatio-temporal data has been
generated and recorded in smart cities. Forecasting the evolution patterns of spatio-temporal …

Knowledge graph contrastive learning for recommendation

Y Yang, C Huang, L Xia, C Li - … of the 45th international ACM SIGIR …, 2022 - dl.acm.org
Knowledge Graphs (KGs) have been utilized as useful side information to improve
recommendation quality. In those recommender systems, knowledge graph information …

Are graph augmentations necessary? simple graph contrastive learning for recommendation

J Yu, H Yin, X Xia, T Chen, L Cui… - Proceedings of the 45th …, 2022 - dl.acm.org
Contrastive learning (CL) recently has spurred a fruitful line of research in the field of
recommendation, since its ability to extract self-supervised signals from the raw data is well …

Mind the gap: Understanding the modality gap in multi-modal contrastive representation learning

VW Liang, Y Zhang, Y Kwon… - Advances in Neural …, 2022 - proceedings.neurips.cc
We present modality gap, an intriguing geometric phenomenon of the representation space
of multi-modal models. Specifically, we show that different data modalities (eg images and …

Improving graph collaborative filtering with neighborhood-enriched contrastive learning

Z Lin, C Tian, Y Hou, WX Zhao - … of the ACM web conference 2022, 2022 - dl.acm.org
Recently, graph collaborative filtering methods have been proposed as an effective
recommendation approach, which can capture users' preference over items by modeling the …

Graph neural networks: foundation, frontiers and applications

L Wu, P Cui, J Pei, L Zhao, X Guo - … of the 28th ACM SIGKDD Conference …, 2022 - dl.acm.org
The field of graph neural networks (GNNs) has seen rapid and incredible strides over the
recent years. Graph neural networks, also known as deep learning on graphs, graph …