Multimodal learning with graphs

Y Ektefaie, G Dasoulas, A Noori, M Farhat… - Nature Machine …, 2023 - nature.com
Artificial intelligence for graphs has achieved remarkable success in modelling complex
systems, ranging from dynamic networks in biology to interacting particle systems in physics …

[PDF][PDF] A survey of multi-agent reinforcement learning with communication

C Zhu, M Dastani, S Wang - arXiv preprint arXiv:2203.08975, 2022 - researchgate.net
Communication is an effective mechanism for coordinating the behavior of multiple agents.
In the field of multi-agent reinforcement learning, agents can improve the overall learning …

FOCAL: Contrastive learning for multimodal time-series sensing signals in factorized orthogonal latent space

S Liu, T Kimura, D Liu, R Wang, J Li… - Advances in …, 2024 - proceedings.neurips.cc
This paper proposes a novel contrastive learning framework, called FOCAL, for extracting
comprehensive features from multimodal time-series sensing signals through self …

M3AE: multimodal representation learning for brain tumor segmentation with missing modalities

H Liu, D Wei, D Lu, J Sun, L Wang… - Proceedings of the AAAI …, 2023 - ojs.aaai.org
Multimodal magnetic resonance imaging (MRI) provides complementary information for sub-
region analysis of brain tumors. Plenty of methods have been proposed for automatic brain …

Self-weighted contrastive learning among multiple views for mitigating representation degeneration

J Xu, S Chen, Y Ren, X Shi, H Shen… - Advances in Neural …, 2024 - proceedings.neurips.cc
Recently, numerous studies have demonstrated the effectiveness of contrastive learning
(CL), which learns feature representations by pulling in positive samples while pushing …

Multi-level contrastive learning: Hierarchical alleviation of heterogeneity in multimodal sentiment analysis

C Fan, K Zhu, J Tao, G Yi, J Xue… - IEEE Transactions on …, 2024 - ieeexplore.ieee.org
Recently, multimodal fusion efforts have achieved remarkable success in Multimodal
Sentiment Analysis (MSA). However, most of the existing methods are based on model-level …

Identifiability results for multimodal contrastive learning

I Daunhawer, A Bizeul, E Palumbo, A Marx… - arXiv preprint arXiv …, 2023 - arxiv.org
Contrastive learning is a cornerstone underlying recent progress in multi-view and
multimodal learning, eg, in representation learning with image/caption pairs. While its …

Geometric-inspired graph-based Incomplete Multi-view Clustering

Z Yang, H Zhang, Y Wei, Z Wang, F Nie, D Hu - Pattern Recognition, 2024 - Elsevier
Multi-view clustering methods group data into different clusters by discovering the
consensus in heterogeneous sources, which however becomes difficult when partial views …

Learning missing modal electronic health records with unified multi-modal data embedding and modality-aware attention

K Lee, S Lee, S Hahn, H Hyun… - Machine Learning …, 2023 - proceedings.mlr.press
Abstract Electronic Health Record (EHR) provides abundant information through various
modalities. However, learning multi-modal EHR is currently facing two major challenges …

Enhanced multimodal representation learning with cross-modal kd

M Chen, L Xing, Y Wang… - Proceedings of the IEEE …, 2023 - openaccess.thecvf.com
This paper explores the tasks of leveraging auxiliary modalities which are only available at
training to enhance multimodal representation learning through cross-modal Knowledge …