Multimodal co-learning: Challenges, applications with datasets, recent advances and future directions

A Rahate, R Walambe, S Ramanna, K Kotecha - Information Fusion, 2022 - Elsevier
Multimodal deep learning systems that employ multiple modalities like text, image, audio,
video, etc., are showing better performance than individual modalities (ie, unimodal) …

A systematic literature review on multimodal machine learning: Applications, challenges, gaps and future directions

A Barua, MU Ahmed, S Begum - IEEE Access, 2023 - ieeexplore.ieee.org
Multimodal machine learning (MML) is a tempting multidisciplinary research area where
heterogeneous data from multiple modalities and machine learning (ML) are combined to …

Hierarchical consensus hashing for cross-modal retrieval

Y Sun, Z Ren, P Hu, D Peng… - IEEE Transactions on …, 2023 - ieeexplore.ieee.org
Cross-modal hashing (CMH) has gained much attention due to its effectiveness and
efficiency in facilitating efficient retrieval between different modalities. Whereas, most …

Proactive privacy-preserving learning for cross-modal retrieval

PF Zhang, G Bai, H Yin, Z Huang - ACM Transactions on Information …, 2023 - dl.acm.org
Deep cross-modal retrieval techniques have recently achieved remarkable performance,
which also poses severe threats to data privacy potentially. Nowadays, enormous user …

Less is better: Exponential loss for cross-modal matching

J Wei, Y Yang, X Xu, J Song, G Wang… - IEEE Transactions on …, 2023 - ieeexplore.ieee.org
Deep metric learning has become a key component of cross-modal retrieval. By learning to
pull the features of matched instances closer while pushing the features of mismatched …

Geometric Correspondence-Based Multimodal Learning for Ophthalmic Image Analysis

Y Wang, L Zhen, TE Tan, H Fu, Y Feng… - … on Medical Imaging, 2024 - ieeexplore.ieee.org
Color fundus photography (CFP) and Optical coherence tomography (OCT) images are two
of the most widely used modalities in the clinical diagnosis and management of retinal …

Cross-Modality Knowledge Calibration Network for Video Corpus Moment Retrieval

T Chen, W Wang, Z Jiang, R Li… - IEEE Transactions on …, 2023 - ieeexplore.ieee.org
Video corpus moment retrieval has become a hot topic recently, which aims to localize a
consequent video moments highly relevant to the given query language description from …

EDMH: Efficient discrete matrix factorization hashing for multi-modal similarity retrieval

F Yang, X Ding, F Ma, D Tong, J Cao - Information Processing & …, 2023 - Elsevier
Hashing has been an emerging topic and has recently attracted widespread attention in
multi-modal similarity search applications. However, most existing approaches rely on …

Deep supervised dual cycle adversarial network for cross-modal retrieval

L Liao, M Yang, B Zhang - … on Circuits and Systems for Video …, 2022 - ieeexplore.ieee.org
Cross-modal retrieval tasks, which are more natural and challenging than traditional
retrieval tasks, have attracted increasing interest from researchers in recent years. Although …

Evolutionary multi-objective model compression for deep neural networks

Z Wang, T Luo, M Li, JT Zhou… - IEEE Computational …, 2021 - ieeexplore.ieee.org
While deep neural networks (DNNs) deliver state-of-the-art accuracy on various applications
from face recognition to language translation, it comes at the cost of high computational and …