Reconstructing the mind's eye: fMRI-to-image with contrastive learning and diffusion priors
We present MindEye, a novel fMRI-to-image approach to retrieve and reconstruct viewed
images from brain activity. Our model comprises two parallel submodules that are …
images from brain activity. Our model comprises two parallel submodules that are …
Rethinking federated learning with domain shift: A prototype view
Federated learning shows a bright promise as a privacy-preserving collaborative learning
technique. However, prevalent solutions mainly focus on all private data sampled from the …
technique. However, prevalent solutions mainly focus on all private data sampled from the …
Contrastive learning reduces hallucination in conversations
Pre-trained language models (LMs) store knowledge in their parameters and can generate
informative responses when used in conversational systems. However, LMs suffer from the …
informative responses when used in conversational systems. However, LMs suffer from the …
Improving contrastive learning by visualizing feature transformation
Contrastive learning, which aims at minimizing the distance between positive pairs while
maximizing that of negative ones, has been widely and successfully applied in unsupervised …
maximizing that of negative ones, has been widely and successfully applied in unsupervised …
A unified analysis of mixed sample data augmentation: A loss function perspective
We propose the first unified theoretical analysis of mixed sample data augmentation
(MSDA), such as Mixup and CutMix. Our theoretical results show that regardless of the …
(MSDA), such as Mixup and CutMix. Our theoretical results show that regardless of the …
Un-mix: Rethinking image mixtures for unsupervised visual representation learning
The recently advanced unsupervised learning approaches use the siamese-like framework
to compare two" views" from the same image for learning representations. Making the two …
to compare two" views" from the same image for learning representations. Making the two …
Dream: Visual decoding from reversing human visual system
In this work we present DREAM, an fMRI-to-image method for reconstructing viewed images
from brain activities, grounded on fundamental knowledge of the human visual system. We …
from brain activities, grounded on fundamental knowledge of the human visual system. We …
A simple data mixing prior for improving self-supervised learning
Abstract Data mixing (eg, Mixup, Cutmix, ResizeMix) is an essential component for
advancing recognition models. In this paper, we focus on studying its effectiveness in the …
advancing recognition models. In this paper, we focus on studying its effectiveness in the …
M-mix: Generating hard negatives via multi-sample mixing for contrastive learning
Negative pairs, especially hard negatives as combined with common negatives (easy to
discriminate), are essential in contrastive learning, which plays a role of avoiding …
discriminate), are essential in contrastive learning, which plays a role of avoiding …
Motion-aware contrastive video representation learning via foreground-background merging
In light of the success of contrastive learning in the image domain, current self-supervised
video representation learning methods usually employ contrastive loss to facilitate video …
video representation learning methods usually employ contrastive loss to facilitate video …