State of the art on diffusion models for visual computing

R Po, W Yifan, V Golyanik, K Aberman… - Computer Graphics …, 2024 - Wiley Online Library
The field of visual computing is rapidly advancing due to the emergence of generative
artificial intelligence (AI), which unlocks unprecedented capabilities for the generation …

Spectral motion alignment for video motion transfer using diffusion models

GY Park, H Jeong, SW Lee, JC Ye - arXiv preprint arXiv:2403.15249, 2024 - arxiv.org
The evolution of diffusion models has greatly impacted video generation and understanding.
Particularly, text-to-video diffusion models (VDMs) have significantly facilitated the …

Video Editing via Factorized Diffusion Distillation

U Singer, A Zohar, Y Kirstain, S Sheynin… - arXiv preprint arXiv …, 2024 - arxiv.org
We introduce Emu Video Edit (EVE), a model that establishes a new state-of-the art in video
editing without relying on any supervised video editing data. To develop EVE we separately …

Videoshop: Localized Semantic Video Editing with Noise-Extrapolated Diffusion Inversion

X Fan, A Bhattad, R Krishna - arXiv preprint arXiv:2403.14617, 2024 - arxiv.org
We introduce Videoshop, a training-free video editing algorithm for localized semantic edits.
Videoshop allows users to use any editing software, including Photoshop and generative …

Video Diffusion Models are Training-free Motion Interpreter and Controller

Z Xiao, Y Zhou, S Yang, X Pan - arXiv preprint arXiv:2405.14864, 2024 - arxiv.org
Video generation primarily aims to model authentic and customized motion across frames,
making understanding and controlling the motion a crucial topic. Most diffusion-based …

I2VEdit: First-Frame-Guided Video Editing via Image-to-Video Diffusion Models

W Ouyang, Y Dong, L Yang, J Si, X Pan - arXiv preprint arXiv:2405.16537, 2024 - arxiv.org
The remarkable generative capabilities of diffusion models have motivated extensive
research in both image and video editing. Compared to video editing which faces additional …

HOI-Swap: Swapping Objects in Videos with Hand-Object Interaction Awareness

Z Xue, M Luo, C Chen, K Grauman - arXiv preprint arXiv:2406.07754, 2024 - arxiv.org
We study the problem of precisely swapping objects in videos, with a focus on those
interacted with by hands, given one user-provided reference object image. Despite the great …

Diffusion Models and Representation Learning: A Survey

M Fuest, P Ma, M Gui, JS Fischer, VT Hu… - arXiv preprint arXiv …, 2024 - arxiv.org
Diffusion Models are popular generative modeling methods in various vision tasks, attracting
significant attention. They can be considered a unique instance of self-supervised learning …

ReVideo: Remake a Video with Motion and Content Control

C Mou, M Cao, X Wang, Z Zhang, Y Shan… - arXiv preprint arXiv …, 2024 - arxiv.org
Despite significant advancements in video generation and editing using diffusion models,
achieving accurate and localized video editing remains a substantial challenge …

Edit-Your-Motion: Space-Time Diffusion Decoupling Learning for Video Motion Editing

Y Zuo, L Li, L Jiao, F Liu, X Liu, W Ma, S Yang… - arXiv preprint arXiv …, 2024 - arxiv.org
Existing diffusion-based video editing methods have achieved impressive results in motion
editing. Most of the existing methods focus on the motion alignment between the edited …