[引用][C] Moving offthe-grid: Scene-grounded video representations

S van Steenkiste, D Zoran, Y Yang, Y Rubanova… - 2024 - NeurIPS

Moving Off-the-Grid: Scene-Grounded Video Representations

S van Steenkiste, D Zoran, Y Yang, Y Rubanova… - arXiv preprint arXiv …, 2024 - arxiv.org
Current vision models typically maintain a fixed correspondence between their
representation structure and image space. Each layer comprises a set of tokens arranged"
on-the-grid," which biases patches or tokens to encode information at a specific spatio (-
temporal) location. In this work we present Moving Off-the-Grid (MooG), a self-supervised
video representation model that offers an alternative approach, allowing tokens to move" off-
the-grid" to better enable them to represent scene elements consistently, even as they move …
以上显示的是最相近的搜索结果。 查看全部搜索结果