M³vit: Mixture-of-experts vision transformer for efficient multi-task learning with model-accelerator co-design

Z Fan, R Sarkar, Z Jiang, T Chen… - Advances in …, 2022 - proceedings.neurips.cc
Multi-task learning (MTL) encapsulates multiple learned tasks in a single model and often
lets those tasks learn better jointly. Multi-tasking models have become successful and often …

Mtformer: Multi-task learning via transformer and cross-task reasoning

X Xu, H Zhao, V Vineet, SN Lim, A Torralba - European Conference on …, 2022 - Springer
In this paper, we explore the advantages of utilizing transformer structures for addressing
multi-task learning (MTL). Specifically, we demonstrate that models with transformer …

Vision transformer adapters for generalizable multitask learning

D Bhattacharjee, S Süsstrunk… - Proceedings of the …, 2023 - openaccess.thecvf.com
We introduce the first multitasking vision transformer adapters that learn generalizable task
affinities which can be applied to novel tasks and domains. Integrated into an off-the-shelf …

Indiscernible object counting in underwater scenes

G Sun, Z An, Y Liu, C Liu, C Sakaridis… - Proceedings of the …, 2023 - openaccess.thecvf.com
Recently, indiscernible scene understanding has attracted a lot of attention in the vision
community. We further advance the frontier of this field by systematically studying a new …

Paco: Parameter-compositional multi-task reinforcement learning

L Sun, H Zhang, W Xu… - Advances in Neural …, 2022 - proceedings.neurips.cc
The purpose of multi-task reinforcement learning (MTRL) is to train a single policy that can
be applied to a set of different tasks. Sharing parameters allows us to take advantage of the …

Addressing negative transfer in diffusion models

H Go, Y Lee, S Lee, S Oh, H Moon… - Advances in Neural …, 2024 - proceedings.neurips.cc
Diffusion-based generative models have achieved remarkable success in various domains.
It trains a shared model on denoising tasks that encompass different noise levels …

[HTML][HTML] Universal representations: A unified look at multiple task and domain learning

WH Li, X Liu, H Bilen - International Journal of Computer Vision, 2024 - Springer
We propose a unified look at jointly learning multiple vision tasks and visual domains
through universal representations, a single deep neural network. Learning multiple …

Promptonomyvit: Multi-task prompt learning improves video transformers using synthetic scene data

R Herzig, O Abramovich… - Proceedings of the …, 2024 - openaccess.thecvf.com
Action recognition models have achieved impressive results by incorporating scene-level
annotations, such as objects, their relations, 3D structure, and more. However, obtaining …

Prompt guided transformer for multi-task dense prediction

Y Lu, S Sirejiding, Y Ding, C Wang… - IEEE Transactions on …, 2024 - ieeexplore.ieee.org
Task-conditional architecture offers advantage in parameter efficiency but falls short in
performance compared to state-of-the-art multi-decoder methods. How to trade off …

Denoising task routing for diffusion models

B Park, S Woo, H Go, JY Kim, C Kim - arXiv preprint arXiv:2310.07138, 2023 - arxiv.org
Diffusion models generate highly realistic images through learning a multi-step denoising
process, naturally embodying the principles of multi-task learning (MTL). Despite the …