Towards in-context scene understanding

I Balazevic, D Steiner… - Advances in …, 2024 - proceedings.neurips.cc
In-context learning––the ability to configure a model's behavior with different prompts––has
revolutionized the field of natural language processing, alleviating the need for task-specific …

Images speak in images: A generalist painter for in-context visual learning

X Wang, W Wang, Y Cao, C Shen… - Proceedings of the …, 2023 - openaccess.thecvf.com
In-context learning, as a new paradigm in NLP, allows the model to rapidly adapt to various
tasks with only a handful of prompts and examples. But in computer vision, the difficulties for …

Learning to retrieve prompts for in-context learning

O Rubin, J Herzig, J Berant - arXiv preprint arXiv:2112.08633, 2021 - arxiv.org
In-context learning is a recent paradigm in natural language understanding, where a large
pre-trained language model (LM) observes a test instance and a few training examples as …

X-detr: A versatile architecture for instance-wise vision-language tasks

Z Cai, G Kwon, A Ravichandran, E Bas, Z Tu… - … on Computer Vision, 2022 - Springer
In this paper, we study the challenging instance-wise vision-language tasks, where the free-
form language is required to align with the objects instead of the whole image. To address …

Meta-in-context learning in large language models

J Coda-Forno, M Binz, Z Akata… - Advances in …, 2023 - proceedings.neurips.cc
Large language models have shown tremendous performance in a variety of tasks. In-
context learning--the ability to improve at a task after being provided with a number of …

Placing language in an integrated understanding system: Next steps toward human-level performance in neural language models

JL McClelland, F Hill, M Rudolph… - Proceedings of the …, 2020 - National Acad Sciences
Language is crucial for human intelligence, but what exactly is its role? We take language to
be a part of a system for understanding and communicating about situations. In humans …

Compositional exemplars for in-context learning

J Ye, Z Wu, J Feng, T Yu… - … Conference on Machine …, 2023 - proceedings.mlr.press
Large pretrained language models (LMs) have shown impressive In-Context Learning (ICL)
ability, where the model learns to do an unseen task simply by conditioning on a prompt …

Learning visually grounded words and syntax for a scene description task

DK Roy - Computer speech & language, 2002 - Elsevier
A spoken language generation system has been developed that learns to describe objects
in computer-generated visual scenes. The system is trained by a 'show-and-tell'procedure in …

[PDF][PDF] Taskprompter: Spatial-channel multi-task prompting for dense scene understanding

H Ye, D Xu - The Eleventh International Conference on Learning …, 2022 - drive.google.com
Learning effective representations simultaneously from multiple tasks in a unified network
framework is a fundamental paradigm for multi-task dense visual scene understanding. This …

Focused transformer: Contrastive training for context scaling

S Tworkowski, K Staniszewski… - Advances in …, 2024 - proceedings.neurips.cc
Large language models have an exceptional capability to incorporate new information in a
contextual manner. However, the full potential of such an approach is often restrained due to …