Inductive biases for deep learning of higher-level cognition

A Goyal, Y Bengio - Proceedings of the Royal Society A, 2022 - royalsocietypublishing.org
A fascinating hypothesis is that human and animal intelligence could be explained by a few
principles (rather than an encyclopaedic list of heuristics). If that hypothesis was correct, we …

Semantic memory: A review of methods, models, and current challenges

AA Kumar - Psychonomic Bulletin & Review, 2021 - Springer
Adult semantic memory has been traditionally conceptualized as a relatively static memory
system that consists of knowledge about the world, concepts, and symbols. Considerable …

Do as i can, not as i say: Grounding language in robotic affordances

M Ahn, A Brohan, N Brown, Y Chebotar… - arXiv preprint arXiv …, 2022 - arxiv.org
Large language models can encode a wealth of semantic knowledge about the world. Such
knowledge could be extremely useful to robots aiming to act upon high-level, temporally …

Guiding pretraining in reinforcement learning with large language models

Y Du, O Watkins, Z Wang, C Colas… - International …, 2023 - proceedings.mlr.press
Reinforcement learning algorithms typically struggle in the absence of a dense, well-shaped
reward function. Intrinsically motivated exploration methods address this limitation by …

Grounding large language models in interactive environments with online reinforcement learning

T Carta, C Romac, T Wolf, S Lamprier… - International …, 2023 - proceedings.mlr.press
Recent works successfully leveraged Large Language Models'(LLM) abilities to capture
abstract knowledge about world's physics to solve decision-making problems. Yet, the …

Interactive language: Talking to robots in real time

C Lynch, A Wahid, J Tompson, T Ding… - IEEE Robotics and …, 2023 - ieeexplore.ieee.org
We present a framework for building interactive, real-time, natural language-instructable
robots in the real world, and we open source related assets (dataset, environment …

Learning transferable visual models from natural language supervision

A Radford, JW Kim, C Hallacy… - International …, 2021 - proceedings.mlr.press
State-of-the-art computer vision systems are trained to predict a fixed set of predetermined
object categories. This restricted form of supervision limits their generality and usability since …

Navigating to objects in the real world

T Gervet, S Chintala, D Batra, J Malik, DS Chaplot - Science Robotics, 2023 - science.org
Semantic navigation is necessary to deploy mobile robots in uncontrolled environments
such as homes or hospitals. Many learning-based approaches have been proposed in …

[PDF][PDF] Vima: General robot manipulation with multimodal prompts

Y Jiang, A Gupta, Z Zhang, G Wang… - arXiv preprint …, 2022 - authors.library.caltech.edu
Prompt-based learning has emerged as a successful paradigm in natural language
processing, where a single general-purpose language model can be instructed to perform …

Emergent world representations: Exploring a sequence model trained on a synthetic task

K Li, AK Hopkins, D Bau, F Viégas, H Pfister… - arXiv preprint arXiv …, 2022 - arxiv.org
Language models show a surprising range of capabilities, but the source of their apparent
competence is unclear. Do these networks just memorize a collection of surface statistics, or …