What can transformers learn in-context? a case study of simple function classes

S Garg, D Tsipras, PS Liang… - Advances in Neural …, 2022 - proceedings.neurips.cc
In-context learning is the ability of a model to condition on a prompt sequence consisting of
in-context examples (input-output pairs corresponding to some task) along with a new query …

[HTML][HTML] Leveraging large language models for predictive chemistry

KM Jablonka, P Schwaller… - Nature Machine …, 2024 - nature.com
Abstract Machine learning has transformed many fields and has recently found applications
in chemistry and materials science. The small datasets commonly found in chemistry …

Transformers as algorithms: Generalization and stability in in-context learning

Y Li, ME Ildiz, D Papailiopoulos… - … on Machine Learning, 2023 - proceedings.mlr.press
In-context learning (ICL) is a type of prompting where a transformer model operates on a
sequence of (input, output) examples and performs inference on-the-fly. In this work, we …

Large language models as general pattern machines

S Mirchandani, F Xia, P Florence, B Ichter… - arXiv preprint arXiv …, 2023 - arxiv.org
We observe that pre-trained large language models (LLMs) are capable of autoregressively
completing complex token sequences--from arbitrary ones procedurally generated by …

Promptcast: A new prompt-based learning paradigm for time series forecasting

H Xue, FD Salim - IEEE Transactions on Knowledge and Data …, 2023 - ieeexplore.ieee.org
This paper presents a new perspective on time series forecasting. In existing time series
forecasting methods, the models take a sequence of numerical values as input and yield …

Language models are weak learners

H Manikandan, Y Jiang… - Advances in Neural …, 2023 - proceedings.neurips.cc
A central notion in practical and theoretical machine learning is that of a weak learner,
classifiers that achieve better-than-random performance (on any given distribution over …

Large language models for time series: A survey

X Zhang, RR Chowdhury, RK Gupta… - arXiv preprint arXiv …, 2024 - arxiv.org
Large Language Models (LLMs) have seen significant use in domains such as natural
language processing and computer vision. Going beyond text, image and graphics, LLMs …

Contrast everything: A hierarchical contrastive framework for medical time-series

Y Wang, Y Han, H Wang… - Advances in Neural …, 2024 - proceedings.neurips.cc
Contrastive representation learning is crucial in medical time series analysis as it alleviates
dependency on labor-intensive, domain-specific, and scarce expert annotations. However …

Multimodal llms for health grounded in individual-specific data

A Belyaeva, J Cosentino, F Hormozdiari… - Workshop on Machine …, 2023 - Springer
Foundation large language models (LLMs) have shown an impressive ability to solve tasks
across a wide range of fields including health. To effectively solve personalized health tasks …

The expressive power of low-rank adaptation

Y Zeng, K Lee - arXiv preprint arXiv:2310.17513, 2023 - arxiv.org
Low-Rank Adaptation (LoRA), a parameter-efficient fine-tuning method that leverages low-
rank adaptation of weight matrices, has emerged as a prevalent technique for fine-tuning pre …