Foundational challenges in assuring alignment and safety of large language models

U Anwar, A Saparov, J Rando, D Paleka… - arXiv preprint arXiv …, 2024 - arxiv.org
This work identifies 18 foundational challenges in assuring the alignment and safety of large
language models (LLMs). These challenges are organized into three different categories …

Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks

S Jain, R Kirk, ES Lubana, RP Dick, H Tanaka… - arXiv preprint arXiv …, 2023 - arxiv.org
Fine-tuning large pre-trained models has become the de facto strategy for developing both
task-specific and general-purpose machine learning systems, including developing models …

Eclipse: A resource-efficient text-to-image prior for image generations

M Patel, C Kim, S Cheng, C Baral… - Proceedings of the …, 2024 - openaccess.thecvf.com
Abstract Text-to-image (T2I) diffusion models notably the unCLIP models (eg DALL-E-2)
achieve state-of-the-art (SOTA) performance on various compositional T2I benchmarks at …

How capable can a transformer become? a study on synthetic, interpretable tasks

R Ramesh, M Khona, RP Dick, H Tanaka… - arXiv preprint arXiv …, 2023 - arxiv.org
Transformers trained on huge text corpora exhibit a remarkable set of capabilities, eg,
performing simple logical operations. Given the inherent compositional nature of language …

Why do animals need shaping? a theory of task composition and curriculum learning

JH Lee, SS Mannelli, A Saxe - arXiv preprint arXiv:2402.18361, 2024 - arxiv.org
Diverse studies in systems neuroscience begin with extended periods of training known as'
shaping'procedures. These involve progressively studying component parts of more …

Skews in the Phenomenon Space Hinder Generalization in Text-to-Image Generation

Y Chang, Y Zhang, Z Fang, Y Wu, Y Bisk… - arXiv preprint arXiv …, 2024 - arxiv.org
The literature on text-to-image generation is plagued by issues of faithfully composing
entities with relations. But there lacks a formal understanding of how entity-relation …

Initialization is Critical to Whether Transformers Fit Composite Functions by Inference or Memorizing

Z Zhang, P Lin, Z Wang, Y Zhang, ZQJ Xu - arXiv preprint arXiv …, 2024 - arxiv.org
Transformers have shown impressive capabilities across various tasks, but their
performance on compositional problems remains a topic of debate. In this work, we …

Who's in and who's out? A case study of multimodal CLIP-filtering in DataComp

R Hong, W Agnew, T Kohno, J Morgenstern - arXiv preprint arXiv …, 2024 - arxiv.org
As training datasets become increasingly drawn from unstructured, uncontrolled
environments such as the web, researchers and industry practitioners have increasingly …

[HTML][HTML] Report on the AAPM Grand Challenge on deep generative modeling for learning medical image statistics

R Deshpande, VA Kelkar, D Gotsis, P Kc, R Zeng… - ArXiv, 2024 - ncbi.nlm.nih.gov
Background: The findings of the 2023 AAPM Grand Challenge on Deep Generative
Modeling for Learning Medical Image Statistics are reported in this Special Report. Purpose …

CUPID: Contextual Understanding of Prompt‐conditioned Image Distributions

Y Zhao, M Li, M Berger - Computer Graphics Forum, 2024 - Wiley Online Library
We present CUPID: a visualization method for the contextual understanding of prompt‐
conditioned image distributions. CUPID targets the visual analysis of distributions produced …