BabyStories: Can Reinforcement Learning Teach Baby Language Models to Write Better Stories?

X Zhao, T Wang, S Osborn, A Rios - arXiv preprint arXiv:2310.16681, 2023 - arxiv.org
Language models have seen significant growth in the size of their corpus, leading to notable
performance improvements. Yet, there has been limited progress in developing models that …

Mind your Language (Model): Fact-Checking LLMs and their Role in NLP Research and Practice

AS Luccioni, A Rogers - arXiv preprint arXiv:2308.07120, 2023 - arxiv.org
Much of the recent discourse within the NLP research community has been centered around
Large Language Models (LLMs), their functionality and potential--yet not only do we not …

Position: Key Claims in LLM Research Have a Long Tail of Footnotes

A Rogers, S Luccioni - Forty-first International Conference on …, 2023 - openreview.net
Much of the recent discourse within the ML community has been centered around Large
Language Models (LLMs), their functionality and potential--yet not only do we not have a …

Bootstrapping Small & High Performance Language Models with Unmasking-Removal Training Policy

Y Yang, E Sulem, I Lee, D Roth - Proceedings of the 2023 …, 2023 - aclanthology.org
BabyBERTa, a language model trained on small-scale child-directed speech while none of
the words are unmasked during training, has been shown to achieve a level of …

Emergent Abilities in Reduced-Scale Generative Language Models

S Muckatira, V Deshpande, V Lialin… - arXiv preprint arXiv …, 2024 - arxiv.org
Large language models can solve new tasks without task-specific fine-tuning. This ability,
also known as in-context learning (ICL), is considered an emergent ability and is primarily …

LocalTweets to LocalHealth: A Mental Health Surveillance Framework Based on Twitter Data

V Deshpande, M Lee, Z Yao, Z Zhang… - arXiv preprint arXiv …, 2024 - arxiv.org
Prior research on Twitter (now X) data has provided positive evidence of its utility in
developing supplementary health surveillance systems. In this study, we present a new …

Iterative improvements from feedback for language models

Y Li - ScienceOpen Preprints, 2023 - scienceopen.com
Iterative improvements from feedback is a general approach for many, if not all, successful
systems. Ground-truth-in-the-loop is critical. Language models (LMs) like ChatGPT are …

[PDF][PDF] Des petits aux grands modèles de langage: État des lieux et perspectives

MN Marwa, MYM Kamel, MSLL ESI, MB Riyadh - researchgate.net
Résumé Nous sommes actuellement témoins d'une révolution technologique majeure,
potentiellement la plus significative de notre époque. En effet, le domaine du traitement du …

[PDF][PDF] Étude Exploratoire des Grands Modèles de Langage à Travers le Développement de Petits Modèles de Langage

MN Marwa, MYM Kamel, MSLL ESI, MB Riyadh - researchgate.net
Résumé L'avènement des grands modèles de langage, tels que GPT-3, a non seulement
captivé l'attention du public, mais a également redéfini les frontières du possible dans la …