Why does surprisal from larger transformer-based language models provide a poorer fit to human reading times?
BD Oh, W Schuler - Transactions of the Association for Computational …, 2023 - direct.mit.edu
This work presents a linguistic analysis into why larger Transformer-based pre-trained
language models with more parameters and lower perplexity nonetheless yield surprisal …
language models with more parameters and lower perplexity nonetheless yield surprisal …
Large-scale evidence for logarithmic effects of word predictability on reading time
During real-time language comprehension, our minds rapidly decode complex meanings
from sequences of words. The difficulty of doing so is known to be related to words' …
from sequences of words. The difficulty of doing so is known to be related to words' …
What artificial neural networks can tell us about human language acquisition
A Warstadt, SR Bowman - Algebraic structures in natural …, 2022 - taylorfrancis.com
Rapid progress in machine learning for natural language processing has the potential to
transform debates about how humans learn language. However, the learning environments …
transform debates about how humans learn language. However, the learning environments …
Robust effects of working memory demand during naturalistic language comprehension in language-selective cortex
To understand language, we must infer structured meanings from real-time auditory or visual
signals. Researchers have long focused on word-by-word structure building in working …
signals. Researchers have long focused on word-by-word structure building in working …
[HTML][HTML] Comparison of structural parsers and neural language models as surprisal estimators
Expectation-based theories of sentence processing posit that processing difficulty is
determined by predictability in context. While predictability quantified via surprisal has …
determined by predictability in context. While predictability quantified via surprisal has …
Incremental language comprehension difficulty predicts activity in the language network but not the multiple demand network
What role do domain-general executive functions play in human language comprehension?
To address this question, we examine the relationship between behavioral measures of …
To address this question, we examine the relationship between behavioral measures of …
The plausibility of sampling as an algorithmic theory of sentence processing
Abstract Words that are more surprising given context take longer to process. However, no
incremental parsing algorithm has been shown to directly predict this phenomenon. In this …
incremental parsing algorithm has been shown to directly predict this phenomenon. In this …
Large-scale benchmark yields no evidence that language model surprisal explains syntactic disambiguation difficulty
Prediction has been proposed as an overarching principle that explains human information
processing in language and beyond. To what degree can processing difficulty in …
processing in language and beyond. To what degree can processing difficulty in …
Differential tracking of linguistic vs. mental state content in naturalistic stimuli by language and theory of mind (ToM) brain networks
Abstract Language and social cognition, especially the ability to reason about mental states,
known as theory of mind (ToM), are deeply related in development and everyday use …
known as theory of mind (ToM), are deeply related in development and everyday use …
Overview of the CLEF 2022 SimpleText Lab: Automatic simplification of scientific texts
Although citizens agree on the importance of objective scientific information, yet they tend to
avoid scientific literature due to access restrictions, its complex language or their lack of prior …
avoid scientific literature due to access restrictions, its complex language or their lack of prior …