Multimodal language processing in human communication

J Holler, SC Levinson - Trends in Cognitive Sciences, 2019 - cell.com
The natural ecology of human language is face-to-face interaction comprising the exchange
of a plethora of multimodal signals. Trying to understand the psycholinguistic processing of …

[HTML][HTML] Why we should study multimodal language

P Perniss - Frontiers in psychology, 2018 - frontiersin.org
What do we study when we study language? Our theories of language, and particularly our
theories of the cognitive and neural underpinnings of language, have developed primarily …

Processing language in face-to-face conversation: Questions with gestures get faster responses

J Holler, KH Kendrick, SC Levinson - Psychonomic bulletin & review, 2018 - Springer
The home of human language use is face-to-face interaction, a context in which
communicative exchanges are characterised not only by bodily signals accompanying what …

Methodology for multimodality

I Mittelberg - Methods in cognitive linguistics, 2007 - torrossa.com
Multimodality, besides entailing at least two modes of expression, also engages at least two
modes of interpretation. For instance, when language and gesture coincide, the body of the …

[PDF][PDF] Gestures orchestrate brain networks for language understanding

JI Skipper, S Goldin-Meadow, HC Nusbaum, SL Small - Current Biology, 2009 - cell.com
Although the linguistic structure of speech provides valuable communicative information,
nonverbal behaviors can offer additional, often disambiguating cues. In particular, being …

Language as a multimodal phenomenon: implications for language learning, processing and evolution

G Vigliocco, P Perniss… - … Transactions of the …, 2014 - royalsocietypublishing.org
Our understanding of the cognitive and neural underpinnings of language has traditionally
been firmly based on spoken Indo-European languages and on language studied as …

Multiple temporalities of language and body in interaction: Challenges for transcribing multimodality

L Mondada - Research on language and social interaction, 2018 - Taylor & Francis
The article focuses on the principles of multimodal CA, the way they can be operationalized
in a transcription system, and the analytical and conceptual consequences of transcription …

The language network as a natural kind within the broader landscape of the human brain

E Fedorenko, AA Ivanova, TI Regev - Nature Reviews Neuroscience, 2024 - nature.com
Abstract Language behaviour is complex, but neuroscientific evidence disentangles it into
distinct components supported by dedicated brain areas or networks. In this Review, we …

[HTML][HTML] Language processing as cue integration: Grounding the psychology of language in perception and neurophysiology

AE Martin - Frontiers in psychology, 2016 - frontiersin.org
I argue that cue integration, a psychophysiological mechanism from vision and multisensory
perception, offers a computational linking hypothesis between psycholinguistic theory and …

Turn-taking in human communication–origins and implications for language processing

SC Levinson - Trends in cognitive sciences, 2016 - cell.com
Most language usage is interactive, involving rapid turn-taking. The turn-taking system has a
number of striking properties: turns are short and responses are remarkably rapid, but turns …