Multimodality and the origin of a novel communication system in face-to-face interaction

V Macuch Silva, J Holler… - Royal Society open …, 2020 - royalsocietypublishing.org
Face-to-face communication is multimodal at its core: it consists of a combination of vocal
and visual signalling. However, current evidence suggests that, in the absence of an …

Language and perception: introduction to the special issue “Speakers and listeners in the visual world”

M Vulchanova, V Vulchanov, I Fritz… - Journal of Cultural …, 2019 - Springer
Abstract Language and perception are two central cognitive systems. Until relatively
recently, however, the interaction between them has been examined only partially and not …

The role of iconic gestures and mouth movements in face-to-face communication

A Krason, R Fenton, R Varley, G Vigliocco - Psychonomic bulletin & …, 2021 - Springer
Human face-to-face communication is multimodal: it comprises speech as well as visual
cues, such as articulatory and limb gestures. In the current study, we assess how iconic …

The fixation and processing of the iconic gestures that accompany talk

G Beattie, K Webster, J Ross - Journal of Language and …, 2010 - journals.sagepub.com
In everyday communication, semantic information is transmitted by both speech and the
gestures that accompany speech. Listeners, therefore, need to monitor two quite different …

Multi-modal communication of common ground

J Holler, J Bavelas - Why gesture, 2017 - books.google.com
Until recently, the literature on common ground depicted its influence as a purely verbal
phenomenon. We review current research on how common ground influences gesture. With …

Processing language in face-to-face conversation: Questions with gestures get faster responses

J Holler, KH Kendrick, SC Levinson - Psychonomic bulletin & review, 2018 - Springer
The home of human language use is face-to-face interaction, a context in which
communicative exchanges are characterised not only by bodily signals accompanying what …

The influence of the visual modality on language structure and conventionalization: Insights from sign language and gesture

P Perniss, A Özyürek, G Morgan - Topics in cognitive science, 2015 - Wiley Online Library
For humans, the ability to communicate and use language is instantiated not only in the
vocal modality but also in the visual modality. The main examples of this are sign languages …

Gesture is the primary modality for language creation

N Fay, B Walker, TM Ellison… - … of the Royal …, 2022 - royalsocietypublishing.org
How language began is one of the oldest questions in science, but theories remain
speculative due to a lack of direct evidence. Here, we report two experiments that generate …

Facial signals and social actions in multimodal face-to-face interaction

N Nota, JP Trujillo, J Holler - Brain Sciences, 2021 - mdpi.com
In a conversation, recognising the speaker's social action (eg, a request) early may help the
potential following speakers understand the intended message quickly, and plan a timely …

Hearing and seeing meaning in speech and gesture: Insights from brain and behaviour

A Özyürek - … Transactions of the Royal Society B …, 2014 - royalsocietypublishing.org
As we speak, we use not only the arbitrary form–meaning mappings of the speech channel
but also motivated form–meaning correspondences, ie iconic gestures that accompany …