Deep transfer learning & beyond: Transformer language models in information systems research

R Gruetzemacher, D Paradice - ACM Computing Surveys (CSUR), 2022 - dl.acm.org
AI is widely thought to be poised to transform business, yet current perceptions of the scope
of this transformation may be myopic. Recent progress in natural language processing …

Shared computational principles for language processing in humans and deep language models

A Goldstein, Z Zada, E Buchnik, M Schain, A Price… - Nature …, 2022 - nature.com
Departing from traditional linguistic models, advances in deep learning have resulted in a
new type of predictive (autoregressive) deep language models (DLMs). Using a self …

The neural architecture of language: Integrative modeling converges on predictive processing

M Schrimpf, IA Blank, G Tuckute… - Proceedings of the …, 2021 - National Acad Sciences
The neuroscience of perception has recently been revolutionized with an integrative
modeling approach in which computation, brain function, and behavior are linked across …

Fine-tuning pretrained language models: Weight initializations, data orders, and early stopping

J Dodge, G Ilharco, R Schwartz, A Farhadi… - arXiv preprint arXiv …, 2020 - arxiv.org
Fine-tuning pretrained contextual word embedding models to supervised downstream tasks
has become commonplace in natural language processing. This process, however, is often …

Getting aligned on representational alignment

I Sucholutsky, L Muttenthaler, A Weller, A Peng… - arXiv preprint arXiv …, 2023 - arxiv.org
Biological and artificial information processing systems form representations that they can
use to categorize, reason, plan, navigate, and make decisions. How can we measure the …

Toward a realistic model of speech processing in the brain with self-supervised learning

J Millet, C Caucheteux, Y Boubenec… - Advances in …, 2022 - proceedings.neurips.cc
Several deep neural networks have recently been shown to generate activations similar to
those of the brain in response to the same input. These algorithms, however, remain largely …

Feature-space selection with banded ridge regression

TD La Tour, M Eickenberg, AO Nunez-Elizalde… - NeuroImage, 2022 - Elsevier
Encoding models provide a powerful framework to identify the information represented in
brain recordings. In this framework, a stimulus representation is expressed within a feature …

Neural language taskonomy: Which NLP tasks are the most predictive of fMRI brain activity?

SR Oota, J Arora, V Agarwal, M Marreddy… - arXiv preprint arXiv …, 2022 - arxiv.org
Several popular Transformer based language models have been found to be successful for
text-driven brain encoding. However, existing literature leverages only pretrained text …

Joint processing of linguistic properties in brains and language models

SR Oota, M Gupta, M Toneva - Advances in Neural …, 2024 - proceedings.neurips.cc
Abstract Language models have been shown to be very effective in predicting brain
recordings of subjects experiencing complex language stimuli. For a deeper understanding …

[HTML][HTML] The underwater acoustic target timbre perception and recognition based on the auditory inspired deep convolutional neural network

J Li, H Yang - Applied Acoustics, 2021 - Elsevier
Targeted at solving the problem in extracting the line spectrum features of ship radiated
noise in complex marine environments, this paper proposes a new deep convolutional …