Factorized visual representations in the primate visual system and deep neural networks

JW Lindsey, EB Issa - Elife, 2024 - elifesciences.org
Object classification has been proposed as a principal objective of the primate ventral visual
stream and has been used as an optimization target for deep neural network models (DNNs) …

Modular representations emerge in neural networks trained to perform context-dependent tasks

WJ Johnston, S Fusi - bioRxiv, 2024 - biorxiv.org
The brain has a well-known large scale organization at the level of brain regions, where
different regions have been argued to primarily serve different behavioral functions. This …

AERO: Softmax-Only LLMs for Efficient Private Inference

NK Jha, B Reagen - arXiv preprint arXiv:2410.13060, 2024 - arxiv.org
The pervasiveness of proprietary language models has raised privacy concerns for users'
sensitive data, emphasizing the need for private inference (PI), where inference is performed …

ReLU's Revival: On the Entropic Overload in Normalization-Free Large Language Models

NK Jha, B Reagen - arXiv preprint arXiv:2410.09637, 2024 - arxiv.org
LayerNorm is a critical component in modern large language models (LLMs) for stabilizing
training and ensuring smooth optimization. However, it introduces significant challenges in …

Poisson Variational Autoencoder

H Vafaii, D Galor, JL Yates - arXiv preprint arXiv:2405.14473, 2024 - arxiv.org
Variational autoencoders (VAE) employ Bayesian inference to interpret sensory inputs,
mirroring processes that occur in primate vision across both ventral (Higgins et al., 2021) …

Coordinated Response Modulations Enable Flexible Use of Visual Information

R Srinath, MM Czarnik, MR Cohen - bioRxiv, 2024 - pmc.ncbi.nlm.nih.gov
We use sensory information in remarkably flexible ways. We can generalize by ignoring task-
irrelevant features, report different features of a stimulus, and use different actions to report a …

Single-unit activations confer inductive biases for emergent circuit solutions to cognitive tasks

P Tolmachev, TA Engel - bioRxiv, 2024 - biorxiv.org
Trained recurrent neural networks (RNNs) have become the leading framework for modeling
neural dynamics in the brain, owing to their capacity to mimic how population-level …

Parsing the Geometry of Distributed Representations

M Alleman - 2024 - search.proquest.com
The progression of neuroscience relies on the discovery of structure in the brain. From the
discovery of neurons to the structure of the potassium channel, and, in recent years, the …

Contrastive-Equivariant Self-Supervised Learning Improves Alignment with Primate Visual Area IT

TE Yerxa, J Feather, EP Simoncelli… - The Thirty-eighth Annual … - openreview.net
Models trained with self-supervised learning objectives have recently matched or surpassed
models trained with traditional supervised object recognition in their ability to predict neural …

Delays in generalization match delayed changes in representational geometry

X Zheng, K Daruwalla, AS Benjamin, D Klindt - UniReps: 2nd Edition of the … - openreview.net
Delayed generalization, also known as``grokking'', has emerged as a well-replicated
phenomenon in overparameterized neural networks. Recent theoretical works associated …