Causal interpretation of self-attention in pre-trained transformers

RY Rohekar, Y Gurwicz… - Advances in Neural …, 2024 - proceedings.neurips.cc
We propose a causal interpretation of self-attention in the Transformer neural network
architecture. We interpret self-attention as a mechanism that estimates a structural equation …

Demystifying deep reinforcement learning-based autonomous vehicle decision-making

H Wan, P Li, A Kusari - arXiv preprint arXiv:2403.11432, 2024 - arxiv.org
With the advent of universal function approximators in the domain of reinforcement learning,
the number of practical applications leveraging deep reinforcement learning (DRL) has …

OCDB: Revisiting Causal Discovery with a Comprehensive Benchmark and Evaluation Framework

W Zhou, H Huang, G Zhang, R Shi, K Yin, Y Lin… - arXiv preprint arXiv …, 2024 - arxiv.org
Large language models (LLMs) have excelled in various natural language processing tasks,
but challenges in interpretability and trustworthiness persist, limiting their use in high-stakes …

Transformer-aided dynamic causal model for scalable estimation of effective connectivity

S Nag, K Uludag - Imaging Neuroscience, 2024 - direct.mit.edu
Abstract Dynamic Causal Models (DCMs) in functional Magnetic Resonance Imaging (fMRI)
decipher causal interactions, known as Effective Connectivity, among neuronal populations …

Causal Representation Learning in Temporal Data via Single-Parent Decoding

P Brouillard, S Lachapelle, J Kaltenborn… - arXiv preprint arXiv …, 2024 - arxiv.org
Scientific research often seeks to understand the causal structure underlying high-level
variables in a system. For example, climate scientists study how phenomena, such as El Ni …

Fail-Active Autonomy in Unknown-Unknown Situations for Deep Space Missions

C Debrunner, E Dixon, B Hockman… - 2024 IEEE …, 2024 - ieeexplore.ieee.org
This paper discusses the environment anomalies that the CARL/COLDArm system was
subjected to utilizing the OceanWATERS simulator to demonstrate" fail-active" autonomy …

Towards Causal Representations of Climate Model Data

J Boussard, C Nagda, J Kaltenborn, CEE Lange… - arXiv preprint arXiv …, 2023 - arxiv.org
Climate models, such as Earth system models (ESMs), are crucial for simulating future
climate change based on projected Shared Socioeconomic Pathways (SSP) greenhouse …

Verification, Validation, and Calibration Through a Causal Lens

R Gonzales, D Mandelli, C Wang… - Verification and …, 2024 - asmedigitalcollection.asme.org
While typical validation and verification approaches focus on identifying the associations
between data elements using statistical and machine learning methods, the novel methods …