The future is log-Gaussian: ResNets and their infinite-depth-and-width limit at initialization
Theoretical results show that neural networks can be approximated by Gaussian processes
in the infinite-width limit. However, for fully connected networks, it has been previously …
in the infinite-width limit. However, for fully connected networks, it has been previously …
Towards training without depth limits: Batch normalization without gradient explosion
Normalization layers are one of the key building blocks for deep neural networks. Several
theoretical studies have shown that batch normalization improves the signal propagation, by …
theoretical studies have shown that batch normalization improves the signal propagation, by …
Zero initialization: Initializing neural networks with only zeros and ones
Deep neural networks are usually initialized with random weights, with adequately selected
initial variance to ensure stable signal propagation during training. However, selecting the …
initial variance to ensure stable signal propagation during training. However, selecting the …
Zero initialization: Initializing residual networks with only zeros and ones
Deep neural networks are usually initialized with random weights, with adequately selected
initial variance to ensure stable signal propagation during training. However, there is no …
initial variance to ensure stable signal propagation during training. However, there is no …
Neuroadaptive Fault-Tolerant Control Embedded With Diversified Activating Functions With Application to Auto-Driving Vehicles Under Fading Actuation
Z Gao, W Yu, J Yan - IEEE Transactions on Neural Networks …, 2023 - ieeexplore.ieee.org
This article presents a neuroadaptive fault-tolerant control method for path tracking of
multiinput multioutput (MIMO) systems in the presence of modeling uncertainties and …
multiinput multioutput (MIMO) systems in the presence of modeling uncertainties and …
Artificial intelligence methods for security and cyber security systems
RN Rudd-Orthner - 2022 - etheses.whiterose.ac.uk
This research is in threat analysis and countermeasures employing Artificial Intelligence (AI)
methods within the civilian domain, where safety and mission-critical aspects are essential …
methods within the civilian domain, where safety and mission-critical aspects are essential …
Parseval Regularization for Continual Reinforcement Learning
Loss of plasticity, trainability loss, and primacy bias have been identified as issues arising
when training deep neural networks on sequences of tasks--all referring to the increased …
when training deep neural networks on sequences of tasks--all referring to the increased …
Leveraging Sub-Optimal Data for Human-in-the-Loop Reinforcement Learning
C Muslimani, ME Taylor - arXiv preprint arXiv:2405.00746, 2024 - arxiv.org
To create useful reinforcement learning (RL) agents, step zero is to design a suitable reward
function that captures the nuances of the task. However, reward engineering can be a …
function that captures the nuances of the task. However, reward engineering can be a …
Marginalizable density models
Probability density models based on deep networks have achieved remarkable success in
modeling complex high-dimensional datasets. However, unlike kernel density estimators …
modeling complex high-dimensional datasets. However, unlike kernel density estimators …
Toward Generate-and-Test Algorithms for Continual Feature Discovery
P Rahman - 2021 - era.library.ualberta.ca
The backpropagation algorithm is a fundamental algorithm for training modern artificial
neural networks (ANNs). However, it is known the backpropagation algorithm performs …
neural networks (ANNs). However, it is known the backpropagation algorithm performs …