Implicit regularization of dropout
It is important to understand how dropout, a popular regularization method, aids in achieving
a good generalization solution during neural network training. In this work, we present a …
a good generalization solution during neural network training. In this work, we present a …
Initialization is Critical to Whether Transformers Fit Composite Functions by Inference or Memorizing
Transformers have shown impressive capabilities across various tasks, but their
performance on compositional problems remains a topic of debate. In this work, we …
performance on compositional problems remains a topic of debate. In this work, we …
Adaptive Methods through the Lens of SDEs: Theoretical Insights on the Role of Noise
Despite the vast empirical evidence supporting the efficacy of adaptive optimization methods
in deep learning, their theoretical understanding is far from complete. This work introduces …
in deep learning, their theoretical understanding is far from complete. This work introduces …
Initialization is Critical to Whether Transformers Fit Composite Functions by Reasoning or Memorizing
Transformers have shown impressive capabilities across various tasks, but their
performance on compositional problems remains a topic of debate. In this work, we …
performance on compositional problems remains a topic of debate. In this work, we …