Artificial intelligence (AI), and, in particular, deep learning as a subcategory of AI, provides
opportunities for the discovery and development of innovative drugs. Various machine …

The past year has witnessed rapid advances in sequence-to-sequence (seq2seq) modeling
for Machine Translation (MT). The classic RNN-based approaches to MT were first out …

In this paper we propose to study generalization of neural networks on small algorithmically
generated datasets. In this setting, questions about data efficiency, memorization …

In humans, Attention is a core property of all perceptual and cognitive operations. Given our
limited ability to process competing sources, attention mechanisms select, modulate, and …

Artificial intelligence (AI) has undergone a renaissance recently, making major progress in
key domains such as vision, language, control, and decision-making. This has been due, in …

We have devised and implemented a novel computational strategy for de novo design of
molecules with desired properties termed ReLeaSE (Reinforcement Learning for Structural …

A major limitation for the broader scope of problems solvable by transformers is the
quadratic scaling of computational complexity with input size. In this study, we investigate …

Neural networks are a family of powerful machine learning models. This book focuses on the
application of neural network models to natural language data. The first half of the book …

The reparameterization trick enables optimizing large scale stochastic computation graphs
via gradient descent. The essence of the trick is to refactor each stochastic node into a …

The original" Seven Motifs" set forth a roadmap of essential methods for the field of scientific
computing, where a motif is an algorithmic method that captures a pattern of computation …