Transformer is the state-of-the-art model in recent machine translation evaluations. Two
strands of research are promising to improve models of this kind: the first uses wide …

Abstract We show that Neural Ordinary Differential Equations (ODEs) learn representations
that preserve the topology of the input space and prove that this implies the existence of …

We introduce the framework of continuous--depth graph neural networks (GNNs). Graph
neural ordinary differential equations (GDEs) are formalized as the counterpart to GNNs …

Partial differential equations (PDEs) are commonly derived based on empirical
observations. However, recent advances of technology enable us to collect and store …

Deep learning achieves state-of-the-art results in many tasks in computer vision and natural
language processing. However, recent works have shown that deep networks can be …

Single image super-resolution, as a high dimensional structured prediction problem, aims to
characterize fine-grain information given a low-resolution sample. Recent advances in …

Residual neural networks can be viewed as the forward Euler discretization of an Ordinary
Differential Equation (ODE) with a unit time step. This has recently motivated researchers to …

Neural ordinary differential equations (ODEs) have been attracting increasing attention in
various research domains recently. There have been some works studying optimization …

In this paper, we introduce a modular deep neural network (DNN) framework for data-driven
reduced order modeling of dynamical systems relevant to fluid flows. We propose various …

The Transformer architecture is widely used in natural language processing. Despite its
success, the design principle of the Transformer remains elusive. In this paper, we provide a …