Transformers are arguably the main workhorse in recent natural language processing
research. By definition, a Transformer is invariant with respect to reordering of the input …

Self-supervised deep language modeling has shown unprecedented success across natural
language tasks, and has recently been repurposed to biological sequences. However …

Large language models have been shown to struggle with multi-step reasoning, and do not
retain previous reasoning steps for future use. We propose a simple method for solving both …

Attentional mechanisms are order-invariant. Positional encoding is a crucial component to
allow attention-based deep model architectures such as Transformer to address sequences …

Abstract The evolution of Neural Machine Translation (NMT) has been significantly
influenced by six core challenges (Koehn and Knowles,) that have acted as benchmarks for …

Personalized chatbots focus on endowing chatbots with a consistent personality to behave
like real users, give more informative responses, and further act as personal assistants …

Transformer-based sequence-to-sequence architectures, while achieving state-of-the-art
results on a large number of NLP tasks, can still suffer from overfitting during training. In …

Short texts (STs) present in a variety of scenarios, including query, dialog, and entity names.
Most of the exciting studies in neural machine translation (NMT) are focused on tackling …

Position representation is crucial for building position-aware representations in
Transformers. Existing position representations suffer from a lack of generalization to test …

Large-scale crop type classification is a task at the core of remote sensing efforts with
applications of both economic and ecological importance. Current state-of-the-art deep …