Despite their wide adoption, the underlying training and memorization dynamics of very
large language models is not well understood. We empirically study exact memorization in …

Machine translation systems based on deep neural networks are expensive to train.
Curriculum learning aims to address this issue by choosing the order in which samples are …

Abstract Spelling Error Correction (SEC) that requires high-level language understanding is
a challenging but useful task. Current SEC approaches normally leverage a pre-training …

Curriculum learning methods typically rely on heuristics to estimate the difficulty of training
examples or the ability of the model. In this work, we propose replacing difficulty heuristics …

Highlights Corn damage detection was possible using advanced deep learning and
computer vision techniques trained with images of simulated corn lodging. RetinaNet and …

We present a generic and trend-aware curriculum learning approach that effectively
integrates textual and structural information in text graphs for relation extraction between …

A curriculum is a planned sequence of learning materials and an effective one can make
learning efficient and effective for both humans and machines. Recent studies developed …

Cross-lingual continual learning aims to continuously fine-tune a downstream model on
emerging data from new languages. One major challenge in cross-lingual continual learning …

We introduce the problem of curriculum discovery and describe a curriculum learning
framework capable of discovering effective curricula in a curriculum space based on prior …

Replay methods have shown to be successful in mitigating catastrophic forgetting in
continual learning scenarios despite having limited access to historical data. However …