State-of-the-art models in natural language processing rely on separate rigid subword
tokenization algorithms, which limit their generalization ability and adaptation to new …

What are the units of text that we want to model? From bytes to multi-word expressions, text
can be analyzed and generated at many granularities. Until recently, most natural language …

Tokenization is an important text preprocessing step to prepare input tokens for deep
language models. WordPiece and BPE are de facto methods employed by important …

The SIGMORPHON 2022 shared task on morpheme segmentation challenged systems to
decompose a word into a sequence of morphemes and covered most types of morphology …

Most current popular subword tokenizers are trained based on word frequency statistics over
a corpus, without considering information about co-occurrence or context. Nevertheless, the …

This paper presents a comprehensive examination of the impact of tokenization strategies
and vocabulary sizes on the performance of Arabic language models in downstream natural …

Multilingual pretrained models are effective for machine translation and cross-lingual
processing because they contain multiple languages in one model. However, they are …

Abstract Domain adaptation through fine-tuning is a well-established strategy to tailor a
neural network model trained on a general-domain for a specific target-domain. During the …

We propose a new technique for computational language representation called
elementwise embedding, in which a material (semantic unit) is abstracted into a horizontal …

This paper proposes a method to optimize tokenization for the performance improvement of
already trained downstream models. Our method generates tokenization results attaining …