What are the units of text that we want to model? From bytes to multi-word expressions, text
can be analyzed and generated at many granularities. Until recently, most natural language …

Sub-word segmentation is an essential pre-processing step for Neural Machine Translation
(NMT). Existing work has shown that neural sub-word segmenters are better than Byte-Pair …

Subword segmenters like BPE operate as a preprocessing step in neural machine
translation and other (conditional) language models. They are applied to datasets before …

Subwords have become the standard units of text in NLP, enabling efficient open-
vocabulary models. With algorithms like byte-pair encoding (BPE), subword segmentation is …

Unsupervised Chinese word segmentation (UCWS) has made progress by incorporating
linguistic knowledge from pre-trained language models using parameter-free probing …

Morphological segmentation and stemming are foundational tasks in natural language
processing. They have become effective ways to alleviate data sparsity in agglutinative …

We show that unsupervised sequence-segmentation performance can be transferred to
extremely low-resource languages by pre-training a Masked Segmental Language Model …

Word segmentation task is the cornerstone of text processing. There are 7111 languages
worldwide, most of which are low-resource languages. This paper attempts to solve the …

Existing subword segmenters are either 1) frequency-based without semantics information
or 2) neural-based but trained on parallel corpora. To address this, we present BERTSeg, an …

Abstract Language models have always been a fundamental NLP tool and application. This
thesis focuses on open-vocabulary language models, ie, models that can deal with novel …