What are the units of text that we want to model? From bytes to multi-word expressions, text
can be analyzed and generated at many granularities. Until recently, most natural language …

Objective This study aims at realizing unsupervised term discovery in Chinese electronic
health records (EHRs) by using the word segmentation technique. The existing supervised …

We propose a segmental neural language model that combines the generalization power of
neural networks with the ability to discover word-like units that are latent in unsegmented …

Sub-word segmentation is an essential pre-processing step for Neural Machine Translation
(NMT). Existing work has shown that neural sub-word segmenters are better than Byte-Pair …

Segmentation remains an important preprocessing step both in languages where" words" or
other important syntactic/semantic units (like morphemes) are not clearly delineated by white …

We propose an unsupervised word segmentation model, in which for each unlabelled
sentence sample, the learning objective is to maximize the generation probability of the …

Subword segmenters like BPE operate as a preprocessing step in neural machine
translation and other (conditional) language models. They are applied to datasets before …

Subwords have become the standard units of text in NLP, enabling efficient open-
vocabulary models. With algorithms like byte-pair encoding (BPE), subword segmentation is …

Unsupervised Chinese word segmentation (UCWS) has made progress by incorporating
linguistic knowledge from pre-trained language models using parameter-free probing …

This article introduces an approach that learns segment-level context for sequence labeling
in natural language processing (NLP). Previous approaches limit their basic unit to a word …