Deep neural-network-based language models (LMs) are increasingly applied to large-scale
protein sequence data to predict protein function. However, being largely black-box models …

State-of-the-art models in natural language processing rely on separate rigid subword
tokenization algorithms, which limit their generalization ability and adaptation to new …

What are the units of text that we want to model? From bytes to multi-word expressions, text
can be analyzed and generated at many granularities. Until recently, most natural language …

Most current popular subword tokenizers are trained based on word frequency statistics over
a corpus, without considering information about co-occurrence or context. Nevertheless, the …

The Reinforcement Learning from Human Feedback (RLHF) plays a pivotal role in shaping
the impact of large language models (LLMs), contributing significantly to controlling output …

Abstract Many Natural Language Processing (NLP) tasks are labeled on the token level,
forthese tasks, the first step is to identify the tokens (tokenization). Becausethis step is often …

Abstract Domain adaptation through fine-tuning is a well-established strategy to tailor a
neural network model trained on a general-domain for a specific target-domain. During the …

This paper proposes a method to optimize tokenization for the performance improvement of
already trained downstream models. Our method generates tokenization results attaining …

Is preferred tokenization for humans also preferred for machine-learning (ML) models? This
study examines the relations between preferred tokenization for humans (appropriateness …

In recent years, the use of distributed representations has been a fundamental technology
for natural language processing. However, Japanese has multiple compound words, and …