Comparing neural‐and N‐gram‐based language models for word segmentation

Y Doval, C Gómez‐Rodríguez - Journal of the Association for …, 2019 - Wiley Online Library
Word segmentation is the task of inserting or deleting word boundary characters in order to
separate character sequences that correspond to words in some language. In this article we …

[PDF][PDF] Transition-based neural word segmentation

M Zhang, Y Zhang, G Fu - … of the 54th Annual Meeting of the …, 2016 - aclanthology.org
Character-based and word-based methods are two main types of statistical models for
Chinese word segmentation, the former exploiting sequence labeling models over …

Mingmatch—a fast n-gram model for word segmentation of the ainu language

K Nowakowski, M Ptaszynski, F Masui - Information, 2019 - mdpi.com
Word segmentation is an essential task in automatic language processing for languages
where there are no explicit word boundary markers, or where space-delimited orthographic …

A new unsupervised approach to word segmentation

H Wang, J Zhu, S Tang, X Fan - Computational Linguistics, 2011 - direct.mit.edu
This article proposes ESA, a new unsupervised approach to word segmentation. ESA is an
iterative process consisting of 3 phases: Evaluation, Selection, and Adjustment. In …

Transition-based neural word segmentation using word-level features

M Zhang, Y Zhang, G Fu - Journal of Artificial Intelligence Research, 2018 - jair.org
Character-based and word-based methods are two different solutions for Chinese word
segmentation, the former exploiting sequence labeling models over characters and the latter …

Incorporating word attention into character-based word segmentation

S Higashiyama, M Utiyama, E Sumita… - Proceedings of the …, 2019 - aclanthology.org
Neural network models have been actively applied to word segmentation, especially
Chinese, because of the ability to minimize the effort in feature engineering. Typical …

Adaptive Chinese word segmentation

J Gao, A Wu, M Li, CN Huang, H Li, X Xia, H Qin - 2004 - microsoft.com
This paper presents a Chinese word segmentation system which can adapt to different
domains and standards. We first present a statistical framework where domain-specific …

[PDF][PDF] Feature-based neural language model and chinese word segmentation

M Mansur, W Pei, B Chang - Proceedings of the Sixth International …, 2013 - aclanthology.org
In this paper we introduce a feature-based neural language model, which is trained to
estimate the probability of an element given its previous context features. In this way our …

[PDF][PDF] Word-based and character-based word segmentation models: Comparison and combination

W Sun - Coling 2010: Posters, 2010 - aclanthology.org
We present a theoretical and empirical comparative analysis of the two dominant categories
of approaches in Chinese word segmentation: word-based models and character-based …

[PDF][PDF] Dependency-based gated recursive neural network for chinese word segmentation

J Xu, X Sun - Proceedings of the 54th Annual Meeting of the …, 2016 - aclanthology.org
Recently, many neural network models have been applied to Chinese word segmentation.
However, such models focus more on collecting local information while long distance …