We posed the shared task of assigning sentence-level quality scores for a very noisy corpus
of sentence pairs crawled from the web, with the goal of sub-selecting 1% and 10% of high …

Abstract Following the WMT 2018 Shared Task on Parallel Corpus Filtering, we posed the
challenge of assigning sentence-level quality scores for very noisy corpora of sentence pairs …

Following two preceding WMT Shared Task on Parallel Corpus Filtering (Koehn et al., 2018,
2019), we posed again the challenge of assigning sentence-level quality scores for very …

Abstract This paper describes Prompsit Language Engineering's submissions to the WMT
2018 parallel corpus filtering shared task. Our four submissions were based on an automatic …

Web-crawled data provides a good source of parallel corpora for training machine
translation models. It is automatically obtained, but extremely noisy, and recent work shows …

This paper introduces OpusFilter, a flexible and modular toolbox for filtering parallel corpora.
It implements a number of components based on heuristic filters, language identification …

This paper shows the utility of two open-source tools designed for parallel data cleaning:
Bifixer and Bicleaner. Already used to clean highly noisy parallel content from crawled …

Machine translation is highly sensitive to the size and quality of the training data, which has
led to an increasing interest in collecting and filtering large parallel corpora. In this paper, we …

This paper describes the Microsoft submission to the WMT2018 news translation shared
task. We participated in one language direction--English-German. Our system follows …

In this work we introduce dual conditional cross-entropy filtering for noisy parallel data. For
each sentence pair of the noisy parallel corpus we compute cross-entropy scores according …