Palm: Scaling language modeling with pathways

A Chowdhery, S Narang, J Devlin, M Bosma… - Journal of Machine …, 2023 - jmlr.org
Large language models have been shown to achieve remarkable performance across a
variety of natural language tasks using few-shot learning, which drastically reduces the …

Beyond english-centric multilingual machine translation

A Fan, S Bhosale, H Schwenk, Z Ma, A El-Kishky… - Journal of Machine …, 2021 - jmlr.org
Existing work in translation demonstrated the potential of massively multilingual machine
translation by training a single model able to translate between any pair of languages …

Findings of the 2019 conference on machine translation (WMT19)

L Barrault, O Bojar, MR Costa-Jussa, C Federmann… - 2019 - zora.uzh.ch
This paper presents the results of the premier shared task organized alongside the
Conference on Machine Translation (WMT) 2019. Participants were asked to build machine …

Survey of low-resource machine translation

B Haddow, R Bawden, AVM Barone, J Helcl… - Computational …, 2022 - direct.mit.edu
We present a survey covering the state of the art in low-resource machine translation (MT)
research. There are currently around 7,000 languages spoken in the world and almost all …

Pre-training multilingual neural machine translation by leveraging alignment information

Z Lin, X Pan, M Wang, X Qiu, J Feng, H Zhou… - arXiv preprint arXiv …, 2020 - arxiv.org
We investigate the following question for machine translation (MT): can we develop a single
universal MT model to serve as the common seed and obtain derivative and improved …

Improving neural machine translation by bidirectional training

L Ding, D Wu, D Tao - arXiv preprint arXiv:2109.07780, 2021 - arxiv.org
We present a simple and effective pretraining strategy--bidirectional training (BiT) for neural
machine translation. Specifically, we bidirectionally update the model parameters at the …

On compositional generalization of neural machine translation

Y Li, Y Yin, Y Chen, Y Zhang - arXiv preprint arXiv:2105.14802, 2021 - arxiv.org
Modern neural machine translation (NMT) models have achieved competitive performance
in standard benchmarks such as WMT. However, there still exist significant issues such as …

Towards continual learning for multilingual machine translation via vocabulary substitution

X Garcia, N Constant, AP Parikh, O Firat - arXiv preprint arXiv:2103.06799, 2021 - arxiv.org
We propose a straightforward vocabulary adaptation scheme to extend the language
capacity of multilingual machine translation models, paving the way towards efficient …

Knowledge distillation for multilingual unsupervised neural machine translation

H Sun, R Wang, K Chen, M Utiyama, E Sumita… - arXiv preprint arXiv …, 2020 - arxiv.org
Unsupervised neural machine translation (UNMT) has recently achieved remarkable results
for several language pairs. However, it can only translate between a single language pair …

Shallow-to-deep training for neural machine translation

B Li, Z Wang, H Liu, Y Jiang, Q Du, T Xiao… - arXiv preprint arXiv …, 2020 - arxiv.org
Deep encoders have been proven to be effective in improving neural machine translation
(NMT) systems, but training an extremely deep encoder is time consuming. Moreover, why …