Gronroos , S-A , Virpioja , S & Kurimo , M 2021 , ' Transfer learning and subword sampling for asymmetric-resource one-to-many neural translation ' , Machine Translation , vol. 34 , pp. 251-286 . https://doi.org/10.1007/s10590-020-09253-x
Title: | Transfer learning and subword sampling for asymmetric-resource one-to-many neural translation |
Author: | Gronroos, Stig-Arne; Virpioja, Sami; Kurimo, Mikko |
Contributor organization: | Department of Digital Humanities Language Technology |
Date: | 2021-01-30 |
Language: | eng |
Number of pages: | 36 |
Belongs to series: | Machine Translation |
ISSN: | 0922-6567 |
DOI: | https://doi.org/10.1007/s10590-020-09253-x |
URI: | http://hdl.handle.net/10138/330171 |
Abstract: | There are several approaches for improving neural machine translation for low-resource languages: monolingual data can be exploited via pretraining or data augmentation; parallel corpora on related language pairs can be used via parameter sharing or transfer learning in multilingual models; subword segmentation and regularization techniques can be applied to ensure high coverage of the vocabulary. We review these approaches in the context of an asymmetric-resource one-to-many translation task, in which the pair of target languages are related, with one being a very low-resource and the other a higher-resource language. We test various methods on three artificially restricted translation tasks—English to Estonian (low-resource) and Finnish (high-resource), English to Slovak and Czech, English to Danish and Swedish—and one real-world task, Norwegian to North Sámi and Finnish. The experiments show positive effects especially for scheduled multi-task learning, denoising autoencoder, and subword sampling. There are several approaches for improving neural machine translation for low-resource languages: monolingual data can be exploited via pretraining or data augmentation; parallel corpora on related language pairs can be used via parameter sharing or transfer learning in multilingual models; subword segmentation and regularization techniques can be applied to ensure high coverage of the vocabulary. We review these approaches in the context of an asymmetric-resource one-to-many translation task, in which the pair of target languages are related, with one being a very low-resource and the other a higher-resource language. We test various methods on three artificially restricted translation tasks-English to Estonian (low-resource) and Finnish (high-resource), English to Slovak and Czech, English to Danish and Swedish-and one real-world task, Norwegian to North Sami and Finnish. The experiments show positive effects especially for scheduled multi-task learning, denoising autoencoder, and subword sampling. |
Subject: |
113 Computer and information sciences
6121 Languages |
Peer reviewed: | Yes |
Rights: | unspecified |
Usage restriction: | openAccess |
Self-archived version: | acceptedVersion |
Funder: | European Commission / Horizon 2020 European Commission |
Grant number: | 771113 771113 |
Total number of downloads: Loading...
Files | Size | Format | View |
---|---|---|---|
2004.04002.pdf | 646.5Kb |
View/ |
|
2020_specialissue.pdf | 511.0Kb |
View/ |