Overview
We collected a large amount of web data and created JParaCrawl, a large-scale Japanese-English parallel corpus of over 10 million words. Small-scale parallel data from a specific field, such as medicine or finance, can be combined with JParaCrawl, a large general-purpose translation data bank, making it easy to create machine translation systems geared toward specific fields.
Background / Issues
The use of neural networks has greatly improved the accuracy of machine translation. Daily conversations and short sentences can be translated in practical accuracy. However, there are problems in highly specialized fields such as medicine, finance and law, where the lack of available translation data has made it difficult to obtain sufficient translation accuracy.
Advantages of this technology
- One of the world’s largest Japanese-English parallel corpus, including text from a wide range of fields collected from the web
Use Scene
- Neural machine translation models trained on the Japanese-English parallel corpus, JParaCrawl, can be adapted to the fields of medicine, finance, etc.
Explanatory chart
Technical explanation
We created large-scale Japanese-English parallel corpus JParaCrawl by crawling websites that contain large volumes of parallel data, detecting translated pairs of web pages and extracting pairs of sentences that are translations of each other.
By using JParaCrawl to create a translation model in advance, and then using parallel data from a specific target field to fine-tune the model, it is possible to create a highly accurate translation system with a short training time.
Department in charge
NTT Communication Science Laboratories - Innovative Communication Laboratory