RIBES: Rank-based Intuitive Bilingual Evaluation Score

What's RIBES?

RIBES is an automatic evaluation metric for machine translation, developed in NTT Communication Science Labs.
This website distributes its implementation in Python.

Download

The program is distributed based on GNU General Public License (ver.2)．
You have to confirm the agreement with the license terms before download.
Click to Download : Latest version 1.03.1 (2014/9/8) -- fixed a compatibility problem of Python 2's split / introduced a new option -z/--emptryref to allow blank lines in references
Notice (2014/10/01) Fixed a download link problem for version 1.03.1. Please download it again for the latest version.
Click to Download : Version 1.03 (2014/8/13) -- supports Python 2.6 or later / only supports utf-8 / fixed the multibyte white space issue (Thanks to Graham Neubig)
Click to Download : Version 1.02.4 (2013/12/18) -- fixed a problem in word alignment
Click to Download : Version 1.02.3 (2012/2/23)
Click to Download : Version 1.01 (2011/8/10)
* Requires Python 2.6 or later (Python 3.0 is not supported. RIBES-1.02.4 or earlier only works with Python 3.1 or later)．

How to Use

$ python RIBES.py -r REFERENCE_TRANSLATION EVALUATING_TRANSLATION

Also you can see other options by the following:

$ python RIBES.py -h

File Format

Reference and evaluating files are plain text files, not SGML or XML.
Reference and evaluating files must have the same number of lines.
Each line must be tokenized including punctuations (The program does not change tokenization).

References

Hideki Isozaki, Tsutomu Hirao, Kevin Duh, Katsuhito Sudoh, Hajime Tsukada
Automatic Evaluation of Translation Quality for Distant Language Pairs (PDF)
Conference on Empirical Methods on Natural Language Processing (EMNLP), Oct. 2010.

Contact: (the name of our evaluation measure, lowercased)@lab.ntt.co.jp