Autores: | Jesús Giménez and Lluís Màrquez |
URL: | http://www.lsi.upc.edu/~nlp/IQMT |
Contacto: | Jesús Giménez <jgimenezlsi.upc.edu> |
Descripción
A common workbench on which automatic MT evaluation metrics can be robustly used and combined for the purpose of MT system development. Current version includes a rich set of metrics operating at different linguistic levels (lexical, shallow syntactic, syntactic, and shallow semantic).
Funcionalidad
Tecnología
Perl.
Requisitos técnicos
The followinf requirements are optional: Individual metrics (BLEU, NIST, GTM, ROUGE, METEOR, …); Linguistic processors (C&C Tools); SwiRL; BIOS; Charniak Parser; MINIPAR dependency parser; SVMTool.
Módulos
Metrics: Lexical Similarity; Syntactic Similarity (Shallow parsing, Dependency parsing, Constituency parsing); Semantic Similarity (Named entities, Semantic roles, Discourse representations); QARLA Framework.
Innovación
IQmt allows for evaluation of translation quality aspects at different linguistic levels. Metric scores can also be combined into a single measure of quality, either through QARLA, of via arithmetic mean.
Desarrollo
Publicaciones
- Jesús Giménez and Enrique Amigó. IQMT: A Framework for Automatic Machine Translation Evaluation. Proceedings of the 5th International Conference on Language Resources and Evaluation (LREC’06). Genoa, Italy, 22-28 May. 2006. Departament Research Technical Report (LSI-07-29-R).
- Jesús Giménez and Lluís Màrquez. Linguistic Features for Automatic Evaluation of Heterogeneous MT Systems. Proceedings of WMT 2007 (ACL’07).
- Jesús Giménez and Lluís Màrquez. A Smorgasbord of Features for Automatic MT Evaluation. Proceedings of the 3rd ACL Workshop on Statistical Machine Translation (shared evaluation task).