MetaHOPE: metaphor translation evaluation framework investigating open-source LLMs and state-of-the-art neural translation models

Liang, J.; Han, L. (2026)

Research paper

In this paper, we propose MetaHOPE, an error severity-aware annotation framework for evaluating metaphor translations. 
Metaphors present challenges for machine translation (MT) and natural language understanding and processing (NLU, NLP), because it presents the features of semantic complexity, contextual dependency, and cultural embeddings that can lead to ambiguity issues for NLP models. 
To investigate how state-of-the-art NLP models perform on translating metaphors, we select three representative systems, i.e., GoogleMT, GPT5.4, and Hunyuan-7b as Neural MT (NMT) models and LLMs. We used two human-annotated metaphor corpora, including VUAMC and PSUCMC, for English-to-Chinese and Chinese-to-English translation purposes.
The original corpora we used are monolingual, where we carried out error annotation using the MetaHOPE framework and also produced the human post-edited gold reference for bilingual use as a new resource.
We believe the MetaHOPE evaluation framework for metaphor translation annotation, the parallel corpora resources, and the error analysis on SOTA automatic translation models can be useful and shed some light for the field of metaphor translation study.
We share our \textit{resources} online (via \url{https://github.com/Jiahui84/MetaHOPE}).

• Metaphor translation
• Translation evaluation
• Evaluation framework
• Human and automatic evaluation

• All authors: Liang, J.; Han, L.
• Date: 2026
• Title of host publication: 9th International Conference on Natural Language and Speech Processing (ICNLSP 2026)
• Link: https://doi.org/10.48550/arXiv.2607.00848