Towards a resource for news domain bilingual metaphor-annotated corpus: LLMs walk on thin ice when translating metaphors

Abstract

Metaphor remains challenging for machine

translation (MT), as its interpretation often

depends on discourse context and culturally

embedded meanings that cannot be recovered from literal expressions alone. However, systematic research on metaphor translation is constrained by the limited availability of bilingual resources combining source metaphor annotations, cross-lingual alignments, and fine-grained human evaluations. To address this gap, we introduce \textsc{NewsMeta}, a bidirectional English--Chinese metaphor-annotated resource, and extend MetaHOPE, a metaphor-oriented human evaluation framework previously examined in a 20-segment pilot study. \textsc{NewsMeta} comprises 400 metaphor-bearing source sentences, with 200 in each translation direction, containing 565 English-source and 368 Chinese-source metaphor-related word occurrences. The resource integrates context-aware machine translations, source-to-target metaphor alignments, human post-edited references, and annotations of five translation error categories and their severity. We evaluate GPT-5.4, Google Translate, and Hunyuan-MT-7B at both metaphor and sentence levels. Across both directions, GPT-5.4 receives the lowest overall mean sentence-level MetaHOPE penalties, followed by Google Translate and Hunyuan-MT-7B. Inter-annotator agreement is higher in the English-to-Chinese rollout than in the earlier pilot, although this comparison is descriptive; reliability remains uneven across error categories, with mistranslation judgments showing comparatively greater consistency. These findings highlight both the value of fine-grained metaphor-sensitive evaluation and the continuing challenges of human annotation. \textsc{NewsMeta} provides a reusable foundation for investigating cross-lingual metaphor realisation, translation strategies, and metaphor-related errors in MT and large language models.