Calibrating Translation Decoding with Quality Estimation on LLMs

Thanks for your careful reading and insightful suggestions, and for noting our contribution. We summarize and address your concerns as follows:

"l. 150 the statement that the translation quality has not been used in previous works is not exactly accurate, see for example "A Preference-driven Paradigm for Enhanced Translation with Large Language Models" (Zhu et al., NAACL 2024)."

We agree and appreciate the clarification. We had misread the reference, thinking it referred to the original PL model. In fact, they reweight each pairwise comparison by the score difference. Thank you for pointing this out. We will revise our wording accordingly and include a discussion in the background section.

"If I understand correctly, the proposed approach uses listwise preference data through the proxy of the Pearson Loss, which is not really a fair comparison to e.g. CPO which only leverages a single rejected hypothesis. A fairer comparison would be towards A Preference-driven Paradigm for Enhanced Translation with Large Language Models (Zhu et al., NAACL 2024) where we leverage the metric scores as replacement for human annotations and optimize directly on listwise rejected hypothesis."

Thank you for your suggestion. We agree that a direct comparison would ideally strengthen our work. Unfortunately, [Zhu et al., 2024] did not release their codebase, and their implementation is based on weaker models such as LLaMA-1 or Mistral. To address your concern, we apply our method on Mistral using their data (WMT data and human annotations). Also, we report our results when applying calibration to the top-performing translation model, i.e., TowerMistral-7B [1], using their data.

To ensure comparability, we use the same beam search size of 4. The results are evaluated on the WMT22 test set using COMET as shown in the following table:

+ Best is supervised fine-tuning using best scoring samples, and + PL is the result of their implementation of the PL model. + Calibration is applying our method. It is easy to see that

1. On the weaker base models, such as Mistral-Ins, Calibration yields clear improvements over both SFT and PL. We provisionally attribute this gain to Calibration’s broader consideration of factors compared to PL. Specifically: (1) Calibration normalizes both likelihood and quality scores within a group, making the objective invariant to scale and shift. This promotes stable and robust optimization across diverse input distributions. (2) Calibration directly integrates translation quality via the metric function q(⋅∣x), and optimizes at the distribution level using the Pearson correlation. In contrast, [Zhu et al.] introduce reweighting for each pairwise comparison in PL modeling. In extreme cases, this approach may treat a comparison between ratings 2 (bad) and 1 (nonsense) the same as one between 4 (nearly perfect) and 3 (good), which might be overly heuristic.
2. Even when applying calibration to the top-performing system, TowerMistral, using human-annotated data, our system still achieves clear, albeit relatively smaller, gains, demonstrating that calibration training is also feasible with human annotations rather than machine-generated signals.

"The presented approach is only evaluated on the sentence level and doesn't venture into paragraph or document level machine translation which may come with different learning dynamics."

To address your concern, we collect paragraph-level data in four high-resource language pairs (en-de, en-fr, en-zh, and en-ru) from WMT24++ [1] for evaluation, following current representative practices for long-form data [2]. We merge sentence-level segments into paragraphs using the WMT24++ metadata. This process yields 221 paragraphs, each capped at 150 space-separated tokens in English to ensure compatibility with current evaluation metrics [3]. We follow all settings the same as those in Table 1 based on TowerMistral-7B. We report the results in three metrics, i.e., KIWI-XL, KIWI-XXL, and COMET, as follows:

It is easy to see that our calibration method consistently outperforms both SFT and CPO across all metrics, demonstrating its effectiveness in paragraph-level evaluation. That said, document-level evaluation remains an open challenge in current MT research. Practical measures are still under development and need to account for broader aspects such as consistency. These issues are beyond the scope of this paper and are left for future work.

[1] WMT24++: Expanding the Language Coverage of WMT24 to 55 Languages & Dialects

[2] Translating Step-by-Step: Decomposing the Translation Process for Improved Translation Quality of Long-Form Texts

[3] Training and Meta-Evaluating Machine Translation Evaluation Metrics at the Paragraph Level

"l. 218ff. / l.333f.: I don't think it is justified to say that the quality is "comparable to that of the current top-performing system, that is Tower-70B-v2 equipped with 100-time-sampling MBR/TRR"as there are still significant gaps towards that from Table 1 which are as big (and sometimes even bigger) than the gain achieved through this approach compared to CPO."

Let us clarify the comparison. In Table 1, our model, ​TowerInstruct-Mistral-7B + Calibration​, compared to ​Tower-70B-v2 + MBR/TRR​, achieves scores of 72.4 vs. 72.3 in en-de, 73.9 vs. 74.5 in en-es, 73.6 vs. 74.2 in en-ru, and 72.6 vs. 72.6 in en-zh. On average, it slightly lags behind Tower-70B-v2 + MBR/TRR (73.1 vs. 73.4). However, our model outperforms CPO across these four directions by +1.0 (73.1 vs. 72.1), and by +1.0 across all directions (73.9 vs. 72.9).

It is worth noting that Tower-70B+MBR/TRR is far from practical: (1) the base model is 10 times larger than our setting, and (2) MBR/TRR requires 100 times sampling followed by reranking. In contrast, our results rely solely on efficient MAP decoding, using beam search with a beam size of 5.

---

Thank you again for your careful reading of our paper and for your insightful suggestions and questions. Hopefully, our response addresses your concerns. We will incorporate this discussion in our next version. Also, we are more than happy to provide further explanation or discussion if needed.

Thanks for your insightful suggestions and for noting our contribution. We summarize and address your concerns as follows:

The relevance and differences compared to the SLiC paper.

Thank you for your careful reading and valuable suggestions. Our paper indeed shares part of the motivation with SLiC [Zhao et al., 2023][1]. We agree that a direct comparison would ideally strengthen our work. Unfortunately, [Zhao et al., 2023] did not release their codebase, and the implementation complexity, along with a lack of certain details, currently prevents us from conducting a direct comparison.

Broadly speaking, SLiC can be viewed as a preliminary variant of PL-based preference optimization. Although indirect, it operates in a similar manner to [Zhu et al., 2024][2], a PL-based PO method applied in the machine translation domain. To address your concern, we provide a direct comparison with this approach to demonstrate the effectiveness of our method.

To ensure comparability, (1) we follow their setting and apply our method on Mistral using their data (WMT data and human annotations). Also, we report our results when applying to the top-performing translation model, i.e., TowerMistral-7B [3], using their data. (2) We use the same beam search size of 4 in [Zhu et al., 2024]. The results are evaluated on the WMT22 test set using COMET as shown in the following table:

+ Best is supervised fine-tuning using best scoring samples, and + PL is the result of their implementation of the PL model [2]. + Calibration is applying our method. It is easy to see that

1. On the weaker base models, such as Mistral-Ins, Calibration yields clear improvements over both SFT and PL.
2. Even when applying calibration to the top-performing system, TowerMistral, our system still achieves clear, albeit relatively smaller, gains, demonstrating that calibration training is also feasible with human annotations rather than machine-generated signals.

Further, our work encompasses broader underlying concepts compared to [Zhao et al., 2023] in motivation, modeling, and analysis. We summarize and discuss them as follows:

1. [Zhao et al., 2023] aim to optimize candidate ranking based on their similarity to a target output, where similarity is computed using token or sentence representations. In contrast, our work proposes optimizing the Pearson correlation between translation likelihood and quality, rather than similarity. Moreover, the Pearson objective normalizes both likelihood and quality scores within a group, making it invariant to scale and shift. Our calibration approach directly incorporates translation quality via the metric function q(⋅∣x), optimizing at the distribution level using the Pearson correlation. By contrast, [Zhao et al., 2023] focus on pairwise comparisons, lacking a more direct utilization of absolute quality values at a distribution level.
2. In Section 5.2, we show that our objective, Pearson correlation itself, shares the goal with translation meta-evaluation. It provides a theoretical foundation for unifying translation quality optimization and quality evaluation within a single model.
3. In Section 5.3, we analyze the connection between calibration method and quality-aware decoding, and show that calibration captures the benefits of Best-of-N sampling (a key form of QAD) during training time. This, in turn, enhances the effectiveness of MAP decoding strategies such as beam search.

[1] Calibrating Sequence Likelihood Improves Conditional Language Generation

[2] A Preference-driven Paradigm for Enhanced Translation with Large Language Models

[3] Tower: An Open Multilingual Large Language Model for Translation-Related Tasks

“Another small concern is that the translation quality improvement of human evaluation shown in Figure 4 is far more significant than the automatic evaluation improvement in Table 1. Does it mean that the automatic evaluation metrics like XCOMET and KIWI-XL could not fully reflect the improvement of this method? If so, is it problematic (or can be improved) to use COMET as the translation quality score for likelihood calibration.”

Thanks for these insightful questions. We try to address them as follows:

1. Firstly, in Table 1, our method achieves +2.8 KIWI-XL and +2.7 XCOMET improvements. It is worth noting that these gains are indeed big. [Kocmi et al., 2024][4] provide a collective understanding of the meaning of differences in different metrics, emperically showing how big a metric difference is reflected in human annotator judgments. According to their finding, improvements of ≥ 1.99 in XCOMET or ≥ 0.94 in COMET scores correspond to at least 90% estimated agreement in human judgment—both of which are achieved by our method. To some extent, this aligns with our human annotation in EN-ZH and EN-RU, where the win rates are also high, e.g., 106 (win) / 32 (loss), and 93 (win) / 44 (loss).
2. Secondly, each metric has its own downside; COMET series scores are the same. The training data for COMET models comes from human annotations of each year's WMT data, which means they may be effective only within a limited range of translation quality, as WMT submissions are generally not bad. For example, [Lo et al., 2023][5] show that the COMET series struggles to evaluate translations when quality falls below a certain level. Accordingly, we are not sure, but it might also lack of ability to measure a system that outperforms previous years' systems, e.g., our method's output. We are not sure whether it fully reflects our system improvement. That is also a motivation for us to conduct human annotation.
3. Broadly speaking, although not perfect, using COMET-like neural metrics as quality signals for calibration is a successful practice, at least for mid- and high-resource languages that are well supported by current neural models. A potential improvement could be to combine multiple metrics to better exploit their complementary strengths and mitigate individual biases. Incorporating recent advanced metrics, such as LLM-based MQM, may also offer additional benefits. That said, we acknowledge that our method may not generalize well to certain low-resource languages, such as Bhojpuri, where neural metric support is lacking. Addressing these challenges will require collaborative efforts from the broader community, particularly in advancing evaluation metrics, which lie beyond the scope of this paper.

[4] Navigating the Metrics Maze: Reconciling Score Magnitudes and Accuracies

[5] Metric Score Landscape Challenge (MSLC23): Understanding Metrics’ Performance on a Wider Landscape of Translation Quality

---

Thank you again for your insightful suggestions on our paper. We hope that our additional experiments and explanations address your concerns. We will incorporate our discussion into the next version. Please don’t hesitate to let us know if further clarification or discussion is needed; we’d be happy to provide it.

Thanks for your insightful suggestions on our paper and highlighting its contributions. We address your concerns as follows:

"The experimental baselines are insufficient, only comparing the single baseline of CPO. Fine-tuning work based on translation quality preferences, such as the work [1] referenced by Section 2, should be considered for inclusion in the main experiments."

Thank you for your suggestion. We agree that a direct comparison would ideally strengthen our work. Unfortunately, [Zhu et al., 2024] did not release their codebase, and their implementation is based on weaker models such as LLaMA-1 or Mistral. To address your concern, we apply our method on Mistral using their data (WMT data and human annotations). Also, we report our results when applying calibration to the top-performing translation model, i.e., TowerMistral-7B [1], using their data.

To ensure comparability, we use the same beam search size of 4. The results are evaluated on the WMT22 test set using COMET as shown in the following table:

+ Best is supervised fine-tuning using best scoring samples, and + PL is the result of their implementation of the PL model [2]. + Calibration is applying our method. It is easy to see that

1. On the weaker base models, such as Mistral-Ins, Calibration yields clear improvements over both SFT and PL. We provisionally attribute this gain to Calibration’s broader consideration of factors compared to PL. Specifically: (1) Calibration normalizes both likelihood and quality scores within a group, making the objective invariant to scale and shift. This promotes stable and robust optimization across diverse input distributions. (2) Calibration directly integrates translation quality via the metric function q(⋅∣x), and optimizes at the distribution level using the Pearson correlation. In contrast, [2] introduces reweighting for each pairwise comparison. In extreme cases, this approach may treat a comparison between ratings 2 (bad) and 1 (nonsense) the same as one between 4 (nearly perfect) and 3 (good), which could be overly heuristic.
2. Even when applying calibration to the top-performing system, TowerMistral, using human-annotated data, our system still achieves clear, albeit relatively smaller, gains, demonstrating that calibration training is also feasible with human annotations rather than machine-generated signals.

We will include these analyses in our next version.

[1] Tower: An Open Multilingual Large Language Model for Translation-Related Tasks [2] A Preference-driven Paradigm for Enhanced Translation with Large Language Models

"The experimental setup mentions that the presented results show the best outcomes under different learning rate settings. Sensitivity analysis regarding learning rates should be added to provide more information about the method’s robustness."

Thank you for your careful reading. Yes, as mentioned in lines 195–196, to ensure robust results, we experiment with learning rates ranging from 1e-5 to 1e-4, in steps of 1e-5, for each setting.

Due to space limitations, we present a subset of these results here. The following table (aligned with experiments in Table-1) provides the details, using TowerMistral-7B as the base model and CometKIWI-XL as the evaluation metric.

The average results for all 10 steps are as follows.

During our experiments, we found that SFT is not particularly sensitive within the explored learning rate range, which we attribute to the extensive high-quality SFT already performed on TowerMistral-7B. For CPO and Calibration, the performance tends to fluctuate, often degrading at relatively small or large learning rates. The optimal learning rate for both is generally around 5e-5 across different evaluation settings.

"The comparison between the MBR of the Tower-70B model and the additionally trained 7B model is not univariate. It is suggested to supplement with an SFT fine-tuned or fine-tuned with the proposed method on 7B model to perform MBR, to better control variables and demonstrate the effect of the proposed method when superimposed with MBR."

Thank you for your suggestion. Indeed, directly conducting experiments on the base model with MBR applied would better demonstrate the effect of our method. However, applying the same magnitude of MBR (e.g., 100-fold sampling) to TowerMistral is challenging due to our limited computational resources. (That said, we are currently running such an experiment based on your suggestion, and we hope to provide updated results before the rebuttal period ends.)

Although indirect, in Section 5.3, we provide a comparison of calibration with BoN (sampling and reranking using QE model), another key form of quality-aware decoding that is similar to MBR. We summarize them as follows:

In Section 5.3, we present direct results under settings of (1) base-model + beam-search, (2) base-model + BoN, (3) calibration-model + beam-search, and (4) calibration-model + BoN. To ensure fairness, we adopt a cross-metric evaluation: Candidates are reranked using KIWI-XXL, while the results are measured using a reference-based metric, Comet22:

In short, we can easily see that:

1. Our calibrated model with beam search (beam=5) achieves clearly better performance than base model + BoN (100-time sampling);
2. When controlling the model as Tower-Base, beam search (beam=5) leg behind that of BoN, even with only 10-time sampling;
3. When controlling the model as our calibrated model, beam search (beam=5) achieves comparable results with that of BoN (100-time sampling).

These results form the basis of our conclusion in Section 5.3: calibration improves the effectiveness of maximum a posteriori decoding and offers clear efficiency advantages over QAD.

---

Thank you again for your careful reading and meaningful suggestions. We hope our additional explanation and experiments address your concern. We look forward to your further discussion!

Thank you for carefully reading our paper and highlighting its contributions. We address your concerns as follows:

"Unclear to me how well the calibration holds under domain shifts or when scaling to low-resource languages with low-quality, noisier signals. It'd be interesting to report the results for mid/low-resourced language directions."

In fact, our experiments with ALMA already include a low-resource language: Icelandic. As shown in Tables 4 and 5, ALMA achieves notable improvements for EN→IS (+0.9 COMET, +2.6 COMETKIWI, +1.7 XCOMET) and IS→EN (+1.2 COMET, +1.7 COMETKIWI, +4.0 XCOMET), consistently outperforming both SFT and CPO baselines.

To further demonstrate the effectiveness of calibration in low-resource settings, we conducted one additional experiment. Since Tower and ALMA primarily target high-resource languages, we selected Gemma-3-12B (google/gemma-3-12b-it) as our testbed, due to its strong multilingual base capabilities and broader language coverage.

Settings: (1) We applied our calibration method directly to Gemma-3-12B across 10 languages directions from this year’s WMT25 competition (ongoing), covering low-resource languages (Icelandic, Estonian) and mid-resource languages (Serbian, Ukrainian), as well as six other 6 high-resource languages. (2) We used the same data generation pipeline described in our paper, originally based on ​Flores; (3) We reused the same training hyperparameters as those used for ​TowerMistral-7B. The results are as follows:

We can see that in this setting, for mid- and low-resource languages (the last four columns), calibration yields clear improvements over both SFT and CPO across both reference-free and reference-based metrics, aligning with our paper. That said, we acknowledge that for certain low-resource languages lacking support from neural metric models, such as Bhojpuri, our method is likely to fail. Addressing these directions will require joint efforts from the broader community, particularly in developing better evaluation metrics, which fall beyond the scope of this paper.

"This work would benefit from deeper comparisons to related methods like quality-aware decoding [Fernandes et al., 2022]. Have you ever done such experiments?

Yes, we do have such experiments. Quality-aware decoding (QAD), particularly in the context of [Fernandes et al., 2022], typically involves reranking or voting strategies. A common instantiation of QAD is Best-of-N (BoN) sampling, where translations are scored and ranked using a reference-free metric model (e.g., COMETKiwi-XXL). In Section 5.3, we present direct results under settings of (1) base-model + beam-search, (2) base-model + BoN, (3) calibration-model + beam-search, and (4) calibration-model + BoN.

To ensure fairness, we adopt a cross-metric evaluation: Candidates are reranked using KIWI-XXL, while the results are measured using a reference-based model, Comet22. To be fair, for calibration, we also use KIWI-XXL as the training signal. We summarize them as follows:

In short, we can easily see that:

1. Our calibrated model with beam search (beam=5) achieves clearly better performance than base model + BoN (100-time sampling);
2. When controlling the model as Tower-Base, beam search (beam=5) leg behind that of BoN, even with only 10-time sampling;
3. When controlling the model as our calibrated model, beam search (beam=5) achieves comparable results with that of BoN (100-time sampling).

These results form the basis of our conclusion in Section 5.3: ​Calibration enhances the effectiveness of maximum a posteriori decoding. Also, compared to QAD, it offers clear efficiency benefits. Note that 100-time sampling is far from practical for online services. From another perspective, calibration can be seen as a training time optimization that captures the benefits of QAD.

"Do you have any insights into search dynamics of the proposed method? Compared to MAP decoding, does calibration impact beam search diversity or risk repetitive patterns or hallucination unnecessarily?"

Thanks for your insightful questions! In our paper, we did not dig deep into the search dynamics or changes in the decoding space. But we can share with you some observations during our experiments.

1. Hallucination: We did not observe any hallucinations (lexicon/pattern repetition) during our experiments, at least not in Chinese, German, Dutch, and Russian, where we conducted detailed human analysis. In fact, even the base model (TowerMistral-7B) exhibited only minimal instances of hallucination. While not rigorously, we have a feeling that hallucinations are largely due to limited instruction-following ability, which can be effectively mitigated through SFT with a few translation examples, at least for high- and mid-resource languages. Notably, in our additional experiments with the instruction-tuned Gemma-12B model, we also found no hallucination issues, even without SFT stage.
2. Beam Search Diversity: Broadly speaking, calibration is to reorder the hypothesis likelihood within decoding space, while its impact on the shape of the space (flatter or sharper) is hard to say. We believe it depends on multiple aspects, like the sampling strategy. Additionally, we read your question about diversity as a concern that calibration might reduce the diversity of the decoding space, potentially diminishing the benefits of using a larger beam size. To investigate this, we conducted an experiment using the TowerMistral-7B model, selecting four language directions (EN-DE, EN-ES, EN-RU, EN-ZH) and varying the beam size to compare its impact on both the base and our calibrated model as follows. It is easy to see that: (1) When the beam size exceeds 5, we do not observe further gains. (2) From Beam-1 to Beam-5, the gains for the SFT model and Calibrated models are comparable, 0.7 vs. 0.8 in KIWI-XL and 0.4 vs. 0.6 in Comet22. Thus, we can assume that calibration does NOT change the shape of the space to a degree that affects search effectiveness.

---

Thank you again for your careful reading of our paper and for your insightful suggestions and questions. Hopefully, our response addresses your concerns. We will incorporate this discussion, particularly for the low-resource and decoding space aspects, in our next version. Also, we are more than happy to provide further explanation or discussion if needed.