Align after Pre-train: Improving Multilingual Generative Models with Cross-lingual Alignment

Thank you for your detailed review! You evidently spent a substantial time reviewing our paper, and we are thankful for your efforts during the challenging review period. We will explain your concerns point by point.

Weakness 1: The innovativeness of this paper is limited.

We acknowledge that the concepts of cross-lingual contrastive learning and instruction tuning are not novel. The novelty of this work comes from the following two parts:

(1). Natural language generation task is different from natural language understanding task, there are fewer studies to investigate effectiveness of the contrastive learning in generation tasks. In this work, we find that using multilingual contrastive learning alone cannot enhance the cross-lingual ability of multilingual generative model, and in most cases, it will impair the performance of the model, e.g., the inferior results in the 'w/MCL' row of Table 8 in the ablation experiment (Section 3.5). We argue that the generation ability of the model may be affected by the multilingual contrastive learning method.

(2). Our cross-lingual instruction fine-tuning differs from multilingual instruction tuning, where models usually learn and reply in the same language. It requires models to reply in different languages. The work mentioned, which is "Few-shot Learning with Multilingual Generative Language Models"[1], investigates the impact of prompt temple in the demonstrations, such as same-language-prompting and source-language-prompting, rather than the instruction tuning process.

Weakness 1&Question 1: Is there any related work in cross-lingual instruct tuning that could be added as the compare system?

Of course, we add the CoIT (Cross-lingual Instruction Tuning) method of xLLaMA[2] for comparison using the same training data in EN-ZH bilingual instruction tuning, and the experimental results are as follows:

Table 1. In-context learning performance on XNLI and PAWS-X.

It can be found that AFP achieves a higher improvement than CoIT. The difference between AFP and CoIT lies in the additional multilingual contrastive learning and cross-lingual answering format, which is in line with findings in the ablation study that multilingual contrastive learning further boosts the performance of cross-lingual instruction tuning.

Weakness 2&Question 2: For the MT experiment, is it possible to add the parallel data used in AFP framework into the baseline model for a fair comparison?

Sure, we supplement the results of the baseline model fine-tuned on the parallel data ('+FT'):

Table 2. Translation results on FLORES-101 devtest set(ZH-EN).

It can be found that fine-tuning on the same parallel data benefits the machine translation task (+1.1 BLEU), but AFP can bring further improvement(+1.9 BLEU).

Typo in Sec 3.3: BLUE --> BLEU

Thank you for reminding us! We will correct this typo in the next edition.

[1] Lin, Xi Victoria, et al. "Few-shot Learning with Multilingual Generative Language Models." EMNLP 2022.

[2] Zhu, Wenhao, et al. "Extrapolating Large Language Models to Non-English by Aligning Languages." arXiv:2308.04948 (2023).

Thank you for your review! We will explain your concerns point by point.

Weakness 1: The motivation for AFP and its value.

There may be a misunderstanding here. The motivation of AFP lies in the performance bias toward high-resource languages, which is also found in the native multilingual LLMs from scratch, including XGLM, BLOOM, and the mentioned PolyLM[1]. Experimental results demonstrate that our method greatly improves the cross-lingual ability of generative models, including multilingual ones (XGLM and BLOOM) and models pre-trained on English corpus (LLaMA), by using less than 1M parallel samples.

Thank you for your recommendation to analyze PolyLM. And we conduct preliminary bilingual alignment experiments on $PolyLM_{1.7B}$:

Table 1. In-context learning performance on XNLI and PAWS-X.

Table 2. In-context learning performance on XCOPA, XStoryCloze, and XWinograd.

We find that AFP improves the cross-lingual performance of $PolyLM_{1.7B}$ (+2.0% on average), which further proves the effectiveness of our method.

Weakness 2: Evaluated tasks are simple. The generative abilities in multilingual setup are not properly measured. Are other multilingual tasks, such as summarization/QA, more appropriate for evaluation?

We acknowledge that the five datasets you mentioned are simpler than generation tasks. However, current multilingual generative models have not achieved satisfactory results on these simpler datasets, where the best result in this paper is still lower than 90%. It means that they can still reflect the improvement in the cross-lingual performance of the model, especially in the ability to cross-lingual understanding. Moreover, similar datasets such as MMLU are widely adopted to evaluate LLMs.

Although the generation task can better reflect the generation ability of language models, the evaluation of the generation task, e.g., open domain QA, is still an open question, especially in the cross-lingual condition. Therefore, we only adopt a commonly used generation task, which is machine translation, to evaluate the cross-lingual generation ability of the model. In addition, in order to better reflect the improvement in cross-lingual generation ability, we supplement the evaluation results of cross-lingual summarization tasks:

Table 3. One-shot in-context learning performance on EN-ZH cross-lingual summarization task[2] (Rouge1).

In line with the results of the machine translation task, $XGLM_{7.5B}$ also obtains an improvement on the cross-lingual summarization task after using AFP.

Weakness 3: The cross-lingual finetuning step leverages machine translation outputs, which is prone to error.

Thank you for reminding us! In future work, we will add a data evaluation process to reduce the error from machine translation.

Weakness 4: The existing cross-lingual datasets are perhaps outdated in the age of LLMs with emergent abilities.

Similar to the response above, current multilingual generative models have not achieved satisfactory results on these datasets, where the best result in this paper is still lower than 90%. It means that they can still reflect the improvement in the cross-lingual performance of the model, especially in the ability to cross-lingual understanding.

Moreover, thank you for your suggestion on the research of multilingual generation model! We agree that it is important to build an adequate multilingual benchmark for LLMs, which is part of our future work.

[1] Wei, Xiangpeng, et al. "Polylm: An open source polyglot large language model." arXiv:2307.06018 (2023).

[2] Zhu, Junnan, et al. "NCLS: Neural Cross-Lingual Summarization." EMNLP-IJCNLP 2019.

Dear Reviewer GUFo,

We wish to express our sincere gratitude once again to you for the feedback. We would like to gently bring to your attention that the discussion phase between authors and reviewers is closing to the end (within 12 hours).

We hope our response has addressed your concerns. Should you have any further insights to share, we are more than willing to sustain our discussion until the deadline.

Thank you for your detailed review! You evidently spent a substantial time reviewing our paper, and we are thankful for your efforts during the challenging review period. We will explain your concerns point by point.

Weakness 1: The novelty of the proposed method is limited.

We acknowledge that the concepts of cross-lingual contrastive learning and instruction tuning are not novel. The novelty of this work comes from the following two parts:

(1). Natural language generation task is different from natural language understanding task, there are fewer studies to investigate effectiveness of the contrastive learning in generation tasks. In this work, we find that using multilingual contrastive learning alone cannot enhance the cross-lingual ability of multilingual generative model, and in most cases, it will impair the performance of the model, e.g., the inferior results in the 'w/MCL' row of Table 8 in the ablation experiment (Section 3.5). We argue that the generation ability of the model may be affected by the multilingual contrastive learning method.

(2). Our cross-lingual instruction fine-tuning differs from multilingual instruction tuning, where models usually learn and reply in the same language. It requires models to reply in different languages.

The experimental results prove the effectiveness of the cross-lingual instruction tuning task:

(1). The cross-lingual performance of BLOOM model with 167k parallel corpus surpasses that of BLOOMZ (Table 1, 3, and 7), which is trained with 78M multilingual instructions.

(2). As shown in Figure 4(b), given the EN-ZH bilingual samples, the results of fine-tuning multilingual instruction tuning ($p_{src}$=1) are worse than that of fully cross-lingual instruction tuning ($p_{src}$=0).

Weakness 2&Question 1: Does cross-lingual instruction tuning work better than multilingual instruction tuning?

According to our empirical results on $BLOOM_{560M}$ and $XGLM_{564M}$, cross-lingual instruction tuning performs better than multilingual instruction tuning.

Firstly, under the condition of EN-ZH bilingual instruction tuning, we compare the results of multilingual instruction tuning ($p_{src}$=1, which is also called monolingual instruction tuning for each language) and fully cross-lingual instruction tuning ($p_{src}$=0). We find that the performance of cross-lingual instruction fine-tuning is 0.9% higher on average (Figure 4(b) in Section 3.2.3).

To further prove the effectiveness of cross-lingual tuning(CIT), we supplement the results of model tuning with 5 times multilingual instruction-tuning(MIT) samples in the multilingual condition of five languages for comparison(EN, ZH, TH, TR, SW):

Table 1. In-context learning performance on XNLI.

Table 2. In-context learning performance on XCOPA.

From the above results, we can find that cross-lingual instruction tuning is still better than multilingual instruction tuning by 0.7% on average.

As for the influence of the sample size, Chen et al.(2023) found that fine-tuning alpaca with 9k samples performs better than the one with 52k data[1]. Similarly, other work also found that more data (Nx) does not necessarily bring better performance when fine-tuning on multilingual instructions, and even the model trained with down-sampled multilingual samples(1x) obtains better robustness[2].

[1] Chen, Lichang, et al. "Alpagasus: Training a better alpaca with fewer data." arXiv:2307.08701 (2023).

[2] Chen, Pinzhen, et al. "Monolingual or Multilingual Instruction Tuning: Which Makes a Better Alpaca." arXiv:2309.08958 (2023).

Thanks for addressing my comments on the ablation experiments of CIT.

However, I would note that the BLOOM with AFP and BLOOMZ in Table 1 and Table 3 are not directly comparable because the AFP is under a bilingual setup but BLOOMZ is multilingual. The setup in Table 7 is fair for BLOOM w/ AFP vs BLOOMZ, and the proposed methods slightly outperform BLOOMZ while using much fewer data as mentioned in your response.

Although the combination of contrastive learning and instruction tuning is simple, "using multilingual contrastive learning alone cannot enhance the cross-lingual ability" is a good point.

This clarifies some points for me, and I have updated the rating.

Some suggestions:

• Highlight the finding: "using multilingual contrastive learning alone cannot enhance the cross-lingual ability".
• Clarify the experimental setups, and distinguish the bilingual and multilingual setups.