All Languages Matter: On the Multilingual Safety of Large Language Models

Question

Q1. You say that “Bengali, Hindi, and Japanese are […] generally are the most low-resource languages in the preatraining data of LLMs”. However, is this really the case or are these just the languages with least resources amongst the one that you consider? Generally, LLMs see much less data in, e.g. Tibetan, Burmese, Shan, etc, than, e.g. in Japanese.

These languages are the languages with the least resources among the ones that we consider in this work. We modified the claim in the revised paper.

Q2. Are the prompts that you propose in 4.3.1 always provided in English or are they also translated?

All the prompts are in English. [1] finds that using the instruction and examples in English performs better for multilingual tasks. Hence, we follow this setting and use all the system prompts in English.

[1] Language models are multilingual chain-of-thought reasoners.

Q3. How come that the XLing1 prompt actually increases the unsafe ratio for Hindi?

The increase of 0.5% can be counted as a variance. The finding we want to highlight is that XLing1 cannot increase the safety of conversation in Hindi.

Q4. When you prompt the model to “think” how do you implement this? Does the model “think” by generating first an output in English and then respond in the target language (similar to chain-of-thought prompting) or do you expect it to internally do this thinking?

We simply modify the system prompt, for example, by stating, "Please think in English and then generate the response in the original language." We anticipate that the model will internally perform this thought process. As expected, the model with the modified prompt directly generates responses in the target languages.

Weakness

W1. “The Resulting XSAFETY Benchmark” paragraph is not all clear. I don’t see why “if each unique Chinese phrase is consistently translated into the same phrase across instances for another language, the datasets of the two languages should share similar data distributions”. Especially considering that different languages have different grammar and large part of the tokens is not topic-specific. You say “Both quantitative and qualitative analyses show that the XSAFETY benchmark shares similar data distributions across languages, indicating the possibly consistent translation as expected.” but where is this analysis? Furthermore, the “translation” of ませ and すれ is not a translation but rather transliteration. They are both parts of grammatical constructs and hence not translatable. Overall, this paragraph seems unmotivated and poorly defended. Won’t the issue be resolved by simply having multiple translations and variations per sample? And similarly augment the original corpus? This would also make the results more robust to the choice of words and phrasing.

Thank you for your insightful suggestion. We deleted this paragraph in the revised paper and used the proofreading statistic to show the reliability of the translation. In addition, we include more details about the benchmark construction with the vacated space.

W2. The “Models” paragraph seems to be mostly pure guessing of the language distribution in the pretraining data. It is not clear that the GPT-3 distribution is representative for ChatGPT, and furthermore, depending on whether one uses the 3.5 or 4 version of ChatGPT the number could be even more different. The values for PaLM 2 seem to be not “estimated” but “guessed”. The issue is further complicated because of language similarity: German, French, Italian, Spanish, Dutch, etc. can result in cross-language learning while others, e.g. Korean cannot. Overall, this analysis does not seem to be necessary for the conclusions of the paper, so I am not sure what purpose it serves.

Thank you for pointing it out. We will treat these data as references rather than concrete evidence and statistics.

We use the data distributions to show that although the 10 languages studied in this work are most used in real world, they are not necessarily high-resource languages in the pretraining data of LLMs. We will move the content to Appendix, and leave the vacated space for more necessary details in the revised paper.

W3. I am missing a discussion on the limitations of the work.

Our paper presents several limitations:

1. Our benchmark relies on a dataset translated from English and Chinese, which may result in biases toward English and Chinese cultures and under-representation of safety issues within the respective cultures.
2. We employ a self-evaluation method using ChatGPT to determine the safety of LLMs' responses. Although we incorporate human annotations to demonstrate the reliability of this method, it is not entirely accurate, potentially compromising the soundness of our findings.
3. Our proposed improvement methods are not sufficient to resolve this issue. Further investigation is required to enhance the handling of multilingual safety concerns.

We include the limitations in the revised paper.

W4. Neither the dataset nor the results (actual model responses) have been provided.

We will release all the data and results to facilitate future research on LLMs safety, which we clarify the claim in the contributions in the revised paper. We uploaded all the data and results in the Supplementary Material in the revised paper.

W5. The dataset offered consists of translated samples of two prior works. However, there is no mention of their licences and whether they allow for such a use or not.

The datasets are apache-2.0 license, which can be used and modified for research usage.

Weakness

W1. The benchmark is based on the work of (Sun et al., 2023), which hasn’t been peer-reviewed.

We systematically review all the safety benchmarks for LLMs, from different fields including NLP, Security, and AI. The dataset provided by Sun et al. (2023) is the most comprehensive benchmark including 14 kinds of safety scenarios. We do not choose widely-used benchmarks, especially the dataset from OpenAI and Anthropic, due to the data contamination issue (e.g. these data may have already been trained and aligned).

W2. The benchmark is mainly derived from the Chinese dataset in (Sun et al., 2023). This could potentially be biased toward safety assessment in Chinese rather than English. It’s unclear to me why this dataset is a good starting point for building the multilingual benchmark. I would expect much of the text dedicated to describe and convince the reader that the benchmark is adequate and well constructed.

There are several methods we have conducted to avoid our benchmark bias toward Chinese cultures. First, we filter out the Chinese-Culture-Related cases and select a high-quality subset from Sun's benchmark. Second, we ask the translator to make necessary modifications on the translation to adapt the benchmark to different languages.

W3. Using chatGPT as an evaluator for safety doesn’t seem like a good idea to me. The authors have stated that safety is a very important issue, to which I agree. As such an important issue, human evaluation should be done rather than using another LLM. Human evaluation is only conducted on 50 samples from Crimes and Illegal Activities and Goal Hijacking scenarios. What languages do these 50 samples come from? What is the quality of the translation of the responses? 50 samples seem too small for human evaluation and to make a statement about multilingual performance of LLMs. Moreover, it’s unclear if we can trust chatGPT on other scenarios in the taxonomy.

We annotated the automatic evaluation of LLMs' responses in both Chinese and English. To further validate its reliability, we expanded the scope of human annotation from 200 cases covering 2 languages and 2 safety issues to 1,400 cases, encompassing all 10 languages and 14 safety issues. The accuracy is 88.5%, demonstrating the effectiveness of this automatic evaluation method.

In addition, we utilized a more advanced GPT-4 as the evaluation model. Specifically, we employed GPT-4 to evaluate responses in English, Chinese, and Hindi, with 100 cases randomly selected and annotated where ChatGPT and GPT-4 had differing judgments. The annotation results reveal that ChatGPT is correct in 76 cases, while GPT-4 is correct in 24 cases (primarily due to its over-sensitivity, which led to classifying 70 safe responses as unsafe).

Both experiments provide evidence that our current self-evaluation method using ChatGPT is reliable.

W4. The authors tried to provide some estimate about the percentage of language data in each LLMs. While I appreciate this effort, I don’t think Table 4 makes sense. Training data for GPT-2 is NOT the training data for chatGPT. And PALM2 training data is NOT the training data of PALM.

Thank you for pointing it out. We follow your suggestions to modify the claims in the revised paper.

W5. I have a meta-concern/question about the setup. If culture specific is removed from the safety benchmark then is it an interesting problem to study? If everything can be mapped to English and the model chooses to respond or not based on its safeguard then is it just a machine translation problem? In the experiment when the prompt asks chatGPT to think in English then answer, is it just a specific instance of implicit translation?

Even if culture-specific aspects are removed from the safety benchmark, studying the problem can still be interesting. While it may seem like a machine translation problem, the focus here is on the model's capacity to generalize the safety alignment from English to other languages. As stated in the GPT-4 report (page 61), "The majority of pretraining data and our alignment data is in English. While there is some evidence that safety mitigations can generalize to other languages, they have not been robustly tested for multilingual performance." Our research addresses this gap and reveals that safety mitigations cannot flawlessly generalize to other languages using the constructed multilingual safety benchmark. In other words, current LLMs cannot flawlessly map everything, such as safety mitigations, to English.

"Think in English then answer" can be regarded as a specific instance of implicit translation. Implicitly translating the query to English can better evoke the LLMs' capacity of safety mitigation, which is mainly trained in English data.

Overall, the goal is to ensure that AI systems are safe, useful, and respectful of user values in the multilingual scenario, which goes beyond just translation.

Weakness

W1. All of the text is Google translated from English or Chinese. There is a possibility that translating this data across cultures would result in it not being representative of the harms that appear in those cultures. There is a good example of removing the China-specific safety questions from the Sun et al. dataset.

We do consider the cultural differences between different languages. During the two rounds of translation proofreading, we ask the annotators to adjust the sentences if they do not follow the harmful representation of the culture of the languages.

W2. There is a manual evaluation of 50 instances only from the Crimes and Illegal Activities and Goal Hijacking scenario. And it is not clear what language the model was queried in.

We annotated the automatic evaluation of LLMs' responses in both Chinese and English. To further validate its reliability, we expanded the scope of human annotation from 200 cases covering 2 languages and safety issues to 1,400 cases, encompassing all 10 languages and 14 safety issues. The accuracy is 88.5%, demonstrating the effectiveness of this automatic evaluation method.

In addition, we utilized a more advanced GPT-4 as the evaluation model. Specifically, we employed GPT-4 to evaluate responses in English, Chinese, and Hindi, with 100 cases randomly selected and annotated where ChatGPT and GPT-4 had differing judgments. The annotation results reveal that ChatGPT is correct in 76 cases, while GPT-4 is correct in 24 cases (primarily due to its over-sensitivity, which led to classifying 70 safe responses as unsafe).

Both experiments provide evidence that our current self-evaluation method using ChatGPT is reliable.

W3. There are no actual qualitative examples or data provided with the submission, so I am not able to further validate and understand whether I test the benchmark results or not.

We upload all the data and results in the Supplementary Material in the revised paper.

Below are some example responses in different languages:

W4. I believe that the authors did not mention that they would release any software or data.

We will release all the data and results to facilitate future research on LLMs safety, which we clarify the claim in the contributions in the revised paper. We uploaded all the data and results in the Supplementary Material in the revised paper.

Question

Q1. The second line below Table 3 mentioned XLingPrompt3, but it's never introduced in this paper.

Thank you for pointing it out. XLingPrompt3 is another setting that was omitted in our submitted version. We have incorporated the necessary changes in our revised paper.

Q2. Can LLMs understand user input and generate coherent responses in non-English languages, especially Hindi and Bengali, considering their limited multilingual capabilities?

The multilingual capacity of LLMs, such as ChatGPT and LLaMa2, has been studied by previous works, showing that LLMs have the capability to communicate with other languages, although not as good as in English [5][6]. For example, ChatGPT has COMET score of 76.7 for English-Hindi and 89.2 for Hindi-English, and LLaMa2-7B has 44.6 for English-Hindi and 80.1 for Hindi-English.

In our experiments, we found that ChatGPT can understand user input and generate coherent responses in non-English languages, including Hindi and Bengali.

[5] ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning

[6] Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis

Weakness

W1. The current selection of translated languages primarily focuses on high-resource languages, which limits the comprehensiveness and representativeness of the evaluation.

We select 10 languages based on the following criteria: 1) the languages should have a sufficient number of native speakers in the real world, which means more people could be harmed when unsafe responses in these languages occur; and 2) current LLMs have enough capability to chat in these languages.

Note that though these languages are not low-resource languages in the real world, they are relatively low-resource in the pre-training data of LLMs. The LLMs may fail to process real-world low-resource languages, which hardly occur in the pre-training data.

In addition, we are the first multilingual safety benchmark with the largest number of languages. We will leave building more comprehensiveness benchmarks for future work.

W2. The proposed prompting methods lack novelty and only exhibit marginal improvements on models other than ChatGPT.

We want to highlight our main contribution is that our paper is the first to identify and benchmark the multilingual safety issue of LLMs. Although our mitigation method is straightforward and demonstrates some enhancement, it indicates that current prompting techniques are far from resolving this problem. We hope our paper serves as a catalyst for motivating future research.

W3. Ensuring accurate annotation in a multilingual dataset is a challenging task that requires a rigorous verification process. The annotation process employed by XSAFETY and the validation of automatic evaluation lack essential details for standard data validation, such as cross-validation, and inter-annotator agreement.

In the revised paper (Section 3), we have provided more detailed information on how we ensured the quality of creating multilingual data. Specifically, we conducted two rounds of proofreading by professional translators from a commercial data annotation company to guarantee the accuracy of data translated from Google. The modification rate for the first round was 15.5%, while the second round had a 3.4% modification rate. Subsequently, we randomly inspected 10% of the data, achieving a pass rate greater than 99%.

W4. Self-evaluation approach diminishes the reliability of the assessment. To enhance the reliability, additional steps or more advanced models should be explored.

We would like to note that employing LLM as an evaluator is a commonly adopted approach in recent papers [1-4]. Furthermore, based on our human annotation, the reliability of self-evaluation using ChatGPT is deemed acceptable (i.e. 94%).

To further address the reviewer's concerns, we have conducted two additional experiments:

1. We expanded the scope of human annotation from 200 cases covering 2 languages and 2 safety issues to 1,400 cases, encompassing all 10 languages and 14 safety issues. The accuracy is 88.5%, demonstrating the effectiveness of this automatic evaluation method.
2. We utilized a more advanced LLM, GPT-4, as the evaluation model. Specifically, we employed GPT-4 to evaluate responses in English, Chinese, and Hindi, with 100 cases randomly selected and annotated where ChatGPT and GPT-4 had differing judgments. The annotation results reveal that ChatGPT is correct in 76 cases, while GPT-4 is correct in 24 cases (primarily due to its over-sensitivity, which led to classifying 70 safe responses as unsafe).

Both experiments provide evidence that our current self-evaluation method using ChatGPT is reliable.

[1] Vicuna: An Open-Source Chatbot impressing GPT-4 with 90%* ChatGPT Quality

[2] Alpacaeval: An automatic evaluator of instruction-following models

[3] Shadow Alignment: The Ease of Subverting Safely-Aligned Language Models

[4] Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!