CULTURE-GEN: Revealing Global Cultural Perception in Language Models through Natural Language Prompting

W1: Automatic cultural markers discovery

We did obtain the cultural markers from inspection. To automatically obtain markers, one may look for unigrams statistically different among cultures, such as (https://aclanthology.org/2023.acl-long.84.pdf). Their method requires a “marked”/“default” group which we do not have. To approximate, we used nationality pairs that are English-speaking/non English-speaking, eg. Canadian/Armenian, American/Albanian, on “favorite music”. We find that “traditional” is frequently in the top 10 words with the highest z-score, validating our inspection.

Given our generations are < 10 tokens, it is unlikely to discover more cultural markers from them. Parentheses, however, can’t be discovered using word-based methods. For future work we can explore automated ways to discover cultural markers in long texts.

W2: Bias mitigation with RLHF

We will add a discussion section about culture-bias mitigation. Our thoughts:

Works (https://arxiv.org/pdf/2402.15018) show that monotonic alignment create disparities between default and marginalized groups and that reward models do not impact OOD behaviors as much as data. Hence, we need: 1. Expand global cultural coverage of data 2. Pluralistic alignment (https://arxiv.org/pdf/2402.05070). To resolve conflicts between different cultures, we decide the method based on the intended usage of the model:

For general-purpose models, one may prefer models that output the whole spectrum of reasonable responses; for models for an audience in mind, one may prefer models that can be steered for attributions preferred by that group; for data generation models, one may prefer models who can generate an array of answers during sampling.

W3: cultural proxies

Thank you for informing us of this contemporary work! We will add a paragraph in the discussion section. Our thoughts:

The semantic proxies defined in this mentioned work mainly encompass words that have cross-lingual reference, while our work discovers entities that are unique to certain cultures. We try our best to match the topics to the defined proxies:

picture, statue -> the house; food -> food and beverage; clothing -> clothing and grooming. A new category “practices and objects” may best fit “exercise routine”, “music”, “movie”, and “music instrument”.

We used countries and regions as demographic proxies and we acknowledged in P4 1st paragraph that within one country there may be more granularly defined cultures.

W1: Collecting data from biased LLMs

Generating data from LLMs for any purpose not limited to bias studies all face similar concerns in correctness, diversity, and hallucination[1]. For other works that aim to collect comprehensive cultural data from LLMs and use it in downstream training or tuning language models, one will need to take extra care to mitigate bias in the data. However, as explained in the paragraph after the one we discussed the trade-off, our work focuses on analyzing the cultural perceptions of the LLMs, and does not intend the cultural symbols to be used in downstream training. That’s why we chose the continue-generation task and expected the result to be biased.

[1] Guo, Xu, and Yiqiang Chen. "Generative AI for Synthetic Data Generation: Methods, Challenges and the Future."

Q1: filter out the invalid phrases

We deem “traditional Albanian music” as invalid, because this is a boilerplate answer that can be applied to all cultures, such as “traditional Algerian music”, “traditional Nigerian music”, etc. So we will filter for culture symbol candidates that contain “traditional” or “[culture]”, where [culture] is the culture for which this phrase is generated. Therefore, if for culture A the model generates “[culture B] music,” we will keep the answers.

Q2: Typo

Thanks for catching that! Will fix.

Q1: Incorporate the audience into the prompting.

We quickly tried this prompt: “Describe the favorite music of your neighbor. I am Azerbaijani. My neighbor is Azerbaijani. My neighbor probably likes listening to” and asked GPT-4 to generate 10 times. Only one of them has parentheses: “Azerbaijani traditional folk music, Meykhana (a traditional Azerbaijani music characterized by catchy rhythm and spontaneous lyrics), Mugham (a”. Seems like adding an audience from the same nationality does not completely remove markedness, although it may reduce the frequency that it occurs.

We will add a discussion paragraph and a more systematic experiment in the camera-ready version.

Q2: estimation for the markedness.

We examined the markedness of culture-agnostic generations, and found that GPT-4 has 0 vocabulary or parentheses markedness for most topics (exceptions are picture and statue, with only 1 generation each), llama2-13b has as many as 2 vocabulary and parentheses markedness among all topics, and mistral-7b has as many as 3 parentheses markedness for 3 out of 8 topics: markedness behavior does not appear when the model is only asked to talk about a certain culture.

One could count the frequency of parentheses in an indexed pre-training data, similar to what we did in 5.2. But our indexing server does not index punctuations, so we cannot get an estimate for parentheses. For reference, # documents with the word “traditional” is 51563138, 0.25% out of 20B documents of RedPajama-v2, quite a big portion.

We will add these two results to the camera-ready version.

Q3: LLMs that are not only English-based.

This is indeed out of scope of the current work, but it is an interesting topic that previous works have touched upon. [1] evaluated multilingual models and monolingual Arabic models for cultural biases, and found that even monolingual Arabic-specific LMs exhibit Western bias, potentially because part of the pretraining-data, albeit solely in Arabic, often discusses Western topics. They also found that multilingual LMs show even stronger Western bias, suggesting that multilingual training could impact cultural relevance of non-western languages.

Based on the takeaways from previous works, we speculate that there are shared trends between English-only and non-English models, but we will defer the accurate study on this topic for another work.

[1] Naous, Tarek, et al. "Having beer after prayer? measuring cultural bias in large language models."

Q1: Table 1

See P4 2nd paragraph.

Q2: representativeness of cultural symbols

We were not studying what a language model can generate, but what a language model will generate. It is expected that the current set of generations show bias towards certain subpopulations. We will study subpopulation biases in future work.

We tried mistral-7b on 3 topics with the culture with most symbols in each geographic region.

We report hit rate, avg percentage of collected culture symbols that are generated in each condition, and new rate, avg percentage of generated symbols that are not culture symbols previously detected.

TLDR: quite representative:

Model only generates part of all culture symbols extracted (hit rate), i.e. the generated culture symbols become more homogenous. Only a tiny part of generations have unseen symbols (new rate). Note that some of these symbols are not culture symbols, rather unique expressions.

Q3: picture & statue

For pictures and statues, we intended to extract culture symbols that do not belong to the other topics but still have cultural significance, eg. animals, religious symbols and historical figures. In downstream tasks like story generation, one may ask the model to describe decorations containing objects of cultural significance, but those objects may be hard to categorize.

Q4: traditional & markedness

“Traditional music” or “traditional clothes” may overall increase the frequency of “traditional”, but the difference in model usage of these combinations manifests markedness. For the “default” cultures, “traditional” is not the top choice in a model’s decoding process of the first word. As low as 1% GPT-4 generations contain markers for those cultures.

Q5: geographic regions

We report averaged metrics for each region as normalization.

The number of countries in each geographic region will impact the variance of the # culture symbols. The box plots show high IQR regions, for which highlighted the importance of studying variance within each region.