Do LLMs have Consistent Values?

• The paper does not adequately address why it is a valuable addition to the already crowded discussion around values in LLMs. Several papers this year have explored similar themes of value consistency and expression through various methods (e.g., role-playing, moral beliefs, and novel-based approaches). The authors need to provide a clearer explanation of how their use of the Value Anchor prompt and focus on value correlations sets this study apart from others. It would strengthen the paper if the authors cited these related works more extensively and articulated how their approach advances the conversation rather than merely replicating it.

• One thing that I believe this paper fails to answer is that the paper leans heavily on the idea that LLMs should be compared to human value systems as the benchmark for consistency. However, it does not explore whether LLMs should necessarily be held to human standards, or whether they could develop a distinct and equally valid form of value coherence that differs from human psychology. By only focusing on human comparison, the paper misses an opportunity to explore how LLMs might create unique, non-human patterns of consistency that could still be valuable.

1. The experimental results could be made stronger by analyzing whether minor variations in the same prompt could elicit the same results. Since language models often respond differently to small changes in wording, showing how the results hold up with different prompts would add a lot of value. A bit more discussion around this could help understand how stable the findings really are.

2. It would be great to see the value rankings and correlation structures explored in generation tasks as well, not just in classification. Since the goal here is to simulate different values and perspectives across populations, showing that language models can pick up on these differences in more open-ended tasks would make the results feel even more real and convincing.

3. I would like to see more formal statistical tests to make the paper stronger. For example, the authors use Spearman's rank correlation to compare LLM value rankings to human rankings, but they don't report statistical significance (p-values) which could potentially add another layer of rigor to the analysis.

The study of whether LLMs share the same value structure as humans do is interesting, but the practical uses and the influences on how to build better LLMs remain a little unclear. It might be more interesting to shed some light on how the results could help improve LLM behaviors.

In addition, the groundtruth human responses for comparisons may exhibit certain biases. As written in line 245-247, the mean age of participants was 34.2 with 59% females. Does it cover a fuller spectrum of human subjects, e.g., from children (primary school students) to elderly (people over 60 years old), whose values are of equal importance to study?

That LLMs may mimic a population like these participants may fail to show if the models resemble values of people on the ends of a spectrum, but may just suggest that the models are trained on web data dominated by young adults.

Also, the potential impacts and limitations of this study are not clearly discussed. For example, it could be potentially easy for people to fake their answers in the value questionnaires. Will an LLM do similar things? What if an LLM misrepresents itself as a person holding positive values, but instigates people to hurt themselves?

The statement “little research has been done to study the values exhibited in text generated by LLMs” in the abstract (and echoed repeatedly in the introduction) overly downplays the amount of attention this area of research has received in the past five years. That is, it really seems to ignore all of the research that was around even in the era of BERT family models. Some prominent examples from the past five years: Social Chemistry 101 by Forbes et al. in 2020, Argyle et al. 2022’s work on simulating human samples, and Durmus et al. 2023’s work on world opinions. I see that the authors do cite Argyle et al., but it’s just strange to frame the paper as something entirely novel given such extensive related literature on the topic. See Ma et al. 2024’s “The Potential and Challenges of Evaluating Attitudes, Opinions, and Values in Large Language Models” for a recent survey. The authors may also be interested in looking at Angelina Wang’s 2024 work on language models portraying identity groups and Mingqian Zheng’s work on personas in system prompts.

The paper is written in an unclear and messy manner. It’s difficult to understand the motivation behind certain decisions (such as the varying ways they prompt the models) or grasp the substantive implications of results that are presented. Some things, like the inclusion of a sine function in Figure 4, feel very arbitrary.

As one concrete example of why the experimental choices made in this paper do not make much sense to me, let’s take for example the results shown in Table 1. Table 1 seems to show that including personas based on different types of human values results in model outputs that best fit different human value judgements. Personas based on other characteristics related to different people do not fit as well to human value judgements. This is like saying, water is water, and if we try to pretend some other non-water substance is water, it is not as water-like. The authors could look into prior literature on measurement modeling (a.k.a. how social science researchers link observed behavioral data to latent theoretical concepts) to see why the outcome of their experimental setup is unsurprising.

This paper would be stronger if it showed how its prompting approach contributes to some sort of downstream task involving values, e.g. “replicate known findings … or pretest novel hypotheses” as suggested in lines 477-478. As a model for how to do this, the authors could consider looking at Park et al’s 2022 paper on social simulacra. Their paper concludes with a study where digital platform designers use their approach for simulating social media communities.

Finally, prompting large language models with an established survey and reporting what they output is not very methodologically interesting for an ICLR audience. Not every AI/ML paper needs to showcase great methodological novelty to be a great paper, but given that this paper is not conceptually novel, either, it makes me wonder what its key contributions are. It’s possible that I may have misunderstood some key strength of this paper due to how it is written/presented; thus, I’m very open to carefully reading over the authors’ response to this review.

Minor comment: Line 402: “Llamma” -> “Llama”