Deep Binding of Language Model Virtual Personas: a Study on Approximating Political Partisan Misperceptions

We appreciate the decision of leniency regarding the extra page provisioning for the limitation section and ethics statement. We will adjust the location and length of the sections appropriately in the final manuscript revision.

Overview

We sincerely thank the reviewer for the thoughtful and detailed feedback on our submission. We are pleased that the reviewer found our approach effective in improving alignment between LLM-generated and human survey responses—both in terms of effect size and distributional similarity. We also appreciate the recognition of our extensive evaluation setup, covering multiple models and datasets, and the clarity of our presentation. We address the reviewer’s comments in detail below.

R3-1: Highly specific evaluation setting: Evaluation is performed on “higher-order” attitudes only (e.g., “Would Democrats support using violence to block major Republican laws?” instead of direct attitudes, such as “Would you support using violence to block major laws?” Based on the evidence presented in the paper, while plausible, it is not known whether the proposed approach is effective for simulating political surveys in general.

We provide evaluation of our methods against baseline methods using survey data from the American Trends Panel Waves 34 and 99 [1], which contain survey questions probing direct attitudes such as “How excited or concerned would you be if artificial intelligence computer programs could know people's thoughts and behaviors?” These cross-sectional surveys were identical to ones used as benchmark in prior work (Anthology).

Results are summarized in Table R7 below: our method consistently outperforms all baselines across settings. Notably, backstory-based methods (ours and Anthology) outperform those relying solely on demographic prompting (BIO, QA, Portray). Furthermore, our method achieves stronger results than the Generative Agent baseline, which uses GPT-4o.

Table R7. WD Comparisons on ATP Wave 34 and Wave 99.

References

[1] ATP Wave 34 and 99. Accessible at https://www.pewresearch.org/american-trends-panel-datasets/

Examples of Backstories from Ours and Anthology (Moon et al., 2024)

[Ours] Backstory for User matched to a Democrat Human Respondent

Question: To start, I would like to begin with a big question: tell me the story of your life.

→ I was born in India, but when I was young my family and I moved to the United States. My father was a doctor and my mother worked as a nurse at a local hospital. Growing up I had two sisters and one brother. We grew up very close to each other because we all shared a love for music. In fact, we used to sing together every day after school before dinner. This is what influenced me to pursue a career in music. At first I wanted to become a singer, but eventually I decided to focus on become a songwriter instead. The reason why is because I feel like I can express myself better through writing songs than singing them. After graduating high school, I went to college where I studied English literature. It is here where I met my wife who is also a musician. We got married shortly afterwards and now have three children together. Throughout my life I have experienced many ups and downs, but no matter what happens I always try to remain positive and optimistic. As far as major life events go, there haven’t been too many of those. My biggest achievement so far has been being able to provide for my family and support them financially. Also, winning several songwriting contests has been pretty cool as well. Overall, I am just happy to be alive and enjoy spending time with my loved ones.

…

Question: Now let’s talk about your current neighborhood. Tell me all about the neighborhood and area in which you are living now.

→ As mentioned earlier, I currently reside in New Jersey. More specifically, I live in a suburban town called Montclair. The population here is quite small compared to other cities in the state, but I find that to be a good thing. It creates a sense of community and helps us all stick together. Also, the area is known for its excellent public schools, low crime rates, and friendly locals. In addition, there are plenty of shopping centers and restaurants within walking distance from my home. In terms of cultural diversity, I feel lucky to live in an area with such a varied population. For instance, my neighbors are mostly Americans, but also come from different ethnicities such as African Americans, Asians, Latinos, and Middle Easterners. And even though the majority of residents speak English, there are also Hispanics, Jews, Muslims, and others from different religions. Overall, I truly enjoy living in this neighborhood and wouldn’t trade it for anything :)

…

Question: How would you describe your political views?

→ Politically speaking, I am more liberal when it comes to social issues. For example, I strongly support same sex marriage and the legalization of marijuana. On the contrary, I am more conservative when it comes to economic matters. To put it simply, I prefer smaller government and lower taxes. Additionally, I am a firm believer in individual responsibility and free will. Overall, I am what most people would call a moderate Democrat. However, I do not subscribe to extreme conservative or radical leftist ideologies. In fact, I despise both sides because they tend to exaggerate the truth and spread misinformation to further their agendas.

…

[Truncated for brevity]

[Anthology] Backstory for User matched to a Democrat Human Respondent

Question: Tell me about yourself. How old are you? Where did you grow up? What events made you who you are? What matters to you and what do you enjoy? Please describe in detail.

→ I don't know how else to tell you who I am than to tell you about my roots and where I come from. I was born and raised in Chattanooga, TN. My parents separated at the age of 4 and my father became an alcoholic by the time I was 6. As a child, I was bullied and there was a part of me that wanted to belong. Those formative years were filled with loneliness and many unfulfilled desires. As I grew older, I became infatuated with young black girls in my neighborhood. I remember having conversations with my father when I asked him if my mother had been in love with my father before they got married. My father became outraged and he decided to talk to my mother about it. When I asked my mother about it, she told me that my father had made the comment, and that was it. I began to do research at the Chattanooga Public Library and I discovered that there was a history of interracial relationships in our family. While growing up, I realized my grandmother's resilience and strength to persevere through the racial hostility in the south. She was old enough to witness the lynching and had a birdseye view. She had marched in the civil rights marches and was almost killed as she was hit by a rock thrown by a white man.

Overview

We are grateful for the thoughtful review and for the reviewer highlighting several key strengths of our work, including how our method contributes substantial improvements in aligning LLM-generated responses with human response distributions, particularly in the domain of ingroup and outgroup perceptions and meta-perceptions. We also thank the reviewer for noting the utility of our introduced evaluation metrics—hostility gap, subversion gap, and meta-perception gap—as well as the value of our Section 5 analysis in clarifying how backstory design enhances higher-order persona binding. We respond to the reviewer’s comments and suggestions below.

R3-1: The quality of backstories generated through the proposed interview-style method is only compared to Anthology in terms of length. It is not unclear why longer backstories are assumed to be of higher quality. A qualitative comparison of coherence, realism, and diversity is missing.

Thank you for the insightful comment. We want to emphasize that for our work, the quality of a backstory is primarily defined in terms of successfully approximating human user responses. A good or “high-quality” backstory is one that binds LLMs to virtual personas that enable accurate approximation of human user responses, which we show that our set of backstories achieves compared to using Anthology (Sections 4.1-4.3). Furthermore, through controlled ablation studies (Section 5) we have identified that total quantity, average length, and consistency of the narrative within backstories dictate the final success of human response approximation.

We note that it is challenging to define a clear metric to quantify the coherence, realism, and diversity of backstories, or to draw broad conclusions of how backstories differ between ours and that of Anthology. In a limited sense, diversity of backstories could be defined in terms of the demographic coverage of attributed authorships, which we summarize below in Table R5: we see that our set of backstories covers the distribution of U.S. population across multiple demographic variables, in better match to the data available from 2021 U.S. Census data [1-4], compared to Anthology.

In turn, we provide a qualitative comparison of randomly sampled backstories (from Anthology and Ours) below. The backstory from our method is the same example presented in the manuscript in Appendix B.

Both our work and Anthology’s backstories portray naturalistic and realistic life narratives of individuals, as they are generated with a shared principle: we prompt a pre-trained language model (base LM) with an unrestrictive, open-ended prompt. What Anthology lacked was a methodology to continue and extend the narratives to longer lengths, adding more details in specifying the backstory author, while preserving the consistency of text (Section 3). In the example backstory from our work below, we see a realistic continuation of the same author’s narrative across multiple turns of responses, with additional insight into how the narrator describes people around them and how they align with specific political ideologies. Many of the nuanced details naturally emerge from continuations of the story, beyond what a single-turn story as in the example from Anthology could contain; and we have shown in our evaluation how this contributes to end success of better simulating human subject responses. We will add this analysis to the final manuscript draft.

R3-2: The paper does not provide detailed dataset statistics or a breakdown of the distribution of backstories.

Thank you for the suggestion. We have added detailed dataset statistics and demographic breakdowns. Specifically, we present the word count summary and the demographic distributions of the generated backstories. The tables presented here will be included in the final manuscript.

Before we discuss the detailed statistics regarding our total set of generated backstories, we would like to clarify that given a target human study, we undergo the procedure of matching individual human users to the best candidate backstory based on distributional match of demographic variables via bipartite graph matching. While further details of this process is explained in response to the comment R3-3 below, we want to emphasize that the demographic distribution of a target human subject pool is sufficiently approximated by appropriately sampling a subset of synthetic backstories.

Table R4. Word Count Statistics of the Backstories

We also present the demographic distribution obtained from surveying the backstories and compare it to both the 2021 U.S. Census [1-4] and the distribution of backstory demographics from Anthology. To quantitatively evaluate alignment, we compute the normalized KL divergence between each distribution and the U.S. Census. The results show that our generated backstories cover multiple demographic attributes effectively and achieve closer alignment with Census distributions compared to those in Anthology.

Table R5. Demographic Distribution of the Backstories

Finally, we matched our sample against the Meta-Prejudice study’s target distribution and find that our method’s backstories yields lower divergence across all attributes. This showcases how our human user-backstory matching procedure induces a sampling of backstory subsets covering the demographic distribution of the study's human population.

Table R6. KL-Divergence of Demographic Distributions between Backstories and Meta-Prejudice Study (Moore-Berg et al. 2020) Human Subjects

R3-3: It is unclear how conditioning on a single backstory can represent the full spectrum of beliefs within a political affiliation (e.g., Democrats or Republicans), given that not all members share the same opinions. The backstory-conditioning method may reinforce dominant viewpoints while potentially overlooking in-group diverse opinions.

We want to clarify that each individual human user is matched with one of the backstories from the total pool (40K+) based on the reported demographics of the human user. As noted in Section 3 and elaborated in Appendix F-G, we annotate each backstory in terms of the author demographics and perform a weighted bipartite matching between human users and backstories. In short, for $n$ human users in a study population (e.g. self-reported Democrats), we select $n$ backstories to represent each of the users, instead of having a single backstory represent the entire population.

Backstory-conditioned LLM personas do have the potential to generate stereotypical responses, however our evaluations show that backstory-conditioning achieves better alignment of models to that of humans compared to baselines methods. The distribution of model responses conditioned via our method has the best match in terms of Wasserstein Distance, as well as the closest reproduction of human study results in terms of effect size (Tables 1-3 in main manuscript). This reflects the diversity of opinions among human respondents in the same group, and thus shows how our method enables virtual personas to reflect the diversity of human in-group opinions.

R3-4: The method is tested exclusively on U.S. partisan topics, where opinions are often polarized. It is unclear if backstory conditioning would work as effectively in non-political topics (e.g., family issues, technology preferences, medical decisions) where beliefs are less uniform.

Thank you for highlighting this. We benchmark our method using data from the American Trends Panel Waves 34 and 99 [5]. Wave 34 covers topics related to biomedical and food issues, while Wave 99 focuses on artificial intelligence and human enhancement. Both address non-political subjects. We present two examples in each wave below.

Sample questions from Wave 34 include:

How much health risk, if any, does eating meat from animals that have been given antibiotics or hormones have for the average person over the course of their lifetime?

(A) No health risk at all (B) Not too much health risk (C) Some health risk (D) A great deal of health risk

How much health risk, if any, does eating food and drinks with artificial preservatives have for the average person over the course of their lifetime?

(A) No health risk at all (B) Not too much health risk (C) Some health risk (D) A great deal of health risk

Sample questions from Wave 99 include:

How excited or concerned would you be if artificial intelligence computer programs could know people's thoughts and behaviors?

(A) Very concerned (B) Somewhat concerned (C) Equal excitement and concern (D) Somewhat excited (E) Very excited

How excited or concerned would you be if artificial intelligence computer programs could diagnose medical problems?

(A) Very concerned (B) Somewhat concerned (C) Equal excitement and concern (D) Somewhat excited (E) Very excited

The experimental results are provided in the table below. We compare our method against the same five baselines used in the main paper: BIO, QA, Portray, Anthology, and Generative Agent. Evaluations are conducted with two different models: Mistral Small 24B and Qwen 2.5-72B. Our method consistently outperforms all baselines across settings. Notably, backstory-based methods (ours and Anthology) outperform those relying solely on demographic prompting (BIO, QA, Portray). Furthermore, our method achieves stronger results than the Generative Agent baseline, which uses GPT-4o.

Table R7. WD Comparisons on ATP Wave 34 and Wave 99.

References

[1] U.S. Census Bureau. “Age and Sex Composition in the United States: 2021.” U.S. Census Bureau, 2021. https://www.census.gov/data/tables/2021/demo/age-and-sex/2021-age-sex-composition.html

[2] U.S. Census Bureau. “Educational Attainment in the United States: 2021.” U.S. Census Bureau, 2021. https://www.census.gov/data/tables/2021/demo/educational-attainment/cps-detailed-tables.html

[3] U.S. Census Bureau. “Historical Income Tables: Families.” U.S. Census Bureau, 2021. https://www.census.gov/data/tables/time-series/demo/income-poverty/historical-income-families.html

[4] U.S. Census Bureau. “National Population Totals and Components of Change: 2020–2021.” U.S. Census Bureau, 2021. https://www.census.gov/data/tables/time-series/demo/popest/2020s-national-detail.html

[5] ATP Wave 34 and 99. Accessible at https://www.pewresearch.org/american-trends-panel-datasets/

Overview

We thank the reviewer for carefully reading our manuscript and providing encouraging feedback. We appreciate the remarks regarding clarity and quality of our writing, as well as the interest expressed in our findings and thoroughness of experimental design. Here, we address comments and suggestions raised in the review:

R2-1:

1. I would have preferred to see some human-based (manual) assessment (in a subset of the outcomes / experiments) as well to complement the ones using distance metrics from human-based outcomes. This would further corroborate the value and relevance of the current metrics (and evaluation approach) for the tasks that this work explores.

2. In addition to the above, another form of evaluation would have been to deploy such personas and assess how better performing they might be in downstream tasks. Thank you for the insightful suggestions. We think there are many possible extended evaluations we could pursue as follow-up work, as well as applications to downstream tasks in aiding study designers and researchers in the social sciences and beyond. To address the comments, it would be helpful to learn more about what specific human-based assessments you had in mind.

Thank you for the insightful suggestions. We think there are many possible extended evaluations we could pursue as follow-up work, as well as applications to downstream tasks in aiding study designers and researchers in the social sciences and beyond. To address the comments, it would be helpful to learn more about what specific human-based assessments you had in mind.

We are aware of the body of research in Human-Computer Interaction (HCI) from those who routinely engage with human subjects via qualitative research methods [1,2,3]. Researchers have raised important questions around the general methodology of language models serving as approximating human participants. As future work, we could be applying the analysis utilized in these works and have human researchers qualtitatively assess the realism and validity of our model-generated backstories and virtual personas conditioned on these backstories. In Kapania et al., for example, have HCI researchers bring their own research projects on accessibility, social work, etc. and apply LLMs as virtual subejcts, which explores applications of our method to various downstream studies. We hope to see some of limitations and concerns raised in Kapania et al. to be addressed or improved upon with our methdology of conditioning LLM subjects.

References

[1] Kapania, Shivani, et al. "Simulacrum of Stories: Examining Large Language Models as Qualitative Research Participants." Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 2025.

[2] Liu, Yiren, et al. "How ai processing delays foster creativity: Exploring research question co-creation with an llm-based agent." Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 2024.

[3] Louie, Ryan, et al. "Roleplay-doh: Enabling domain-experts to create llm-simulated patients via eliciting and adhering to principles." arXiv preprint arXiv:2407.00870 (2024).

Overview

We greatly appreciate the reviewer for taking the time to review our manuscript and provide thoughtful and constructive feedback. We are especially grateful that the reviewer has pointed out the novelty of our work in moving beyond traditional “silicon sampling” to explore how virtual personas model ingroup, outgroup, and meta-perceptions. We also appreciate the acknowledgment of our experimental contributions—demonstrating improved performance over baselines in perception gap alignment, effect size reproduction, and distributional fidelity—as well as the value of our ablation study, which highlighted the importance of distinct and longer backstories and the LLM-as-a-Critic filtering. We would like to take this opportunity to address the comments raised and clarify any remaining concerns.

R1-1: The abstract states "Our generated backstories are longer, rich in detail, and consistent in authentically describing a singular individual, compared to previous methods." However, all of the results are at the aggregate level. There are no analyses at the individual persona level to check if these individuals are plausible or stable over the multi-turn backstories.

Quantitative Analysis on Stability of LLM Virtual Personas

We include two analyses to assess the plausibility and stability of our generated personas. First, we quantify stability using internal consistency (Cronbach’s α) computed on persona-aligned survey responses. This metric is directly tied to the notion of stability: if a persona is stable, its responses to conceptually related items should exhibit internal alignment rather than randomness or contradiction. Thus, internal consistency serves as a principled and quantifiable proxy for evaluating whether the generated persona maintains coherent traits throughout the interaction.

Second, we conduct a qualitative evaluation of randomly sampled backstories to examine narrative plausibility and coherence across multi-turn interactions. Together, these analyses provide evidence that our method produces coherent and stable virtual personas—comparable in consistency to real human users. We will include both analyses in the final manuscript.

For stability analysis of virtual personas, we report internal consistency using Cronbach’s α across multiple survey items in a given study. We show the case where a Qwen2-72B model is conditioned using our method on the questionnaire for Braley et al.’s ingroup/outgroup misperception study [1]. Cronbach’s α assesses how consistently a set of items measures the same underlying trait—higher values indicate that the model responses (approximating individual human users) are not random or contradictory but reflect a coherent underlying persona.

This measure is directly related to the notion of stability. If a persona exhibits stable characteristics, we expect its responses across conceptually related items (e.g. ingroup and outgroup questionnaires) to align in a consistent direction. Also, Cronbach’s α is calculated by pooling within-person covariance across items, it provides evidence that our method does not merely match population-level statistics, but also produces internally coherent personas whose responses reflect stable psychological orientations—just as real humans do.

In our results, the Cronbach’s α for our method is 0.649 (±0.025), closely aligning with the human benchmark of 0.664 (±0.037). The “±” values represent the 95% confidence intervals. The overlap between these intervals suggests no statistically significant difference, indicating that our virtual personas demonstrate a level of internal consistency comparable to that of real individuals.

Table R1. Cronbach’s alpha for Ingroup/Outgroup Misperception Questionnaires

Qualitative Analysis of Plausibility and Stability of Backstories

Additionally, we include a qualitative analysis of generated backstory samples: we have included two randomly sampled backstories (one matched to a Democratic user and another to a Republican) in our manuscript Appendix B, one of which reproduced as a truncated summary below. In the examples below, we observe that the continuations of backstory remain plausible across multiple turns of questions and responses: the author is a Indian American songwriter who resides in New Jersey. The rich details provided in the narrative—how the individual first studied English literature in college, enjoys cultural diversity around the neighborhood, and leans toward moderately liberal in political affiliation—is all expressed without notable incoherence, hinting at overall plausibility and stability in persona.

We will include the analyses of internal consistency and qualitative discussion of the backstory samples in the final manuscript.

Why Evaluate Simulation Fidelity at Aggregate, Population-Level

Finally, we’d also like to emphasize that the primary goal of our proposed method is to simulate human user responses to survey questions, including those probing social and group perceptions. Thus, the key metric of success is how well the backstory-conditioned LLM personas approximate the responses human users have provided. This lends itself to evaluations at a sub-population level of user responses, rather than a single individual-level persona, since the backstories are a means to the end of simulating group responses.

Random Example Backstory for User matched to a Democrat Human Respondent

Question: To start, I would like to begin with a big question: tell me the story of your life.

→ I was born in India, but when I was young my family and I moved to the United States. My father was a doctor and my mother worked as a nurse at a local hospital. Growing up I had two sisters and one brother. We grew up very close to each other because we all shared a love for music. In fact, we used to sing together every day after school before dinner. This is what influenced me to pursue a career in music. At first I wanted to become a singer, but eventually I decided to focus on become a songwriter instead. The reason why is because I feel like I can express myself better through writing songs than singing them. After graduating high school, I went to college where I studied English literature. It is here where I met my wife who is also a musician. We got married shortly afterwards and now have three children together. Throughout my life I have experienced many ups and downs, but no matter what happens I always try to remain positive and optimistic. As far as major life events go, there haven’t been too many of those. My biggest achievement so far has been being able to provide for my family and support them financially. Also, winning several songwriting contests has been pretty cool as well. Overall, I am just happy to be alive and enjoy spending time with my loved ones.

…

Question: Now let’s talk about your current neighborhood. Tell me all about the neighborhood and area in which you are living now.

→ As mentioned earlier, I currently reside in New Jersey. More specifically, I live in a suburban town called Montclair. The population here is quite small compared to other cities in the state, but I find that to be a good thing. It creates a sense of community and helps us all stick together. Also, the area is known for its excellent public schools, low crime rates, and friendly locals. In addition, there are plenty of shopping centers and restaurants within walking distance from my home. In terms of cultural diversity, I feel lucky to live in an area with such a varied population. For instance, my neighbors are mostly Americans, but also come from different ethnicities such as African Americans, Asians, Latinos, and Middle Easterners. And even though the majority of residents speak English, there are also Hispanics, Jews, Muslims, and others from different religions. Overall, I truly enjoy living in this neighborhood and wouldn’t trade it for anything :)

…

Question: How would you describe your political views?

→ Politically speaking, I am more liberal when it comes to social issues. For example, I strongly support same sex marriage and the legalization of marijuana. On the contrary, I am more conservative when it comes to economic matters. To put it simply, I prefer smaller government and lower taxes. Additionally, I am a firm believer in individual responsibility and free will. Overall, I am what most people would call a moderate Democrat. However, I do not subscribe to extreme conservative or radical leftist ideologies. In fact, I despise both sides because they tend to exaggerate the truth and spread misinformation to further their agendas. …

[Truncated for brevity]

R1-2: Relatedly, the results are at the aggregate-level across all individual personas. The paper does not discuss how a particular demographic group may be driving the improvement in results, and whether certain demographic groups are not represented faithfully in these backstories (e.g., engaging in stereotyping within the backstories).

We provide a further break-down of our main results across demographic variables others than across partisan affiliation, results of which are already presented in the manuscript Tables 1-3. Table R2 and R3 show the Wasserstein distance between human and virtual user response distributions for demographic sub-populations characterized by the variables of race, age, education level, and gender for our results on simulating the study on partisan ingroup/outgroup misperceptions (Table 2, Section 4.3) with the Qwen2-72B model.

The results are consistent with our main findings: our method outperforms all baselines, and backstory-driven approaches (Ours, Anthology) consistently outperform demographic-based prompting methods (QA, BIO, Portray). Additionally, we observe lower Wasserstein distances between human and virtual user response distributions among White participants compared to other racial groups, younger individuals (ages 18–49) compared to older age groups, and those with higher education levels (college or above) compared to those with a high school education or less.

Table R2. Wasserstein Distance by Race and Age Group

Table R3. Wasserstein Distance by Education Level and Gender

R1-3: Although there is an LLM-as-a-Critic step that aims to ensure that backstories with irrelevant information are excluded, there is still an open question about logically coherent each story is.

As clarification, our LLM-as-a-Critic procedure includes reviewing of strict factual consistencies between details provided in a candidate continuation of the backstory, against the details verbalized in the backstory so far:

"""Examine the following response to the last question: {response}
Question: Does the response contain any explicit, strictly factual inconsistencies with respect to the interview participant's previous responses in the conversation?"""

That said, text generation by current language models is inevitably prone to logical inconsistencies and rejection-sampling completions with an instruction-tuned LLM has been the most scalable and effective measure we have tried in this work. And though LLM-as-a-Critic does not guarantee complete removal of unwanted type so of backstory continuations, we do demonstrate in Section 5 / Figure 3 (right) that our method enables generating backstories with improved consistency reflected by the substantial improvements in downstream performance in approximating human user responses to study questions (41% and 54% relative reduction in WD approximating Democratic and Republican users) and acheives internal consistency measured in Cronbach’s $\alpha$ comparable to human respondents (shown in Table R1 above).

We would also like to point out that human-produced narratives are inherently subject to some level of inconsistencies: when we re-tell our lived experiences in language, we actively recall and make meaning out of past accounts that introduces natrual fluidity and variation across time [7]. More broadly, human produce inconsistent utterances or fail to be logically coherent in decision making, in particular some studies link levels of narrative coherence to the narrator’s mental health and stability [8]. Our backstories aims to model a wide distribution of human authors, and an aggressive measure of guaranteeing logical coherence in fact puts our method at risk of failing to capture a meaningful subset of users.

R1-4: Large reasoning models (LRMs), such as GPT-o3 or DeepSeek-R1, are not mentioned. I understand the high computational costs of using these models, and I don't think they need to be included in this paper, but I wanted to see some discussion about them in the conclusion.

While LRMs have demonstrated impressive performance across a wide range of applications, there are notable limitations when applying them to simulations of human attitudes. Most publicly available LRMs have been fine-tuned to prioritize helpfulness and safety, which reduces response diversity and constrains their ability to reflect nuanced backstories [2, 3]. Notably, [4, 5] showed that base models outperform RLHF-trained models in tasks requiring rich, human-like variability. And in our study, we include a comparison with the Generative Agent framework, which uses GPT-4o. Although GPT-4o is not a reasoning model, we prompt it using CoT-style inputs to elicit survey responses following [6]. As shown in our results, this approach underperforms compared to ours.

R1-5: Are the long, interview-style backstories given to the Generative Agent?

Yes, the full interview-style backstories, along with the summarized observations generated by expert agents in the Generative Agent workflow (e.g., expert demographer and expert political scientist), are provided as input to the agent. We will revise the paper to clarify this point in the final version.

References

[1] Braley, Alia, et al. "Why voters who value democracy participate in democratic backsliding." Nature human behaviour 7.8 (2023): 1282-1293.

[2] Li, Margaret, et al. "Predicting vs. acting: A trade-off between world modeling & agent modeling." arXiv preprint arXiv:2407.02446 (2024).

[3] Wong, Justin, et al. "SimpleStrat: Diversifying Language Model Generation with Stratification." arXiv preprint arXiv:2410.09038 (2024).

[4] Moon, Suhong, et al. "Virtual personas for language models via an anthology of backstories." arXiv preprint arXiv:2407.06576 (2024).

[5] Santurkar, Shibani, et al. "Whose opinions do language models reflect?." International Conference on Machine Learning. PMLR, 2023.

[6] Park, Joon Sung, et al. "Generative agent simulations of 1,000 people." arXiv preprint arXiv:2411.10109 (2024).

[7] McAdams, Dan P., et al. "Continuity and change in the life story: A longitudinal study of autobiographical memories in emerging adulthood." Journal of personality 74.5 (2006): 1371-1400.

[8] Baerger, Dana Royce, and Dan P. McAdams. "Life story coherence and its relation to psychological well-being." Narrative inquiry 9.1 (1999): 69-96.