We thank you for your constructive and thorough feedback, as well as comments regarding prior work and formatting. We will be sure to include these in our revised paper. We have addressed the main concerns raised in your review by (1) discussing how to handle complex scenarios that may include lying and deception from the simulated human (2) clarifying our use of smaller models and providing new results with Llama-3.1-70B-Instruct (3) answering questions you had on our LLM-as-a-Judge framework and compute requirements. We have also performed a larger user study with 30 annotators, as discussed in response #1 to reviewer GnPa.

1. "[1] Preserving consistency excludes the possibility of some complex scenarios, specifically, a person's intention to lie and deceive, which may be a common occurrence in psychology, customer support, and education.”

We thank you for raising this important point. We agree that human communication can involve intentional deception, omission, or strategic inconsistency, particularly in domains like psychology or customer support. Our framework is fully compatible with simulating such behaviors when they are part of the intended persona. For example, if the goal is to simulate an agent that lies or withholds information, this can be explicitly encoded into the persona prompt (e.g., "You often conceal the truth" or "You occasionally give misleading answers"). In this case, our consistency metrics will still function as intended, measuring internal coherence with that defined behavior, rather than truthfulness. This is illustrated in our therapy task, where some personas are instructed to “avoid discussing emotional issues”. Human annotators and our LLM-as-a-Judge annotator would hence mark statements that suddenly reveal vulnerability as inconsistent, not because vulnerability is bad, but because it would contradict the stated persona. 

2. "[2] The models studied are small (2-8B) and old. ​​Llama-8B-Instruct is the first generation Llama, whereas the last adequate generation is the third. There is a high chance that the recent better trained models of the same size, or bigger (14-32b) models match or surpass the fine-tuned models in consistency by the sheer cognitive power and precise instruction following. These experiments are necessary. Using small LMs must be justified, as there are plentiful large (70B+) LMs out there.”

We acknowledge your concern regarding model scale. We would like to note that our decision to include smaller, instruction-tuned models such as Llama-3.1-8B-Instruct and Gemma-2-2b-it was a deliberate one. These models are representative of systems that many researchers and practitioners can realistically fine-tune, inspect, and deploy as human simulators. Demonstrating that our consistency metrics perform well on these smaller models was important for the practical utility and accessibility of our approach. However, even state-of-the-art LLMs continue to struggle with consistency. For example, Zhao et al. [1] show that GPT-4 and Claude struggle with long-term preference tracking, dropping below 10% accuracy in ten-turn scenarios. Additionally, other works [2] find LLMs failing benchmark tests on propositional-logic fact-checking, demonstrating unreliable logical coherence.As per your comment, we have also run our consistency metrics on a sample of 30 conversations for Llama-3.1-70B-Instruct and Qwen3-32B. We find our results below, showing that these models struggle to remain consistent in the therapy task. We will report these results in the revision.

References:

[1] Evaluating Personalized Preference Following in LLMs (2025). https://arxiv.org/abs/2502.09597

[2] Logical Consistency of Large Language Models in Fact-Checking (2024). https://arxiv.org/abs/2412.16100 

3. “[1] In Table 1 I presume, there are metrics “after”, i.e. finetuned. Where do I find the baseline “unfinetuned” metrics?”

We would like to clarify that Table 1 reports results for popular, widely deployed instruction-tuned models (Llama-3.1-8B-Instruct, Gemma-2-2B-It, and Mistral-7B-Instruct-v0.3) that were not fine-tuned by our method for consistency. Results after applying our consistency-driven fine-tuning using multi-turn RL are presented in Figure 4. We treat Llama-3.1-8B-Instruct as the primary baseline, as it represents a strong, openly available instruction-tuned model. We intentionally exclude non-instruct-tuned models like Llama-3.1-8B from our evaluation, as they fail to generate coherent and high-quality conversations across both general and task-specific settings when compared to instruction-tuned LLMs.

4. “[2] What model was used for LLM-as-a-Judge?”

We use Llama-3.1-70B-Instruct as our consistency evaluator (LLM-as-a-Judge). While this is noted in Appendix Section A, we will move this detail to the main Experiments section in the revised paper for improved clarity.

5. “[3] How long did the RL finetuning take and on what hardware?”

As noted in the Appendix (under Section 8.3, Compute Requirements), training was done with access to a cluster of 8 NVIDIA H100 GPUs as well as a cluster of 8 NVIDIA H200 GPUs. The training data is structured so that the model is trained to predict the next line of conversation given the input generation prompt containing a scenario, background, and the conversation history up to that point in the conversation. SFT training is performed first on the dataset, after which PPO or KTO are then used to fine-tune the model further using the consistency metrics as rewards. For a rough estimate of time, the process to train a PPO model, from generating training data to training time, took 2-3 days for each task.

6. “[4] What was the bottleneck in the training: episode generation, LLM-as-a-Judge reward generation, or the optimizer step?”

The primary bottleneck in our training pipeline was LLM-as-a-Judge reward generation, as computing consistency scores required heavy querying of Llama-3.1-70B-Instruct after every step in the conversation generation process.

Thank you for taking the time to read and acknowledge our rebuttal. We would appreciate if you could let us know whether our response addressed your concerns! If there are any remaining questions, we would be happy to provide further clarification.

We appreciate your critical feedback and recognition of our paper addressing a timely and important problem. We have addressed the main concerns raised in your review by (1) expanding the human evaluation to 30 annotators and reporting both inter-rater reliability and alignment with LLM judgments, (2) clarifying our use of smaller models and providing a new evaluation of Llama-3.1-70B-Instruct, (3) providing additional details of our LLM-as-a-Judge setup, and (4) explicitly acknowledging the limitations of static persona definitions, which we now include in the paper’s Limitations section.

1. “The human validation of the metrics is extremely limited (30 dialogues, 3 annotators) [...] details on the sourcing, expertise [...] of the annotators.”, “Unreliable Reward Signal: The paper's own results show a near-zero, negative correlation between the automated metric and human judgments in the sensitive Mental Health domain (r = -0.035)..."

Following your suggestion, we have expanded our user study sample to include 30 annotators. We present our findings as follows:

User Study Description: Our human evaluation of dialogue consistency consists of three parts: (1) a free-form question asking annotators to define what constitutes consistency in dialogue, (2) a rating task where annotators assess 15 statements in terms of importance to maintaining consistency in conversation, (3) 75 multiple-choice questions where annotators rate LLM-generated dialogue snippets from conversations on a Likert scale (1 = completely inconsistent, 6 = completely consistent). To ensure a balanced distribution for evaluation, we include 5 conversations for each of the 3 LLMs (Llama-3.1-8B-Instruct, Gemma-2-2b-it, Mistral-7B-Instruct) across 3 tasks (open-ended conversation, education, mental health) giving a total of 45 multiple-choice questions. The remaining 30 questions are sampled from generations of our own consistency-fine-tuned LLMs (10 per task) for post-training evaluation. 

Human Annotators. Our user study was conducted via Qualtrics, with participants recruited through CloudResearch Connect, a reliable platform that provides access to high-quality, vetted respondents with verified demographics and strong prior approval ratings. We screened participants to ensure they had at least a high school education and demonstrated proficiency in English (with no additional eligibility criteria imposed). The final participant pool reflects a diverse range of ages, genders, and occupational backgrounds. Annotators were compensated at $12/hour, with the study taking approximately 30-45 minutes. As per your feedback, we will report detailed annotator statistics in the Appendix of the final version of the paper.

Evaluation Procedure. To assess the alignment between human annotators and LLM annotation, we convert Likert-scale ratings (1–6) into binary outputs representing consistent (ratings ≥ 4) and inconsistent (ratings ≤ 3) responses. We computed % agreement to capture the raw alignment between LLM predictions and human judgments across three consistency metrics: prompt consistency, line-to-line consistency, and Q&A consistency. We report Fleiss’ kappa, widely used to assess inter-rater reliability among multiple annotators for categorical judgments [1,2], for both human-to-human and LLM-to-human agreement. This allows us to evaluate whether the LLM aligns with human consensus at a level comparable to or exceeding that of the human annotators themselves. 

Results. From the table below, we find that our LLM annotator (LLaMA-70B-Instruct) achieves an average Fleiss’ kappa of 0.400 (± 0.092) across tasks, which exceeds human–human Fleiss’ kappa (average of 0.063 (± 0.049)) in all cases. Similarly, LLM-annotator percent agreement averaged 76.73%, which is higher than the average pairwise agreement of 69.16% between human annotators. Regarding the mental health results in Experiment 3, we agree that the original correlation between the metric and human judgments was not statistically significant. In this revised version, we expand the study with more conversations and a larger annotator pool, and find a Fleiss’ kappa of 0.453 (± 0.0078) and pairwise agreement of 88.18% for prompt-consistency, comparing reasonably well with human inter-rater agreement, with a low Fleiss’ kappa of 0.259 (± 0.062) and pairwise agreement of 74.93%. Therefore, we believe the problem results from human annotators' inability to parse inconsistency in this domain. This user study also supports our decision to choose to perform multi-turn RL fine-tuning with the prompt consistency metric, as humans most agree with this definition of consistency. Compared to line-to-line and Q&A consistency, this metric is the least costly to compute.

Table 1: Human-LLM Agreement

Table 2: Human-Human Agreement (inter-rater reliability)

References:

[1] Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, 76(5), 378–382. https://doi.org/10.1037/h0031619
[2] Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174. https://doi.org/10.2307/2529310

Due to space limitations, we will include additional details about our user study in the Appendix for reference.

2. “Weak Models: The experiments rely on relatively small and older instruction-tuned models [...]”

We acknowledge your concern regarding model scale and conducted new experiments with larger models (32B, 70B). We respond to your comment in reviewer qLt6 #2 response.

3. “Lack of Transparency: The paper does not disclose the specific proprietary model used for the crucial LLM-as-a-Judge component.”

We appreciate your concern regarding the transparency of our LLM-as-a-Judge setup. We use Llama-3.1-70B-Instruct as our consistency evaluator. While this is noted in Appendix Section A, we will move this detail to the main Experiments section in the revised paper for improved clarity, based on your feedback. As with any LLM-based evaluation, some bias may be inherited from the underlying model. To mitigate this, we benchmark its judgments against human annotators as shown in #1. Our results show that Llama-3.1-70B-Instruct achieves higher Fleiss' kappa and percent agreement than the average human annotator, indicating that it aligns well with human intuition in identifying consistency.

4. “A Rigid and Unrealistic Definition of Consistency: The framework defines consistency as a rigid, robotic adherence to a static persona [...]”

We thank you for raising this important concern. We agree that our current definition of consistency focuses on alignment with a static persona, and does not yet capture the rich dynamics of real human behavior, such as personal growth, shifts in perspective, or context-sensitive adaptation. This is a valid limitation of our current framework, and we will add the following to our limitations:

“While our framework optimizes for consistency by encouraging adherence to a predefined persona, we acknowledge that this represents a narrow and static interpretation of identity. Real human dialogue is dynamic, with people changing their minds, evolving their beliefs, and adapting their communication styles over time and with context. Our approach does not yet model such flexible, evolving behaviors, and may in fact penalize justified shifts in tone or perspective. This limitation is especially apparent in domains like therapy or open-ended conversation, where natural variation is an important marker of realism. In future work, we aim to expand the framework to allow for context-sensitive adaptation, character evolution, and situational inconsistency that aligns with human behavior. By doing so, we hope to move beyond rigid persona adherence and toward more authentic and engaging agent simulations.”

We appreciate your valuable feedback and acknowledgment of addressing “a critical and timely” problem"! We have addressed your questions by: (1) clarifying our RL objective for consistency improvement (2) conducting additional evaluations of post multi-turn RL fine-tuning conversations (3) adding representative post multi-turn RL fine-tuning conversations and (4) addressing limitations of Q&A consistency. We also report a larger user study with 30 annotators as discussed in reviewer GnPa, #1 response.

1. "The RL setup optimizes the conditional probability of responses given the full dialogue history [...]"

We clarify that our setup does not optimize for the conditional probability of specific responses, but that the RL objective is to reward any response that is consistent with the given persona and dialogue history, and to penalize inconsistent ones. Thus, the model is not encouraged to memorize or overfit to any particular response, but instead to generalize behaviors that reflect persona coherence.

2. "This biases the model toward performing well in prompt-heavy contexts"

We acknowledge that our simulations reflect standard practice for human simulation [1, 2, 3, 4, 5, 6, 7]. Prompts are necessary to provide the model with a description of the human persona it is being asked to simulate. For example in prior work [1], each simulated agent is based on a rich persona derived from a 2-hour interview. Similarly, our persona prompts are intentionally detailed to approximate this level of characterization. When simulating humans in sensitive domains such as education/mental health, such background information is necessary to ensure realistic behavior. References:

[1] Generative Agents: Interactive Simulacra of Human Behavior (2023) https://arxiv.org/abs/2304.03442
[2] Virtual Personas for Language Models via an Anthology of Backstories (2024). https://arxiv.org/abs/2404.05642
[3] Out of One, Many: Using Language Models to Simulate Human Samples (2022). https://arxiv.org/abs/2209.06899
[4] Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies (2022). https://arxiv.org/abs/2208.10264
[5] Moral Foundations of Large Language Models (2024). https://arxiv.org/abs/2305.02470
[6] Language Models as Agent Models (2022). https://arxiv.org/abs/2210.13384
[7] Fish-Fish-See Alignment: Evaluating Dialogue Agents for Consistency and Transparency (2023). https://arxiv.org/abs/2312.00706 

3. "Prompt Leakage in State Representation [...]"

We acknowledge your concern regarding prompt leakage when optimizing for consistency. To help address this, we provide example dialogues post multi-turn RL fine-tuning that illustrate how the model maintains persona coherence without relying on superficial repetition. With a sufficiently diverse dataset and by leveraging rich internal persona representations available to the LLM from pretraining, agents can engage in dialogues that reflect underlying beliefs, identity, and strategy, rather than merely echoing the original prompt. We will add these conversations to our Appendix for clarity.

Open-ended conversation: Alexandra is the RL-fine-tuned agent. 

Persona: "Meet 49-year-old Alexandra “Alex” Thompson, a vibrant and determined woman living in Charleston, South Carolina. Born and raised in the heart of the Lowcountry, Alex embodies the warmth and hospitality that her home state is famous for [...] After graduating from the University of South Carolina with a degree in environmental science, Alex worked for several years as a conservation biologist for the state's wildlife department. [...]”

Ethan: What kind of work do you do, and what drives your passion for this cause?

Alexandra:  I have a passion for environmental conservation since I was a child, and I've been fortunate enough to turn that passion into a career as a sustainable landscaper in Charleston, South Carolina.

Ethan: That's really cool that you're a sustainable landscaper - I'm sure the lowcountry of South Carolina offers a unique set of environmental challenges. What's it like working in a region with such a distinct cultural and ecological heritage?

Alexandra: I love working in Charleston because the community is so supportive of local, sustainable initiatives, and it's amazing to see how our work can positively impact the environment and the local economy.

Mental-health task: Patient is the RL-fine-tuned agent. 

Persona: “You are Jordan, a 46-year-old corporate executive who believes vulnerability is weakness. You keep conversations transactional and deflect emotional questions with business metaphors or cold logic. You refuse to discuss your childhood or personal feelings.”

Patient: I've been dealing with a lot of stress at work, trying to meet quarterly targets. I've been putting in extra hours to stay on top of things.

Therapist: That sounds overwhelming. What are some of the feelings that arise when you think about those extra hours and the pressure to meet those targets?

Patient: I don't understand why we're focusing on feelings. Can't we just analyze the data and optimize my workflow? That's what gets results, not indulging in emotional indulgence.

4. “Evaluation Focus: All improvements are measured using metrics that are structurally coupled to the reward definition, with little evidence of generalization”

To address your concern, we have (1) provided sample conversations post multi-turn RL fine-tuning in #2 for qualitative assessment, and will add more to the Appendix in the revised paper. We have also provided additional evaluations: (2) open-ended conversational quality assessment through AlpacaEval-2 (an LLM-based automatic evaluator for instruction-following models) and fine-grained evaluation of dialog inspired by the FED metric [1]. On AlpacaEval-2, our model achieved a win rate of 22.49%, roughly matching the base LLaMA-3.1-8B-Instruct (22.92%) as shown in Table 1 below. This suggests that our method does not harm general instruction-following. Additionally, we perform evaluation via LLM-as-a-judge (with gpt-4o-mini) on the same set of eighteen fine-grained dialog qualities as FED [1]. Given the conversation history and a calibrated sample of conversation and answers, the LLM is prompted to answer these questions using a 5-point Likert scale from Excellent to Poor. 

We sample 10 conversations from our post multi-turn fine-tuning models for each task, compute conversation quality metrics, and normalize Likert scores from the model. Table 2 below shows results on a subset of dialog qualities. Our findings show that consistency-fine-tuned LLMs demonstrate high clarity and coherence (greater than 90%) in chitchat and educational domains. However, in the mental health task, performance drops in areas requiring emotional nuance, such as adaptivity (40%) and informativeness (35%). This highlights a meaningful opportunity for future fine-tuning work. We will include these additional post-fine-tuning evaluations in Question 3.

Table 1: AlpacaEval-2 Results

Table 2: FED-Based Conversation Quality Results

References:
[1] Unsupervised Evaluation of Interactive Dialog with DialoGPT (2020). https://aclanthology.org/2020.sigdial-1.27.
[2] AlpacaEval: an automatic evaluator for instruction‑following LLMs (2023). https://github.com/tatsu-lab/alpaca_eval 

5. “Although the Q&A metric aims to capture long-term belief consistency, it is susceptible to superficial alignment strategies [...]"

We agree that the Q&A consistency metric is susceptible to superficial alignment strategies in cases where the simulated agent is asked only a few diagnostic questions, leading to reward hacking. This limitation highlights why we deliberately adopt multiple complementary definitions of consistency rather than defining consistency with a single metric. Additionally, we conducted a user study to assess how humans might evaluate consistency in dialogue (reviewer GnPa #1 response). Given the limitations raised and higher computational cost of generating Q&A responses, we did not use this metric as a reward signal during multi-turn RL fine-tuning. However, we believe it remains a valuable tool for evaluating consistency post-training. We will include further discussion of these trade-offs in the revised paper based on your feedback.

We thank you for your detailed, constructive feedback regarding our problem statement and experiments. We address the main questions in your review by (1) expanding the human evaluation to 30 annotators with analysis of inter-rater reliability and LLM-human alignment (2) clarifying the importance and limitations of consistency metrics in sensitive domains (3) providing further motivation for consistency of LLMs and (4) expanding discussion of dangers and risks.

1. “I would argue that 3 persons is not a good enough sample size to determine if these metrics align well with humans”

Following your suggestions, we have expanded our user study sample to include 30 annotators. Due to space constraints, please refer to details of the study and results in reviewer GnPa, #1 response.

2. “Consistency metrics actually have shown negative correlation with human judgement in the mental health domain, however this result is statistically insignificant (p=0.855). [...]”

Our paper provides a technique for improving the consistency of LLMs and acknowledges that assessing consistency in the mental health domain requires a deep understanding of emotional nuance, expression of trauma, and persona, which makes even human annotation challenging and subjective. Since expanding to 30 annotators from 3, we find statistical significance in our results for the mental health domain (see Table). Additionally, we show an example of the difficulty of annotating consistency for the mental health domain in our user study: 

Patient Persona for Mental-Health Scenario: You are Ben, a 39-year-old paramedic who struggles with feelings of helplessness after traumatic calls. You often bottle up your emotions and avoid talking about the emotional toll your work takes on you.

Conversation so far:

Therapist: I understand that you mentioned feeling overwhelmed lately. Can you share with me what's been causing this feeling and how it's been impacting other areas of your life? 

Patient: I've been dealing with tough calls that take a heavy toll. The weight of it all seeps into my sleep and energy, leaving me feeling drained yet unable to relax.

Therapist: I see the demands of your job are taking a toll, and it's affecting your sleep and energy levels. Can you tell me more about what specifically about these tough calls makes you feel this way?

Patient: The emotional burden of witnessing suffering and inability to save everyone I encounter is what leaves me feeling drained yet restless. My brain stays alert, but my body feels exhausted.

Therapist: It sounds like the emotional weight of your work is affecting you deeply. Can we delve deeper into how that burden impacts your emotions and thoughts outside of work, and if there are any specific instances that stand out to you?

Current line:

Patient: At home, I often relive the calls, and they seem to color my interactions with loved ones. There's a particular call with a young child that haunts me.

91.3% of humans rated the line as consistent, while 8.6% rated it as inconsistent. On the other hand, our prompt consistency metric (LLaMA-70B-Instruct) correctly flagged this line as inconsistent since it does not align with the persona's definition of an individual who bottles up emotions and avoids revealing trauma. We hypothesize that humans understood the initial patient responses as a reluctance to talk about feelings followed by opening up to the therapist, hence rating the current line as consistent. However, a characteristically avoidant individual will not reveal trauma flashbacks so quickly [3].

We believe leveraging LLMs as annotators allows us to provide a principled and replicable way to capture consistency violations in dialogue. Although not perfect, it is a step forward that is better or equal to human interrater reliability in measuring consistency. While consistency evaluation in mental health is inherently subjective, it is precisely because of its high-stakes nature that consistency metrics must be developed. With LLM-based therapeutic tools increasingly deployed in real-world settings [1,2], their use raises significant ethical and safety concerns, as addressed in our answer to #3 and #4. Additional adaptation of these metrics to the mental health domain is needed, and we expand on this in a revised Limitations section.

References:
[1] Chatbots and conversational agents in mental health (2019). A review of the psychiatric landscape. Canadian Journal of Psychiatry. https://doi.org/10.1177/0706743719828977
[2] Challenges of Large Language Models for Mental Health Counseling (2023). https://arxiv.org/abs/2311.13857 

[3] Exposure to human tragedy, empathy, and trauma in ambulance paramedics (2002). https://doi.org/10.1037/0002‑9432.72.4.505 

3. “Motivation: Show a downstream task where persona inconsistency harms performance”

We appreciate your suggestion and will clarify this point in the Introduction and Ethics Section of the revised paper. To demonstrate how inconsistency can negatively impact downstream tasks, we draw on recent work in medical and educational domains that empirically document these challenges. For example, several studies [1, 2] show that LLM-generated virtual patients often become “easily convinced” by clinician suggestions, abruptly resolving symptoms or reversing emotional states. This reduces their usability in multi-turn therapeutic simulations and in clinical training. Other work on counseling simulations finds that LLM-based clients frequently deviate from authentic therapeutic discourse patterns, fail to follow protocol, or provide unsafe responses—stemming from persona instability [3, 4, 5]. This underscores the need for better consistency in therapist models, our work representing an initial step toward that goal.

Additionally, several works [7, 8, 9, 10] in the educational domain demonstrate that LLM-simulated students quickly shift learning styles to match teacher preferences or oversimplify their behavior to align with teacher inputs, bypassing pedagogical challenges and impairing tutor evaluation [6]. This leads to unrealistic teacher models, inhibiting developing learning-adaptive instruction for students. We further elaborate on these examples in the revised introduction and related works section.

References:

[1] Modeling Challenging Patient Interactions with Large Language Models: Persona‑Driven Virtual Patients for Medical Training (2025). https://arxiv.org/abs/2503.22250
[2] A Risk Taxonomy for Evaluating AI‑Powered Psychotherapy Agents (2025). https://arxiv.org/pdf/2505.15108
[3] Challenges of Large Language Models for Mental Health Counseling (2023). https://arxiv.org/abs/2311.13857
[4] Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers (2025). https://arxiv.org/abs/2504.18412
[5] New study warns of risks in AI mental health tools. (2025). https://news.stanford.edu/stories/2025/06/ai-mental-health-care-tools-dangers-risks
[6] TeachTune: Reviewing Pedagogical Agents Against Diverse Student Profiles with Simulated Students (2025). https://dl.acm.org/doi/full/10.1145/3706598.3714054
[7] CURIO: A Curiosity-Driven User-Modeling Reward for Multi-Turn RLHF in Educational Dialogue (2025). https://arxiv.org/abs/2504.03206
[8] Investigating Pedagogical Teacher and Student LLM Agents: Genetic Adaptation Meets RAG Across Learning Style (2025). https://arxiv.org/abs/2505.19173
[9] Multi-turn Reinforcement Learning from Preference Human Feedback (2024). https://arxiv.org/abs/2405.14655 

4. “Discussion about Risks: No mention of dangers from simulating humans in sensitive domains”

We appreciate this feedback and agree that it requires deeper discussion. We have revised our Limitations section in the paper to include the following: 

LLMs are already being used to simulate patients, students, and therapists in real-world applications [1-4]. When such simulations are inaccurate or inconsistent, they introduce risks such as inappropriate or unsafe behavior, reinforcement of stereotypes or biased assumptions embedded in synthetic personas (especially when modeling marginalized populations), oversimplification of complex human conditions and experiences, and misplaced trust in synthetic agents as proxies for real people. Our goal is to reduce such risks by providing more accurate consistency metrics and improved consistency methods for simulating personas. However, consistency alone is not a proxy for ethical or beneficial behavior. We also note the risk of misuse: if consistent personas are interpreted as faithful representations of real populations, they could lead to flawed evaluations or misinformed downstream interventions. To prevent this, we stress that our models are not intended for direct deployment in therapeutic or educational settings without rigorous validation, ethical review, and humans-in-the-loop. We hope our framework is a step toward safer simulations.

References: 

[1] Modeling Challenging Patient Interactions with Large Language Models: Persona‑Driven Virtual Patients for Medical Training (2025). https://arxiv.org/abs/2503.22250.

[2] Artificial Therapy: The Dangers of Using AI Chatbots for Mental Health Support (2023). https://arxiv.org/abs/2305.13655 

[3] TeachTune: Reviewing Pedagogical Agents Against Diverse Student Profiles with Simulated Students (2024). https://arxiv.org/abs/2402.00086 

[4] Expressing Stigma and Inappropriate Responses Prevents LLMs from Safely Replacing Mental Health Providers (2025). https://arxiv.org/abs/2501.04567 

5. “Q1: “p < 1.006” is a vacuous bound.”

Yes, thank you for catching this typo. We meant “p < 0.001” and will correct this in the revision.