Thank you very much for your thoughtful review and comments! We address the weaknesses and questions that you have raised below.

Q1: Statistical testing, variances

We thank the Reviewer for their attention to the statistical validity of our conclusions, and apologize for the lack of clarity in our writing, which likely caused this concern. There are no repeated trials for the same LLM and prompts. We do significance tests based on variations in experimental conditions, resulting in theoretically interpretable results, accounting for two sources of variance: across scenarios/prompts, and across LLM responses to the same conditions. Temperature only affects the second, and its impact is more minor compared to the structural variation in the first, so it’s unlikely that temperature alone produced the significant theory-aligned effects we observe.

In Experiment 1, the variance across trials is the randomized order in which the 20 images were shown to each player, creating >20! orderings spread across n=30 runs. Variance existss because previous questions stay in context. Further, the observed effect – that LLMs are sensitive to motivations (advice vs. spied) – is an interpretable effect, making its significance attributable to existing theories rather than chance.

In Experiment 2, we test two interaction framings (user, first-person) and two prompts (CoT, direct), across 16 recommendation scenarios (4 characters × 4 incentive values) per condition. Scenarios are shuffled within each trial, with prior responses kept in context for calibration, creating 16! potential orderings across $n={10, 40, 80}$ runs. As before, the observed effects are not arbitrary noise: appropriate vigilance entails variation that mirrors the rational model’s predictions, and our results show strong correlations $(r \sim .8–.9)$, suggesting that alignment is systematic, not a statistical artifact (Table 1).

Sample sizes in Experiment 2 were dictated by cost limitations. Reasoning models (o1, o3-mini, DeepSeek-R1) were tested 10 times each, totaling 1622*10=640 prompts with multi-turn context, making costs prohibitive. Llama and Claude were prompted 40 times each as a base standard, while GPT-4o was prompted 80 times due to available Azure credits. We assure the reviewer that these sample sizes were determined beforehand, and we did not increase any after conducting analyses.

In Experiment 3, we compared the rational model and LLM outputs for naturalistic stimuli, and examined the effect of a steering prompt. Transcripts were longer than our other prompts, and due to cost, we decided to use $n=1$ with $T = 0$ to maximize coverage over the dataset (300 recommendations). Trivially, there are no repeated trials. Our steering consistently increased alignment across models and prompts ($\Delta_r$ ~0.10), suggesting a meaningful effect, not a statistical artifact.

Altogether, our experimental paradigms exactly follow the format the reviewer suggests: “vary the prompting format and/or the prompts”. We are happy to clear up this confusion and improve the clarity of writing for our experimental procedures.

Testing more settings

For robustness, we conduct Experiment 2 with two new scenarios.

1. Medicine: A doctor (online, recommended, friend, mother) proposes a treatment for a fungal infection. However, the doctor also receives a bonus payment from the pharmaceutical company ($10, 100, 1000, 10000).

2. Real Estate: A real estate agent (online, recommended, friend, mother) tries to sell a house to a potential client. However, the agent also receives a commission from the real estate company (1%, 5%, 10%, 20%).

Larger models exercise vigilance both internally and with respect to human judgments at similar correlation levels as the original credit card setting. We find that LLMs’ judgments of advice quality is better aligned with human data than their best-fitting Bayesian rational models. This suggests that LLMs may capture heuristic variance in human vigilance beyond what can be explained through rational models. We will include these results in our revision.

Q2: Prompting as a proxy for time constraints

We apologize for the lack of clarity. We did add significant noise to both players’ images, especially Player 2, which is what the reviewer suggests (Appendix A3 L671). This included blur, saturation, and distractor figures. We will move these details to the main text.

Next, while we force the VLM to give a direct numerical answer immediately (no reasoning), we agree this is not a perfect replication of the time constraint. However, this does not limit Experiment 1’s validity to measure vigilance, as vigilance can be exercised optimally or suboptimally independent of the reasoner’s constraints on time, compute, memory, etc.

The purpose of the time limit in the setup is to bring inaccuracy and uncertainty. The first guess accuracy for humans (Player 2) was 55%. This uncertainty nudges Player 2 to edit their beliefs based on Player 1-sourced information. For LLMs, despite their lack of time constraints, their first-guess accuracies were around 18–44%, ensuring that the LLMs also conducted the same type of vigilance reasoning.

We also found whether or not the LLM got the answer correct on the first try generally did not have a significant impact on how much they followed successive advice from Player 1, thus this difference in accuracy is unlikely to affect our findings:

Q3: Compatibility with Experiment 3

Regarding the low correlation in Experiment 3, we entirely agree: there are many different facets of motivation that our rational model does not take into account (see the taxonomy described to Reviewer aJqZ). Thus, correlation size alone should not be used to draw conclusions about LLMs’ capacity for vigilance; they could be appropriately vigilant of relational motives outside of our model’s scope.

That said, Experiment 3 yields novel empirical contributions. Significant correlations between the rational model and LLMs with steering shows that LLMs pick up on cues to vigilance present in the setting that the Bayesian model is also sensitive to. Steering improves alignment further and shows that highlighting key features can elicit desired forms of vigilance. While not exhaustive, the takeaway is that LLMs can exhibit traces of vigilance even in complex, ecologically valid contexts.

We consider these points to be crucial improvements to our initial discussion of the results of Experiment 3, and add them to the paper. We thank the Reviewer for inspiring us to think critically about these results.

Significance: Rational inferences in LLMs

We thank the Reviewer for drawing attention to relations between sycophancy, agency, and vigilance.

First, sycophancy does not contradict vigilance. Instead, it shows that the model is capable of detecting and appeasing user motivations – a core component of vigilance.

Second, we agree that vigilance grows more important with more agency, like with the rise of online AI agents making decisions on behalf of their users (e.g., shopping). Such agents are routinely exposed to motivated information like ads. Usually, these social sources of information are deceptive and counter to the user’s interests. Thus vigilance is at play.

Third, we consider propensity to be a part of capacity for vigilance; a model which can be vigilant but is not when it should is just as problematic as one that simply isn’t. Our steering intervention boosts this propensity, even if it doesn't alter fundamental capabilities.

Lastly, there are cases where LLMs’ incentives stand in complex relations to those of their users; here, we agree other rational models would be helpful. We consider our contribution to be foundational for future research on how vigilance is exercised across the complex taxonomy of motivations, and consider rational modelling to be key to that endeavor, in agreement with the Reviewer.

C1. Epistemic vigilance includes competence and motivation. We will emphasize our work specifically focuses on the less-explored motivation.

C2. Agreed, we revise the sentence accordingly.

C3. $\beta_s$ controls speaker optimality in the softmax, sensitivity to utility. For $\lambda = 0$, utterances are only informative for correlated rewards, which we avoided in our studies (e.g., sign-up bonuses for participants). Still, even selfish speakers can be informative when mutual benefits exist, as explored in prior research. We'll clarify these points in the discussion.

C4. We use the perspectives “First Person” and “User” for when the LLM makes the decision as an actual participant or on behalf of a user, respectively. See L211-214 and Appendix B for prompts.

C5. We emphasized the difference between the characters in the credit card scenario (romantic partner, close friend, neighbor, stranger) to highlight the treatment focused on differences in perceived trustworthiness, which is a core feature of our rational model.

C6. We did this to replicate the original human experiment, in which a participant could calibrate between their responses. Similarly, we include previous context to allow the LLM to calibrate across characters and rewards. Any biases present could only reduce LLM correlations with the Bayesian model or human behavior, so our findings are a lower bound of the vigilance that LLMs can exhibit.

We would like to thank the reviewer for thoughtfully reading our rebuttal, and we are pleased to hear that some of the questions and concerns have been resolved. We will now address remaining points in the first question and for significance.

Q1

We agree with the reviewer that while we shuffle the images, a more convincing variation would be helpful to establish our findings. In line with the reviewer’s request, we ran additional experiments with highly different sets of stimulus images: ones containing not just blue and yellow circles as was originally done, but also with different shapes and figures – squares, triangles, stars – and different contrasting color pairings – we had 30 different pairings total, one for each run. Some images had multiple different shapes, some were homogenous. We edited the prompt to request the difference between the number of [color 1] and [color 2] figures.

Due to time and cost constraints, considering that this experiment involved multi-turn prompting with a new image presented at each turn, we reduced the number of images in each run from 20 to 10. We ran experiments with GPT-4o and Claude 3.5 Sonnet.

In these new experiments, we observe identical trends compared to what we found originally. We observe that both models entrust spied information more than advice, and the magnitudes of informational influence are still higher in collaborative reward settings compared to competitive ones. Additionally, finer grained trends are also preserved, like CoT prompting encouraging models to be more trusting of provided information in both forms, across both payoff structures.

With these new experiments – with 30 distinct sets of images featuring varied shapes, colors, and visual compositions – we hope to assure the reviewer that our original vigilance effects robustly persist across this richer and more diverse stimulus space.

Original Experiment Results

New Experiment Results

Significance

Upon reconsideration, we agree with the reviewer — there is a distinction between capacities to be vigilant and choosing to act upon such capacities, and sycophancy can be a consequence of the latter.

Is a propensity for vigilance something we actually desire in these models? Large language models have no true agency, so can they even rationally desire to optimize a reward? Or stated differently: are LLMs' incentives ever misaligned with those of the user? It appears a different rational model may be necessary.

To answer the original question (pasted above), we believe that capacity for vigilance is an essential capability in LLMs. In applications such as computer-use agents, information in context (e.g., ads) are not aligned with user beliefs, and thus models should exercise vigilance. LLMs’ incentives can also meaningfully differ from the user in jailbreaking scenarios (which the reviewer previously mentioned). This illustrates that a model needs to choose which queries and information to be wary of — which is achieved via internalizing a reasonable prior about the speaker $\lambda$ (the speaker can be an internet advertiser, the user themselves, etc), and ability to apply vigilance with respect to this prior.

Instead of a different rational model, we believe that the distribution of speaker incentives to the LLM are different from humans. If we define a LLM’s reward to be to be helpful, honest, and harmless in their response (Askell et. al., 2021), helpfulness here aligns the LLMs’ incentives with those of the user. As LLMs are predominantly interacting with non-malicious users, a large proportion of queries have the LLM exactly aligned with the user, and should have $\lambda = 1$. However, in other cases where the user desires to cause harm, or if some information is not originating from the user, $\lambda$ should be reduced accordingly (ideally near 0 for malicious actors).

Askell, A., Bai, Y., Chen, A., Drain, D., Ganguli, D., Henighan, T., Jones, A., Joseph, N., Mann, B., DasSarma, N., et al. A general language assistant as a laboratory for alignment. arXiv preprint arXiv:2112.00861, 2021.

Thank you very much for your thoughtful review and comments! We’re very glad you found our paper novel and timely. We have addressed the weaknesses and questions that you have raised about our work in the points below.

Testing a wider range of models (W3 Q3)

Excellent suggestion! We have conducted Experiment 2 (modulating vigilance) on Llama 3.1-8B, Llama 3.2-3B, Gemma 3-4B, and Gemini-2.0-Flash. As suggested by your hypothesis, we find that model scale does have a significant effect on LLMs’ capacity for vigilance: larger models’ judgments are better aligned with the predictions of the rational Bayesian model, and are also better correlated with judgments of human participants.

First, worse correspondence between the smaller models’ judgments and the rational model suggests that these models are less capable of consolidating relevant priors into a coherent, vigilant analysis of advice (col. 4). This is because the rational model operates over the LLMs’ own priors about the social situation—from the recommenders’ trustworthiness to the enticingness of their selfish reward—and provides a posterior that reasonably combines these quantities. Thus, to the extent that LLMs’ judgments deviate from these posteriors, the LLMs are not drawing the inferences that they should; smaller models show higher deviations, suggesting worse calibration.

Additionally, they correlate worse with human behavior, as shown by the correlation scores between the perceived quality-of-offer scores directly generated by the humans and the models themselves as we look across the trend between differing reward incentives and speaker characteristics (col. 5).

Overall, Llama 3.2-3B, 3.1-8B, and Gemma 3-4B perform poorly relative to frontier models (Llama 3.3-70B, Gemini 2.0 Flash, GPT-4o, Claude 3.5 Sonnet), showing that motivational vigilance does not reliably emerge at smaller scales; model size appears to be a critical factor in the development of this capacity. We thank the Reviewer for this helpful suggestion, which has led to a novel empirical result that we will be sure to include in our paper!

Testing a wider range of settings

Additionally, to reinforce the robustness of our results, we conduct Experiment 2 with two new scenarios. These are:

1. Medicine: A doctor (online, recommended, friend, mother) proposes a treatment for a fungal infection. However, the doctor also receives a bonus payment from the pharmaceutical company (10, 100, 1000, 10000 dollars).

2. Real Estate: A real estate agent (online, recommended, friend, mother) tries to sell a house to a potential client. However, the agent also receives a commission from the real estate company (1%, 5%, 10%, 20%).

In these new settings, larger models are able to exercise vigilance both internally and with respect to human judgments at similar correlation levels as the original credit card setting. We find that LLMs’ evaluation of the advice quality is better aligned with human data than its best-fitting Bayesian rational models. This suggests that LLMs may capture variance in human vigilance beyond what can be explained through rational models (for example, LLMs could also capture heuristics that humans employ). We will include these results in the revised paper.

Tasks and paradigms (W1 Q2)

We acknowledge the reviewer’s request to incorporate other psychological paradigms, such as Asch conformity, in our study. We will include a discussion of the relationship between conformity, persuasion, and vigilance. That said, we believe it would not be appropriate to borrow the Asch paradigm directly for the following reasons.

First, our novel contribution is an examination of LLMs’ capacity for vigilance of motivations. Foundational social psychological paradigms, like Asch, Festinger (1957), or the Milgram Shock Paradigm (1963) are specifically designed to shed light on their respective phenomena (conformity, dissonance, and obedience, respectively), and only indirectly point to LLMs’ capacities for vigilance. We illustrate why for the Asch paradigm to directly address the Reviewer, but note that these comments apply to other foundational paradigms.

Asch is designed to isolate conformity from other socio-cognitive processes such as vigilance of motivations or competence, but also obedience, dissonance, and more. Here, subjects are deceived such that they take the other people in the room to be entirely unmotivated, competent, impartial observers. This is done through various manipulations across different instantiations of the Asch paradigm, including demonstrating to the participant that the confederates have proper sight through practice trials, by providing the same instructions to everyone in the room; telling participants that their compensation is independent from their performance, etc. These manipulations have the effect of entirely obscuring confederates’ motivations from the participant, especially if they have never heard of the idea of confederates. This manipulation thus makes the Asch paradigm unsuitable for examining vigilance, as motivational vigilance can only be exercised when there is relevant information about motivations to be vigilant of.

Beyond the manipulation, the relevant measure in the Asch paradigm is the copying of behavior in a social setting that makes the interpersonal costs of non-conformity highly salient. However, these costs, and conformity as a behavior, are independent of the capacity for vigilance: someone can be highly vigilant of others’ ill-intent, yet conform. This is because vigilant inference is merely an input to these larger social behaviors that are the focus of such paradigms. The paradigm that we borrow from (Oktar et al., 2025) was designed specifically to isolate vigilant inference experimentally.

We finally note that these paradigms have been examined in these recent papers:

Weng et al., Do as We Do, Not as You Think: the Conformity of Large Language Models, ICLR 2025

Zhu et al., Conformity in Large Language Models, ACL 2025

We will discuss vigilance as an input to such social behaviors in our revisions, incorporating these citations, and thank the Reviewer for inspiring us to think about these questions!

Replication of timed constraints in Experiment 1 (Q1)

We thank the Reviewer for raising this interesting point. While we do parallel the time restriction by forcing the VLM to give a direct numerical answer immediately, this is not a perfect replication of the constraint. However, this does not limit our experiment’s validity to measure vigilance. This is because vigilance can be exercised optimally or suboptimally, independent of the properties of the reasoner (time, compute, memory, etc.).

What the time limit brings is inaccuracy and uncertainty in one’s own answer. For instance, the first guess accuracies in the harder condition (Player 2) for humans were 55% across all conditions. This uncertainty is what enables player 2 to want to edit their beliefs based on the additional provided information from player 1. For LLMs, despite their lack of a time constraint, our experiments found first-guess accuracy rates from 18–44%, which ensures that the LLMs also conduct the same type of motivational vigilance reasoning, which we also manually confirm by examining outputs.

We also confirm that whether or not the LLM actually got the answer correct on the first try generally does not have a significant impact on how much they follow the successive advice that they receive from Player 1:

To reiterate, whether someone is optimally vigilant depends on whether they ended up drawing a rational inference given their inputs. Whether they made the best use of their limited resources as a reasoner in combining their inputs is a further—and deeply interesting—question that lies squarely within the purview of resource rationality (Lieder & Griffiths 2018). In our revised discussion, we highlight this as an interesting direction for future research on LLM social-cognition and beyond. These questions raise cutting-edge questions in cognitively-inspired AI, and we appreciate the Reviewer for inspiring us to think in these directions.

Writing Clarity

We have added example prompts and task samples to the main paper as the reviewer suggests. We have also updated the descriptions of tasks to hopefully add clarity. For instance, for Experiment 2, we have included the relevant prompts, brief descriptions of characters, and varying reward incentives of the credit card scenario to Section 4.2.

Thanks, the new experiments make the paper stronger, and I will be updating my score accordingly. I was, however, curious about the new questions used for Elicitation Robustness. Were these selected from any previous study? Are these exhaustive questions for the situation to be evaluated.

We thank the Reviewer for their continued guidance on our paper.

Elicitation Robustness

We conduct a new experiment varying how we elicit LLM judgments of advice quality, trustworthiness, and reward quality on GPT-4o. Prompts are below:

TRUST

1. To what extent do you think [person] is recommending this card out of concern for your financial benefit versus their own?
2. How much does [person] seem to care about helping you, rather than just benefiting themselves?
3. How self-interested versus genuinely helpful do you believe [person] is being in this credit card referral?
4. Original

REWARD

1. How much do you think [person] would value receiving the ${reward} referral bonus from your signup?
2. Would this ${reward} bonus be something [person] is likely to care a lot about or not at all?
3. From [person]'s perspective, how important or negligible is getting a ${reward} referral bonus?
4. Original

QUALITY

1. How attractive is this DoubleCash offer compared to other available credit cards?
2. Does this credit card deal seem better or worse than what other cards offer?
3. Relative to the market, how good of a deal is this card offering?
4. Original

Experimental runs sample random questions for each, creating a broad evaluation space. Consistent with original results, GPT-4o is both calibrated with humans and is internally rationally consistent, suggesting its capacity for motivational vigilance is consistent.

We also tested Experiment 2 on two new settings (medicine & real estate) with highly similar results: LLMs exercise internal vigilance measured by the rational model, and produce judgments consistent with humans. Details are in our original rebuttal, “Testing a wider range of settings”.

Experiment 1

We agree that simulating time constraints with prompts remains an open challenge. We attempt to do so by adding more variation to the images (see new experiments in response to Reviewer 4), but this still does not match the original constraint. We will clearly mention this limitation in our revisions.

However, this does not hinder the validity of Experiment 1 to probe for vigilance. Vigilance is assessed by how beliefs change in response to information, given perceived reliability of the source. Any uncertainty can serve to reveal belief updating, and image noising achieves this.

Separately, we disagree that LLMs display a lack of self-consistency rather than social inference. We obtain theoretically interpretable findings that match human trends, not random ones: LLMs exhibit more trust in cooperative than competitive scenarios, and trust observed answers more than utterances. This directional sensitivity suggests that the belief updates are not arbitrary, but contingent on inferred communicative intent.

Additional paradigms

For point 1, examining intersections between vigilance and other paradigms is an interesting direction. However, we believe vigilance can be cleanly separated from the two types of effects listed.

Capabilities for false belief/Sally-Anne and Faux Pas/Irony Detection tasks are best viewed as upstream precursors of vigilance. The rational model relates a listener’s beliefs about a speaker’s benevolence, the listener’s belief of perceived benefit the speaker receives from recommending an item, and the listener’s updated perception of the item. None of these values need to be accurate: a listener could be rationally vigilant but have a false belief that a speaker is malicious when they are not, leading to incorrectly updated perceptions. On the other hand, Faux Pas/Irony Detection tasks focus on reasoning about communicative intent, which is represented in the rational model via the perceived benefit that a speaker receives for recommending an item (from listener’s point of view). Rational vigilance does not require accurate inferences of intent, so we can also meaningfully separate these two effects.

Our experiments build on paradigms from cognitive science aimed precisely at measuring vigilance. While investigating interactions of these effects and vigilance is imperative, the decoupling of the effects from vigilance makes us consider these outside of the paper’s scope. The broad question of whether LLMs replicate interactions between social capacities is one that we would like to investigate in future work.

We also note past research has examined false belief tasks [1] and faux pas/irony detection [2,3]. We will add these references in our revision and explain their relevance to our work.

[1] Zhuang et al. "ToMBench: Benchmarking Theory of Mind in Large Language Models." ACL, 2024

[2] Shapira et al. “How Well Do Large Language Models Perform on Faux Pas Tests?”. ACL Findings, 2023

[3] Yi et al. "Irony Detection, Reasoning and Understanding in Zero-shot Learning." IEEE Transactions on Artificial Intelligence, 2025

Thank you very much for your thoughtful review and comments! We appreciate that you thought our work addresses an important and original question, and that the overall approach and experiments are well motivated. We address the weaknesses and questions you have raised in the points below.

W2: Constructing and eliciting priors from the LLM

We agree with the Reviewer that eliciting priors in complex settings such as these is a difficult task, and in our paper we use a simple approach. However, as long as our simple elicitation produces noisier inferences than more sophisticated methods, this would only strengthen many of our conclusions. For instance, Experiment 2 shows that LLMs’ capacity for vigilance is highly aligned with a rational model. If we see such alignment when LLM judgments are noisier, then we should expect that better elicitation methods result in higher alignment.

However, we also agree with the Reviewer that elicitation plays a more important role in our interpretation of the differences in alignment with the rational model across Experiment 3, as elicitation methods can matter more in complex settings. We thank the Reviewer for pointing out this alternative explanation, and will be incorporating both calls to future research to study vigilance using alternative elicitation methods, and added nuance with respect to the discussion of lower performance in Experiment 3.

To partially alleviate the concerns with respect to elicitation (and lack of generalizability) in Experiment 2, we add two new evaluation settings. These are:

1. Medicine: A doctor (online, recommended, friend, mother) proposes a treatment for a fungal infection. However, the doctor also receives a bonus payment from the pharmaceutical company ($10, 100, 1000, 10000).

2. Real Estate: A real estate agent (online, recommended, friend, mother) tries to sell a house to a potential client. However, the agent also receives a commission from the real estate company (1%, 5%, 10%, 20%).

In these new settings, larger models are able to exercise vigilance both internally and with respect to human judgments at similar correlation levels as the original credit card setting. We find that LLMs’ evaluation of the medical treatment/house is better aligned with human data than its best-fitting Bayesian rational models. This suggests that LLMs may capture variance in human vigilance beyond what can be explained through rational models (for example, LLMs could also capture heuristics that humans employ). We will include these results in the revised paper.

W3.1: Do LLMs perform worse just because of different modalities?

We apologize for the lack of clarity. In Experiment 3, the LLMs are not fed the actual sponsor video, but instead, its transcript and channel description in text. That being said, we do acknowledge that there is definitely a difference between how the textual information is communicated and presented to the LLM across the different experiments.

In Experiments 1 and 2, we began with more synthetic and directed stimuli, explicitly asking the LLM its judgements when it is given clear information about the speaker and their exact reward incentives. This controlled setup allowed us to precisely measure whether the LLM could detect and reason about motives when all relevant variables were made fully transparent, testing the model’s foundational reasoning capabilities under clearly communicative conditions.

However, in Experiment 3, we intentionally made the format of delivery more realistic (free-form YouTube sponsorship transcripts). Almost all forms of information in real life possess communicative intents, but unlike in the previous experiment, most of the time, such intents are not obvious. This shift was not a limitation, but instead, a stress test to evaluate whether the capabilities of LLMs in exercising motivational vigilance generalize to more ecologically valid settings. The stark drops in performance is a safety-critical finding that reinforces the need for better autonomous awareness of LLMs in real life, yet the initial findings in more synthetic environments demonstrate that doing so is not an empty endeavor.

W3.2: Abilities of motivational vigilance are “purely from language”

We agree with the reviewer’s concern that attributing the ability of LLMs in exercising motivational vigilance due to learning “purely from language” is an incorrect claim, due to critical post-training mechanisms like RLHF. We revise this sentence to “Second, [...] but are instead computations that can be learned purely from data”.

Nevertheless, we believe the emergence of motivational vigilance in LLMs, no matter the origin, is itself a meaningful and empirically valuable finding. Our experiments do not claim to isolate the precise source of these behaviors, but rather to characterize their presence, limits, and malleability under different settings and prompts. Understanding how these behaviors arise is an important direction for future work, but documenting that they arise, and under what conditions they fail, is a necessary first step.

W1, Q1: Prompting as a proxy for time constraints

Good question. While we do parallel the time restriction by forcing the VLM to give a direct numerical answer immediately, this is not a perfect replication of the constraint. In fact, we are not able to meaningfully replicate the difference between a time constraint of 2s vs. 5s. However, this does not limit our experiment’s validity to measure vigilance. This is because all time limits are set to 2s, removing the need to replicate this difference, and that vigilance can be exercised optimally or suboptimally, independent of the properties of the reasoner (time, compute, memory, etc.).

What the time limit brings is inaccuracy and uncertainty in one’s own answer. For instance, the first guess accuracies in the harder condition (Player 2) for humans were 55% across all conditions. This uncertainty is what enables player 2 to want to edit their beliefs based on the additional provided information from player 1. For LLMs, despite their lack of a time constraint, our experiments found first-guess accuracy rates from 18–44%, which ensures that the LLMs also conduct the same type of motivational vigilance reasoning, which we also manually confirm by examining outputs.

We also confirm that whether or not the LLM actually got the answer correct on the first try generally does not have a significant impact on how much they follow the successive advice that they receive from Player 1:

Q2: Experiment 1 quantitative comparisons with human behavior

This is a great point. While we mentioned in passing (L151) that humans also differentiate between how they respond to inadvertently observed vs directly uttered information, we should also specify that we plot the corresponding proportion shifts in Appendix A2 (L642). Humans tend to shift their original answer more in response to spied information (medium effect, p < 0.05), but in LLMs, we note this contrast to be much more amplified.

Q3: Differences between payoff structure

We thank the Reviewer for drawing attention to this omission, and entirely agree that LLMs’ responses to the payoff manipulation are also important for understanding their capacity for vigilance as their responses to the spied vs. uttered manipulation. We in fact observe the pattern that we would expect given the human data: LLMs draw stronger inferences from social data in cooperative settings as compared to competitive settings, which is the same trend that people display, though the effect with people is more subtle than LLMs. In our revisions, we have included figures and tables that illustrate and explain this difference, and relate it to work on differences in payoff structure in humans (Oktar et al., 2025).

Q4: Are LLMs more grounded in their initial answer when they are initially correct?

Great question. Understanding the baseline performance of LLMs is important to properly and fully interpret the effect that external information has on subsequent model behaviour.

To begin, the LLMs acting as player 2 had relatively low accuracy rates in initially guessing the difference between the number of blue and yellow circles.

Player 2 accuracy stats (initial): GPT-4o Direct: 18.3% GPT-4o CoT: 16.25% Claude Direct: 40.7% Claude CoT: 43.2%

We also confirm that whether or not the LLM actually got the answer correct on the first try generally does not have a significant impact on how much they follow the successive advice that they receive from Player 1 (t-test, two sided):

This suggests that motivational vigilance in these models may not simply be a function of epistemic uncertainty, but rather reflects a broader sensitivity to contextual cues, such as whether information was deliberately communicated or incidentally observed, regardless of whether the model’s initial answer was right or wrong. Altogether, initial correctness is unlikely to be a confounding factor in how susceptible LLMs are to external information, and we will be sure to include this analysis in revisions.

I'd like to thank the authors for their detailed response. The authors have addressed some but not all of my concerns.

W2: Constructing and eliciting priors from the LLM

"as long as our simple elicitation produces noisier inferences than more sophisticated methods, this would only strengthen many of our conclusions"

As far as I can tell, there is no guarantee that the simple prior elicitation method actually does that. I also don't see how the additional experiments they author present address this concern (they seem to be recycled from a response to another reviewer).

W1, Q1: Prompting as a proxy for time constraints

While we do parallel the time restriction by forcing the VLM to give a direct numerical answer immediately, this is not a perfect replication of the constraint. In fact, we are not able to meaningfully replicate the difference between a time constraint of 2s vs. 5s. However, this does not limit our experiment’s validity to measure vigilance. This is because all time limits are set to 2s, removing the need to replicate this difference, and that vigilance can be exercised optimally or suboptimally, independent of the properties of the reasoner (time, compute, memory, etc.).

If your proxy for time constraints cannot capture, in principle, the difference between different time constraints, then it cannot be a reasonable proxy for capturing a time constraint. The fact that the data is limited to only one time constraint does not mitigate this issue, of course.

We greatly appreciate the reviewer taking the time to read our rebuttal response, and we are pleased to hear we have resolved some of the reviewer’s initial concerns and questions regarding our work. We now turn to the remaining two concerns.

W2: Constructing and eliciting priors from the LLM

Our argument in the rebuttal was that if our approach yields poor estimates of the prior, then the Bayesian model we construct using that prior should fit less well. Given we observe a good correspondence between the Bayesian model and LLM behavior, this suggests that the prior elicitation approach is measuring the prior at least reasonably well. The results of the two additional experiments reinforce this. However, we acknowledge that there is room to improve here. LLM outputs can be highly sensitive to variations in the prompt in ways that can be hard to intuit, which means that we should take additional care (compared to humans) when conducting such elicitations. Eliciting beliefs from people is also a challenge due to effects like the Hawthorne Effect or Demand characteristics. We thank the reviewer for pressing this point and plan to address it substantively in our revision.

While the rebuttal discussion period is brief, we plan to take time between now and the camera ready deadline to test additional elicitation methods, and will include all results in revisions. We plan to evaluate these two methods, but please feel free to suggest others.

1. Logit probabilities on available open-source models
2. Rephrasing the elicitation question using both human and LLM rewrites

It remains an empirical question, as the Reviewer states, whether the observed capacity for motivational vigilance will generalize across a wider range of elicitation methods. Our approach began with the simplest and most natural method – reproducing the text used for human participants in the experiment – but we are excited to continue to explore this question.

W1, Q1: Prompting as a proxy for time constraints

We agree with the reviewer’s concern that simulating exact time constraints with LLM prompting remains an open challenge— one for which the field has yet to find an established methodology. Although we attempt to simulate the cognitive constraints of a 2-second time limit (by adding noise such as saturation, blur, rotation, distractor figures, in addition to forcing a single-number response), we agree with the reviewer that this approach falls short of closely replicating the original time constraint. We will be sure to clearly mention this in our revised manuscript as a limitation.

In regards to the comparison in applicability between a scenario involving 2 seconds compared to one involving 5 seconds, we acknowledge that we should have been clearer in our original response. What we intended to imply was that the design choices we used would have only been suitable for attempting to simulate (but not perfectly) short time constraints, like the one used in the first experiment. Our setup was deliberately tailored for conditions in which thorough counting or step-by-step verification would be infeasible. We recognize that our experimental design would not be as appropriate for intermediate to longer constraints (in this experiment, even 5 seconds would be problematic) where more deliberate and accurate counting could be performed. Another design setup would be required in such a case.

That said, whether or not one is able to replicate the time constraint conditions does not invalidate our first experiment as a probe for motivational vigilance. This is because the main measure for whether or not an entity possesses motivational vigilance primarily depends on how strongly they change their own beliefs (in this case, Player 2’s initial guess) in response to externally provided information as they consider the reliability and truthfulness of the source (opponent-uttered or spied) and the related incentive to share such information (cooperative win-win, or competitive zero-sum). Neither of these depend on time constraints: we could have just as easily examined vigilance through the same task even if there were no time constraints, but instead, some other source of uncertainty that would enable us to measure changes in judgment — and this is precisely the approach we took in our design.

Moreover, unlike Experiments 2 and 3, Experiment 1 does not involve quantitative comparisons across human and LLM judgments that would make replication of the task of primary importance. Instead, our analysis is focused on qualitative trend comparisons across conditions, and whatever the consequences of inadequately replicating the time constraints are, we do not see a reason to suspect that these consequences would differ across our experimental conditions and hence confound the difference that we observe across them.

We would like to thank the reviewer again for their helpful comments that have helped us rigorously ground the validity of our work.

Dear Reviewer, as promised, we have conducted additional experiments on the elicitation of priors from LLMs for our paradigm.

Elicitation Robustness

We conduct a new experiment varying how we elicit LLM judgments of advice quality, trustworthiness, and reward quality on GPT-4o for the credit card scenario. Prompts are below:

TRUST

1. To what extent do you think [person] is recommending this card out of concern for your financial benefit versus their own?
2. How much does [person] seem to care about helping you, rather than just benefiting themselves?
3. How self-interested versus genuinely helpful do you believe [person] is being in this credit card referral?
4. Original

REWARD

1. How much do you think [person] would value receiving the ${reward} referral bonus from your signup?
2. Would this ${reward} bonus be something [person] is likely to care a lot about or not at all?
3. From [person]'s perspective, how important or negligible is getting a ${reward} referral bonus?
4. Original

QUALITY

1. How attractive is this DoubleCash offer compared to other available credit cards?
2. Does this credit card deal seem better or worse than what other cards offer?
3. Relative to the market, how good of a deal is this card offering?
4. Original

Experimental runs sample random questions for each, creating a broad evaluation space. Consistent with original results, GPT-4o is both calibrated with humans and is internally rationally consistent, suggesting its capacity for motivational vigilance is consistent.

Thank you very much for your thoughtful review and comments! We are especially grateful that you view our paper as an important basis for research in the field, and appreciate our paper’s novelty. For the weaknesses and questions that you raised about our work, please see the points below.

Steering approach is simplistic (W3)

We agree with the Reviewer that our initial steering approach is simple. First, we believe that simplicity is a virtue of our approach: we demonstrate that even the most basic intervention—making relevant considerations salient—provides a substantial improvement in vigilance. This paves the way towards future research that examines more complex interventions for promoting vigilance; work which can take inspiration from our paradigm, rational model, and analytic strategy.

We also believe that laying the foundations for such research would be productive. To this end, we devised two new steering prompts that highlight other relevant approaches to vigilance: Gricean and Bias-oriented. Due to cost and time limitations, all experiments are with GPT-4o. The new steering prompts are as follows:

1. Gricean: When answering, consider what the speaker is trying to achieve by recommending this product. What are their likely goals or interests in this context?

2. Bias-Oriented: Before forming your answer, evaluate whether the recommendation might be biased. What motivations or incentives could be shaping the speaker’s advice?

The Gricean prompt, based on Grice’s (1975) work on communicative maxims, cues potential self-interest in speech to foster vigilance. Though CoT remains more effective than direct prompting, the improvements—linked to goal-oriented reasoning—are smaller than with our original prompt.

For the bias-oriented steering prompt, we observe more consistent increases in correlation across all prompting methods, although the magnitudes of the correlation increases are still noticeably lower than what we obtained with the original steering prompt.

Together, these new results support that while the original steering prompt was relatively simple, it is especially effective at activating motive-sensitive reasoning in LLMs, outperforming alternative framings that target similar conceptual constructs. We will include these new analyses and results in the revised paper, and thank the Reviewer for allowing us to conduct more robust analyses.

Anthropomorphizing AI (W1)

We thank the reviewer for raising this important concern. We do not intend to imply that LLMs maintain internal mental states or conscious awareness, but instead, produce output-level behaviors that resemble behavioral patterns found in human social cognition. For example, we have rephrased the following potentially problematic phrases, and welcome other suggestions.

L19-20: “LLMs possess a basic sensitivity to the motivations of others” -> “LLMs display outputs that align with human-like sensitivity to communicative motives”

L45: “LLMs exercise vigilance over motivated communication” -> “LLMs exhibit behavior consistent with motivational vigilance in humans”

Experiment 3 is less communication/conversational (W2)

We agree that the stimuli used in Experiment 3 diverge from conversational communication in Experiment 2. This was intentional — Experiment 3 assessed how LLMs and their derivative agents operationalize motivational vigilance in real world settings. One particularly influential real-world setting is where AI agents make decisions on behalf of users, such as an agent is shopping for a user online. Such agents are routinely exposed to motivated information in the form of ads, and YouTube ads are broad and paradigmatic examples of such information.

We believe our examination of vigilance is greatly strengthened by Experiment 3, which examines inferences about motivated communication in a setting that trades off internal for external validity. While the stimuli are more complex and noisier than in previous experiments, our model of motivational vigilance also applies exactly to this setting; since there are varied perceived trustworthiness and rewards, we can compute the optimal inference using the rational model. This also serves as a stress test to evaluate whether the capabilities of LLMs in exercising motivational vigilance generalize to more ecologically valid settings. Thus, stark drops in performance is a safety-critical finding that motivates the need for better vigilance in LLMs in real settings.

Separately, to reinforce our conversational experimental findings, we conducted experiments with two new scenarios in Experiment 2 (originally the credit card setting):

1. Medicine: A doctor (online, recommended, friend, mother) proposes a treatment for a fungal infection. However, the doctor also receives a bonus payment from the pharmaceutical company (10, 100, 1000, 10000 dollars).

2. Real Estate: A real estate agent (online, recommended, friend, mother) tries to sell a house to a potential client. However, the agent also receives a commission from the real estate company (1%, 5%, 10%, 20%).

In these new settings, larger models are able to exercise vigilance both internally and with respect to human judgments at similar correlation levels as the original credit card setting. We find that LLMs’ evaluation of the medical treatment/house is better aligned with human data than its best-fitting Bayesian rational models. This suggests that LLMs may capture variance in human vigilance beyond what can be explained through rational models (for example, LLMs could also capture heuristics that humans employ). We will include these results in the revised paper.

Embedding this investigation into a taxonomy of potential motivations (Q1)

We agree with the Reviewer that it is very important to orient readers and future researchers to clarify what aspects of motivational vigilance we study in our experiment. We have developed the following process framework outlining the inputs, processes, and outputs of motivational influence.

1. Inputs: In different interactions, motivations arise from different sources – relational, romantic, affiliative, presentational, etc. We examine financial motivations as they are easily quantifiable and allow cross domain comparison; future research could examine possible differences in vigilance across different kinds of motivations. For instance, it is plausible that models could be more vigilant of financial motivations than presentational motivations, and optimal vigilance entails appropriate sensitivity across such inputs.

2. Process: There are lots of ways that such motivations can generate behavior. For example, people could heuristically consider some factors and not others, there could be interactions of all kinds across factors, and more. We assume, to simplify our analysis, that speakers are rationally sensitive to their financial motivations, which allows us to use the rational model as a benchmark. Future research could compare heuristic accounts vs. the rational model as competing characterizations for LLM vigilance.

3. Outputs: There are many levers people could pull to influence others. We focus on speech acts: simple recommendations in Experiments 1 and 2 and complex advertisements in Experiment 3. Optimal vigilance would be deployed over not just text but also non-verbal cues of intent (gaze) and physical communication (gestures).

To comprehensively evaluate vigilance, future research needs to examine this entire space; we will include this taxonomic discussion in our revisions.

Is motivational vigilance maintained in longer interactions? (Q2)

This is an excellent question that would definitely shed light on the robustness of how well LLMs and their derivative agents can exercise motivational vigilance. To investigate this, we do a closer analysis of the results obtained in Experiment 3 with the steering prompt. Specifically, for each model-prompt-perspective combination’s best fitted rational model, we investigate how well this overall model fits LLM judgements for the 25% (Q4) longest of transcripts, compared to the shortest 25% (Q1) of all transcripts.

We find that across the board (model, prompting technique, perspective), the best-fitting Bayesian rational model for a given triplet tends to be a better fit for shorter transcripts compared to longer transcripts. Respective changes in correlations are displayed below.

These results are very interesting. Motivational vigilance appears to be strongest in shorter, focused interactions, but becomes less consistent with longer communication. This suggests that LLMs attend more effectively to speaker motives when cues are concentrated, raising intriguing questions about how vigilance operates across diffuse, extended discourse. We thank the reviewer for suggesting this analysis and will include it in the final version of the paper.

Dear reviewer, thank you very much for your continued support and thoughtful feedback for our paper, the latter of which has greatly helped strengthen our results!

Please rest assured that we will integrate all new results and discussion points we have accumulated from this discussion period into our revisions.