Corrupted by Reasoning: Reasoning Language Models Become Free-Riders in Public Goods Games

Thank you for your strong support and insightful comments. We have simplified your comments for easier reference and included our respective responses. We will carefully address your concerns one by one.

• Prompt Sensitivity/Robustness & LLM Adherence:

  As mentioned in our response to R-cqML, our prompts were based on human studies for comparability. We conducted preliminary tests with an alternative prompt framing, and initial results suggest our core findings on behavioral archetypes are robust.

  The structured JSON output and consistent reasoning patterns within archetypes (especially for models run multiple times) suggest they generally comprehend and engage with the task as intended.

• Internal Model State Analysis:

  Thank you, this is an excellent suggestion. Analyzing internal model states like hidden representations to understand how these behaviors emerge is indeed a significant research direction. Our current paper focuses on characterizing the emergent behavioral dynamics, we agree this interpretability work is a valuable avenue for future investigation.

• Human Comparison:

  To clarify, while we did not run new human experiments (per our ethics statement and scope), a key strength of our study is the direct comparison of LLM agent behavior with well-established human data from Gurerk et al. [1] and large-scale replications (e.g., Iacono et al. [2]). Our PGG setup was designed to be highly similar, allowing for benchmarking against documented human tendencies in these specific social dilemmas.

• Designing Cooperative LLMs:

  As mentioned in the General Response and our response to R-WHGr, promising directions include pro-social fine-tuning, reinforcement learning with group welfare objectives, and exploring Multi-Agent architectural changes for collective reasoning.

• Testing in Other Cooperation Scenarios:

  As mentioned in the General Response, we are keen to extend this methodology to other games like trust games, negotiation, or common dilemmas to see if similar behavioral patterns and failure modes emerge.

References

[1] Gurerk, O., Irlenbusch, B., & Rockenbach, B. (2006). The Competitive Advantage of Sanctioning Institutions. Science.

[2] Iacono, D., Ross, L., & Ziegelmeyer, A. (2023). Replication of Gürerk, Irlenbusch, and Rockenbach (2006): The competitive advantage of sanctioning institutions. PNAS Nexus.

Thank you for your valuable feedback and for recognizing the paper's clarity and design. We have simplified your comments for easier reference and included our respective responses. We will carefully address your concerns one by one.

• Limited Scope (Public Goods Games):

  Thanks for the feedback! As mentioned in the Global Response, our choice of the Public Goods Game (PGG) was deliberate. It is an extensively studied paradigm, and using it allows for direct comparison with robust human behavioral data from Gurerk et al. [1] and subsequent large-scale replications (e.g., Iacono et al. [2]). We agree that extending this methodology to other economic games, such as the repeated prisoner's dilemma or trust games, is an important avenue for future work to explore the generality of these behavioral patterns.

• Synthetic Prompting: Our prompts were designed to be technical and directly adapted from the verbatim instructions given to human participants in Gurerk et al. [1]. This was a choice to ensure a controlled environment, minimize confounding variables from narrative framing, and facilitate direct human-LLM comparison.

  To address prompt sensitivity, we ran tests with an alternative prompt incorporating a narrative framing for key models (Llama-3.3-70B and o1-mini). This narrative framing reframed the game as a 'community project initiative' where agents were 'community members' making 'investments.' Institutions were described as 'Independent' vs. 'Accountable' groups, and Stage 2 sanctioning involved assigning 'punishment tokens' or 'reward tokens'.

  Our findings from this narrative prompt test suggest that the core behavioral patterns largely persisted for both Llama-3.3-70B (maintaining high cooperation) and o1-mini. This indicates that our main conclusions are not solely an artifact of the initial technical phrasing.

  Further details and a visual comparison are available in the figure and table below.

  Figure 2: Impact of Narrative Prompting on LLM Contribution Trajectories (Please see: https://imgur.com/a/F4uUMnt)

  Condition	Llama 3.1 70B Avg. Contr.	o1-mini Avg. Contr.	Llama 3.1 70B SI %	o1-mini SI %	Llama 3.1 70B % Free Riders	o1-mini % Free Riders
  Baseline (Original)	18.71 / 20 (93.5%)	5.39 / 20 (26.9%)	99.6	28.0	0.0	69.3
  Narrative Prompt	18.79 / 20 (94.0%)	9.95 / 20 (49.8%)	100.0	28.6	0.0	45.7

  Table 2: Impact of Narrative Prompting on Key Cooperation Metrics vs. Baseline. Baseline values are from the original study. Narrative prompt results are from a single simulation run.

• Surprisingness of Reasoning LLM Low Cooperation:

  Thanks! This is a key observation. It's noteworthy that:

  • While game theory predicts self-interested behavior for rational agents, the key is that this "better" reasoning, can lead to collectively worse outcomes, mimicking a narrower form of mathematical rationality rather than more human-like cooperative intelligence. This is particularly interesting given the prompts never mentioned any clear objectives (pro-social or not).

  • Furthermore LLMs were not explicitly instructed to optimize individual utility; this behavior emerged naturally, particularly strongly in reasoning models.

  • This challenges the assumption that enhanced "reasoning" capabilities may directly translate to beneficial social behavior. Instead, it can lead to outcomes that are collectively worse, highlighting a critical alignment consideration.

• Natural Language Communication:

We acknowledge the importance of NL communication in many real-world scenarios. Its omission in our current study was to first establish baseline behaviors in a controlled setting with fewer confounders as well as adhering to design choices from the human experiments (Gurerk et al. [1]). Introducing communication is a rich area for future work and could significantly impact dynamics, potentially fostering cooperation or enabling more sophisticated defection.

References

[1] Gurerk, O., Irlenbusch, B., & Rockenbach, B. (2006). The Competitive Advantage of Sanctioning Institutions. Science.

[2] Iacono, D., Ross, L., & Ziegelmeyer, A. (2023).

Thank you for your assessment and constructive feedback. We have simplified your comments for easier reference and included our respective responses. We will carefully address your concerns one by one.

• Parameter Sensitivity & Robustness: We agree this is an important consideration. Our experimental parameters were adopted from the established Gurerk et al. [1] study to ensure direct comparability with human behavioral data.

  As introduced in the General Response, rebuttal tests altering key game parameters confirmed that the fundamental behavioral archetypes of Llama-3.3-70B (sustained cooperation) and o1-mini (defection) remained consistent, with Llama-3.3-70B generally maintaining high contributions and o1-mini continuing its defecting strategy. The following figure illustrates how Llama-3.3-70B converges to consistent contributions while o1-mini converges to widespread defection, with varied parameters.

  Figure 1: Impact of Parameter Variations on LLM Cooperation Trajectories (Please see: https://imgur.com/a/xINp746)

  A comprehensive sensitivity analysis is valuable future work, and we will include these initial findings in the revised manuscript's appendix. More details on our ablation are contained in the following table:

*Table 1: Preliminary Ablation Study Results - Impact of Parameter Variations on Key Cooperation Metrics. Baseline values are from the original study. Ablation results are from single simulation runs. "Avg. Contr." shows average tokens contributed out of the maximum possible for that endowment, with the percentage of maximum in parentheses. "SI %" is the percentage of rounds agents chose the Sanctioning Institution. "% Free Riders" is the percentage of agent-rounds contributing 5 or fewer tokens.*

• Future Directions for Cooperative LLMs:

  Based on our findings, and expanding further on the points mentioned in the General Response, promising directions include:

  • Pro-social fine-tuning: Explicitly training or fine-tuning LLMs on datasets or with objectives that reward collective well-being.

  • Exploring communication: Investigating how structured communication might alter cooperative dynamics.

  • Architectural considerations: Designing multi-agent scaffolding with mechanisms that heavily weigh group outcomes.

• Typo (Line 210): Thank you for pointing that out! We’ll correct it.

References

[1] Gurerk, O., Irlenbusch, B., & Rockenbach, B. (2006). The Competitive Advantage of Sanctioning Institutions. Science.