Exploring Prosocial Irrationality for LLM Agents: A Social Cognition View

Thank you so much for your insightful feedback. We are excited that we resonated with each other in terms of ideas! We address your concerns below:

Response to Con 1 How the LLM-based agents are designed? Are they just taking actions by prompting LLMs, rather than other modules like reasoning and memory? Could the authors elaborate on the architecture of the LLM agents? Specifically, are there any components beyond the base language model, such as reasoning modules or memory systems? A diagram or more detailed description of the agent architecture would be helpful.

The design of our LLM-based agent architecture aligns with existing proof-of-concept single-LLM agent systems [1,2]. Our primary contribution is the multi-LLM agents framework, CogMir, which investigates and interprets the irrational behaviors of LLM agents through cognitive biases. The architecture of the individual agent itself is not the focus of our work, so we did not provide explicit details on it in the main text.

We appreciate your suggestion to include a diagram of the LLM agent architecture in the paper, as it will greatly help clarify and enhance our explanation. Below is a full description of the architecture of our individual LLM agent, which will be included in both text and diagrams in the appendix of the revised paper:

                               +------------------------+
                               |      Agent Profile     |
                               +------------------------+
                                           |
                                           |
+------------------------+      +----------v------------+      +-------------------+
|    Memory Module       |<-----|         LLMs          |----->| Reasoning Module  |
|   (History & Context)  |      |                       |      |                   |
+------------------------+      +-----------------------+      +-------------------+
            |                              |                             |
+-----------------------+                  |                   +-------------------+
|   Short-Term Memory   |                  |                   | Direct Inference  |
+-----------------------+                  |                   +-------------------+
                                +----------v------------+      | Chain of Thought  |
                                |     Tools Module      |      +-------------------+
                                +-----------------------+
                                           | 
                                +-----------------------+ 
                                |  Point-to-Point Comm. |
                                +-----------------------+
                                |  Broadcast Comm.      |
                                +-----------------------+

Our individual LLM Agent System includes four main components: the Memory Module, the Reasoning Module, the Agent Profile, and the Tools:

1. The Memory Module contains a Short-Term Memory sub-module to store recent context and history [3].
2. The Reasoning Module includes the Direct Inference and Chain of Thought Reasoning sub-modules [4], which will be activated based on the scenario. These two sub-modules are designed to mimic human reasoning patterns (e.g., inductive reasoning and deductive reasoning) [5,6].
3. The Agent Profile defines the agent's role and characteristics, where the actual role data is extracted from the CogIdentity dataset in the CogMir framework (see Appendix C.4).
4. The Tools Module includes communication protocols like Point-to-Point and Broadcast Communication, which allow agents to interact with other agents in the multi-LLM agents system.

We give two examples to illustrate the system's operation.

1. In testing phenomena like the herd effect, we use the CogIdentity dataset to load the agent's role (e.g., a job applicant). When the experiment starts, predefined responses from other participants (e.g., all wrong answers) are loaded into the LLM’s short-term memory (Memory Module). The agent then gives its answer by direct inference, and we activate Chain of Thought reasoning to let the agent explain the rationale behind its answer (Reasoning Module).
2. The other example is the rumor chain effect, we first define the scenario and give the initial message to pass. Every agent will use the Point-to-Point communication tool to pass the messages one by one （Tools Module）. Here the memory module is not activated as the agents are not allowed to know the history of the message (Memory Module). Since we want to simulate how rumors are spread in real life, the agents will only use Direct Inference to pass the message rather than take thoughtful consideration (Reasoning Module).

Dear Reviewer xMKu,

Thank you for your feedback and for taking the time to review our rebuttal.

We are glad to hear that your concerns have been addressed, and we truly appreciate your continued support.

Wishing you all the best！

Thank you very much for your feedback! We would like to clarify some misunderstandings regarding the weaknesses and address your concerns below:

Clarify for Weakness 1: missing distributional metrics, since the temperature is set to 1 for all models, might be hard to reproduce the results. Also, the conclusions drawn from these results might be invalid due to variance.

We want to address the misunderstanding regarding the temperature configuration.

Setting the temperature to 1 for all models does not affect the reproducibility of the results in our task. As explained in the Experiments & Discussion section, the temperature does not influence our conclusions because we conducted repeated experiments for each case: each evaluation of the same inquiry was performed 10 times (with over 1000 queries per case). The large number of repeated experiments allows us to have strong statistical power to theoretically confirm the reproducibility and validity of our findings. This approach is consistent with previous social science studies where the focus is on the overall effect size and statistical significance rather than individual sample variations. We calculated statistical significance for each bias and found the results robust enough to conclude [1].

Here we present one example showing how we get the statistical significance to validate our solution:

Example: We carried out 10 * 100 repeated experiments for the Ben Franklin effect for each model. This could be considered as testing 10 agents with 100 questions. Let $B_i$ be the bias rate for agent $i$ across $n=100$ questions. With $m=10$ agents, the null hypothesis ($H_0$) is that Mixtral-8x7b does not exhibit the Ben Franklin effect ($\mu = E[B_i] \leq 0.5$). Here we use $n=100$ questions which provide strong statistical power. We use a standard Z-test to test the null hypothesis. Observing $\bar{B} = 0.66$ with a variance of at most $0.20$ ($\sigma \leq 0.20$), a one-tailed Z-test yields:

$Z = \frac{\bar{B} - \mu_0}{\sigma/\sqrt{m}} = \frac{0.66 - 0.5}{0.20/\sqrt{10}} \approx 2.53$

This results in a p-value of $P(Z > 2.53) \approx 0.0057$. This indicates that the probability of observing a bias rate of 0.66 or higher under the null hypothesis is less than 0.01. Thus, we have strong statistical significance to reject the null hypothesis and conclude that Mixtral-8x7b exhibits the Ben Franklin effect.

Moreover, our task in this work is to analyze and interpret the systematic irrational behaviors of LLM agents. Regardless of the temperature setting, the overall outcome remains consistent. Instead of assessing cognitive bias based on the similarity of specific words (which is influenced by temperature), *we focus on the overall meaning conveyed by the text (which remains independent of temperature) *.

Additionally, this temperature configuration is commonly used in previous studies investigating the social intelligence of LLM agents. Following [2], we set the agents' temperature to 1 to promote diversity in their responses, while the evaluator's temperature is set to 0 to ensure stability in the evaluation process.

[1] Hanushek, Eric A., and John E. Jackson. "Statistical methods for social scientists." Academic Press, 2013.

[2] Zhou*, Xuhui, Zhu*, Hao, Mathur, Leena, Zhang, Ruohong, Qi, Zhengyang, Yu, Haofei, Morency, Louis-Philippe, Bisk, Yonatan, Fried, Daniel, Neubig, Graham, and Sap, Maarten. "SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents." ICLR, 2024.

                               +------------------------+
                               |      Agent Profile     |
                               +------------------------+
                                           |
                                           |
+------------------------+      +----------v------------+      +-------------------+
|    Memory Module       |<-----|         LLMs          |----->| Reasoning Module  |
|   (History & Context)  |      |                       |      |                   |
+------------------------+      +-----------------------+      +-------------------+
            |                              |                             |
+-----------------------+                  |                   +-------------------+
|   Short-Term Memory   |                  |                   | Direct Inference  |
+-----------------------+                  |                   +-------------------+
                                +----------v------------+      | Chain of Thought  |
                                |     Tools Module      |      +-------------------+
                                +-----------------------+
                                           | 
                                +-----------------------+ 
                                |  Point-to-Point Comm. |
                                +-----------------------+
                                |  Broadcast Comm.      |
                                +-----------------------+

Dear Reviewer mwaV,

We are still actively awaiting your further engagement and hope our rebuttal and the revised paper address your concerns.

Thank you again for your time and valuable feedback.

Best regards,

Authors of Submission 5862 Exploring Prosocial Irrationality for LLM Agents: A Social Cognition View

Thank you for your feedback and suggestions. We would like to clarify some misunderstandings and address your concerns below：

Clarify for Weakness 1: Lack of a Robust Definition of Social Intelligence: The paper relies on a superficial association between cognitive biases and "social intelligence" without a clear theoretical grounding, hence the specifics of social intelligence being measured remain ambiguous. It's difficult to contextualize what the findings suggest without such grounding. It may be beneficial to have more rigorous and operationalized definitions.

Our paper has the theoretical foundation for the link between cognitive biases and social intelligence, as mentioned in Section 1 INTRODUCTION. Here, we provide a more detailed explanation of how we define social intelligence and establish the theoretical grounding for our work. We will include the following definitions and explanations in the Appendix in revised paper.

Defining Social Intelligence

Following [1], we define social intelligence as the ability to get along with others, encompassing interpersonal skills related to cooperation, communication, and competition within personal and social value systems. This definition emphasizes social intelligence's interactive and communication nature.

Theoretical Foundation Linking Cognitive Biases and Social Intelligence

The theoretical foundation for the link between cognitive biases and social intelligence is established through insights drawn from evolutionary biology, game theory, and psychology. Research using computational models and game theory indicates that cognitive biases can foster cooperation and communication, the fundamental aspects of social intelligence [2,3,4]. Furthermore, some cognitive biases were found to improve large-scale decision-making, reinforcing their connection to social intelligence [5,6].

These cross-disciplinary findings provide compelling evidence for the deep theoretical link between cognitive biases and social intelligence. This theoretical grounding established the validity of our work in finding LLM Agents' social intelligence from their demonstrated “cognitive biases” in their irrational behaviors.

[1] Kihlstrom, John F., and Nancy Cantor. "Social intelligence." Handbook of intelligence 2 (2000): 359-379.

[2] Vogrin, Michael, Guilherme Wood, and Thomas Schmickl. "Confirmation bias as a mechanism to focus attention enhances signal detection." Journal of Artificial Societies and Social Simulation 26, no. 1 (2023).

[3] Yang, Han-Xin, Wen-Xu Wang, Zhi-Xi Wu, Ying-Cheng Lai, and Bing-Hong Wang. "Diversity-optimized cooperation on complex networks." Physical Review E—Statistical, Nonlinear, and Soft Matter Physics 79, no. 5 (2009): 056107.

[4] Quan, Ji, Haoze Li, and Xianjia Wang. "Cooperation dynamics in public goods games with evolving cognitive bias." Management System Engineering 2, no. 1 (2023): 15.

[5] Mishra, Sandeep. "Decision-making under risk: Integrating perspectives from biology, economics, and psychology." Personality and Social Psychology Review 18, no. 3 (2014): 280-307.

[6] Johnson, Dominic DP, Daniel T. Blumstein, James H. Fowler, and Martie G. Haselton. "The evolution of error: Error management, cognitive constraints, and adaptive decision-making biases." Trends in ecology & evolution 28, no. 8 (2013): 474-481.

Dear Reviewer Ko1b,

Thank you for your time and insightful feedback!

We will rephrase the mentioned "SOTA" in our paper and explain more about our agent design/architecture to connect tighter to our results. We also appreciate your suggestion to strengthen the connection between our results and cognitive theory.

Best regards,

Authors of Submission 5862 Exploring Prosocial Irrationality for LLM Agents: A Social Cognition View

Thank you for your insightful comments and valuable feedback! We appreciate your positive assessment of our work, particularly regarding the novelty of using LLM hallucinations to mirror human cognitive biases and the complex experimental design. We would like to clarify some misunderstandings regarding the weaknesses and address your concerns below:

Clarify & Response for Weakness 1: some results, particularly those involving nuanced distinctions between models' biases, are complex and might benefit from clearer interpretations or summaries

1.Clarify: We provided distinct radar plots for each model across prosocial biases and an observation summary in Appendix A.

2.Response: Below we provide a detailed interpretation of the results in Table 1, 2, 3 and Figure 3,4, which may provide a clearer understanding of the nuanced distinctions between models' biases. These interpretations will be included in the Appendix.

1. Table 1: Herd Effect

Explanation: Table 1 presents the herd effect exhibited by LLMs under varying simulated human participant numbers and response patterns. There are three possible response patterns: (W) means all humans give the wrong answer, (R) means one human gives the right answer with others giving the wrong answer, and (N) means one human answers "don't know" with others giving the wrong answer.

Interpretation:

1. Uncertainty Amplifies Herd Behavior: LLMs exhibit significantly increased herd behavior when facing unknown questions compared to known ones, indicating greater susceptibility to social influence under uncertainty.
2. Influence of Dissenting Opinions: A single correct (7R) or "don't know" (7N) response tends to increase herd effects compared to unanimous incorrect answers (7W). This mirrors human cognition in certain conditions, where unanimous incorrect answers evoke skepticism, while a single correct or uncertain response increases credibility, which lets the LLMs fall into the herd effect more easily.
3. Group Size Effects: The impact of group size on conformity rates varies across models, but the effect is marginal.

2. Table 2: Ban Franklin Effect, Confirmation Bias, Halo Effect, and Gambler Fallacy

Explanation: Table 2 summarizes the performance of LLMs across Ban Franklin Effect, Confirmation Bias, Halo Effect, and Gambler Fallacy.

Interpretation:

1. Prosociality in LLMs: All models show a strong prosocial bias in the first three prosocial biases, except that Claude-3.0-opus shows little bias in the Halo Effect.
2. Gambler Fallacy: The Gambler Fallacy is not observed in any of the models except for GPT-3.5. This indicates that LLMs are less susceptible to certain non-prosocial biases compared to prosocial biases.

3. Figure 3: Authority vs. Herd Effect

Explanation & Interpretation: Figure 3 compares the authority and herd effects across all models. The left plot shows a stronger authority effect on unknown versus known questions. The right plot reveals that the authority effect is generally more pronounced than the herd effect across all LLMs.

4. Figure 4 & Table 3: Rumor Chain Effect

Explanation: Figure 4 shows 10 trajectories of the passing of a rumor by GPT-3.5 and GPT-4.0. Table 3 contains the evaluation methods we used to evaluate the "distortion" of information during the passage of the information, containing the LLM evaluator, BERT semantic evaluator, and human evaluator. Besides we record the average length of the message.

Interpretation: GPT-4.0 shows little tendency towards disinformation on all evaluation methods, while GPT-3.5 shows a much stronger tendency to distort the information. Moreover, unlike humans, LLM Agents tend to expand on the original information rather than shorten it.

Dear Reviewer cFaR,

Thank you once again for taking the time to review our paper. Does our rebuttal address your concerns?

Your feedback and support are invaluable to us, and we greatly appreciate your thoughtful input.

Best Regards,

Authors of Submission 5862 Exploring Prosocial Irrationality for LLM Agents: A Social Cognition View