LLM Strategic Reasoning: Agentic Study through Behavioral Game Theory

Thank you for the review. Below are our responses to your questions and comments.

Weakness 1 Response:

The concept of reasoning depth (sometimes also referred to as “levels of reasoning” or “strategic sophistication”) is a longstanding and standard practice in behavioral game theory and experimental economics. Classic works (e.g., Camerer, Ho, & Chong 2004; Costa-Gomes, Crawford, & Broseta 2001; Wright & Leyton-Brown 2010) use one-shot games specifically to benchmark and quantify these capacities.

Secondly, dynamic, multi-stage interactions are commonly analyzed as sequences of one-shot decisions, each defined by the agent’s reasoning given the available information at that stage. This decomposition is a standard approach in game theory and behavioral economics (Camerer 2003; Fudenberg & Tirole 1991), where each round is modeled as a one-shot decision conditioned on the current information set. Similar to current practice with LLMs, multi‑round analyses are implemented by appending the transcript of each preceding round to the prompt, allowing the model to revise its decisions and adapt to its opponent. From the LLM’s perspective, a dynamic strategic game thus remains a single‑shot decision problem presented within an ever‑growing context window. Although the model’s finite context length eventually limits how many rounds can be accommodated, this constraint does not undermine our framework: the core evaluation metric, reasoning depth, captures the model’s capabilities across all relevant dimensions and therefore remains valid regardless of prompt length.

Adaptation and learning over repeated play are valuable research avenues, but they differ from the baseline strategic reasoning that our framework measures. Recursive reasoning asks, “How many steps ahead can the agent think, given its priors?” Learning dynamics ask, “How efficiently and quickly can it update those priors?” Our framework targets the former. Moreover, whether the agent relies on innate priors or on newly acquired knowledge of the game, its “thinking paradigm” remains the same.

Weakness 2 Response:

Thanks for the suggestions. Adding comparison using syntactic recursion for CoT parsing or using LLM-as-judge are promising ideas, and we will definitely consider them in our ongoing and future studies. Nonetheless, we want to point out that we opted for experienced human annotators in the current study for greater interpretability and established validity, and because there is currently no robust evidence that LLMs can accurately or validly assess the reasoning depth of other LLMs. Employing LLMs as judges would require substantial new validation and would shift the focus of our work away from behavioral evaluation and interpretability. In current work, using LLMs to judge other LLMs is challenging also because it could introduce circular logic, as both may share training data, inductive biases, or flaws. We agree that using additional analysis is value-added, but it can only be considered as parallel analysis.

CoT is a major focus of our work. In this work, we systematically collected diverse reasoning traces from each model and anonymized them for “blind review.” Our researchers independently classified the reasoning styles using a structured coding rubric, with cross-validation to ensure reliability. The procedure directly links behavioral patterns (e.g., “self-interested” or “distrustful” models struggling to cooperate) to CoT explanations. Please check Section 4.2 for more details.

We are happy to include your proposed judges for comparison.

Weakness 3 Response:

There might be a misunderstanding here. The TQRE model used in our analysis is not a data-driven machine learning model, but a well-established structural model from behavioral economics and experimental game theory. TQRE does not “learn” its parameters from the data in a flexible way, but rather imposes a theoretically motivated structure to interpret observed choices, which is an approach widely used for both human and agent behavior. If you are referring to the maximum log likelihood of TQRE fits, for individual models and games, the average negative maximum log likelihood per game is typically in the range of 0–2, which is comparable to prior experimental studies (Camerer, Ho, & Chong, 2004).

We adapted TQRE as the foundation of our quantitative framework based on rigorous justification. It builds upon and extends earlier behavioral game theory models, such as the Cognitive Hierarchy model families and the Quantal Response Equilibrium models, by incorporating bounded rationality, noisy response, and empirically realistic distributions of reasoning depth. It is already the most comprehensive and realistic model compared to other CH-family and QRE models, as shown in comparative studies (e.g., Wright & Leyton-Brown, 2010, AAAI), to provide the best fit to actual agent behavior.

Questions Response:

We address precisely this issue in Section 4.2 of our paper, where we analyze model behavior and internal reasoning processes across a diverse set of game types. Our results show that models do exhibit typical patterns in different types of games, and each model tends to have its own areas of strength and weakness given their reasoning styles, and these patterns are reflected in both their behavioral choices and internal chain-of-thought reasoning.

It is important to recognize that the “task complexity” you refer to is not an absolute measure. For instance, a cooperative game might be straightforward for a model with a preference toward trust, but much more complex for a model that assumes opponents are untrustworthy, requiring more layers of strategic reasoning and anticipation of adversarial play.

The TQRE framework is specifically designed to accommodate this heterogeneity. Rather than assuming a fixed hierarchy of task difficulty, TQRE estimates τ based on how an agent actually navigates the strategic landscape of each game, given its own reasoning process, beliefs, and biases. This means that an increase in τ for a particular agent in a given game reflects that agent’s need for deeper recursion, and not an externally imposed notion of complexity.

Our findings confirm that models display distinct patterns across game types rather than a universal progression from “simple” to “complex.” This agent-relative perspective is fundamental in behavioral game theory and essential for interpreting τ as a measure of context-sensitive strategic depth.

Reference:

Wright, J., & Leyton-Brown, K. (2010, July). Beyond equilibrium: Predicting human behavior in normal-form games. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 24, No. 1, pp. 901–907).

Camerer, C. F., Ho, T.-H., & Chong, J.-K. (2004). A cognitive hierarchy model of games. Quarterly Journal of Economics, 119(3), 861–898.

Costa-Gomes, M. A., Crawford, V. P., & Broseta, B. (2001). Cognition and behavior in normal-form games: An experimental study. Econometrica, 69(5), 1193–1235.

Dear Reviewer,

Thanks for your engagement during travel, and for your careful follow-up. We are happy to provide further clarification on the confusion:

Chance-level baseline: In each scenario, we compute log-likelihoods over the joint action profile for simultaneous games, because in each trial we recorded both the row‐player’s and column‐player’s binary choice. For example, for cooperation game, this means the chance-level log-likelihood is ln  0.25 = –1.386, not –0.693 (which would be for a single binary choice). We will clearly state this baseline in the revised tables. For further interpretation, values near the baseline are consistent with spontaneous or low-level play (level-0 reasoners). We agree that interpreting in conjunction with model parameters can give a fuller picture of agent behavior. We will include a detailed appendix table showing these action counts, chance baselines, and model fits for each scenario, along with explanatory note. We appreciate your feedback and agree this will make the results more transparent and easier to interpret.

We also appreciate the suggestion on complementary model comparisons. We will add this to our project pipeline and look forward to exploring these extensions.

We hope these explanation resolves any confusion, and thank you again for your feedbacks, we truly appreciate your engagement and suggestions!

Thank you for the review. Below are our responses to your questions and comments.

Question 1 Response:

Our experiments suggest that dominant reasoning styles are often intrinsic and stable properties of the LLMs we tested. As you mentioned in the question, GPT-o1’s strong maximin preference remains the dominant pattern across most of the 13 games in our library, regardless of specific game context. This intrinsic tendency also explains why GPT-o1 tends to underperform in cooperative or mixed-motive games; it still chooses the secure but collectively inferior outcome.

Our goal is to benchmark the models’ native strategic reasoning. Supplying detailed guidance would blur that line, turning the task into “follow‑the‑instructions” rather than revealing the model’s default reasoning style. We therefore keep prompts minimal and identical across models.

Question 2 Response:

Thanks for bringing up this question. We acknowledge this as an important and interesting problem. The proprietary training corpora and RLHF pipelines of most models are opaque, so we cannot trace a single, provable cause directly.  Any updates to the weights of the model can and will introduce bias to the model’s behavior, and it is quite difficult to pinpoint where the bias is introduced. What we can say, based on our additional “internal‑thinking” audit, is the following:

If we ask the model directly, “Will you treat personas differently?” it would most likely deny the answer. However, the observation would still show the shifts in responses. This suggests that this bias is internal and implicit. 

The most plausible hypothesis is that large-scale text corpora encode stereotypical associations between demographic and behavioral cues. These priors affect downstream decisions even though the explicit request to the model or its rationale is “to be fair”. For example, some cultures have an emphasis on unity and moderation, so the model trained heavily using data in those regions might benefit more from a cooperation game, such as Deepseek. RLHF can either dampen or amplify these latent patterns by bringing researchers’ personal “preferences”. Without access to the non-public data, it’s hard to disentangle these factors.

The aim of this work is therefore not to locate the ultimate cause of the bias, but to demonstrate that the framework is sensitive and robust enough to reveal such latent biases, even when these trends are absent from the models’ verbal justification.

Question 3 Response:

In our analysis, we primarily focus on τ as it directly captures strategic reasoning depth, which is central to our study. γ, on the other hand, is included mainly as a control variable to account for variability and contextual confounders in decision-making. We will add further clarification to the final draft.

Regarding the correlation or trade-off between τ and γ, we did not observe any strong or consistent patterns across models or games in our results, and our data does not support a simple relationship or trade-off between these parameters. Since γ did not reveal consistent or generalizable patterns across models, we chose to focus our main discussion on τ in order to maintain clarity and prioritize our primary research objectives within the available space. We thank the reviewer for highlighting this interesting question, and we will consider a deeper analysis of more complex relationships between these two parameters in future work.

Question 4 Response:

The reasoning chains were obtained directly from the model outputs using the API feature that returns the model’s internal step-by-step reasoning for each decision, rather than asking for an explanation after the fact. This process is analogous to revealing the model’s “thought process” as it happens, so it’s not a post-hoc response. Thank you for raising that up, and we will add a note to the paper to make this point clearer.

Weakness 1 Response:

Our quantitative framework is grounded in TQRE because, in comparative studies such as Wright & Leyton-Brown (2010, AAAI), it has been shown to fit human data better than alternative models—including Cognitive Hierarchy and standard Quantal Response Equilibrium.

TQRE is closely related to the CH family of models, which can include a variety of belief formation rules. Our use of the Poisson distribution in TQRE follows both theoretical precedent and empirical findings that most agents reason at lower levels, with fewer at higher depths (Camerer, Ho, & Chong, 2004; Costa-Gomes, Crawford, & Broseta, 2001; Stahl & Wilson, 1994, 1995).

We will discuss how future work could incorporate and compare alternative models into the final draft. Thank you for the suggestion.

Weakness 2 Response:

Our methodological choice is grounded in the perspective that dynamic, multi-stage interactions can be decomposed into rounds of one-shot decisions, with each round representing a cross-sectional snapshot of agent reasoning given the available information at that stage.

For example, consider a multi-round coordination game:

In the first round, agents have no history or prior information about each other, so they make decisions based solely on the game’s structure and any available cues, just the same as in the one-shot coordination game. In the second round and beyond, agents update their beliefs based on observed past actions. These historical interactions then become part of the information available for subsequent decisions. Thus, each new round can still mirror as a one-shot decision, but conditioned on a richer informational context.

Reference:

Wright, J., & Leyton-Brown, K. (2010, July). Beyond equilibrium: Predicting human behavior in normal-form games. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 24, No. 1, pp. 901–907).

Camerer, C. F., Ho, T.-H., & Chong, J.-K. (2004). A cognitive hierarchy model of games. Quarterly Journal of Economics, 119(3), 861–898.

Costa-Gomes, M. A., Crawford, V. P., & Broseta, B. (2001). Cognition and behavior in normal-form games: An experimental study. Econometrica, 69(5), 1193–1235.

Stahl, D. O., & Wilson, P. W. (1994). Experimental evidence on players' models of other players. Journal of Economic Behavior & Organization, 25(3), 309–327.

Stahl, D. O., & Wilson, P. W. (1995). On players' models of other players: Theory and experimental evidence. Games and Economic Behavior, 10(1), 218–254.

We are pleased to have addressed your earlier questions. As you mentioned a willingness to raise your score if concerns were addressed, we hope our responses provide the needed clarity:

1. Regarding role of gamma

Thank you for your concern. Our analytical approach is well-grounded in established behavioral modeling practice, where γ is routinely treated as a control parameter. This is because γ does not capture a singular, interpretable behavioral trait; rather, it absorbs a wide array of influences, including individual quirks, random variability, and context-driven noise. It tries to encapsulate and isolate the environmental confounding factors that may affect evaluating the τ (reasoning depth). Its main function is to ensure that estimates of τ are not conflated with such noise, thereby allowing τ to provide a clear and interpretable signal of cognitive sophistication.

Given our primary research objective to quantify and compare the reasoning depth of LLMs, it is methodologically appropriate to focus on τ as the most direct and meaningful index of strategic sophistication. Including γ as a control allows us to capture robust and unbiased estimation of τ, while avoiding misinterpretation of behavioral inconsistency as shallow reasoning.

For completeness and transparency, we provide a summary table of γ across models and tasks here, which will also be added to the appendix, which further clarifies its role in the revised manuscript. We could have spent more paragraphs discussing the impacts of γ with these data. Still, we believe the current approach allows our work to provide a focused and rigorous contribution, telling one clear story about reasoning depth, without introducing other ambiguity.

2. Thank you for raising this question. We hope to fully clarify our modeling rationale:

First, our qualitative analysis in Section 4.2 demonstrates that LLMs frequently display step-by-step, multi-level reasoning in their chain-of-thought outputs. This evidence directly supports the relevance and interpretability of behavioral models like TQRE, for characterizing LLM decision-making, not just human reasoning.

Second, our choice of TQRE is under rigorous justification. In behavioral game theory models, Nash Equilibrium, QRE, CH, and TQRE form a clear theoretical lineage, not a set of parallel alternatives. TQRE combines the advances of QRE and CH, and has consistently shown superior empirical fit across human data and LLM data. Previous works have demonstrated the justification of applying and evaluating earlier models like NE to LLMs, as we have pointed out that these earlier models are all special or limiting cases within the broader TQRE framework, making the TQRE framework a comprehensive and robust choice for unified evaluation.

Third, we do not assume LLMs are identical to humans in cognition. Instead, we use TQRE as a principled, interpretable benchmark, enabling meaningful comparison and insight into agentic reasoning across domains. Our empirical findings further support this choice: stepwise reasoning is observed, and the model structure captures the main behavioral regularities in LLM play.

We believe these points demonstrate that our framework is both conceptually grounded and empirically validated for the LLM setting. We are happy to add a clarifying discussion in the final version and are confident that this work provides a strong, unified foundation for future comparative studies. We thank the reviewer for their careful consideration and hope these clarifications address any remaining reservations.

We hope these clarifications meet your expectations and look forward to your positive reassessment.

Thank you for the review. Below are our responses to your questions and comments.

Question 1 and Weakness 1 (about the link between quantitative reasoning depth and qualitative reasoning styles) Response:

This is a very insightful question. Our submission in this “social and economic aspects of machine learning” track has adopted a combined method that is typical in social science research:

Quantitative layer: A data-driven method, let the data tell the story from the observed behaviors, therefore effectively capturing the underlying mechanism and patterns of the agent. Here, we use TQRE as a rigorous, behavioral metric to quantify how deeply each LLM appears to reason in strategic settings. 

Qualitative layer: To complement the quantitative analysis, we mirror the “verbal protocol analysis”, which is a well-established qualitative method in the behavioral sciences. Specifically, each CoT trace generated by the model is treated as an “interview transcript,” and we conduct thematic coding using a pre‑defined rubric, which is a qualitative analysis method for systematically labeling and organizing recurring ideas or patterns in qualitative data, such as text or transcripts (Braun & Clarke, 2006), to identify reasoning styles.

This qualitative layer is essential because, just as with human agents, numbers alone rarely reveal the full narrative or structure of reasoning, and a contextual “story” is needed to interpret the meaning of quantitative signals. The linkage, therefore, is methodological rather than algebraic: the quantitative TQRE analysis tells us how far the agent seems to think; the qualitative “interview” reveals how it uses that depth.

Employing both layers is standard practice when behavioural‑decision research aims to connect statistical regularities with interpretive understanding, and it is precisely this synthesis that our framework contributes to LLM evaluation. We will revise the text to make it clearer how this synthetic approach works.

Question 2 and Weakness 2 (about the method to avoid subjectivity and keep reliability) Response:

Yes, thank you for asking. We took the following steps to ensure the robustness and avoid subjectivity: We collected a diverse set of reasoning traces from different models and anonymized them before analysis. The researchers who coded each CoT trace for reasoning style did so without knowing which model produced it, analogous to a “blind review” process. This helps to minimize potential post-hoc bias. We then used a pre-defined structured coding rubric to classify reasoning styles.

Then we did the cross-validation to mitigate subjectivity. Multiple researchers independently coded overlapping subsets of the data, and we compared their classifications to check for consistency.

Question 3 Response:

Our conclusion is not necessarily that “shorter chains are always better”, but that the common assumption “longer CoT → better strategic reasoning” does not always hold true. Therefore, the main takeaway is that the researchers should measure, but not assume the effect of CoT length.

Question 4 Response:

In our analysis, we focused primarily on τ because it is the key metric for strategic sophistication in behavioral game theory, as it directly reflects the depth of recursive reasoning and belief modeling. The γ parameter (decision noise), on the other hand, primarily serves as a control variable. It accounts for contextual confounders within the experimental setup and helps ensure the robustness of our primary measure τ. γ did not provide a consistent, model-wide insight comparable to τ, and due to page limit, we didn’t focus on discussing it and left more space to make the major objective clearer. Nevertheless, we acknowledge the reviewer’s suggestion and will include a detailed summary of γ values in the appendix of the revised paper. For reference, we have provided a table of γ estimates here.

Weakness 3 Response:

In illustrating how our unified framework combines qualitative and quantitative methods, we focused our deeper analysis mainly on top-performing models for several reasons: (1) these models consistently provide explicit internal reasoning steps (Chain-of-Thought traces), whereas many smaller or less capable models do not provide reasoning steps as an API feature; and (2) as top performers, their behavior offers stronger and more discernible features, making trends more interpretable. For example, while many intermediate models perform moderately across games, the three models we selected demonstrate clear strengths and weaknesses (e.g., DeepSeek R1 excels in cooperative games but underperforms elsewhere), making it more intuitive to link reasoning patterns to their successes and failures. However, studies and results about lower-performing models are also included in the paper in section 4.3. We will elaborate more about those parts in the final version.

We do not claim that the observed effects of CoT prompting are universal across all LLMs; indeed, each model exhibits its own reasoning style and behavioral pattern. Rather, our goal is to demonstrate that the unified framework can reveal these trends within individual models, particularly where reasoning traces are available and informative. We will clarify this point and its limitations in the revised manuscript as well.

Reference:

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101.

Thank you for the review. Below are our responses to your questions and comments.

Weakness 1 Response:

The paper presents a two-stage study, but not just a combination of two separate experiments. The “social effect” study is not an independent effort, but a direct extension and application of our evaluation framework to real-world, human-centered challenges. It should be viewed as the second stage of the same framework:

Stage 1: We use TQRE to estimate each model’s intrinsic reasoning depth, establishing a behavioral baseline.
Stage 2: We enhance the estimation through introducing demographic personas or human factors, allowing us to isolate and diagnose how social cues shift model reasoning and expose demographic biases

This staged approach demonstrates that an effective evaluation framework must move beyond quantitative performance alone to explicitly address the social and ethical dimensions of agentic reasoning. We can further clarify this linkage more accessible in a revision, making explicit how the social study is motivated by and only made possible through the unified framework introduced earlier.

Weakness 2 Response:

We adapt TQRE from behavioral economics and show that our framework completely subsumes NE and CH baselines. We emphasize that methodological innovation at this level of breadth and rigor constitutes a major scientific contribution. To our knowledge, no prior work has introduced a comprehensive methodology and extensive quantitative evaluation of LLM reasoning through the behavioral decision-theory lens. Our framework addresses major shortcomings of previous work, such as restrictive or unrealistic assumptions, by offering a rigorous, behavioral approach that directly quantifies reasoning depth and social effects. The issue of improving reasoning depth and alignment across diverse scenarios is a broad, ongoing challenge faced by the entire field, including leading research labs and companies. Our work provides, for the first time, a rigorous and quantitative behavioral lens to measure and compare reasoning depth, laying the necessary foundation for any principled model improvements in the future.

By exposing and quantifying these limitations in a systematic and interpretable way, our framework enables future work on targeted interventions (e.g., model fine-tuning, training data augmentation, loss function design). We explicitly outline these next steps in our discussion section. Thus, we view our methodological innovation and actionable diagnostics as a crucial advance, enabling and accelerating broader progress on the open problems highlighted by the reviewer. Finding a single solution that aligns all models with one particular reasoning process or style remains a significant challenge. Aligning with one direction sometimes means misaligning with other directions; our framework’s value shows up at where the alignment or misalignment is, but aligning the model with a corresponding target is out of the scope of this work. But of course, we are looking forward to designing methodologies to achieve the ultimate alignment in the future based on the foundation established by this work.

Weakness 3 Response:

Our work goes beyond a dataset or static benchmark by providing new metrics and theoretical synthesis: it introduces a theoretically grounded, behavioral game-theoretic approach for analyzing LLMs as reasoning agents, including both the modeling methodology and actionable diagnostic tools for both developers and theorists. Moreover, we do not simply report scores, but interpret mechanisms of reasoning, model variation, and real-world alignment/fairness risks. This framework serves as a practical diagnostic tool that enables and facilitates researchers to iterate on LLM studies. These aligned with the NeurIPS main track’s goals of advancing both scientific understanding and practical impact in AI.

Specifically, we submitted to the main NeurIPS track under the subsection “Social and economic aspects of machine learning (e.g., fairness, interpretability, human-AI interaction, privacy, safety, strategic behavior)”. Our work provides a methodological and empirical contribution at the intersection of strategic reasoning, human-AI interaction, and fairness in LLMs, all of which are central themes listed in this track. For these reasons, we believe our paper is a perfect fit for this subsection of the main track, rather than the datasets & benchmarks track.

Question 1 Response:

Yes, these are promising future directions. Our findings directly inform future directions in model improvement and lay the foundations for that.

For example, the behavioral metrics produced by our framework allow for targeted fine-tuning to improve reasoning depth in specific game types, as well as the identification and mitigation of social biases via curated data augmentation. We are actively exploring these possibilities and believe our framework can serve as a foundation for iterative cycles of evaluation and model alignment. We see this as an important next step.

Question 2 Response:

This has been answered above in the weakness 3 response.