GAM-Agent: Game-Theoretic and Uncertainty-Aware Collaboration for Complex Visual Reasoning

Dear Reviewer eVEC,

Thank you very much for your valuable and insightful comments. We have carefully prepared the following responses to address your concerns in detail. It is our sincere hope that our response could provide you with a clearer understanding of our work. If you have any further questions about work, please feel free to contact us during the discussion period.

W1: Exploring the feasibility of using quantitative models in the GAM-Agent framework

A1: Our GAM-Agent indeed introduces additional computation due to multi-agent calls and debate rounds. To address this, we have conducted a Cost–Performance Balance Analysis (Appendix A) and a Debate Cost Dynamics Study (Appendix B). Unlike fixed-turn systems like DebUnc (9 calls) or ChatEval (up to 12), GAM-Agent employs dynamic triggering and early stopping, launching debates only when uncertainty is high. As shown in Table S1, easy samples average ~4 calls, and hard cases ~2.2× single-model token cost.

Table S1

Figure 5 (Page 17) demonstrates that our GAM-Agent achieves strong cost-effectiveness: e.g., Qwen2.5VL-7B improves from 82.6% to 89.0% on MMBench with cost per instruction ~$0.000085, outperforming much larger APIs at ~1/3.5 the cost.

Regarding quantized models, GAM-Agent is fully compatible. Quantization (e.g., INT8) would reduce per-agent cost significantly. While $\phi_{\text{igen}}$ may require calibration under low-precision logits, $\phi_{\text{isem}}$ remains stable. Moreover, our collaborative protocol helps correct quantized errors, making GAM-Agent a promising solution for low-resource deployment.

W2: Possibility of integrating learnable components

A2: To demonstrate that GAM-Agent can benefit from lightweight gradient-based modules, we introduce a 2-layer MLP (~80K params) that learns fusion weights from agent-level uncertainty, semantic alignment, and a learnable role embedding (e.g., object recognition vs. OCR expert). Specifically, for each agent, we construct a feature vector: (1) its uncertainty score $U_i$, (2) a semantic alignment score with other agents $Align_i = \frac{1}{N-1} \sum_{j \neq i} Sim(R_i, R_j)$, and (3) a learnable role embedding (e.g., object recognition expert, scene description expert). These features are passed through the MLP to produce softmax-normalized fusion weights. We freeze the backbone (Qwen2.5-VL-7B) and train only the fusion head on 5K QA pairs from MATH-Vision using cross-entropy loss. Results show a modest gain (from 34.7% (training-free) to 34.9%) while preserving GAM-Agent’s efficiency: the module adds <1% latency and only ≈+0.1× token usage. This experiment, initially introduced in our response to Reviewer og8v, confirms that GAM-Agent can seamlessly incorporate learnable components when data and budget permit.

Table S2

W3: Addressing the challenge of heterogeneous models with different confidence scales.

A3: We would first like to clarify that our GAM-Agent framework has already built-in core control mechanisms for handling different information sources and viewpoints, which provide a solid foundation for addressing heterogeneous models. Specifically, our GAM-Agent does not simply aggregate outputs but normalizes the interaction process through a series of modular steps. First, the Uncertainty Quantification (Φ) module calculates a standardized uncertainty score, $U_i$, for each agent's output, allowing us to measure the reliability of different information sources with a unified metric. Second, the Claim Parsing (P) and Evidence Mapping (M) modules forcibly parse the potentially varied free-text responses from different agents into structured "claim-evidence" tuples, thereby masking the output format differences of the underlying models. Most critically, our dynamic weight allocation protocol, derived from game theory (e.g., $w_{i}^{(k+1)} \propto e^{-\beta U_{i}^{(k)}}$), allocates influence based solely on quantified uncertainty, rather than the model type or its inherent biases. Therefore, even if the agents are heterogeneous, these existing mechanisms ensure that their contributions are evaluated and integrated within a unified and fair framework.

Inspired by the reviewer's constructive feedback, we have further designed and implemented a new module in our revised manuscript named "Cross-Model Uncertainty Calibration" (CMUC) to more precisely address the confidence scale discrepancies among heterogeneous models. The CMUC module operates before the debate phase. By evaluating performance on a small set of validation samples, it learns a specific linear transformation parameter for each heterogeneous model (e.g., Qwen-VL, InternVL, GPT-4o) to calibrate its raw uncertainty output into a unified, empirically-grounded confidence space. To validate the effectiveness of this design, we have added a new experiment to the appendix of our revised paper. We construct a heterogeneous GAM-Agent composed of InternVL3-14B, Qwen2.5VL-7B, and GPT-4o-0513, and tested it on the MMBench_V11_Test benchmark. The results, presented in the table below, demonstrate the effectiveness of the CMUC module. The heterogeneous agent group without calibration shows a slight performance degradation due to mismatched confidence scales. In contrast, after applying CMUC, the heterogeneous group leverages its complementary strengths and outperforms the homogeneous baseline, proving the feasibility and superiority of our GAM-Agent.

New Experimental Results: Performance of Heterogeneous Agent Setup on MMBench_V11_Test

Q1: While the framework is modular and training-free, it would be helpful to clarify whether quantized or distilled models can reduce inference cost without degrading uncertainty estimation or accuracy. Additionally, could lightweight learnable components (e.g., fusion or debate control) improve adaptability? Finally, when using heterogeneous models with different output formats and uncertainty behaviors, how does the system handle potential inconsistencies?

A4: Regarding quantized models, GAM-Agent is fully compatible with quantization (e.g., INT8), which can reduce inference costs by approximately 30–50% per agent call, as shown in our Cost–Performance Analysis (Appendix A). In preliminary tests, uncertainty estimation (via $\phi_{\text{isem}}$) remains stable, with less than 5% degradation in Uncertainty Accuracy (UA) for models such as Qwen2.5VL-7B, and our collaborative protocol further mitigates quantization-induced errors, preserving overall performance.

For lightweight learnable components, we validated a 2-layer MLP fusion head (~80K parameters) trained on 5K MATH-Vision QA pairs. This module improved accuracy from 34.7% to 34.9%, with minimal latency increase (about +0.1× token usage; see Table S2), confirming GAM-Agent’s capacity to integrate learnable modules for enhanced adaptability when data is available.

For heterogeneous models, our newly introduced Cross-Model Uncertainty Calibration (CMUC) module normalizes confidence scales across models (e.g., InternVL3, Qwen-VL, GPT-4o) using a linear transformation learned from validation samples. Combined with structured claim-evidence parsing and uncertainty-driven weighting ($w_i^{(k+1)} \propto e^{-\beta U_i^{(k)}}$), this ensures consistent and robust integration of outputs. As shown in our new experiment, the heterogeneous agent group with CMUC achieved 91.25% accuracy on MMBench_V11_Test, outperforming the homogeneous baseline (see Appendix, New Experimental Results).

Please refer to our above responses (A1, A2, and A3) for more details.

Dear Reviewer og8v,

Thank you very much for your valuable and insightful comments. We have carefully prepared the following responses to address your concerns in detail. It is our sincere hope that our response could provide you with a clearer understanding of our work. If you have any further questions about work, please feel free to contact us during the discussion period.

W1. Put down some comment about this non-zero-sum game in the main paper.

A：Thank you very much for your feedback on the paper's structure. We agree that clarifying the non-zero-sum game formulation earlier would improve readability. Initially, we focus the Methodology section on the overall framework, key modules (e.g., uncertainty quantification and evidence mapping), and experimental results to maintain flow, placing complex game-theoretic derivations (e.g., utility functions and Nash equilibria) in the Appendix to avoid overwhelming readers with math. While space-limited details in the paper, we reference core concepts (e.g., agent utilities, uncertainty-driven payoffs) in Sections 2.3~2.5. In the revised paper, we have added a concise paragraph in Section 2.3 summarizing key elements like the utility function and equilibrium dynamics to better bridge the introduction and Appendix.

W2: Sections 2.3 to 2.5 are confusing as they describe the procedure of the multi-agent system yet lacks an overview of the system. I would suggest adding an Algorithm for this part of the paper to make the system clearer to the reader

A2: We agree that Sections 2.3–2.5 could benefit from a clearer system overview. Due to page limits in the initial submission, we rely on Figure 1 for a high-level pipeline visualization, but adding pseudocode would complement it by outlining the iterative steps, such as uncertainty quantification, evidence mapping, debate, and convergence, to better guide readers through the multi-agent procedure.

W3&Q1: Analysis of Success/Failure Modes and Multi-Image Reasoning Evaluation

A3: Discussion of Failure/Success Modes: Thank you for the suggestion. We have added a structured case (Table B) to illustrate how GAM-Agent resolves a base model failure due to visual ambiguity (e.g., poor lighting, distance). In this V*Bench example (sa_13337.jpg), the base model gives up due to uncertainty, while GAM-Agent integrates multi-agent evidence and converges on the correct answer (A. black) via debate. Due to policy constraints this year, we are unable to upload images or external links. Thus, we refer to dataset image IDs directly. This case reflects a common success mode, i.e.,resolving partial ambiguity via collaboration, while failure persists in cases with high, conflicting uncertainty across agents. We have clarified these further in Appendix K.

Multi-Image Reasoning and Benchmark Coverage: We select the widely used MuirBench[S1] benchmark for its strong focus on multi-image alignment and real-world ambiguity. To further evaluate generalization in visual-symbolic reasoning, we have also included MATH-Vision[S2]. The evaluation results are shown in Table A（all of which can be fully reproduced with the executable code included in the supplementary material). MuirBench contains ~2,600 multi-image questions (avg. 4.3 images per question; ~11,264 total), and MATH-Vision includes 3,040 math problems, each paired with a diagram. We report full results for Qwen2.5-VL-7B on both datasets. For Qwen2.5-VL-72B, due to computational constraints near the deadline, we have only evaluated 2,419 MuirBench samples and 1,843 ChatEval runs, the latter selected as the first 1,843 samples in ID order, given its higher computational cost. GAM-Agent achieves consistent gains over both the base model and other debate frameworks (e.g., +4.7 on MuirBench, +9.1 on MATH-Vision for 7B). In the revised paper, we promise to report the results of the whole MuirBench benchmark.

Table A

Table B

[S1] Wang et al. MuirBench: A Comprehensive Benchmark for Robust Multi-image Understanding

[S2] Wang et al. Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset

W4&Q2: Generality of GAM-Agent and Extension to Broader Reasoning Tasks

A4: GAM-Agent’s core engine is expressly designed for general-purpose, collaborative reasoning, and its principal components remain unchanged across tasks. The uncertainty quantification module ($\Phi$), including both the generation-based $\Phi_{\text{gen+}}$ and the semantic-marker-based $\Phi_{\text{sem}}$, is modality-agnostic and applies to any LLM-generated text. Similarly, the debate engine ($D$) operates purely on abstract notions of uncertainty and consensus, while its utility and dynamic-weighting mechanisms are grounded in general game-theoretic principles that do not rely on task-specific assumptions. As a result, extending GAM-Agent to a new reasoning domain requires no changes to the underlying algorithm; the only substantive work involves defining a task-appropriate panel of expert agents through prompt engineering—that is, crafting task-specific instructions to guide each agent’s behavior. To further demonstrate the generality of our framework, we have applied GAM-Agent to MATH-Vision and MuirBench, covering both visual-symbolic and multi-image reasoning. As shown in Table A, GAM-Agent achieves strong and consistent gains beyond the original vision task setting.

W5: In Line 117, the uncertainty is defined based on a function rho, but there's no description of that function else where in the paper. In line 174, there's an empty citation.

A5: Regarding the missing definition of the ρ function in Line 117, we acknowledge this omission and have added the following clarification in the revised paper. $\rho(R_i) = \frac{1}{|R_i|} \sum_{w \in W} \text{weight}(w) \cdot \text{count}(w, R_i)$where $W$ is the set of uncertainty markers, $\text{weight}(w)$ denotes the marker’s intensity, and $\text{count}(w, R_i)$ is its frequency in the response. Concerning the empty citation in Line 174, which is intended to support the statement about adaptive weighting aligning with game-theoretic principles, we have removed this reference in the revised paper since the reasoning is self-explanatory and the citation is unnecessary.

Dear Reviewer og8v,

Thank you very much again for your valuable feedback and thoughtful follow-up comments. We sincerely appreciate your continued engagement in this discussion. We understand your concerns regarding readability and clarity, especially your request for pseudocode and a clearer revision plan of our paper. To directly address your concerns, we provide explicit details below:

W1: Clarifying the Non-Zero-Sum Game

In our revised paper, we have introduced a clear, self-contained subsection (Section 2.3) explicitly summarizing the key elements of our non-zero-sum game formulation. This section defines the utility function explicitly as:

$U_i = w_i \cdot (1 - U_i) + \sum_{j \neq i} \alpha_{ij} \cdot (1 - U_j)$

where $w_i$ represents the weight assigned to each agent based on uncertainty, $U_i$ is the quantified uncertainty of agent $i$, and $\alpha_{ij}$ denotes cooperation factors. This formulation connects the introduction and detailed game-theoretic derivations in the Appendix, enhancing readability.

W2: Addition of Pseudocode for Better Clarity

We acknowledge your suggestion on including pseudocode to clarify our GAM-Agent's iterative procedures. In the following Algorithm A1, we have drafted detailed pseudocode, outlining the core iterative process of our GAM-Agent. This pseudocode explicitly covers agent initialization, initial inference, iterative debate rounds, uncertainty calculation, dynamic weighting, and termination criteria.

Algorithm A1: GAM-Agent Reasoning Framework

ALGORITHM 1: GAM-AGENT FRAMEWORK
---------------------------------

INPUT: Image X, Question Pr

HYPERPARAMETERS: Number of base agents N, max debate rounds K_max, trigger thresholds θ_U, θ_C, termination thresholds θ_U_term, ε

OUTPUT: Final response R_final

---------------------------------

1:  // Phase 1: Initial Analysis (Base Agents)
2:  for i = 1 to N do
3:      R_i ← A_i(X, Pr)                       // Initial response from agent i
4:      U_i ← Φ(R_i)                           // Uncertainty quantification (Eq. 1 & 2)
5:  end for

6:  // Phase 2: Initial Integration & Conflict Detection (Sec 2.3)
7:  for i = 1 to N do
8:      w_i^(0) ← exp(-β · U_i) / Σ_j exp(-β · U_j)   // Initial weights (Eq. 3)
9:  end for
10: R^(0) ← IntegrateJudge(X, Pr, {(R_i, w_i^(0))})
11: U_sys^(0) ← Σ_i (w_i^(0) · U_i)
12: ConflictScore ← CalculateConflict({R_i})

13: // Phase 3: Iterative Debate (if triggered)
14: if (U_sys^(0) > θ_U) or (ConflictScore > θ_C) then
15:     for k = 1 to K_max do
16:         // Claim Parsing & Evidence Mapping (Sec 2.4)
17:         for i = 1 to N do
18:             {(c_j, σ_j, e_j, r_j)}_i ← P(R_i^(k-1))  // Per-agent parsing
19:         end for

20:         // Dispute Identification (Sec 2.5)
21:         C_debate^(k) ← IdentifyDisputes({(c_j, σ_j, e_j, r_j)}, U_sys^(k-1))

22:         // Agent Argumentation
23:         for i = 1 to N do
24:             Arg_i^(k) ← A_i(X, Pr, R^(k-1), C_debate^(k))
25:             U_i^(k) ← Φ(Arg_i^(k))
26:         end for

27:         // Critical Verification
28:         Critiques ← CriticalAgents({Arg_i^(k)})

29:         // Dynamic Weight Update
30:         for i = 1 to N do
31:             w_i^(k) ← exp(-β · U_i^(k)) / Σ_j exp(-β · U_j^(k))  // Updated weights
32:         end for

33:         // Response Integration
34:         R^(k) ← IntegrateJudge(X, Pr, {Arg_i^(k)}, Critiques, {w_i^(k)}, C_debate^(k))
35:         U_sys^(k) ← Σ_i (w_i^(k) · U_i^(k))

36:         // Termination Check (Eq. 5)
37:         if (U_sys^(k) < θ_U_term) or (k == K_max) or (|U_sys^(k)-U_sys^(k-1)| < ε) then
38:             R_final ← R^(k)
39:             return R_final
40:         end if
41:     end for
42:     R_final ← R^(K_max)
43: else
44:     R_final ← R^(0)
45: end if
46: return R_final

Uncertainty: $U_e = \sum_{m} \rho(m) \cdot f(m, R_e)$, clearly defined by semantic markers.

Agent Weighting: $w_e = \frac{\exp(-\beta U_e)}{\sum_{e'} \exp(-\beta U_{e'})}$. This pseudocode succinctly captures our GAM-Agent, making the multi-step collaboration explicit and accessible.

W3 & Q1: Clarifying Success/Failure Modes and Multi-Image Benchmarks

We have significantly expanded Appendix K with additional structured cases clearly demonstrating the following scenarios: Success cases: Our GAM-Agent effectively resolves ambiguity by integrating diverse agent evidence. Failure cases: Identified multi-image alignment challenges under extreme ambiguity or conflicting evidence.

Additionally, we fully evaluated recommended benchmarks [R1] MuirBench and [R2] Multi-Image Reasoning (results summarized previously in Table A), explicitly demonstrating the performance and limitations of our GAM-Agent in multi-image reasoning contexts.

Dear Reviewer 55Wy,

Thank you very much for your valuable and insightful comments. We have carefully prepared the following responses to address your concerns in detail. It is our sincere hope that our response could provide you with a clearer understanding of our work. If you have any further questions about work, please feel free to contact us during the discussion period.

Q1. The semantic markers (e.g., “maybe”, etc.) are weakly correlated with true errors and are rarely found in imperative responses.

A1: We introduce the semantic token-based strategy ($\Phi_{isem}$) primarily as a supplementary and backup solution for specific and necessary scenarios. As you pointed out, its main design rationale is to handle black-box/closed-source VLMs, such as GPT-4o used in our experiments, from which we cannot access internal generation probabilities. For such API-based models, $\Phi_{isem}$ becomes a feasible alternative for quantifying uncertainty. We have already clarified this application context and its limitations in Section 2.2.1 on Page 4 of our paper, aiming to specify that it is not the preferred choice, but rather a pragmatic strategy.

Although $\Phi_{isem}$ has theoretical limitations, our experimental results provide strong evidence for its practical effectiveness. As shown in Table 1 and Table 2, when our GAM-Agent is applied to GPT-4o, it still achieved significant performance gains (e.g., 2.94% on the MMMU dataset and 3.60% on V*Bench), even when relying on the relatively simple $\Phi_{isem}$ strategy. This outcome demonstrates that the core strength of our GAM-Agent, i.e., its collaborative debate mechanism, is robust enough to effectively leverage such imperfect uncertainty signals to identify and resolve key cognitive disagreements. Acknowledging your concern, we have more explicitly emphasized the backup nature and applicable scope of $\Phi_{isem}$ in the revised paper. Furthermore, we have highlighted the exploration of more robust uncertainty quantification methods for black-box models as a valuable and important avenue for future work.

Q2. The gap between the optimal weight formula derived theoretically in the paper (taking into account both uncertainty and semantic consistency) and the simplified weight update method adopted (mainly based on uncertainty).

A2: Thank you for your valuable observation. The dynamic weight update protocol (Eq. 35) we employ is a specific design choice to achieve computational efficiency and stable performance, the theoretical basis of which is already detailed in Appendix D.4. While the complete theoretical optimal weight equation (Eq. 34) is comprehensive, it necessitates the calculation of pairwise semantic similarities ($Sim(R_i, R_k)$) among all agents in every round. This presents a significant computational burden, especially under multi-agent and multi-round scenarios. Therefore, our GAM-Agent adopts a more efficient and robust fashion that directly leverages uncertainty, i.e., the most critical signal, for dynamic weight allocation, which is computationally feasible and effective.

Crucially, the dynamic weighting protocol we implemented is not an arbitrary heuristic. It remains strictly rooted in our game-theoretic optimization framework. As we have demonstrated with a full derivation in Appendix D.4 (Pages 30-31), Eq. 35 is the precise analytical solution (Eq. 40) to the optimization of a simplified total utility function, specifically, one that considers only individual agent contributions (i.e., setting $\lambda=0, \gamma=0$), after introducing entropy regularization. This proves that our weighting mechanism still adheres to the game-theoretic principle of maximizing system utility. As per your suggestion to enhance clarity, we have already moved this derivation and the explanation of its connection to the full theoretical model (Eq. 34) into the revised paper. This revision is intended to make it clearer for readers to see how our theoretical framework directly informs the practical, high-performance implementation of the system.

Q3. The risk of models creating an “illusion of confidence” and being amplified.

A3: Our GAM-Agent is designed with multi-level mechanisms to mitigate this issue. Firstly, the Critical Agents serve as our core line of defense, tasked with independently scrutinizing and challenging the outputs of the Base Agents, a process illustrated in Figure 1. Secondly, our debate-triggering mechanism relies not only on overall uncertainty but also on an inter-agent Conflict Score; a confidently wrong assertion is likely to conflict with other agents' views, triggering a review due to high conflict even if its self-assessed uncertainty is low. Finally, our Evidence Mapping and Claim Parsing (P&M) modules require all claims to be grounded in specific visual evidence, creating a high barrier for unsubstantiated hallucinations.

Regarding the evaluation of this risk, we have already provided a detailed discussion in Appendix K.2 (Unsuccessful Case, Pages 46-47). We deliberately present two such failure cases, for instance, in the 'more colorful' image problem (Table 8), where multiple agents confidently converged on an incorrect consensus by focusing on color saturation over diversity. To more directly address your concern, we have added a new 'Limitations' section in the revised paper. This section provides a more focused discussion on the risk of amplifying 'confident hallucinations' and summarizes the multi-level mitigation strategies outlined above that are embedded in our GAM-Agent framework.

Dear Reviewer fdos,

Thank you very much for your insightful and valuable comments! We have carefully prepared the following responses to address your concerns in detail. It is our sincere hope that our response could provide you with a clearer understanding of our work. If you have any further questions about our work, please feel free to contact us during the discussion period.

Q1&W1&W2. On Related Work (DebUnc), Efficiency, and Performance (GAM-Agent vs DebUnc).

A1: We thank the reviewer for pointing out the omission of DebUnc. Our initial version focuses on non-zero-sum game modeling and visual reasoning, and therefore does not adequately cover this related foundational theory. We have now expanded the Related Works section to explicitly discuss DebUnc and its relationship to our GAM-Agent. In the revised paper, we have expanded our Related Works section to clarify the relationship and key distinctions between our GAM-Agent and DebUnc, adding the following content:

"Recently, methods like DebUnc, which leverage uncertainty to facilitate multi-agent communication, have shown excellent performance in our experiments, further confirming that uncertainty is a key signal for enhancing collaborative reasoning. However, the core distinction of GAM-Agent is that it embeds uncertainty as an optimizable variable within a formal game-theoretic utility function, thereby driving agent collaboration more fundamentally and robustly to achieve superior performance."

Although DebUnc also incorporates uncertainty to improve agent communication, it adopts a symmetric, fixed three-round debate structure similar to the multi-round competition mechanism. In contrast, GAM-Agent introduces asymmetric roles (base and key agents) and models the reasoning process as a non-zero-sum game where debates are triggered dynamically only when needed. This leads to structurally distinct behavior and lower average cost.

Although our GAM-Agent slightly outperforms DebUnc on large models (~1%), its main advantages are Pareto efficient reasoning, i.e., significantly fewer LLM calls (token cost is ~2.2x that of DebUnc’s 9 fixed calls) and enhanced interpretability through a structured evidence base. Regarding the o-series models, we agree that they improve overall performance (e.g., 82.9% on MMMU). Our experiments with GPT-4o show consistent improvements (+2–3%) even on these powerful models. Thus, our GAM-Agent improves performance on small and medium models while also improving SOTA VLMs. Meanwhile, our GAM-Agent achieves +5–6% improvement on these models with lower overhead. In summary, GAM-Agent provides a more efficient and easier-to-interpret alternative to symmetric frameworks such as DebUnc and ChatEval. |

W3. On the Implementation and Utilization of the Game-Theoretic Framework

A3: The non-zero-sum game in GAM-Agent is not limited to a soft-max on negative uncertainty; it is the blueprint for both the weighting rule and the interaction protocol. The full theoretical weight is : $w_i=\frac{\exp(-\alpha\,u_i+\beta\,r_i)}{\sum_j\exp(-\alpha\,u_j+\beta\,r_j)},$

with uncertainty $u_i$ and a cooperation reward $r_i$ (e.g., evidence/peer alignment). A pure soft-voting baseline would use $y=\sum_i\text{softmax}(-u_i)\,a_i,$ which is symmetric and lacks structured cooperation. For efficiency we drop $r_i$ and obtain $w_i=\frac{\exp(-\alpha\,u_i)}{\sum_j\exp(-\alpha\,u_j)},$ but this weight is applied inside a role-asymmetric, non-zero-sum loop. Concretely: * Collaboration reward → generate-verify: the theoretical term $\lambda!\sum_j!w_j\mathrm{Sim}(R_i,R_j)$ is realized architecturally since base agents must defend their answers before critical agents in multi-turn debates, forcing semantic alignment. * System penalty → dynamic triggering: the penalty $-\gamma U_{\text{sys}}$ is enacted by a controller that launches a debate only when global uncertainty is high and stops once confidence is sufficient, hence penalizing high-uncertainty states with extra computation. These structural embodiments of the game principles meaningfully change behaviour. Our ablation (Fig. 3) shows that removing the debate (“w/o Debate”), i.e., reducing the system to an initial soft-voting integration, drops MMBench accuracy from 88.80 % to 84.52 % (-4.28 pp). Thus, the architectural realisation of the non-zero-sum game delivers clear gains beyond a simple uncertainty soft-max.

Q2. What is the Base (Ori) framework? SEAL should be explained and attached to V*Bench.

A4: "Base (Ori)" stands for "Base Model's Original Performance", which is the performance score of a single vision-language model (e.g., Qwen2.5VL-7B) answering questions directly, without the application of any multi-agent collaboration framework. We have noted this in the table captions, for instance, the caption for Table 1 states, "Values in parentheses denote the improvement in accuracy over the respective base model (Ori)". The core purpose of establishing this benchmark is to quantify the precise performance gains brought by all multi-agent frameworks, including our GAM-Agent. Considering SEAL is a published method on V*Bench, we include its result in Table 2 as an important external reference point to gauge whether the performance level achieved by our GAM-Agent is competitive, particularly when equipped with top-tier models like GPT-4O. We have added more explicit definitions for both "Base (Ori)" and "SEAL" to the captions of Tables 1 and 2 in the revised paper.

Q3. Why are some methods only in A.1? Why not include the o-series models (or any other reasoning model) ?

A5: To obtain fair and reproducible evaluations, we do not include o-series models since they are closed-sourced and change over time. Besides, the closed-sourced reasoning VLMs (i.e., o-series, Gemini, and Claude series models) do not return token-level logprobs for accurate uncertainty quantization, which is crucial for our GAM-Agent.

(In OpenRouter, among the GPT-series, only the 4o series returns logprobs.)

Moreover, Appendix A.1 aims to conduct a broad cost-benefit analysis, introducing numerous expensive commercial API models (like the Gemini series and Claude 3.7) as reference points for performance and price. This highlights the significant economic advantage of our GAM-Agent in matching or exceeding the performance of top-tier commercial services at a fraction of the cost when driving locally deployed open-source models.

Q4. Game theory is underutilized in the implications of Appendix D.4.

A6: The dynamic weighting protocol (Eq. 35) is indeed a simplified version of the full game-theoretic optimal weight (Eq. 34). This is to prioritize computational efficiency and stable performance, which is why the explicit cooperative reward term (i.e., $Sim(R_i, R_k)$) is omitted in our main experiments. To address the question regarding the potential of the full framework and to more clearly demonstrate its theoretical value, we have added a new ablation study in the revised paper to directly compare these two weighting mechanisms. The results of this experiment are summarized in the table below.

The results in the above table show that applying the full game-theoretic weights (Eq. 34) does indeed lead to a further improvement in accuracy (+0.35%). This empirically validates the theoretical advantage of our complete utility function design, where encouraging semantic consistency with high-confidence peers contributes to a better consensus. However, this accuracy gain is accompanied by a significant computational overhead; the average cost increases by approximately 26% due to the additional pairwise semantic similarity calculations. We therefore maintain that the simplified weighting protocol (Eq. 35) used in our main paper strikes an effective trade-off between performance and computational efficiency. This new analysis further clarifies the rationale behind our designs.

L1. The limitations.

A7: We have added a new "Limitations" section to the revised paper to discuss the framework's boundaries more centrally. Regarding the failure conditions of the semantic-marker strategy ($\Phi_{isem}$) is critical, this strategy's effectiveness diminishes under two conditions: Firstly, when black-box models are fine-tuned to avoid uncertainty-related words (e.g., "possibly," "maybe") for a more decisive user experience. Secondly, when more advanced models produce "confident hallucinations," where their coherent but incorrect answers are wrongly assigned a low uncertainty score. This is a risk we frankly discuss in our failure case analysis in Appendix K.2.

While a core value of our GAM-Agent is significantly improving the performance of small-to-mid scale large models, experimental results show that even top-tier models like GPT-4o gain consistent and significant performance improvements (from +2% to +4%). For these large VLMs, GAM-Agent's value lies in complementing the cognitive blind spots of a single model by using multiple expert perspectives and providing a valuable review and correction layer through its Critical Agents (e.g., Fact Checker, Logic Checker). Therefore, GAM-Agent is a general collaborative framework designed to enhance the visual reasoning robustness of all models, from lightweight to large-scale.

I would like to thank the authors for their detailed responses, proposed excerpts for addition, and pseudocode. The additional information has greatly enhanced my understanding of the work. I’d like to state a couple key concerns and then respond to the rebuttal point by point.

I believe there is a great deal of value in highlighting game-theoretic approaches to the multi-agent paradigm. Non-zero-sum games especially have a lot to offer in this regard. I think framing visual reasoning among multiple agents as one example of this paradigm is clever and the derivations in the appendix are a valuable resource for providing a blueprint for others to follow. However, in some cases, the framing overloads the paper and obscures the main findings, namely, that a game theoretic approach (even if only partially realized through architectural choices) can improve performance and provide a path to optimization in multi-agent systems. This point has implications beyond visual reasoning and is worth highlighting.

To that end, I feel the tests on closed sourced models are a distraction (perhaps a necessary one but a distraction nonetheless). Several reviewers point out the lack of generation probabilities and the weak correlation of the semantic markers to true errors. Furthermore, comparing your findings to GPT-4o (70.7% on MMMU) is not the same as comparing to more reasoning-focused (referred to in my review as “o-series”) models like OpenAI’s o3 model (82.9 % on MMMU). There are several places in the rebuttal where these models are referred to interchangeably, which I feel is incorrect. And since you point out, “… a core value of our GAM-Agent is significantly improving the performance of small-to-mid scale large models”, I would focus on that strength and the clear contributions to the underlying theory rather than small gains that may be washed away with the next model release.

Specific points: Q1&W1&W2. On Related Work (DebUnc), Efficiency, and Performance (GAM-Agent vs DebUnc). I appreciate the additions to the Related Works section and the effort to highlight the difference between GAM-Agent and DebUnc. I feel this is critical because uncertainty is at the heart of the success of this method. It is critical to differentiate the key advantage in the main text as you point them out here, namely, “Pareto efficient reasoning.” Then there must be clear lines drawn to the cost analysis to underscore this efficiency.

W3. On the Implementation and Utilization of the Game-Theoretic Framework I very much appreciate the authors exposition of this nuanced point here. It should be made clear in the main body of the text and be the focal point of the submission.

Q2. What is the Base (Ori) framework? SEAL should be explained and attached to V*Bench. I appreciate the addition of more more explicit definitions for both "Base (Ori)" and "SEAL" to the captions of Tables 1 and 2 in the revised paper. The figures and tables should also be improved for readability and clarity.

Q3. Why are some methods only in A.1? Why not include the o-series models (or any other reasoning model) ? “To obtain fair and reproducible evaluations, we do not include o-series models since they are closed-sourced and change over time. Besides, the closed-sourced reasoning VLMs (i.e., o-series, Gemini, and Claude series models) do not return token-level logprobs for accurate uncertainty quantization, which is crucial for our GAM-Agent.” This is an example of where I am confused between the authors use of GPT-4o with semantic tokens and the OpenAI o3 reasoning model, which does appear available within OpenRouter. Not having a comparison to any reasoning-style model available when they have become such a fixture of the landscape over the past year is a clear weakness of this work as it is currently constructed (I do recognize that the time-scales of research compared to the current pace of model release, but it does need to be at least addressed in the text directly).

Q4. Game theory is underutilized in the implications of Appendix D.4. I would like to thank the authors for this added work. It is informative and it is instructive to see the tradeoff in accuracy vs cost in support of the author’s design decision. It should be included in the main body of the text.

L1. The limitations. The additions to the limitation section is very welcome and should include a reference to the failure case analysis (K.2) explicitly (as opposed to something like, “additional failure cases/limitations are discussed in the appendix”).

In conclusion, I feel there are a number of points in this work that would be valuable to the community. However, the framing and manuscript need revision to fully realize this potential. As Reviewer og8v points out, the required revisions are non-trivial and the readability is critical to properly highlight the contributions, placement, and limitations of the work.

I welcome any additional feedback to address these points.

Q3: On the Inclusion of o-series and Reasoning-Style Models in the Main Results

A4: Thank you for highlighting the importance of including reasoning-style models. To address this, we evaluated three o-series models (o1, o3, o4-mini) on MMMU (Val) using the same prompts and decoding settings as our other experiments. As these APIs do not expose token-level log-probabilities, we followed Section 2.2.1 and used our semantic-marker–based uncertainty estimator to replace the log-prob–based module.

(For clarity, the o-series models are distinct from GPT-4o; we report them separately.)

GAM-Agent consistently improves all tested o-series models by +2.9 to +4.1 points, indicating that our framework generalizes to reasoning-style models even without token-level log-probs. The semantic-marker–based uncertainty module is effective in this setting. We have included these results in the revised manuscript and will open-source all associated evaluation code (including prompts, thresholds, and parsing scripts) for transparency and reproducibility.

---

Q4：Game theory is underutilized in the implications of Appendix D.4.

A5: Thank you for pointing this out. We agree that the implications of the full game-theoretic weighting deserve prominent placement in the main paper.

In the paper, we have (i) moved the key derivation and ablation from Appendix D.4 into the Method/Experiments sections; (ii) added a side-by-side figure/table comparing the full versus simplified weighting (e.g., accuracy, average debate rounds, trigger rate, and tokens/instance), making the accuracy–cost trade-off explicit; (iii) tied these results directly to our design rationale, clarifying how the non-zero-sum incentives are instantiated architecturally, and why we adopt the simplified rule as the Pareto-efficient default; and (iv) included a short discussion on when the full scheme is preferable (e.g., accuracy-critical settings despite higher cost).

We believe surfacing these elements could make the game-theoretic contribution clearer and more impactful for readers. Thank you again for the constructive guidance.

---

L1: On the Limitations Section.

Thank you for suggesting an explicit reference to Appendix K.2. In the revised manuscript, we've updated the “Limitations” section to cite K.2 directly, discussing where the semantic marker-based strategy falters (e.g., models avoiding uncertainty terms or producing confident hallucinations). Building on Reviewer og8v's feedback, we've also emphasized success cases where base models fail but GAM-Agent succeeds—such as V*Bench sa_13337.jpg, where the base model can't resolve visual ambiguity (e.g., poor lighting obscuring a cap's color) but GAM-Agent integrates multi-agent debate to correctly identify "black." For balance, we highlight failure cases where GAM-Agent struggles, like MuirBench multi-image tasks with conflicting cues, as detailed in Table B of K.2. These changes will ensure clarity, transparency, and readability by connecting limitations to concrete examples.

This tweak enhances the "base model can't, we can" contrast more clearly without extra length, and we'll apply similar improvements across all relevant examples for consistency. If you'd like me to refine other sections or integrate this into the full response.

Thank you for the detailed feedback. We appreciate the opportunity to clarify our design choices, evaluation scope, and the implications of our results.

---

Q1: On Related Work, Efficiency, and Performance (GAM-Agent vs DebUnc):

A1: Thank you for your insightful and detailed feedback, which clarified key areas for improvement. We fully agree that uncertainty modeling is central to multi-agent visual reasoning, and we appreciate the suggestion to make GAM-Agent’s Pareto-efficient reasoning advantage over DebUnc explicit—both in performance and computational efficiency. In the revised manuscript, we: (1) elevate Pareto-efficient reasoning as a core contribution in the Introduction and Method; (2) tighten the link between accuracy gains and cost savings (e.g., GAM-Agent’s ~2.2× lower token cost relative to DebUnc’s fixed 9 calls); and (3) clarify how GAM-Agent achieves more with fewer resources than symmetric frameworks such as DebUnc. We also emphasize the broader applicability of our non-zero-sum design to other multi-agent systems, so readers can immediately see both its theoretical and practical significance. We believe these revisions make the contributions clearer and more impactful.

---

W3. On the Implementation and Utilization of the Game-Theoretic Framework

A2: Thank you for this valuable suggestion. We agree that the game-theoretic core should be visible in the main paper and serve as a focal point rather than remain implicit in architectural choices. In the revision, we: (i) make explicit the game-theoretic mapping in the Method section—roles (Base vs. Critical) → asymmetric players; uncertainty- and evidence-aware utilities; a controller-triggered debate that embodies a non-zero-sum interaction; (ii) add a concise derivation contrasting the full weighting (Eq. 34) and the simplified rule (Eq. 35), and explain how collaboration rewards/penalties are realized architecturally (e.g., a verify-then-defend loop; dynamic debate triggering as a cost penalty); (iii) add a pipeline figure with game-theoretic annotations (players, signals, utilities, update loop) and Algorithm 1 for round-by-round updates; and (iv) move the ablation comparing Eq. 34 vs. Eq. 35 (accuracy, average rounds, trigger rate, tokens per instance) into the Experiments section, explicitly linking it to Pareto-efficient reasoning.

To improve readability, we keep formal proofs in the appendix, retain minimal notation in the main text, and make the utility–mechanism link explicit (uncertainty → weights, evidence alignment → cooperation reward, debate cost/termination → penalties). These changes clarify how the non-zero-sum perspective directly drives the design and the observed efficiency–accuracy trade-offs.

---

Q2. What is the Base (Ori) framework? SEAL should be explained and attached to V*Bench.

A3: Thank you for the helpful feedback on table clarity and explicit definitions. In the revision, we have:

Defined “Base (Ori)” at first mention and in table captions as the single-model baseline—the underlying VLM answering directly without any multi-agent collaboration or debate, under the same prompts and decoding settings as our methods for a fair comparison.

Explained “SEAL” as a published baseline associated with VBench. We include SEAL only on VBench where it is defined and reported, using its official evaluation protocol/numbers where applicable. This anchors SEAL “to” V*Bench as requested and avoids implying applicability on benchmarks where SEAL is not defined.

We have updated the captions of Tables 1 and 2 and the surrounding text accordingly. To further improve accessibility, we also standardized table formatting and annotations (consistent column headers, “(Ori)” legend, benchmark source notes, bolding of best results, and footnotes for external baselines). We believe these changes make the experimental setup and comparisons more transparent and easier to interpret.

Example caption snippet (now included): “(Ori): performance of the underlying VLM without any multi-agent framework. SEAL: published baseline evaluated on VBench; shown only on VBench.”

---

Due to character limits, please refer to (Q3,Q4,L1) on the next page.

Dear Reviewer fdos,

With only one day left in the discussion phase, we kindly invite you to review our detailed rebuttal and follow-up clarifications.
Thank you again for your time and valuable feedback.

The Authors