We thank the reviewer for the thorough feedback; we will use it to improve the clarity of the submission.

Direct Questions

Re: WQMIX

See our response to Reviewer 5gNj. We will clarify this comment in the paper.

Other Comments

Re: QPLEX statement

The corresponding statement for QPLEX is Prop. 2 of [1].

Re: stateful joint values

All common implementations employ stateful joint values; this includes: the QMIX code [3], the QPLEX code [4], and the popular pymarl2 code [5]. The QMIX and QPLEX papers also repeatedly refer to joint history-state values (although their notation is also often inconsistent). We will add these more explicit references to the paper.

We also note “using models of both state and history makes little sense” is a common misconception. The literature of RL [6,7], MARL [8,9], and value-decomposition [2] has shown in both theory and practice that (mis)using models of state (without history) can be highly problematic, and that models of history and state can be more appropriate. Seminal work in MARL is slowly adapting to this misconception, e.g., see Errata in [10].

Re: Definition 4.1

This is indeed a discrepancy between def 3.1 and defs 3.3, 4.1. However, the root cause is that def 3.1 prescribes a unique maximum, while defs 3.3, 4.1 don’t. If defs 3.3, 4.1 were combined with unique maxima, then all definitions would be equivalent. That said, the generalization to multiple maxima is useful, and we agree that defs 3.3, 4.1 should be clarified as requested. We will clarify this point in the paper, but we also note that this has no effect on the rest of the work, as QPLEX, QIGM and QFIX already enforce a double implication.

Re: Eq (19), Lemma 4.2, Thm 4.3

That lemma 4.2 and thm 4.3 follow so naturally from eq (19) is a strength and sign of simplicity; not a weakness. Note: the given example is a special case of our proof for one specific choice of $f$, whereas our thms are valid for any choice of $f$. The more general proof is necessary to derive QFIX by replacing $f$ with any IGM fixee.

Re: Minimalism of QIGM

Our claims of minimalism are informed both quantitatively (by model sizes, Table 1), and qualitatively (eqs (22,25) are simpler than eq (16)). Note: we never claim that QIGM represents a minimal (small) function class; only that it is a minimal (simple) formulation of the IGM class.

Re: proof of Thm 4.3

We understand the confusion. The text states "For any $f$ that satisfies the requirements of eq (19)" We mean the requirements associated with eq (19), not eq (19) itself, i.e., $f$ non-positive and zero iff all inputs are zero. In eq (19), $f$ and $w$ are not codependent. In line 539, $w$ is constructed as a function of $f$ only to prove IGM-completeness; we are not claiming that this is a necessary structure of $w$.

Re: Sufficiently expressive models

This is an appeal to the Universal Approximation Thm, and consistent with other proofs in the literature [1]. We will clarify this point. See our response to Reviewer zepP for a detailed reply.

Re: Ablation

The proposed ablation is only feasible in one case:

• VDN does not use a parameterized mixing model; the requested ablation is impossible for Q+FIX-{sum,lin}.
• QPLEX is already IGM-complete and is never used as a fixee; the requested ablation is again impossible.
• QMIX is the one case where the proposed ablation is feasible; we are working to run this ablation by making Q+FIX-mono fixee smaller and/or QMIX larger. If we get preliminary results within the discussion period, we will post them.

However, note that the performances of all Q+FIX variants are comparable, yet Q+FIX-{sum,lin} employ the smallest mixers by far. This hints at our mixing structure as a core contributor of performance regardless of fixee size; we expect the additional ablation to also confirm this. Also see our response to Reviewer zepP for a related topic.

Rebuttal Summary

We believe we have addressed all concerns and, aside from minor clarifications, we reaffirm the correctness and rigorousness of our theory and results. We hope the reviewer will reconsider their evaluation positively and that they will let us know of any further concern.

1. Wang et al. "QPLEX: Duplex Dueling Multi-Agent Q-Learning" ICLR 2021
2. Marchesini et al. "On Stateful Value Factorization in Multi-Agent Reinforcement Learning" AAMAS 2025
3. github.com/oxwhirl/pymarl
4. github.com/wjh720/QPLEX
5. github.com/benellis3/pymarl2
6. Baisero et al. "Unbiased Asymmetric Reinforcement Learning under Partial Observability" AAMAS 2022
7. Baisero et al. "Asymmetric DQN for Partially Observable Reinforcement Learning" UAI 2022
8. Lyu et al. "A Deeper Understanding of State-Based Critics in Multi-Agent Reinforcement Learning" AAAI 2022
9. Lyu et al. "On centralized critics in multi-agent reinforcement learning" JAIR 2023
10. Foerster et al. "Counterfactual Multi-Agent Policy Gradients" arXiv v3 2024

We thank the reviewer for their feedback.

Direct Questions

Re: WQMIX

The WQMIX theory [1] explicitly assumes fully-observable control (MMDP), and makes assumptions that do not hold for Dec-POMDPs, e.g., that decentralized policies can achieve the same optimal behavior as centralized policies. We will clarify this in the paper.

Re: Similar Performance of Q+FIX variants

We believe the similarity in performance of Q+FIX variants hints at the importance of our mixing:

• When employing our mixing, it does not matter how "(un)sophisticated" the fixee is; We are able to elevate all fixees equitably. This demonstrates the effectiveness of our structure over model size, and suggests that smaller models may be preferable.
• When comparing the stability of Q+FIX to the instability of QPLEX, this implies that the complexity/size of QPLEX is a hindrance and that achieving IGM-completeness via simpler models more effective.

We also note that the models of $w, b$ are not different across Q+FIX variants; only the fixee is. Perhaps the reviewer is referring to the $w, b$ constructed in the proof of thm 4.3; however, that is a construction to prove IGM-completeness, and not a requirement.

Other Comments

Re: Simplicity

Our claims of simplicity are informed quantitatively (Table 1), and qualitatively (eqs (22,25) are simpler than eq (16)). The claims of "fixing" refer to the expansion of the representation class of VDN and QMIX, not performance. We will clarify both points in the paper.

Re: Statistical Significance

We share the reviewer's concern that modern RL suffers from issues of statistical significance due to the increasing complexity of evaluation. While at the time of submission we were limited to 3 seeds per model per task, we have continued to run more evaluations. We are now iterating the 5th seed, and the results are consistent, with higher significance.

We note that [3] does not prescribe a high number of seeds, but rather recognizes the practical limitations of evaluation ("3-10 runs are prevalent in deep RL as it is often computationally prohibitive to evaluate more runs"), and makes recommendations that are "easily applicable with 3-10 runs per task".

We already adhere to all applicable recommendations:

• confidence intervals over point estimates
• aggregate results over tasks to increase significance (though we use mean, not IQM)

When aggregating using IQM, the distinction between QPLEX and QFIX drops (though QFIX still remains in a slight lead); clearly IQM benefits QPLEX by ignoring its unstable runs. We will include the IQM results in the final paper as well, though we believe it's important to note that QPLEX remains less stable.

It's not clear how to apply other recommendations that require each run to be summarized by a single scalar score; e.g., using final or maximal performances are both unfair, respectively against or in favor of QPLEX. We are happy to take suggestions.

Re: Fig 3

We thank the reviewer for pointing out an issue in the presentation of fig 3. The intent was to aggregate results in accordance to [4], first normalizing returns per task separately (to avoid tasks with wider return ranges from dominating over others), then aggregating across tasks.

We have fixed the plot, and it is almost indistinguishable; this is reasonable, as the return ranges are "similar" (same magnitude).

Re: Function Class for MARL

We agree that whether value decomposition methods represent the best function class for coop MARL remains an open question. However, that question falls beyond the scope of our work: our goal is to improve upon value decomposition methods themselves.

Re: Relationship to QPLEX

Though QPLEX and QFIX are both IGM-complete, QPLEX mixing is not strictly a special case of QFIX mixing: it is not possible to take the QPLEX models $w_i, b_i, \lambda_i$ and construct equivalent QFIX models $w, b$ s.t. QFIX and QPLEX values are equal for all inputs; for equality, the QFIX models must also depend on $Q_i$. In that sense, QFIX is not just a reparameterization of QPLEX. We will clarify this distinction in the paper.

Re: Def 3.3

Defs 3.3, 4.1 are framed in accordance to Def 1 from [5]. However, we agree and will reframe both.

Rebuttal Summary

We thank the reviewer for their feedback. We believe to have addressed all the stated concerns, and hope they will revisit their evaluation positively.

1. Tabish et al. "Weighted QMIX: Expanding Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning" NeurIPS 2020
2. Marchesini et al. "On Stateful Value Factorization in Multi-Agent Reinforcement Learning" AAMAS 2025
3. Agarwal et al. "Deep Reinforcement Learning at the Edge of the Statistical Precipice" NeurIPS 2021
4. Papoudakis et al. "Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Tasks" NeurIPS 2021
5. Wang et al. "QPLEX: Duplex Dueling Multi-Agent Q-Learning" ICLR 2020.

We thank the reviewer for their kind words and positive feedback.

Direct Questions

Re: Reason for the Improvement and Model Size

We believe that the empirical results combined with the model sizes of Table 1 provide a compelling argument that the performance of Q+FIX is driven from its mixing mechanism over mixer size, and in almost all cases despite smaller mixer size. Note that Table 1 shows the sizes of the mixing networks alone; this includes the size of the fixee models (when applicable), and excludes the individual value models.

Moreover:

• VDN uses no mixer (hence no entry in Table 1). Any parameterized mixing method must by definition have more parameters; this is unavoidable.
• QPLEX employs by far the most parameters than any other method, including all Q+FIX variants.
• QMIX employs a mixing network of intermediate size (between VDN and QPLEX). Though Q+FIX-mono employs more parameters than QMIX, this is the only case of a parametric Q+FIX mixer being larger than any other parametric baseline mixer; again, Q+FIX-{sum,lin} employ significantly fewer parameters than QMIX.

Finally, we note that the performances of all Q+FIX variants are similar, which suggests that conditioned on the use of our mixing structure, the quality and size of fixee is not an important factor, and that it is specifically the QFIX mixing structure that drives performance. To practitioners, this suggests the use of the smallest models, i.e., Q+FIX-{sum,lin}.

Though we believe the above already provides sufficient evidence for the effectiveness of our mixing structure over model size, we will also run an ablation on model size as requested by reviewer Cnks, for the one case where the ablation is feasible, to further prove this point.

Other Comments

Re: Assumption of sufficient expressive power

The assumption of "sufficient expressive power" is an appeal to the Universal Approximation Theorem (UAT) [1, 2], and this methodology is consistent with other works in the literature, most relevant being Proposition 2 from [3]. Naturally, any concrete architecture may fall short of UAT in practice; this is true of any deep learning model. We will clarify this point better in the text.

Re: hyperparameterization: all baselines (VDN, QMIX, QPLEX) employ the parameterization provided by [4], from the corresponding codebase pymarl2 [5]. As such, the baseline parameterization has already been optimized by prior work. Our mixing models, on the other hand, have not, yet we have found good results even when using the baseline defaults, and without spending significant computational resources towards hyperparameter optimization of Q+FIX.

Rebuttal Summary

We believe we have addressed the reviewer's concerns, that there are no major concerns remaining, and hope the reviewer will revisit their evaluation accordingly. If the reviewer has further concerns, we will gladly use our allotted discussion time to receive further feedback and provide further information.

1. Cybenko "Approximation by superpositions of a sigmoidal function" MCSS 1989
2. Lu et al. "The Expressive Power of Neural Networks: A View from the Width" NeurIPS 2017
3. Wang et al. "QPLEX: Duplex Dueling Multi-Agent Q-Learning" ICLR 2020
4. Ellis et al. "SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learning" arXiv 2023
5. github.com/benellis3/pymarl2