Securing Equal Share: A Principled Approach for Learning Multiplayer Symmetric Games

Thanks a lot for reading our paper and for your insightful comments.

1.The so-called suitable solution concept for learning in multiplayer games is a minimax strategy, which is not new.

We would like to clarify that our proposed solution concept, $\min_{x} \max_{x_1} U_1(x_1, x^{\otimes n-1})$, is indeed new and fundamentally different from the conventional objectives in multiplayer game settings. This formulation is meaningful only through our unique perspective of achieving an equal share. Traditional analyses typically focus on the solution $\max_{x_1} \min_{x_2, \cdots, x_n} U_1(x_1, \cdots, x_n)$, which, as the reviewer notes, is not novel. However, as shown in Proposition 4.1, this conventional approach does not yield the correct solution concept required for our setting (i.e., achieving equal share). Our results in Propositions 4.1 and 4.2 demonstrate that out of the four minimax/maximin formulations we examined, only our proposed solution, $\min_{x} \max_{x_1} U_1(x_1, x^{\otimes n-1})$, successfully achieves an equal share in multiplayer symmetric games. The other three formulations can yield values strictly less than zero under specific game conditions, further highlighting the fundamental differences between two-player zero-sum games and multiplayer symmetric games.

2.The large games mentioned in this paper are not symmetric. Why all asymmetric games can be converted to symmetric games?

We appreciate the reviewer’s observations and agree that games like Mahjong, Poker, Avalon, and Diplomacy are not strictly symmetric because the positions or roles assigned to players may influence their strategic options. However, as noted in footnote 2, we can effectively transform these games into symmetric ones by introducing an initial stage where each player’s position is randomly assigned. This randomization process effectively symmetrizes the game, as each player has an equal probability of assuming any role, ensuring that no player has a consistent positional advantage. Therefore, under this transformed setting, our analysis remains applicable.

To further clarify the mathematical transformation, we enlarge the action space to incorporate both the player's position and their actions. Specifically, for each player $i$, we define $(P_i, a_i)$ where $P_i$ represents the player's position or role in the game, and $a_i$ denotes their action. We then define a symmetric utility function $U^{sym}_i$ as follows:

$U^{sym}_i((P_1, a_1), \ldots, (P_n, a_n))$

$:=U^{asym}\_{\sigma^{-1}(i)}(a_{\sigma^{-1}(1)}, \ldots, a_{\sigma^{-1}(n)}),$

where $\sigma$ is a permutation that reorders player positions such that $\sigma(P_1, \ldots, P_n) = (1, \ldots, n)$. The action space is defined as $\bigcup_{j=1}^n (P_j \times \mathcal{A}_j)$, where $\mathcal{A}_j$ denotes the set of actions available to players when they are in position or role $P_j$. This transformation aligns the players' positions with a standardized symmetric framework, enabling our analysis to apply to games that are asymmetric by nature but can be symmetrized under random initial positioning.

3.This paper shows that the equal share can be secured under two conditions, making a multiplayer game equivalent to a two-player zero-sum game with the opponent fixed. Two-player zero-sum games have been studied well.

We agree with the reviewer that a multiplayer symmetric game can be viewed as equivalent to a two-player zero-sum game under two conditions. However, traditional results from two-player zero-sum games no longer apply directly, as the unique structure of this setting—all opponents sharing the same policy—violates the convexity assumption of the minimax theorem. In Proposition 4.2, we provide a counterexample that contradicts these traditional results, demonstrating that $\max_{x_1} \min_{x} U_1$ can be strictly less than zero, rather than equal to zero, as conventional theory would suggest.

Additionally, we extend our analysis to the adaptive opponent setting in Section 5.2, providing further theoretical insights that strengthen our contributions. Please also refer to our general response for the discussion on the technical contribution.

4.These results may not help solve general multiplayer games.

The challenge of solving general multiplayer games is widely recognized as a difficult problem, and addressing it fully is beyond the scope of this work. However, it would be unfair to overlook our contributions simply because they do not solve this extremely complex problem in its entirety. Progress in any field is made incrementally, with major advances often built upon understanding and solving more tractable subproblems. We see our work as an important initial step in establishing principled methodologies and algorithms that can guide future research in multiplayer games.

For further details regarding the novelty and technical contributions of this paper, please refer to our general response.

Thanks a lot for reading our paper and for your insightful comments.

1.Novelty of the results and the technical contribution. Please refer to our general response.

2.The Success of Self-Play Frameworks.

The primary focus of this paper is to explore how to approach multiplayer games in a principled manner. We use a bottom-up approach, which identifies a suitable objective that extends beyond equilibria---namely, achieving an equal share of rewards---and develop corresponding solution concepts and efficient algorithms within a comprehensive framework.

While the achievements of self-play algorithms in games like multi-player poker are undeniably impressive, it is important to note that these successes are often closely tied to specific contexts and rely heavily on extensive engineering optimizations, often with significant input from human expertise. We agree that examining why these algorithms succeed and whether they can serve as general-purpose solutions across a wide range of game settings is a very compelling future question, which is beyond the scope of this current paper.

Thanks a lot for reading our paper and for your insightful comments.

1.Technical depth and significance of the results. Please refer to our general response.

2.Self-play and clarification of the experiment.

We would like to provide further clarification of the experiment. Our experiment aims to show some worst cases (adversarially constructed) that meta-algorithms (SP_scratch, SP_BC, SP_BC_reg) from prior state-of-the-art systems for multiplayer games, which are shown to converge to NEs in the proposed cases, fail to secure an equal share. Aligned with our theoretical analysis, our proposed games assume that all opponents share the same fixed strategy over all rounds without adaptation. This is arguably the most fundamental scenario in which a principled algorithm should work well, given that in reality human opponents can be adaptive or even adversarial.

For self-play-based algorithms, we first run them to convergence and then evaluate the resulting policy against the fixed opponents' strategy. While SP_scratch cannot access opponents' strategy, we also add SP_BC and SP_BC_reg as baselines for fair comparison, which use the opponents' strategy as initialization or regularization. These design choices originate from modern variants of self-play that achieve state-of-the-art performance on multi-player games. In Section 3 we also highlight that self-play without opponents' strategy can fail easily in simple games such as majority vote.

Thank you for your further thoughtful comments.

1. We compare our methods to self-play algorithms because our goal is to develop an intelligent agent capable of excelling against other players, particularly humans, in multiplayer games. This high-level objective aligns with prior research on games such as Mahjong, Poker, and Diplomacy, where self-play has been established as the state-of-the-art approach.

2. In these tasks, understanding opponents' or human meta-strategies (e.g., average human strategies in Mahjong) is crucial for developing more effective AI agents. Prior state-of-the-art studies on games like Mahjong and Diplomacy have proposed leveraging human strategies as initialization or regularization for self-play, demonstrating improved performance compared to starting self-play from scratch. Therefore, prior self-play methods inherently utilize opponents' strategies. Based on this, we believe our comparison is fair, as we did not rely on additional information beyond what is commonly used.

3. Regarding whether using opponents' strategies as initialization or regularization for self-play guarantees protection against those strategies, we agree with the reviewer that the answer may be "no" for generic multiplayer games. These approaches, as proposed in prior work, are heuristic methods designed to complement self-play and lack theoretical guarantees against opponents' strategies. This precisely highlights the importance of this paper, which develops a principled approach to better leverage the information of opponents' strategy, and provably achieve equal share protecting against opponents' strategy.

Thanks a lot for reading our paper and for your insightful comments.

1. The technique is not strong. Please refer to our general response which discusses the novelty and technical contribution of our work.

2. Some of the literature on the hardness of NE and no-regret learning in games. We thank the reviewer for providing these references about state-of-the-art literature on the hardness of NE and no-regret learning. We will include them in the discussion of related work.