A Cooperative-Game-Theoretical Model for Ad Hoc Teamwork

1. In Theorem 1 it should be made clear exactly what "maximizing the social welfare under the grand coalition" means. I am interpreting it as finding a joint policy, that is, a map $\pi: \mathcal{S} \times \mathbb{P}(\mathcal{N}) \in (s\_{t}, \mathcal{N}\_{t}) \mapsto a\_{t} \in \mathcal{A}\_{\mathcal{N}\_{t}}$ that, for every pair $(s\_{t}, \mathcal{N}\_{t})$, selects $a\_{t}$ to maximize the local social welfare $\sum\_{j \in \mathcal{N}\_{t}} R_{j}(s\_{t}, a\_{t})$. Is that correct? In any case this should be formally stated.

   Reply: Thanks for your suggestion. Your understanding about the result of Theorem 1 for fitting the problem we would like to solve (open ad hoc teamwork) is correct. However, the result from Theorem 1 is general for any preference value profile in CAG. Note that in the original version of CAG, it does not consider action space, while in our theory we extend the definition of preference value with action as an decision variable (i.e. a realization of preference value). Nevertheless, we agree with you that the statement should be linked to our problem more directly to make it better comprehensive. For this reason, we have added a corollary to comprehend the result of Theorem 1 in our case in the revised paper.

2. It seems like the paper is adopting a single-agent perspective, where there is a single learner $i \in \mathcal{N}$ and all other agents' policies are held fixed (is this correct?). But then how can we optimize social welfare, required by Theorem 1, if we only control the single learner $i$? In particular, what if the other agents are acting in such a way that learner $i$ alone cannot achieve social welfare optimality?

   Reply: Thanks for your really thoughtful questions. Yes, in the common setting of ad hoc teamwork, we can only control one agent to collaborate with other teammates on the fly. To make the theoretical analysis simpler, we here assume all other agents' policies are held fixed (i.e., to keep the environment stationary to the learner we control). However, this work can be extended to scenarios where other agents' policies would vary, for which the analysis would become complicated due to the non-stationary envrionment.

   Although we only control the learner $i$, it can affect other agents' decision, since other agents are able to respond the learner's action with regarding the learner's behavior as partial input of their policies. As a result, other agents' individual preference reward as an evaluation of their policies would accordingly change.

   The final question is really insightful. Since the basic assumption of ad hoc teamwork is that all agents are assumed to have a common goal, so the situation you consider is out of scope. Nevertheless, if the situation you state really happens (that is an open question), then for the moment we can only say it could be extended to control a group of agents, where at least one agent is able to influence other uncontrollable agents. We hope our response addresses your concerns.

3. At the beginning of Sec 3.3, I don't understand the point of defining $\mathcal{CS}\_{t}$ only to later set $\mathcal{CS}\_{t} = \mathcal{N}\_{t}$. Does this have a purpose? It seems cleaner to just not define the extra symbol.

   Reply: Thanks for your suggestion. We agree that your suggestion would make the definition more concise. However, the definition of $\mathcal{CS}_{t}$ here has two purposes. The main purpose is that in the future work this theoretical framework can be directly used to extend to the scenarios where multiple agents divided into different coalitions (e.g., to solve multi-task problems) can be controlled. The second purpose is that we would like to keep the tight connection to the past works. This would make the readers understand the overall relationship between our work and past works, which avoids the potential of reinvention in the future.

4. In section "Representation of Preference Q-Values", there is a clause beginning "which is presumed ... goal". What does "which" refer to here? Is this an assumption required by Theorem 2? If so it should be formalized mathematically.

   Reply: Sorry for the confusion. This "which" refers to the condition in Theorem 2 such that $w_{jj'} \geq 0$. We have removed it in the revised paper to keep conciseness and avoid the confusion, since even if $w_{jj'} \geq 0$ does not hold, we can still manually add bias during learning without changing the final result as we discussed in Section 4.4 in the revised paper about practical implementation. Thank you for your mention.

Thank you for the fast response as well. I am willing to engage in discussion about this paper, as should be evidenced by my own fast replies here. As I did say in my review (and you have definitely noticed), this specific area is not very familiar to me. I also cannot become a topic-area expert in a matter of a few days. That said, I do work more broadly in game theory, so I would think that I am at least somewhat in the target audience, enough that I should be able to understand the broad takeaways and motivations of the paper.

I think I am understanding a bit better, but still missing something very crucial. I hope this time though my comments are more pertinent.

Theorem 1 states that the optimal joint policy exhibits strict core stability. So, since you're not finding an optimal joint policy in any sense, Theorem 1 does not apply to your algorithm. Now, assuming I've understood this part correctly, I ask/recommend:

1. I think the above should be explicitly stated in the paper immediately after the statement of Theorem 1, and also in the same place it should be explained why Theorem 1 is even present/relevant to the paper, given that it does not apply to your main algorithm. Otherwise a reader could easily assume (as I did!) that the paper will want to somehow achieve a joint optimal policy, or something like it, in order to apply Theorem 1, and get confused/fixated.
2. The paper very quickly moves from jointly optimizing the social welfare (Theorem 1/Corollary 1) to a single-agent optimization problem (Eq. (3)). The relationship between these should be explained, since solving Eq (3) does not in itself guarantee welfare optimality. This may be related to the previous point.
3. Which theoretical results of the paper do apply to your algorithm? Can you state a concrete result along the lines of "if we run CA-GPL for one agent, with other agents' policies fixed and arbitrary, we will achieve strict core stability"? If so, such a result should be explicitly stated. If not, you should explain what guarantees are achieved by your algorithm, because right now the theoretical results are about strict core stability in general and it is not clear how they relate to your algorithm.
4. For what joint policies do the results of Section 4.2 hold? It looks to me like those results should hold for all joint policies, but if so, that should be explicitly stated. And, in that case, I ask what Section 4.2 has to do with learning a good policy at all, because if all joint policies satisfy the desired property then the satisfaction of the property isn't a constraint on the learning at all.
5. In Section 4.2, it seems that you are making assumptions about the structure of weights $w$ (nonnegative) and rewards ($R_j(s_t, a_{t, j}) = (1/|{-i}|) R_i(s_t, a_{t, i})$). These assumptions should be justified; they seem quite strong to me.

If I have misunderstood something above, I'd be happy for the authors to correct me. I am pretty sure I am not the only reader who will have these confusions, so I do hope that this conversation leads to a stronger, more broadly understandable writeup.

The reply about types makes sense to me. Thank you for clarifying.

Thanks for your so fast reply.

Before answering the further questions, we urge the reviewer to calm down when meeting an unfamiliar area during review and give full respect to this area. We are always patient to answer your questions.

1. (Single-agent perspective) This remark from the authors raises more questions to me than it answers. The assumption that all other agents are playing a fixed policy completely defeats the purpose of investigating a multi-agent system in the first place---you can treat agents with fixed policies as part of the environment and just solve the resulting (single-agent) POMDP. If this paper is really about a single-agent perspective, the authors should do much more to justify why we should believe that this single-agent perspective is relevant to a multi-agent system. (For example, if all agents run your algorithm, what happens? There is a lot of work investigating multi-agent learning dynamics in games, and I believe that the authors should mention that line of work, and fit into it where possible.)

Reply: The main issue is that the reviewer is not familiar with ad hoc teamwork. What this work did follows a topic called ad hoc teamwork which is totally different from multi-agent learning. In ad hoc teamwork, the perspective is to control an agent, while in multi-agent learning, the perspective is to control multiple agents. Also, we have mentioned the difference between ad hoc teamwork and multi-agent learning in Related Works. We hope the reviewer can check the paper carefully before raising questions. As the reviewer said, the ad hoc teamwork can be regarded as POMDP, however, the key point is that POMDP is a very general model. If the details of such a model is unknown, how should we well solve the further complicated problem you raised that "if all agents run your algorithm"? The setting in the reviewer's question would make environment non-stationary (or time-varying from the perspective of control theory), which is impossible to be solved if we know nothing about the environment. This is the reason why we study the agent modelling first with the assumption of fixed teammates' policies, which we believe is the key step towards the more complicated problems. Does the reviewer agree with this?

2. I'm not sure I understand. Are you saying that you're assuming that other agents are playing parts of an (welfare-)optimal joint policy? Or that, if all agents run your algorithm, they are guaranteed to converge to a optimal joint policy? (this would be very surprising to me if it were true, as most algorithms usually do not have this property unless explicitly designed for it!) Or that, regardless of the policies of other agents, there is always a policy for the learner such that the resulting joint policy is optimal? Or something else entirely? In any case, the claim here requires explicit statement and justification. But this comment raises yet another question: what does it mean for agents to have a "common goal"? They don't share a reward function, right? Doesn't that mean that their goals may be misaligned?

Reply: We never say that we would tend to find a joint optimal policy or we finally find the optimal joint policy. The learner's policy obtained (combined with other teammates' policies) is almost a sub-optimal joint policy, which is totally different from multi-agent learning (which aims to optimize a joint policy, so that it can find a joint optimal policy). The common goal can be interpreted into two scenarios: all agents share a reward function to describe the common goal, or each agent has a reward function that is the composition of a shared reward to describe the common goal and an individual function that is not conflicting with the common goal (see Section 2.1 in [1]). The key problem in ad hoc teamwork is that even if all agents have the shared reward, it is difficult to make them collaborate if they were never trained in the same environment before. Does it make more sense now?

3) (on $\mathcal{CS}\_{t}$) I feel that the importance of clear writing in this paper outweighs the concerns raised by the authors; as such, I'd advise streamlining the notation by removing $\mathcal{CS}\_{t}$.

Reply: We respect the reviewer's viewpoint, however, this is not the critical point to decide the acceptance of the paper.

Reference

[1] Mirsky, Reuth, et al. "A survey of ad hoc teamwork research." European Conference on Multi-Agent Systems. Cham: Springer International Publishing, 2022.

(Note: You've numbered your responses off-by-one from my questions. For continuity, I'll switch to your numbering.)

3-4) This answer is somewhat disturbing to me. Are you saying that, without further conditions (e.g., on how other agents are reacting to the learning agent), none of the theoretical results stated in the paper apply to the main algorithm? (Because the algorithm cannot guarantee welfare optimality, which is required for Theorem 1 and therefore according to your response is required in order to say anything about your algorithm).

A previous comment in this thread stated very clearly that this paper is not claiming to find a welfare-optimal joint policy, which I agree with. But yet the theory seems to depend on an optimal joint policy being found. This seems to be a fundamental disconnect here between the algorithm and the theory that severely limits the relevance of the theoretical results.

5) What, formally, is "the assumption of ad-hoc teamwork"? Is it a condition on the rewards/affiinity graph structure in addition to the structure in Section 3.1 and 3.2? A condition on the training procedure (Not sure what precisely that would entail, should specify)? A condition on the learned policies (e.g. that the joint policy is welfare-optimal, as in Theorem 1)? Theorem 2's statement and proof doesn't seem to include any such additional assumption, so if there is an additional assumption it should be explicitly stated.

6) Okay, seems like I've misunderstood something here. Are you saying that your algorithm is allowed to change the weights $w$? Section 3.1 makes it look like $w$ is part of the input/problem definition, so you can't change them as you see fit to achieve a desirable result. If the algorithm is allowed to set the weights $w$, this should be explicitly stated; it is yet another rather strong assumption that the paper seems to be implicitly making.

Actually, regarding the reward assumption, another question: does it have to hold for every $s_t, a_{t, i}, a_{t, j}$? Or only some? If only some, which ones? If every, I think we have a problem:

1. By varying $a_{t, j}$, one immediately concludes that every action for every player must have the same reward, i.e., that the reward must be independent of the action.
2. If there are $n>2$ agents, then applying the condition twice yields $R_j(s_t) = R_i(s_t)/(n-1) = R_j(s_t)/(n-1)^2$ (where I omit the actions because, by the previous point, the reward must be action-independent) but the only reward function that satisfies this is the reward function that is identically zero, which is obviously not very interesting. Am I missing something here?

Regarding Theorem 1 and joint (welfare) optimality: I think you are saying that the "only gap" in the application of Theorem 1 is that the joint optimal policy may not be reached. But, in an earlier message, you stated very clearly that your algorithm does not in any way claim to allow the joint optimal policy to be reached. So, this "gap" is not a small gap at all; rather, it is a fundamental limitation of the algorithm that prevents the application of the main theorem!

To me, this gap is a severe limitation that undermines not only the relevance of Theorem 1, but also the relevance of the whole approach of the paper since, if I understand the authors correctly, all the theoretical results about the main algorithm depend on joint optimality.

Also, if there is some way to formalize what it means to "influence" the other agents, what conditions are being implied here about how the other agents are learning/reacting to the learner's play, and what that means about the eventual joint policy of the players, that would be very important to include. But it is not really directly relevant to my main point above; that point only relies on the authors' own statements that:

1. the main algorithm is not claiming to reach a joint optimal policy.
2. positive results about the main algorithm depend on reaching a joint optimal policy ("If we satify the results from Theorem 1 (optimizing social welfare) and Theorem 2 (confining the weights of the learned affinity graph), then it is possible for us to reach the strict core stability.")

Am I missing something?

Regarding being allowed to change weights: Section 3 is written in such a way that it is implied that the weights are part of the input, i.e., they are not things that are learned by the algorithm. Section 3 should be rewritten to fix this.

Regarding the reward assumption: I see. So, under the assumptions of that section, singleton coalitions are assumed to have reward zero when $n>2$, but non-singleton coalitions can still easily have nonzero reward. Both these points should be stated explicitly in the text.

I think we will agree to disagree here. Here's another way of phrasing where I am stuck: this work is introducing a new framework based on cooperative game theory to achieve ad-hoc teamwork. But, in some sense, one of the main premises of ad-hoc teamwork is that you can't expect to achieve the joint optimal policy because you don't control what the other agents are doing (otherwise, it'd be a MARL problem). So the field attempts to understand what is achievable in this setting when you have so little control. I understand that motivation. But, with that in mind, developing a framework whose entire conceptual/theoretical logic is "if we achieve the thing that we just said is not a reasonable expectation (joint optimality), then we achieve a good outcome (strict core stability)" now seems almost contradictory to the above premise.

I would strongly suggest that the authors revisit the fundamental conceptual premises of the paper, without assuming joint optimality. There could still definitely be interesting directions to investigate at the intersection of cooperative game theory and ad-hoc teamwork. For example, there could be other interesting statements that can be made using cooperative game theory that do not assume joint optimality. Or perhaps, one could experimentally check whether your joint policies exhibit nice properties from cooperative game theory (such as, but perhaps not limited to, core stability), even if they are suboptimal? Of course, I cannot expect this to be done in what remains of the discussion period, so these are suggestions for a future revision.

Reply: Your understanding is generally correct. However, it could be helpful to rearrange the logic.

The paragraph that follows this sentence was not obvious to me. The information/intuition in that paragraph should be explicitly discussed in a revision.

I think I will not have much more significant or new things to say at this point, and I have spent already a very large amount of time on this paper, so this will be my last message here and I will maintain my score. Thank you for being engaging in the review process. I hope that the discussion has been helpful.

1. Can you please elaborate on what do types represent since all teammates always want to collaborate? This is not explained in the text.

   Reply: The types here is for emphasizing the heterogeneity of agents. Even if all teammates always want to collaborate, a group of different types of agents may collaborate in different manners. Thereby, in this paper we apply affinity graph with learnable weights (capture variant collaborative manner) to depict this property.

2. Can you please highlight the main theorem that addresses the motivating question at the bottom of page and top of page 2? I admit that I got lost in some parts of the theoretical presentation.

   Reply: Sorry for making you lost owing to our presentation. For convenience, we will reply following the layout of the revised paper. The answer to the motivating question is constituted of the holistic theory. It starts from the introduction of CAG with the affinity graph and the solution concept for solving the weights of the affinity graph called strict core stability (see Definition 1). Then, we define the affinity graph as a star graph to better describe the scenario of open ad hoc teamwork (see Definition 2). After that, we derive a method to find the solution of weights (see Theorem 1 and the derived Eq. (4)) and an inductive bias for easily finding the weights (see Theorem 2 and Proposition 2). Finally, we show the feasibility of solving Eq. (4) under the open team scenario (see Lemma 1 and Theorem 3). In summary, our theory answers the question that why introducing and learning a graph is necessary and feasible to solve open ad hoc teamwork from the theoretical perspective rather than the pure intuition in the GPL paper, so the method derived from our theory is reliable.

3. An algorithm environment for CA-GPL would have aided the reader to appeciate the novelties over GPL.

   Reply: What do you mean by an algorithm environment? Could you please give some more information about it? We have added an achitecture of CA-GPL to illustrate the novelty over GPL in the revised paper as Figure 1 shows.

4. If I am not mistaken, the notation seems too complicated and redundant. For instance, if I understand correctly, it holds that v_j$C$ = w_ji = R_j and later on R_j = \alpha_j + (other terms). Similarly for the social welfare that is defined as sum (of sums) of such terms. Thus, I had difficulty to follow the proofs. Is my concern valid?

   Reply: Thanks for your suggestion. Your understanding for our own theory is correct only for the teammates $j$. For learner $i$, the instantiation of $v_{i}(C)$ or $R_{i}$ to ad hoc teamwork is different, while the abstract definition of $v_{i}(C)$ is the same as $v_{j}(C)$ in the most general hedonic game. More specifically, $v_{j}(C)$ is the most general definition over any arbitrary coalition $C$ in hedonic game. In our case, we replace $v_{j}(C)$ by $R_{j}$ to make the symbols consistent with convention in reinforcement learning. Using $w_{ji}$ to specify $v_{j}(C)$ is only valid in CAG which is a subclass of hedonic game. $w_{ji} = R_j$ is only valid when we consider the star graph. If we initially define $w_{ji} = R_j$ with ignoring $v_{j}(C)$, this would make our theory too narrow and less valuable to be extended to more complicated scenarios as we mentioned in Conclusion. Similarly, \alpha_j + (other terms) is an instantiation of $w_{ji}$. In summary, our theory is formulated from abstract (the known general theory) to detailed scenarios (the situation of ad hoc teamwork) step by step, with both concerns of the extension of our theory and inheriting past works. We wish you can understand our thought and are happy to have more discussion on it.

5. what is $|-i|$? Is it the cardinality of the set of all agents other than $i$? If yes, then first, this is never explained, and second why not simply write $N_{t} - 1$?

   Reply: Yes, thanks for the suggestion. We do not write it as $N_{t} - 1$ is for the conciseness in expression.

6. what is $b_{j}$ in the bottom of page 2? Is this used later on?

   Reply: $b_{j}$ was originally used to indicate that the singleton coalition value is a constant value greater than or equal to zero. However, we agree with your suggestion and modify it as $v_{j}(\mathcal{C}) = v_{j}(\{j\}) \geq 0$ in the revised paper.

1. when introducing an affinity graph, the values from the singleton coalition are not ported to the new representation, and then they are used when discussing individual rationality.

   Reply: Thanks for the insightful comment. We were also struggled with the definition of the reward for a singleton coalition. After deliberation, we believe that it is unnecessary to define it with a new representation, since we have defined the reward over any arbitrary coalition in Section 3.1, and the reward for a singleton coalition is just a concrete case.

2. in many instances of the paper, assumptions are introduced without justification and they are not properly highlighted (just a "There exitst..."). As an example the existance of an affinity graph is not guaranteed in general as far as I know.

   Reply: Thanks for the interesting comment. "There exists ..." is a type of words to given a boudary, based on which we can analyze the results following logics. As for the existence of affinity graph, we now add discussion on the boundary in the revised paper (See Section 3.2 Affinity Graph for Ad Hoc Teamwork). We sincerely hope you could double check it.

3. Many different probability distributions are introduced (...) in Section 3.1, making really obscure the dependencies across variables, Figure 5 from the appendix and a clearer text may help.

   Reply: Thanks for your comment. In our view, different probabilistic distributions defined here does not obscure the dependencies across probabilities. Instead, it clarifies the dependencies, since compared with expressing the transition function as a whole probability distribution in SBG, it points out the position of intervention (control) during the whole ad hoc teamwork process, following the view of causal inference. Nonetheless, we agree that a figure could help understanding, we have redrawn Figure 5 in the last version and it is now Figure 4 in the revised paper.

4. lack of an example to clarify the settings and its peculiarities.

   Reply: Thanks for the insightful comment. This paper is not like the purpose of popular papers that capture a weakness that a prior work cannot solve and propose a method that can improve the weakness. The research question (motivation) of this paper is that the powerful GPL framework does not have a theoretical understanding of why the incorporation of graph can help. For this reason, we propose to incorporate CAG and surely its theory to mitigate this issue. More importantly, the incorporation of CAG can lead to the possibility the further investigation into more diverse situations (e.g., friend and enermies both exist). Although GPL seemly can address almost every situations, the further improvement of this architecture is a big problem. The main reason is that it does not consider any boudary from theoretical perspective. However, the question is that we do not know whether the design is the best to any specific situation. For example, in this paper, we propose to constrain the range of individual and pairwise Q-values to be greater than or equal to zero, according to our theory. Similarly, we can follow the route of the various and well studied research of cooperative game theory to analyze other scenarios (e.g. friend and foe).

5. the term 'Preference Q-Value' is unneeded, as it is just a Q value over the action space of the agent

   Reply: Thanks for this interesting point. We use the term "Preference Q-Value" is to credit and respect the work of CAG and cooperative game theory as well as to distinguish our work from others, since the Q-value is derived by the theory over preference value which is the terminology in cooperative game theory. If following your logic, it is also unnecessary to define Q-value since it is just a function. We sincerely hope you could reconsider your statement.

As a reviewer also, I believe this is constructive to the whole community, whatever you change the score or not (which is your freedom). Thanks.

By the way, we have to mention that your words "but the paper has to be rebuilt in my opinion." is one of the arrogant words we have never seen before!

We thank the reviewers for response.

The main purpose of our paper is to propose a general theoretical model that can help improve ad hoc teamwork, rather than just fix pitfall existing in GPL. In other words, GPL is just a realization of the algorithm from our theoretical model. For this reason, we do not agree with the reviewer's suggestion.

On the other hand, the purpose of mentioning close relationship to GPL in details in our paper is to show our respect to the GPL work. If this instead distracts the main purpose of our paper, we would consider removing it from the main part of paper and just talking about it in Related Works. Thanks for your feedback to consolidate our determination to drop the unworthy part in our current contents.

Finally, to our best knowledge, if our response has addressed your questions or concerns, it should be deserved to raise your score. This is main purpose of your questions from the policy of review: Think of the things where a response from the author can change your opinion, clarify a confusion or address a limitation. This is important for a productive rebuttal and discussion phase with the authors.

We strongly suggest the reviewer to follow the policy in the next review experience, otherwise, it is unfair to the authors' efforts. Please respect everyone's effort and take the resposibility of avoiding the phenomenon of withdrawing after receiving not positive scores which is not helpful to the community. To keep the healthy development of the community is the obligation of every reviewer.

We sincerely thank you for your effort on the reviewing our work again.

We appologize for our quite aggresive words yesterday, due to our bad mood. Hope you can understand us.

However, we still do not totally agree with your suggestion on emphasizing results (takeaways) through weakening the thinking process. We believe the thought of our paper is more significant than the resulting algorithm. We may have very different philosophy in research, but no one is incorrect.

Nevetheless, we may attempt to find a trade-off between our thoughts in our next revised paper, to convince you (if it is likely that you will be still a reviewer for our paper in the next venue).

Thanks for your service again!

1. Considering that the idea of combining open SBG and CAG is quite natural, and so does the RL formulation.

   Reply: Thanks for your very critical comments. However, we would like to argue that the combination of SBG and CAG is not trivial or natural. The primary reason is that SBG is actually following the non-cooperative game theory framework, while CAG is under the cooperative game theory. These are totally different concepts, where SBG follows the solution concept of Nash equilibrium, whereas OSB-CAG or CAG follows the solution concept of strict core stability. In Section 4, we show the equivalence between these two concepts, then it establishes the connection between SBG and OSB-CAG. Therefore, it not only makes the GPL as an implementation of our framework, but also understands GPL from a theoretical perspective. Note that the original GPL only intuitively (empirically) extends the framework from SBG with a graph to handle open team settings without any theoretical understanding. More importantly, according to the theory from CAG, we propose to set the range of individual utilities and pairwise utilities as greater than or equal to zero, which was never mentioned in GPL.

2. the difference of the outperformance seems marginal.

   Reply: Thanks for your comments again. If possible, could you please mention the concreate cases where our method only outperforms GPL marginally? In our understanding, CAG-GPL (our method) ourperforms GPL in every scenario, and with a large margin in most of scenarios, which is also recognized by Reviewer eRVD.

I thank the authors for the response.

Regarding the margin, I believe Figure 2a, 2b, 2f have almost no margin, and the margin is literally "marginal" in 2c, 2d, 2e. Correct me if you do not believe so.

I also read through other reviews and the discussions. I was a bit surprised that the authors decided to position the work as a "theoretical framework", which seems to be quite ambitious. The presentation did introduce many ambiguities and could this be one of the reasons? It is a natural question how much does this work contributes to theory.

Dear Reviewers and ACs,

Thank you sincerely for the time and effort you invested in reviewing our paper. Your insightful feedback and constructive suggestions have been invaluable in enhancing the quality and clarity of my work. We have carefully considered and incorporated your recommendations to improve our paper. Additionally, we have tried our best to address your concerns.

We sincerely hope our response can help you understand our work better. If you still have concerns, please raise them without hesitation. We are happy to answer any questions.

Your support and guidance are deeply appreciated, and we are grateful for your contribution to advancing knowledge in our field.

Best,

The authors