Regulation Games for Trustworthy Machine Learning

• In Section 1

This is because nowadays ML models are trained, maintained, and audited by separate entities—each of which may pursue their own objectives.
Are there references or real-world use-cases where this is true or implemented? - In California, the Consumer Privacy Act in California stipulates that the Attorney General enforces the privacy of employment background screening. However, the Employment Non-Discrimination Act is enforced by the Department of Justice. These are two separate governmental entities. We believe cross-agency cases such as this should be more prevalent as the use of automated decision making systems grows. Having said that, we are not trained in law so we refrain from making a case reference.

• In Section 1,

... that assumes shared knowledge of a pre-calculated PF between agent objectives.
Can this PF be realized in practice? i.e., how to accurately obtain it? and if only a somewhat inaccurate one can be obtained, what are its implications? - Yes, it is possible to develop a cryptographic zero-knowledge-proof (ZKP) version of ParetoPlay where the agents not only share models but also proof of trustworthiness guarantees (see Shamsabadi et al. 2022 for an example of a ZKP proof of fairness). We expand on this idea in Section J in the appendix, however, this is beyond the scope of the present work which lays the game theoretic foundations for ML Regulation.

- For a discussion on the shared Pareto frontier, please see the general response.

• What is  $i \in I$ in Definition 1?

  • $I$ is the set of objectives. This was left out by mistake and has been added to Definition 1.

• In Section 3,

In this work, we do not consider a competition between regulatory bodies since both are assumed to be governmental agencies.
Even though the regulatory bodies are not set out to compete with each other, the inherent tension between the objectives can lead to competitive strategy profiles and actions, right?

- Yes, that is correct. But the regulators being *governmental* agencies, the higher level agent (the government) can make a decision regarding the fairness and privacy trade-off. We capture this scenario in the first stage of a regulator-led SpecGame in the 2nd paragraph of Section 4.1.

• In that case, what is the significant distinction of "not considering a competition between the regulatory bodies"

  • An alternative game formulation, for example, can consider the regulatory bodies as independent agents (not controlled by a 3rd agent, such as the government).

  • In this setting, the regulators can fight against each other to improve their own metric, possibly to the detriment of the general good. For instance, low-privacy low-fairness can be an equilibrium of such a game, which does not benefit either party. In similar 2-player games, a solution to this is to use a third party mediator to ensure a better equilibrium (see Section 10.7.3 in Shoham and Leyton-Brown 2009 for instance).

  • Note that in our formulation, we already have a natural candidate for such a mediator which is the the government. We have thus shown that a solution to the joint game and subgame reduces to our original game formulation.

• What is B: =\left\\{c_i^{(t)}\right\\}^t   in the overall discounted loss in Section 3.2 ?

  • This is a factor that depends on how the agent perceives its current loss w.r.t its past losses. In the most general case can change over time, $B = B(t)$ ; which is what the expression above models. We have reformulated this general loss as $L_i=\\sum_{t=1}^{\\infty}\\left\\{\\prod^{t}_{i=1}c_i^{(t)}\\right\\} L_i^{(t)}$ which should make this point clearer. Note that, as before, we assume $B$ is constant with time $c_i^{(t)} = c$ which is a common assumption in modeling of repeated games.

• In Section 4,

However, ...the highly non-convex nature of agent losses in
How is the non-convexity addressed?

- By approximating and interpolating the agent losses on Pareto frontier (as we discuss in paragraph titled **Approximation and calibration** in Section 4.1).

• In Section 4,

for $t>1$ , we assume a mapping from the penalty values in $\mathrm{S}$ to trustworthy parameters values $s_{, u y}=(\gamma, \epsilon)$
(How) can this assumption be satisfied in practice ?

- What this assumption says is that for a given pair of parameters $\boldsymbol{s}=(\gamma, \epsilon)$ we can find a penalty to enforce it. Note that this is trivially true for a large enough penalty $C_\text{fair}$ ( $C_\text{priv}$ ) ensuring that fairness (privacy) losses dominate the model builders loss, and loss update in equations 3 and 4. Therefore, for any feasible solution to the joint optimization of training loss under fairness and privacy constraints, this assumption holds; and therefore does not limit the applicability of our algorithm in practice.

• What do the colors represent in Figure 4a?

  • The colors in Figure 4a represent different $C_{fair}$ values. The lightest shade correspond to the lowest $C_{fair}$ , and the darkest the highest. We have adjusted the figure legend and its caption to make this more clear.

• Are the experimental results averaged over multiple trials? If so, is there an analysis of the variation?

  • We run the games on pre-computed Pareto frontiers which we have calculated over many runs of the private and fair learning models we have adopted (FairPATE and DPSGD-Global-Adapt) to ensure we have an accurate Pareto frontier. As a result, the only source of randomization in running ParetoPlay would be the calibration step, where the agents train a model to re-compute the Pareto surface. This pre-calculation of trustworthiness hyper-parameters makes for a relatively smooth (but non-convex) Pareto surface (see Figure 6 in the appendix). As a result we, we have found out that calibration often does not produce a new point on the Pareto surface and therefore we have not observed varied convergence results. The different convergence results we see in the experiments are due to having different starting points and different regulator penalty scalar $C_{fair}$ and $C_{priv}$ .

We once again thank the reviewer for their detailed feedback and kindly ask them to consider raising their score if we have addressed their concerns successfully. We are happy to answer further questions.

We thank the reviewer for their feedback. Below we address each point raised in "Weaknesses" section.

• A key assumption is the common-knowledge of a pre-calculated PF among the considered critera, specifically privacy, fairness and model utility. This can be difficult to satisfy in practice.

  • Please see the general response.

• There are several (simplifying) assumptions made (which can take away the pratical feasibility of the work). For two examples,

  We assume that regulators are able to give penalties for violations of their respective objective which they formulate as a utility (or value) function.

  • Despite nascent privacy laws, there are already many instances of monetary penalties levied for violation of data privacy laws such as GDPR. Therefore, modeling such fines to be proportional to the privacy loss (as we do) is not a limitation beyond the fact that we take differential privacy as our notion of privacy—a choice that we believe is presently justified because differential privacy is the dominant privacy notion for machine learning models.

• We assume the regulators hold necessary information about the task at hand in the form of a Pareto Frontier (PF) which they use to choose fairness and privacy requirements that taken together with the resulting accuracy loss are Pareto efficient:

  • Please see the general response.

We thank the reviewer for their feedback. Below we address each point raised:

• 1-I think the main contribution of the paper is to model the dynamic interaction between the model builder and the regulator. Accordingly, it is more reasonable to think of only fairness or privacy or to possibly even abstract/lump both issues into one. I don't see how having these two considerations has added to the model. One can also consider the safety of the model or its robustness to adversarial manipulations as part of the regulator's concern for example.

  • We agree with the reviewer that our framework is general and can include other trustworthy objectives in future work that builds on our framework.

  • The privacy and fairness regulators can be lumped into one. In fact, we do so at the start of a regulator-led game (in Section 4.1). There we note that considering a joint regulator not resolve the inherent conflicts between the two objectives (fairness and privacy). In practice ML models may be audited by separate entities. Therefore, our goal in keeping the regulators separate was to consider broader and more realistic scenarios. For instance, the Consumer Privacy Act in California stipulates that the Attorney General enforces the privacy of employment background screening. However, the Employment Non-Discrimination Act is enforced by the Department of Justice. A joint regulator tacitly assumes that authorities would coordinate in every decision which is an impractical assumption.

• 2-Why is the paper searching for Pareto Optimality instead of a Nash Equilibrium? Both agents (builder or regulator) are interested in their utility and as a result would deviate to increase it which is what would be captured by a Nash Equilibrium

  • The Nash Equilibria (NE) are not necessarily optimal outcomes for the society. For example, in the US there are fairness regulations that penalize employers for discriminative hiring behavior whereas privacy laws are limited to a few states. This discrepancy can lead the builders to create models that are fair but non-private. The corresponding SpecGame is in a Nash equilibrium $\boldsymbol{s}_1$ because no single party has any alternative strategy that can improve its outcome.

  • Now consider an alternative strategy profile $\boldsymbol{s}_2$ where both fairness and privacy regulators could see an improvement while not hurting the builders utility. Between the two, $\boldsymbol{s}_2$ is a societally optimal strategy profile; and indeed, $\boldsymbol{s}_2$ Pareto dominates $\boldsymbol{s}_1$ .

  • Given that ML regulation intends to achieve societal optimality we search for Pareto efficient points.

• 3-The section “Making a uniform strategy space.” on page 6 is very unclear. What does "consistent" mean? It seems to suggest that the strategies are fixed. Further, it seems that the horizon is n. If that is the case the why does the utility in section 3.2 sum to infinity?

  • By consistent, we mean that the domain of strategies for each stage of the game is the same. Furthermore, the SpecGame is in the class of infinitely-repeated Stackelberg games; therefore $n \rightarrow \infty$ .

  • We have added clarification for both comment to the main text.

• 4-Why is a correlated equilibrium and correlation device well-motivated in this setting? Also the paper mentions that “ This is known as a correlation device. If playing according to the signal is a best response for every player, we can recover a correlated equilibrium (CE). We leave the theoretical proof of this conjecture to future work.” But doesn’t Theorem 1 prove that you have a correlated equilibrium so is it a conjecture?

  • We apologize for the confusion. Indeed the sentence "We leave the theoretical proof of this conjecture to future work." was mistakenly left in the section from a prior submission where we did not have a formal proof).

We once again thank the reviewer for their detailed feedback and kindly ask them to consider raising their score if we have addressed their concerns successfully. We are happy to answer further questions.

• Can the authors clarify the assumption of a pre-calculated Pareto frontier as common knowledge?

  • Please see the general response.

• Can the authors clarify the discrepancy between RQ1 and the contribution claimed at the end of Section 1?

  • We answered this question in the previous section.

• Can the authors provide a more detailed description of the experiments in Section 5, as well as the reasoning for choosing the hyperparameters described in Appendix I? Particularly, the step size discount factor and the loss function weightings λ_fair and λ_priv?

  • We have added a more detailed description of the results as well as more informative captions to the paper (as per reviewer's prior comments). We would be happy to address any other specific confusions about the results.

  • Regarding the discount factor. The $c$ discount factor is one of three scalers which control the convergence behavior; with the other two being $C_*$ and $\lambda_*$ . The discount factor $c < 1$ ensures convergence of the repeated game (as we discuss in Section 3.2). With other factors held constant, as $c$ decreases the dynamics remain the same, but the game will be extended in time.

  • Regarding penalty scalers. $\lambda_*$ are scalars that make the (monetary) penalties comparable to the model loss. Note that $\lambda_*$ s are private information to builder (e.g, 1 million privacy fine compares to the gains from producing a 1-loss-unit lower loss from the trained model). Since regulators do not observe $\lambda_*$ they adjust the impact of their penalties using $C_*$ . In summary, $\lambda_*$ and $C_*$ both control the game dynamics but since we seek policy guidance for regulators and those only control $C_*$ we set $\lambda_*=1$ and run ablations with $C_*$ .

We once again thank the reviewer for their detailed feedback and kindly ask them to consider raising their score if we have addressed their concerns successfully. We are happy to answer further questions.

We thank the reviewer for their feedback. Below we address each point raised in "Weaknesses" section.

• The assumption of a pre-calculated Pareto frontier as common knowledge does not have theoretical proofs and the discussion around the empirical evaluation in Appendix J is lacking.

  • Please see the general response.

• Throughout the main body, the authors sometimes refer to notations that were not defined previously. For example, in Section 3.2, notation $c_i^{(t)}$ and $L_i^{(t)}$ are the first time the "(t)" superscript is used. In Equation 4, the notation $\nabla_s$ is used without a definition.

  • We have added the clarification of all the aforementioned cases (marked in blue in Section 3.2 and Eq. 4)

• The discussion of the experiments in Section 5 is confusing. At the end of Section 1, the author claimed that the experiments would highlight the suboptimality of studying trustworthy ML in a single-agent framework. However, in Section 5, the first research question shows that multi-agent setup leads to sub-optimalities.

  • We agree with the reviewer that wording of RQ1 was confusing. We have re-written it for clarity to:

    (RQ1) if regulators produced specifications for trustworthy parameters $\boldsymbol{s}$ using a single-agent-optimized model, would that recommendation be respected in the multi-agent setting?

    and the description of the results has also been updated to:

    Single-agent-optimized specifications lead to sub-optimalities in the multi-agent settings

• Figure 1 does not have a legend. Overall, most figures are hard to read and the captions do not provide sufficient description of the experiments.

  • We have addressed the reviewer's comment and re-wrote the captions for all experiment figures for clarity.

• Minor typos: ell_build(w) instead of ell_b(w) on page 5 under Equation 3.

  • We thank the reviewer for their attention. We have fixed the typo.

Dear reviewer,

Thank you for your feedback on our work. Your suggestions have improved our work. As the author interaction is coming to an end, we are waiting for your post-rebuttal response. If our response answers your concerns, please consider raising your scores. If not, please let us know of your further questions such that we can answer them in the remaining time.

We thank you for all your efforts!

Dear reviewers,

Thank you for your valuable feedback. We believe it has improved our submission. Given that the author interaction window is coming to an end, we would like to kindly ask you to consider the rebuttal responses and let us know if they address your concerns and, if they do, to consider raising your scores.

It would be our pleasure to answer any further questions.