Robust ML Auditing using Prior Knowledge

Dear Reviewer Q6QF,

Thank you very much for taking the time to read and review our paper. We are delighted that you found such value in our work and agree that it is a very real problem that developers can predict evaluations.

How about more general evaluations? In this paper, we chose to focus on demographic parity because it can be very easily manipulated by a malicious platform (answering uniformly at random gives perfect demographic parity). We are encouraged by the results in this difficult setting. Therefore, testing other evaluation metrics in the manipulation-proof framework (and comparing the brittleness of the metrics we currently use) is, in fact, our end goal. About your question on the title and prior knowledge, we would like to refer you to our answer to Reviewer g7Ct (Security Game paragraph) for more clarification.

Discussion on additional references We thank the reviewer for pointing to the work of Mukobi, we were not aware of it. The other papers discuss different shortcomings of current ML evaluation methods (in terms of safety evaluations for Casper et al. and capacity evaluations for Raji et al.). While we did include a reference to another study on how these evaluations are conducted (Birhane et al.), we agree that this discussion on the gap between evaluations and mitigations deserves more attention. Thus, we will add the references you mentioned along with the discussion to the Related Works section.

We thank you again for your positive evaluation of our work and hope that we have addressed your concerns. Should you have additional questions or remarks, we would be happy to answer them in the final discussion phase.

Dear Reviewer Ddwq,

Thank you for taking the time to read and review our paper. We are delighted that you found our work to be scientifically sound with a thorough experimental design and a mathematically sound theoretical framework that also effectively positions itself within the fairness auditing literature.

To be or not to be a groundbreaking new direction We believe that our work is particularly relevant and of interest to the ICML community because:

• Existing manipulation-proof auditing/fairwashing literature all implicitly use this notion of auditor prior. We formalize it, derive general theorems (Section 3), and introduce a new (and natural) type of prior (Section 4). See for example Yan et al., Aivodji et al., Shamsabadi et al. or Yadav.
• The ML security community is well versed in this type of security game formulation but they try to solve them with cryptographic primitives. We believe that the theoretical ML community can also bring nice audit guarantees. Hence, our auditor prior formulation opens up an avenue to explore better and potentially more complex notions of auditor priors.

Standard fairness repair methods As manipulation strategies, we adapted fairness methods based on their applicability to our setting (binary classification, access to sensitive attributes, and compute overhead) and choose the best performing ones. If you have a specific more innovative or up-to-date method, we would be glad to try it in our experiments before the end of the discussion phase.

Further discussion on formal verification of fairness Formal verification of fairness methods (see Section 6) adapts classical verification frameworks (SSAT, bound propagation...) to fairness auditing. To the best of our knowledge, all require a white box access to the model. Thus, while they are very useful for model providers, existing formal fairness verification methods are not applicable in our setting where the auditor only has black box access.

We have corrected the typos you pointed out in the manuscript. We thank you again for your comments on our work and hope that our contributions are now clearer. Should you have additional questions or remarks, we would be happy to answer them in the final discussion phase.

Dear Reviewer g7Ct,

Thank you very much for taking the time to read and review our paper. We are delighted that you enjoyed reading the paper and that, as reviewer Ddwq and Q6QF, you appreciate the importance of the discussed problem.

We will answer your points on the security game formulation, threshold value, and alternative solutions, but first, we would like to address what we believe is a misunderstanding. You mention that The method is essentially collecting predictions of the model before auditing, whereas the method we instantiate in Section 4 is about collecting ground truth labels before the audit.

Security game To the best of our understanding, a "Security by obscurity" approach would try to hide the verification protocol hoping that the platform could thus not manipulate it. In our setting, the audit protocol is public: the platform knows what metric is measured and knows that the auditor will use a prior (e.g., ground truth labels in Section 4). What is private is "the key": the exact realization of the ground truth label, and thus the exact models in $\mathcal{H}_a$. Similar audit protocols can be found in other domains such as accounting. The protocol is public (the company knows that the auditor will look at their records but not all because of resource constraints) but the exact transactions that are examined by the auditor are not known to the auditee beforehand.

Theorem 4.3 assumptions Since the auditor has no prior bias or belief on the model used by the platform, following the notion of uninformative prior, we assume that the auditor considers all models in $\mathcal{H}_a$ to be equiprobable. This assumption and the closeness axiom (i.e., the auditor prior is a good prior) are justified in lines 174-186 (1st column) and lines 245-248 (2nd column). We will update the Theorem 4.3 statement to include and better justify the uniform model distribution assumption.

Decision threshold $\tau$ value We agree that the threshold choice is also highly dependent on the task and data distribution. It is important in practice for the auditors to understand how to choose $\tau$ and this choice can be very much context- and application-specific. We describe exactly how it is chosen in practice in Section 5.4, lines 374-378. We agree that the paper would benefit from an earlier discussion of this setup. We will add it next to Definition 4.1.

Alternative solutions Finding ways to ask the platform to sign each of its predictions is indeed a very active research area for manipulation proof auditing. In the related works we mention some early efforts in using cryptographic primitives for auditing (Yadav et al., 2024; Shamsabadi et al., 2023; Waiwitlikhit et al., 2024). However, beyond the extremely high computational and infrastructure cost of signing model predictions, our paper provides an alternative with a more learning theoretic view on the manipulation-proof auditing problem by deriving the audit guarantees from the auditor's expertise (i.e., prior).

We have corrected the typos you pointed out in the manuscript. We thank you again for your comments on our work and hope that we have satisfactorily addressed your main concerns. Should you have additional questions or remarks, we would be happy to answer them in the final discussion phase.